flan/HANDOFF-emacs-flake.md

2.8 KiB

The test_emacs flake

The symptom

One run of dune test --root . exited 1; the next run on the same tree exited 0.

  FAIL the client notices a stop nobody asked about
  FAIL and the modeline says so, with the condition
Error: error ("flan dev: cannot reconnect to /tmp/build_997a01_dune/flan-emacs-dev.sock: make client process failed: Connection refused, ...")

What the error text already rules out

Two mechanisms have produced something like this here before: a genuine listen/connect ordering bug, and a five-second await against a cold llc-and-link. Neither fits this text, and the reason is in the branches:

  • A 20s poll deadline that simply expired leaves the connection live. The two FAILs would print and flan-dev-restarts would then succeed. There would be no Error: line at all.
  • The daemon exiting — the child program finishing, serve returning closed, or any OCaml exception — runs Fun.protect's finally in Dev.two_process, which unlinks the socket. The client then takes the (not (file-exists-p ...)) branch and says "nothing is listening on ...; start flan dev program.flan again". A different message.
  • Connection refused on a unix socket needs the path to exist and no socket in listen state behind it. In two_process (lib/dev.ml:2514-2526) bind and listen are adjacent, nothing else ever creates that path, and the only close of the listening fd is in the same finally that unlinks. So it is reachable essentially one way: the daemon died without running its finally — that is, it was killed by a signal.

The machine was loaded (several compiler lanes building at once) and /tmp filled to 100% the day before, so the OOM killer is the first candidate.

The plan

  1. Fix the observability first. test_emacs.ml throws away the one fact that discriminates: the cleanup waitpid matches | _ -> true and drops the status, and the temporary files are removed before the exit code is checked, so the program's output is gone by the time anyone looks.
  2. Reproduce by running _build/default/test/test_emacs.exe in a loop under artificial load, recording the daemon's wait status, df /tmp and dmesg | tail each iteration.
  3. Fix the mechanism the evidence names.
  4. Sweep the siblings — test_repl.ml, test_dev.ml, test_cider.ml. What they share with test_emacs.ml is the discarded wait status, not a missing connect retry: they already have retrying connect helpers.

Open questions

  • lib/cimport.ml:1138 still spells its own cachedir under $TMPDIR, which NEXT.md:694 flags as the last remnant of the cold-cache bug. Another lane holds that file. It is both a per-run cold cache and a plausible filler of /tmp.

Status

Stub. Investigation in progress.