flan/HANDOFF-f3.md

Handoff — a watch for a running program (PORTING.md Tier 1, item 5)

Written because the session was wound down for budget. The work is finished and dune test is green, run twice. This file exists so the reasoning is not re-derived, and so the two things I would have done next are named.

---

The finding that reframes the task

Most of item 5 was already built before this session. The brief reads as though a watch had to be built from nothing; it did not. What existed at 344e571:

• runtime/flan_dev.c — the watch table: 64 slots, a name and rendered text per slot, a per-slot seqlock, an on/off flag so a watch call costs a load and a not-taken branch when nobody is looking, and four scalar entry points (flan_dev_watch_i64/_u64/_f64/_str).
• vendor/agent/flan_agent.c — watch, watch on, watch off on the agent socket.
• lib/dev.ml — watch_read, watch_enable, and the :op "watch" / :op "watch-enable" arms.
• emacs/flan-watch.el — the watch buffer and the inline ghost text, both fed from one reply.
• test/test_dev.ml — a block driving a real daemon and a real running program.

That is watch.clj's spy, end to end. What was missing was spy-num — the numeric accumulator for hot loops, which is precisely the part PORTING.md calls "least obvious and most valuable". That is what this session built.

---

The accumulator decision

This is the expensive part to re-derive, so it is stated in full. The long-form version is in BUILT.md, "A hot loop keeps five numbers, and the window is the editor's"; runtime/flan_dev.c carries it at the code.

What a slot keeps

Count, min, max, last, mean. Five numbers per label.

The scalar watch keeps one value per name. Sampled from a hot inner loop that is nearly useless — you see whichever of the 91,200 cells happened to run last. watch.clj's own docstring says so and is why spy-num exists there.

Each of the five answers a question you can ask without building a query:

• n — how many times the expression actually ran. This is the first thing that is wrong when a loop is wrong. A count that tracks the frame counter rather than the cell count is a loop that is not running.
• min / max — the range. This is the thing a single sample can never show you, and it is what you are looking for when you suspect an index or a velocity is leaving the region it should stay in.
• last — the one sample. Kept because it is what the scalar watch would have given you, and losing it on the way to something richer would be a regression.
• mean — carried as a running sum and divided at read time. A mean accumulated as a mean drifts; a sum does not.

What it deliberately does not keep

A ring, or a history. A small ring of the last N samples was the serious alternative. It loses on the only ground that matters here: N samples out of 91,200 is a sample of the tail of the loop, not of the loop. It answers "what did the last few cells do" when the question is "what did the cells do". Past five numbers, every richer answer is a UI for building a query — and a query builder is the one thing this whole design exists not to be. The ghost text section of BUILT.md already settled the same question the same way for the scalar case ("a watch inside a loop shows the last value written... every better answer is a UI for building a query. Settled, not open.").

The write path does no formatting

This is the feature, not an optimisation. An snprintf per sample at thousands of samples a frame is a HUD that costs more than the game. A sample is: one relaxed load of the epoch, five compares and stores, and the slot's seqlock. The reader — the agent's listener thread, once per editor tick — turns the five numbers into text. watch.clj reaches the same place with a double-array per label and a render function Emacs calls.

This is also why the slot grew a num flag rather than getting a second table: the read path already copies under a seqlock, so it copies five doubles instead of 192 bytes and renders them out of locals after the counter check.

The window is since the last reset — a deliberate divergence from watch.clj

watch.clj is cumulative until reset-spies! is called by hand. This is not, and the disagreement is on purpose.

Cumulative is the wrong default for a frame loop. A min and a max over a whole session reach the session's extremes within a few seconds of play and then never move again — so the two most useful of the five numbers go dead exactly when you start interacting with the thing you are debugging. This tool exists to show you a number while you drag the mouse. So flan-watch--tick sends :reset t beside its read, and what you see is "since you last looked": a fifth of a second, about a dozen frames, which keeps the range tracking the present.

A caller who wants cumulative numbers gets them by not resetting. The setting lives in the editor, not the runtime.

Reset is its own message, never a side effect of reading

A destructive read was the tempting shape and is wrong: it makes looking change what is there. Anything that polls — a test's await, a second editor, a person reading twice — would silently shorten the window and come back with a count that is noise. So watch reset is its own agent command, :reset t is a field on the read op, and dev.ml sends the reset after the read so a tick reports the window it just closed.

This was settled before the test was written rather than after, which is the only reason the test can assert anything about n at all.

The reader never writes the table

Reset bumps one global epoch counter and touches no slot. A slot clears itself on its next sample, when it notices the epoch moved, and does so inside its own odd-generation window so a reader can never catch a half-cleared slot. The game thread stays the only writer of the table — the invariant the whole watch section of flan_dev.c rests on.

The cost: a new window begins when the program next runs, not at the instant of the reset. For a frame loop that is the only moment it could sensibly begin. This bit me in the test (see "What did not work" below) and the test now waits for it and says why.

i64 accumulates as a double

A magnitude past 2^53 loses precision in the sum and in the ends of the range. Recorded rather than designed around: a count, a coordinate and a tile index are what this is pointed at, and a second integer accumulator for a case nobody has would be two code paths for one tool. watch-num-i64 and watch-num-f64 are two entry points only so a program need not cast at the call site.

A whole number prints as one

watch.clj's fmt-num does this and the reason survives the port: a watch on an array index that reads 66.0000 sends you looking for a rounding bug that is not there.

---

What was built, file by file

All of it is working — built, run against a real daemon and a real running program, and covered by the test. Nothing is stubbed and nothing is half-written.

File	Change	State
runtime/flan_dev.c	watch_slot grew num, epoch and five doubles; watch_begin clears num; new section "A number sampled thousands of times a frame" with watch_record, flan_dev_watch_reset, flan_dev_watch_num_i64, flan_dev_watch_num_f64, watch_num_str, watch_render_num; flan_dev_watch_read renders a num slot instead of copying its (unused) text	working
vendor/agent/flan_agent.c	flan_dev_watch_reset declaration; a watch reset command beside watch on/watch off	working
lib/dev.ml	watch_read takes ~reset and sends watch reset after the read; the :op "watch" arm reads a :reset field the same way watch-enable reads :on	working
emacs/flan-watch.el	flan-watch-ghost-call-regexp ? → *; docstring updated; flan-watch--tick sends (:op "watch" :reset t)	working
test/programs/dev-watch.flan	declare-c watch-num-i64; a loop-cells hot loop sampling "cell" eight times a step at 0,3,…,21	working
test/test_dev.ml	assertions inside the existing watch block: the row exists, n > 8 (per-sample not per-step), min < max (a range, not just last), a plain read does not reset, :reset t reopens the window	working
BUILT.md	new section "A hot loop keeps five numbers, and the window is the editor's", after the ghost text one	working
PORTING.md	item 5 struck in house style with the accumulator decision	working
NEXT.md	"What is left on PORTING.md's list" corrected: item 5 done, item 6 is next	working

Verified end to end, not just compiled

Against a real flan dev daemon and the real running program, the "cell" row read back:

n=184  min=0  max=21  last=21  mean=10.5000     (first read)
n=368  min=0  max=21  last=21  mean=10.5000     (second read — a plain read does not reset)
n=16   min=0  max=21  last=21  mean=10.5000     (after :reset t — the window reopened)

The loop is 0,3,…,21, so min, max, mean and the per-step count of 8 are each pinned by a different part of it.

Test result

dune test is green, run twice in full. test_dev.ml's first block — the separately-known-flaky socket bind race — failed on one earlier run with "the daemon never listened" and passed on both final runs. That is exactly the documented flake and another agent is on it; it is not related to anything here.

---

The ghost text path — the brief's assumption, checked

The brief said to reuse the ghost text path rather than invent a second one. It fits, and it needed one character.

The important property is in BUILT.md: ghost text anchors on the string literal in the source, not on a source location carried in the table. flan-watch--ghost-sites scans buffers shown in a window for (<head> "<name>" and matches the name against the rows. So a new kind of watch needs nothing new in the daemon, nothing new in the wire, and no new display path — the accumulator's rendered value is a string like n=368 min=0 max=21 last=21 mean=10.5 and flows through the existing :watch rows untouched.

Is it pull-based? No, and the brief's phrasing is worth correcting for whoever reads this next. The watch is push-based and that is load-bearing: watch.clj is pull (Emacs polls a render function), and flan_dev.c's comment explains at length why that does not port — an evaluation here compiles a module and dlopens it, so a 5Hz poll would be hundreds of .so files a minute. So the program pushes into a table and the editor reads memory. Ghost text is not a poller at all: it is painted from flan-watch--absorb, off the same single reply that paints the buffer, which is what keeps the two from disagreeing and preserves the one-request-in-flight invariant. The accumulator sits inside that unchanged.

The one thing that did not fit. flan-watch-ghost-call-regexp was watch\(?:-[[:alnum:]]+\)? — one optional hyphenated segment. [[:alnum:]] does not match a hyphen, so on (watch-num-i64 "cell" ... the regexp matched watch-num, then required whitespace, found -i64, and backtracked to failure. A numeric watch therefore appeared normally in the watch buffer and got no inline value at all — a silent half-failure. * for ? is the whole fix, verified in emacs --batch against all four head shapes.

That is worth recording as the predictable cost of the decision the ghost text section defends. Anchoring on the name rather than on the head is what makes the head a defcustom, and a defcustom with an enumerated default is a default that needs widening whenever the set of heads grows. Cheaper than the alternative, but not free.

---

What remains

Item 5 is complete, and so is the fixture the previous list asked for: emacs/test-flan-watch.el now has a block "Every head the default regexp claims to cover" pinning watch-num-i64 and watch-num-f64 beside the one-segment heads. It was verified by mutation — restoring the old ?\= in flan-watch-ghost-call-regexp makes exactly those two checks fail and nothing else — so a regression in that regexp is now caught.

One item is left, and it turned into a finding rather than a test.

1. watch_render_num's n=0 branch is unreachable, so the test that was asked for cannot be written. runtime/flan_dev.c:764 renders n=0 last=<v> for a window with no samples. Nothing can present that state to the reader:

   • s->num is set to 1 in exactly one place, watch_record (flan_dev.c:701), and that function always falls through to s->n += 1. The clear (s->n = 0 on an epoch change) and the increment are both inside one odd-generation window, so a reader cannot see between them.
   • flan_dev_watch_begin sets s->num = 0 and never touches s->n, so the scalar path cannot leave a num slot at zero either.
   • flan_dev_watch_read never compares s->epoch to watch_epoch. A reset therefore does not make the slot read as empty.

   The user-visible consequence is worth stating on its own, because it is the thing the branch was presumably written for: after :reset t, a read taken before the program's next sample reports the previous window's five numbers, not an empty one. That is why test_dev.ml's reset assertion has to await the count dropping rather than read once — the test is already working around this.

   Two ways out, and both are decisions rather than test work, which is why neither was taken here (the lane was scoped to tests):

   • Delete the branch. Honest about what the design says: a window begins when the program next runs, and until then the last window is what there is to show.
   • Make the read epoch-aware — compare s->epoch with watch_epoch under the seqlock and render n=0 when they differ. This makes a reset visible immediately without the reader writing the table, and would let the existing await in the test become a single read. It is a behaviour change to what an editor sees in the tick after a reset, so it belongs to whoever owns runtime/flan_dev.c, not to a test pass.

Item 6 (frame rollback as a worked example) is the next real piece of work and NEXT.md points at it.

---

What I tried that did not work

• Two Flan type errors in dev-watch.flan. (* i 3) over a let-bound 0 is i32, and watch-num-i64 wants i64 — fixed with an explicit (i64 …). Then loop-cells was declared i64 and returned the i32 counter; declared i32. Both are ordinary, but they cost two daemon round trips to find because a build error goes to the daemon's stdout, not to the client — worth knowing if you are driving flan dev by hand.

• My first probe script spoke the wrong protocol. Wire.send is length-prefixed ("%d\n%s"), not newline-delimited. A newline-delimited client gets the connection closed with no error text. If you write a throwaway client, read lib/wire.ml:32 first.

• Two attempts at the reset assertion failed before the third stuck, and both failures were informative rather than noise:

  1. "the window did not reopen: 8 after 8" — one step writes eight samples, so two reads back to back leave no room under the count for a reset to show in. The test now runs the accumulator up past 32 first (which is itself the other half of the claim: every one of those polls is a plain read, and a plain read must not reset, or the count could never climb).
  2. "the window did not reopen: 40 after 40" — reading immediately after :reset t still sees the old count, because the reset bumps an epoch and the slot clears on its next sample. That is the design and not a bug; the test now awaits the drop and the comment explains why the laziness is what keeps the reader out of the table.

• Not attempted, deliberately: the (watch "hp" hp) arm in check.ml. Both PORTING.md and flan_dev.c say it is only wanted for composites (a struct, a slice, a union), which need Render.render pointed at the flan_dev_watch_emit_* emitters. A scalar and an accumulator both reach the runtime by plain declare-c and need no compiler change, which is the property that kept this change out of check.ml entirely.

• One worktree note. This worktree was checked out at 2c232dd, an ancient commit, rather than at dev-loop's tip — none of PORTING.md, NEXT.md, lib/dev.ml or runtime/ existed in it. git reset --hard dev-loop on a clean tree fixed it. Also, dune finds the parent repo's root from inside a worktree, so every build here is dune build --root . / dune test --root ..