FIX.org, NEXT.md, DISCUSS.org, docs/DISCUSS.md and the session handoff at the
root are one TODO.org now: 293 entries under seven subsystem headings, each
carrying an org keyword that says where it stands. A DONE entry is a few lines
saying what was decided and what that rules out; the reasoning that would not
compress — the embedding spike and the four reports the hand-written x86
backend was built from — moved into docs/BUILT.md instead, and its entries
point there in one line.
Every entry was checked against the tree before it got a keyword, and the
prose was wrong in both directions. Things the deleted files called open were
built: the first-evaluation stall, main being redefinable, macro parameter
lists, the type-limit constants, the array constructors, the byte fills,
inc/dec, the discard's fontification, the Emacs buffers, rt_die's _exit, the
backtrace surface, and the acceptance failure that could print and still exit
zero. Things they called done were not: the backend reports' no-plan buckets
had gone stale in the other direction, the value-dependent defvar was
superseded rather than built, and macro-expansion source locations are on an
unmerged lane, so that entry is NEXT and names the branch.
Every comment that cited one of the five by name now cites a heading that
exists, in TODO.org or in docs/BUILT.md. The session reports under
docs/handoffs/ keep naming the files they worked on, because rewriting them
would falsify what those sessions did; each carries a note saying where the
content went.
The break buffer captioned every refused restart "below this evaluation", which
is the wrong sentence at a trap: there is no transfer channel, nothing on the
list can be taken, and there is no evaluation to abandon. The terminal listing
had always said the two apart. The wire now does too — a bare ! line ahead of
the entries, because a trap with no restarts at all still has to be able to say
so, and because :abandon being nil means three different things.
The boundary was also the first entry truncation dropped: snap_push walks
innermost first, and the boundary is the outermost frame of the evaluation. A
slot and a name's worth of bytes are kept back for it.
flan_break_resume goes; nothing has called it since choices became positions,
and NEXT.md already said there was no such function. eval_boundary is cleared
between runs beside the two stacks that already were. And the note on a taken
restart stops costing a second round trip: the agent answers ok abandon for the
boundary, which is the end that knows.
Nested boundaries are tested rather than argued: two evaluations, six restarts,
and abandoning the inner leaves the outer with its own still on offer.
A C-x C-e that blew up took the session with it. The expression's break offered
either nothing at all — a bad index establishes no restart, and the program's
own are below the thunk boundary where a transfer has nowhere to land — or a
list on which every entry was refused. That left abort, and abort is _exit(134)
over a mistyped index.
So the boundary offers a restart of its own. The agent pushes a real frame
around every evaluation, after the floor is read so that it lands above it;
taking it unwinds to the thunk, flan_reload_call drops the channel it holds,
and the poll returns to the game loop. It abandons and does not undo, which is
said in the agent's line, the daemon's note, the buffer's row and the manual.
The other half was a silence. The break buffer drew every restart as takeable
and ignored the :unreachable the wire already carried, so a digit on one went
out to be refused and nothing came back. Those rows now carry the reason and
are refused where they are read, and :abandon names the position that drops the
evaluation — a position, because a program may establish a restart of that name
itself.
Not the threading, which is what the report suspected. The thunk does run on
the game thread; a thunk on a thread of its own would have had the same empty
list and the same abort.
Five fixes off the independent review, plus the author's u8 ruling.
x86 parity: the bad-index block always ran x86 (it is flan dev's
default) and now says so with an explicit --x86; the condition render
gets an assertion under the x86 backend too, beside the LLVM one, and
a user error is pinned as carrying no site on both.
ArithError's layout is now pinned: {i32 op; i64 lhs, rhs} in C against
the prelude's defstruct, read field by field through the break loop's
render, driven from the editor through a divide under a restart-case.
That also covers condition and site on LLVM.
Three refusals that were wrong: trap_site tested the prefix "err"
and so ate any site whose path began with those letters; source_line
let Sys_error from input_line escape and take the whole break reply
with it, leaking the handle; and a condition with no fields was
reported as a name no struct has. The daemon now sends its own field
count and the buffer tells the two empties apart.
Nits taken: an over-long site is dropped rather than silently
truncated into a plausible one; the caret pads with the source line's
own tabs; the headline says when it has cut the field list;
flan-cnr-layout is live again as the single spelling of that request
rather than dead beside an inlined copy.
And the ruling: a u8 renders as 97 (\a) where a person is inspecting
and stays 97 where the program is printing.
A break nested inside a trap's break — a fix candidate evaluated at a
bounds stop raising its own error — copied the outer trap's site into
its snapshot, which is a caret pointing at an unrelated line under the
inner condition's name. snap_push now consumes the global: each
snapshot owns its copy, a nested entry that set no fresh site gets
none, and the outer break keeps its own. Pinned end to end.
The fields section says why the values are missing once, above the
rows, instead of repeating the sentence per field; a row keeps its own
reason only when it has one (no printer for its type).
strip_rebind and shown_names get direct coverage, including the clean
strip a real frame almost never reaches.
The break loop used to discard the pointer it was handed, so the buffer
could name a BoundsError's fields and never show 648. Now the snapshot
stashes it, flan_agent_condition hands it back on the stopped thread,
and a daemon-built thunk — locals pointed at the condition — renders
each field. Delivered at-stop, so a resume-and-restop cannot get the
old type read over the new pointer.
The trap sites publish their loc around the hook call, the snapshot
copies it, and break answers :site with the line's text as :source —
the frame lines say where each call was; this is the only record of
the indexing itself.
Compiler temps are hidden from the locals listing rather than refused
as s4; a shadowing rebind strips its ~N except where the outer binding
is on the same list, where both keep their raw spelling.
The bug review found: [start_on] claimed [started] at the top and every
failure exit left it claimed. Under [flan dev] the constructor is the first
caller and reports to nobody, so a path nothing could bind disarmed the
program's own (agent/start ...) as well — it answered 0 with no socket, no
listener and no hooks, where before this lane the explicit form answered -1.
Success reported for nothing at all is worse than the error it replaced.
So every way out that is not a listening socket unwinds: the fd is closed, a
file the bind managed to make is unlinked, and [started] goes back to 0 so a
later start is a real attempt. Pinned by running the zero-argument fixture
with FLAN_AGENT_SOCKET pointing nowhere — constructor fails silently, main's
own call then fails loudly, "cannot listen" and exit 1.
Two arguments to (agent/start) are refused, which nothing held: the macro's
[& args] cannot say "one at most", so what says it is the expansion splicing
every argument into a function that declares one. The message names
agent/start-at and carries the expanded-from note, and that is what the
acceptance row asserts.
And the reply a delivery gets when there is no agent in the process, which
nothing held either. dev-noagent.flan parks, so it was never this case;
dev-noagent-running.flan keeps running, and the answer is a refusal naming the
socket that could not be reached — not install_note's "queued", which would
promise a poll with nothing to drain. Which leaves that note unreachable in
all three shapes rather than merely unpinned, worked through in FIX.org.
FIX.org also now says what an exported FLAN_AGENT_SOCKET would do: start_on
unlinks before it binds, so an agent-linked program started in that
environment takes the path away from whoever bound it first.
A constructor in the agent package binds FLAN_AGENT_SOCKET when it is set,
which is the daemon and nothing else — both shapes set it, before the fork in
--two-process and before the exec in the merged build. So a program under
[flan dev] that calls (agent/poll) and has no (agent/start) in it takes
redefinitions anyway, and one that does call start meets an agent that is
already listening and gets a no-op.
The window this closes was the complaint in DISCUSS.org: a program that opens
a window before starting its agent leaves the daemon waiting on a socket that
does not exist yet. Bound here, the socket exists before main whatever the
program does afterwards — so test_dev.ml's late-agent row asserts the negation
of what it used to. The delivery sent during the sleep no longer carries "the
program has not called (agent/start ...) yet", because that is no longer true
of it; what is still late, and still asserted, is the poll that installs it.
It reaches exactly as far as the linker does. Reach prunes a package nothing
calls into, so a program that mentions the agent nowhere does not link this
file and has no constructor to run: auto-start is for a program that polls and
has dropped its start call, not for one that says nothing about the agent at
all. That limit and the release-build residual are in FIX.org, along with the
daemon branch that can no longer be reached.
agent-nostart.flan is the pin, and its two numbers are the honest ones: 1
before anything could arrive, 1000 after the wait, because a listener bound
before main is still not an install.
The path was ceremony. Under [flan dev] the daemon already decides where it
wants to talk to the program and writes it into FLAN_AGENT_SOCKET, which the
C side has always honoured over whatever the source named — so the argument
was a value nothing read. Outside the daemon any path will do as long as the
program says which one it picked.
So [start] becomes a macro over two functions: no argument picks the
daemon's socket if there is one and otherwise /tmp/flan-agent-<pid>-<clock>.sock,
announced on stderr because a socket nobody can name is a socket nobody can
connect to. The explicit form stays for a program that wants a fixed path.
Extra arguments are spliced into [start-at] rather than dropped, so the arity
refusal is still the checker's, at the call site.
And the socket is removed on the way out. The bind stashes the path it bound
and registers an atexit; the two paths that leave by _exit — the break loop's
[abort] and the orphan handler — unlink it by hand, as the orphan handler
already did for its own copy of the path. Nothing else takes it away: under
the daemon it sits in a temp directory that is still never removed (FIX.org),
and outside there is no daemon at all.
test/programs/dev-loop.flan now names no socket, which puts the whole daemon
block in test_dev.ml behind the zero-argument form; agent-auto.flan is the
standalone half, reached only through the line the program printed, and it
pins the second (agent/start) as a no-op and the socket as gone at exit.
The dev runtime rendered a value twice over: [flan_dev_emit_{u64,i64,f64,str}]
into the result buffer an evaluation is read back from, and
[flan_dev_watch_emit_*] into the current watch slot, with the same four bodies
either side and the sink as the only difference. [flan_dev_result_end] and
[flan_dev_watch_end] were the same ellipsis-and-generation close, comment for
comment, over two buffers.
So the rendering takes the sink as a parameter and the eight entry points are
eight one-line calls into four statics. The exports stay eight: the compiler
emits four of them by name (Session.externs) and a program reaches the watch
four through declare-c. ABI does not collapse because the bodies did.
The escape table is now [flan_escape_char] in flan_rt.c, once: what one byte
reads as inside a quoted string, into a caller's four bytes. A table and not a
printer, because the framings are genuinely different — [flan_escape_bytes]
builds a capped slice to hand back and the dev pair streams into a buffer it
does not own the end of — and the framing is the part that is each caller's.
The dyn printer keeps its own copy, which is docs/SPIKE-DUPLICITY.md §9's one
defended repeat: it is inside the runtime that owns the storage it walks. Its
comment, and flan_rt.c's, no longer tell a reader to change the other two.
Same for the NaN rule, which was spelled four times: [flan_f64_format] is
flan_rt.c's [flan_f64_to_bytes] without the slice, and the REPL emitter and the
watch table call it rather than restating "%g, and nan unsigned".
Byte-identical, checked two ways. A session driven over the daemon's socket
before and after — every arm of the emit family, the escapes, and a string long
enough to reach the truncating close — diffs empty. And a harness linking both
trees' flan_rt.c + flan_dev.c compares 8173 renderings: every byte 0..255
through both string emitters and through flan_escape_bytes, every length across
both caps and the ellipsis either side of them, both NaN signs, both infinities,
i64 and u64 at their extremes. Identical.
Dead code, each verified by its own grep before removal. These are exported C
symbols, so a program could reach one through declare-c; the evidence is that
nothing in the tree does, including the docs that write the surface down.
flan_dev_watch_u64 (flan_dev.c) — one occurrence repo-wide, its own
definition. The i64/f64/str siblings are declare-c'd in
test/programs/dev-watch.flan and written down in emacs/MANUAL.md; this one
appears in neither, and in no other file.
flan_break_resume (flan_rt.c) — the only non-prose reference was a stale
extern in vendor/agent/flan_agent.c with no call under it. Both gone.
[flan_name_id] stays: the bounds and arithmetic conditions still hash through
it. [flan_restart_take]'s comment no longer points at a function that is not
there.
flan_dev_watch_enabled (flan_dev.c) — prototyped in flan_agent.c, never
called. [watch_on] is still read directly by the three sites that gate on it.
clang_stamp (lib/build.ml) — a [lazy] never forced; one occurrence.
marshal (lib/expand.ml) — no reference anywhere. [write], which it wrapped,
is called twice in [call], so the [let rec] group is demoted to keep it.
is_bytes (lib/js.ml) — dead within js.ml. Nothing else in that file is
touched: the JS backend is parked, not dead.
Kept on purpose: [Loc.forget_sources], documented in docs/BUILT.md as
deliberately retained, and [flan_dev_watch_num_f64], which emacs/MANUAL.md
declares as public surface.
Two comments in lib/dev.ml argued the orphan grace in terms of elisp symbols
from before the rename — [flan-dev--open], [flan-dev--connection],
[flan-dev-poll-interval], and a file called emacs/flan-dev.el. None of those
exist. Re-spelled as [flan--open], [flan--connection], [flan-poll-interval] and
emacs/flan.el, which is where they are; the reasoning is load-bearing and is
unchanged.
dune test: exit 0. 59 lines of code out, 24 lines net of the prose that says
why.
`(:op "set")` is `inspect`'s addressing with the arrow turned round: same
frame, same slot index, same path steps, same fingerprint check, and a list of
(path, expression) edits relative to what the buffer is showing. The thunk
stores and then renders the same place, so what comes back is what the program
holds afterwards rather than an echo of what was asked for.
`stopped-only` was not enough for a write. It asks whether the program is
stopped, and for a read that is the whole question — the worst a render can do
against the wrong stop is print something true of a different frame. A module
that *stores* through `flan/dev-slot` reaches its target through whatever
snapshot is on top when the store runs, so a resume and a second stop inside
the ~300ms build window lands it in the same slot index of a different stack.
Not a fault: a plausible shape, in the wrong place, silently. So the agent
grew `at-stop N` beside `stopped-only`, the generation `snap_push` already
mints for the restart machinery, checked on the game thread at the moment the
job is claimed. `stop` answers it, `inspect` carries it out on every reply, and
a write that names a stop the program has left is refused before anything is
built.
Three refusals about where rather than what, and they are in session.ml above
both backends because emit and x86 disagree about two of them. A data type's
case field has no address that does not also settle the tag. An option's
payload has none that does not settle whether there is one. A pointer would be
an address this end made up, which is the blessing the registry exists to
insist on.
`Check.expression` grew a `want` and `Check.expressions` a shared frame: the
first is why `3` into an `f32` field is an f32 three rather than "expected f32,
found i32", and the second is why two edits in one commit are not two `let`s
reading each other's storage.
Under --two-process the compiled program is a child of the daemon, and the
daemon is the only thing that ever ends it: the program is a loop with no
reason of its own to stop, and lib/dev.ml kills it from a Fun.protect finally
block on the way out. That block does not run when the daemon is SIGKILLed —
by a harness tearing down a daemon it gave up on, by a watchdog, by anybody —
and what was left behind was a program with ppid 1, sleeping, holding a socket
nothing will ever connect to. Eight of those were on the machine this was
written on, the oldest six days old, and one had been minted by this suite.
The child now arms PR_SET_PDEATHSIG when it starts its agent, gated on a
FLAN_DEV_PARENT the daemon sets and nothing else does, and closes the race
where the daemon died before the arming by comparing getppid against that pid
rather than against 1 — under a subreaper an orphan is not reparented to init.
The signal is SIGPWR and not SIGTERM, because the daemon's ordinary teardown
already sends the child SIGTERM and the child's stderr is the daemon's own; a
handler there would print "the daemon is gone" into the daemon's output on
every clean close. The handler writes one sentence, unlinks the agent socket
it bound, and exits 0, and it does nothing else because it can run on the game
thread inside stdio's lock.
A merged build arms none of this and must not. There the daemon is the
process, so a dead daemon is a dead program; its parent is whoever typed
flan dev, which is a shell or an emacs and not the session's owner. Merged
daemons do leak on this machine — four of the eight — but for a different
reason, which is that accept_loop has no idea whether its editor still exists.
That wants a mechanism that tracks the client rather than the parent, and it
is not this change.
The daemon's read end of the program's stdout pipe is also cloexec now. The
child used to inherit it, which is both a leak and the reason waiting for EOF
on that pipe could never have been the mechanism: the pipe always had a
reader. A printing program now takes SIGPIPE once its daemon is gone, which is
a second net under chatty programs and not a substitute — the test uses
dev-watch, which is silent, so that only the thing under test can end it.
The test spawns the two-process shape, sends one request so that what it kills
is demonstrably serving, reads the program's pid out of the daemon's
/proc children before killing it — it is a grandchild, so waitpid is ECHILD —
and SIGKILLs the daemon. SIGKILL and not SIGTERM: SIGTERM lets the finally
block run and the assertion would be green with this change reverted. It fails
that way, and passes with it, in well under a second.
Six refusals in the runtime called _exit(134) where every other error had
learned to park: no restart by that name, a restart taken with the wrong
arguments or with none, a defer that invoked one, a null allocator, and
free-all on something with no region. Under a merged flan dev the compiler is
in that process, so a program that named a restart nobody established took the
session down with it, which is the one thing the break loop exists to prevent.
They park now. Not through flan_break_hook, which is what bounds and
arithmetic use: that hook may answer by aiming a transfer channel, and these
six are called by emitted code that falls off the end with no channel anywhere
in the call, so a restart chosen against one would be accepted and dropped.
flan_trap_hook says the other thing instead — stop here, let everything be
read, and refuse the resume with a reason.
All six park, for two reasons rather than one. Four are guards that fire
before the operation they guard, so nothing is half done and the frame reads
like any other. The other two fire mid-transfer, with the frame's defers
possibly half run, and they park only to be looked at: stopping on a torn
unwind is strictly more than exiting before anyone can ask what tore it.
The break loop grew a per-snapshot resumable flag for it. Restarts are still
listed and still numbered, the terminal marks them untakeable and the socket
reports the same positions as unreachable, and the listener refuses a choice
with the trap's own sentence rather than the thunk-boundary one.
Standalone builds die exactly as they did: nothing installs the hook in a
program that did not import the agent, and the acceptance case for free-all
still wants exit 134 and the same message.
The review entry that asked for this named flan_exit_hook, which is normal
termination and not this at all; it is struck out with the correction.
The registry compacted whenever the table was three quarters full, and a
compaction reclaims dead entries and nothing else. A program holding more than
three quarters of the table in live blocks therefore compacted on every
allocation for the rest of its life, reclaiming nothing each time and holding
the table-wide epoch odd while it did. A listing racing that loop lost all
eight of its attempts and answered with zero rows -- "nothing is held", about a
program holding three thousand blocks, from the verb that exists to find a
leak. Measured at 199 wrong answers in 200.
The trigger now also asks whether there is an eighth of a table's worth of dead
to reclaim, which is a count four places maintain: a death, an arena's
free-all, a note written over a dead slot, and the sweep itself. That bounds
the cost from the other side too, since a sweep that runs reclaims at least 512
slots and so cannot run twice in 512 allocations.
Separately, flan_dev_reg_by_type answered a walk it could not take with zero
rows, which is the same number a program that had freed everything gets, and
stepped past slots flan_reg_snap could not copy while still calling the walk
whole. It now counts those slots and returns -1 with the count, the agent
refuses in a sentence the daemon already renders, and the snap contract says
which caller keeps it and why reg_at is allowed not to.
A note that finds no slot is still dropped -- dying because a diagnostic ran out
of room would be the diagnostic shooting the patient -- and now says so on
stderr once, quoting how many entries were dead rather than claiming the table
is all live.
test/dev_limits.c gains three modes, driven from test_reload: 3100 live blocks
read under a writer thread (1 right in 200 before, 200 after), 3000 live with
600 churned on top of them to prove the sweep still runs, and a genuinely full
table that must say so exactly once.
The writer is the game thread, in every allocation and every free; the reader
is the agent's listener, on a program that is running. Nothing stood between
them. The consequence is not a slightly wrong count: a row's type pointer and
its length mean nothing apart, and a reader that takes the new pointer with the
old length reads off the end of a string literal.
Each entry now carries the watch table's seqlock, odd while it is written, and
a reader copies the entry and re-reads the counter before believing it. The
compaction bumps a table-wide counter around itself, because it moves entries
between slots and no per-slot counter can describe that; a scan that sees that
counter move walks again. It clears the table slot by slot rather than with one
memset, since the memset would zero the counters a reader was holding.
The breakdown and the leak report stay answerable while the program runs, which
is the moment they are for. reg at does not: whether one address is still live
is exactly what a running program is changing, so it is refused the way every
break verb is refused, which is what the daemon already did on its own side.
The merged daemon now unlinks its socket on the ways out it does not
control as well as the one it does: an atexit for exit(3), which is what a
runtime trap takes, and by hand in die_now and in main's fallback, which
are _exit and skip the chain on purpose.
The client says so too. A refusal on a path that exists is a leftover, not
a daemon declining, and the raw 'Connection refused' has now misdirected
two investigations.
Separately, and it is separate: dev-repl.flan gets dev-robust.flan's
24000-tick budget. Twenty seconds of program under a two-minute test is a
second flake waiting its turn, and it is not the one fixed above -- that
one fails honestly, saying the program exited.
Three follow-ups to the registry reader.
The agent's "reg types" handler held its four row buffers as statics. The lock
argument for that was right — request_lock is held across the whole of
handle_line from both entry points — but the BSS was not: 8KB in every build
this package is linked into, including a release build of a game that imports
the agent and never writes a row. That is flan_dev.c's own argument against a
fixed table at a thirty-second of the size. Malloc'd and freed per request,
like the watch handler beside it.
A followed pointer parses as an opaque node, so the inspector refused it with
"no structure for this type" — false and unhelpful at once. It has structure,
it is drawn, and the reason you cannot step in is that the step would start
from the pointee. Said that way instead.
And the three new commands have elisp tests: the address root's wire, that a
path is refused rather than dropped, the listing's rows and totals, the
overflow warning, and a build with no registry refusing rather than showing an
empty table that reads like a program holding nothing.
The allocation registry had a recording side and half a reader. This is the
rest of the reader: point at any heap address, a breakdown by type, what is
still held, and the test that stops dev-ptr.flan's header from being read by
hand.
The recorded name, back to a type. The table records a string and has to —
the note is built where the concrete type exists and what crosses into the
runtime is bytes. What closes it is that the string is Types.to_string, which
is the source spelling, so the round trip is the language's own reader,
Parse.texpr and Check.resolve. No table of spellings is written down, so
nothing can fall behind Types.to_string, and a name that is not a type —
"pool slots" — is refused with the name quoted rather than defaulted.
The address root renders a (Ptr T) and not the pointee, which puts it through
render.ml's pointer arm: permission is asked in one place in the compiler, and
an address root and a slot root reach the same two answers by the same code.
Flan has no integer-to-pointer cast, so flan_dev_reg_addr is an extern beside
flan_agent_frame_slot, for the same reason.
One walk and two questions: a leak report is a breakdown with the dead left
out, so flan_dev_reg_by_type is one function and the agent formats it.
"At exit" is not a hook. A program killed by a signal runs no handler, which
is how a game under the editor ends, so (:op "leaks") is the authoritative
reader and can be asked at any moment including the one before the kill. The
atexit hook is for the program that returns from main, is registered from
inside flan_dev_reg_enable rather than by a file-scope destructor so that a
release build does not grow a third not-free place, and is off unless
FLAN_DEV_LEAKS is set because the acceptance table reads stderr.
The memcheck half of item 6 is deliberately not here.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
RESULT_MAX was written down twice, once in flan_dev.c and once in the agent,
with a run-time check that the two had not drifted. That second copy was a
buffer sized to be sent through a socket, and it is gone: the agent asks
flan_dev_result_cap() and allocates, which it can do because it is never the
game thread.
The bound itself stays, and the reason is the rule everything else here is
built around. result is what the game thread writes into, from a render thunk
at a frame boundary; a growable one is the frame thread calling realloc, which
is an allocation in the one place this design exists to keep allocation out of.
It would also break the seqlock, which is a protocol about torn contents and
assumes the address it copies from neither moves nor goes away underneath the
reader. Growing on the writer's side is a use-after-free the counter cannot
see.
So it is a render budget and not a wire size, and it only looked like one
because the agent had a copy of it. Removing the bound is a redesign of the
read -- probe, allocate, re-read, validate, retry -- and belongs with moving
the read to a frame boundary.
Also: flan_agent_request answers "err path too long" where the socket does,
instead of nothing. One verb table is only worth having if the two callers
cannot be told apart.
The measurement table in BUILT.md is retaken back to back across all four
shapes, because this machine drifts by more than the thing being measured. The
internal socket was ~41us of a 21ms redefinition; --two-process is unchanged
and is not slower than the merged build in any column.
flan dev is one process, so the unix socket between the compiler and the
program was a connect, a write and a read that never left the machine's own
memory. A delivery is now a call into the agent's verb table.
flan_agent.c's handlers write into a sink that is either an fd or a growing
buffer, so serve() and the new flan_agent_request() share one verb table rather
than two that would drift -- every answer in there is assembled from several
pieces, so the seam had to be the writing. Dev.deliver, Dev.result and Dev.ask
were three copies of the same socket dance and are now three lines over one
Dev.request.
Which path is taken is the linker's answer, not a flag: flan_agent_request is
weak in dynload_stubs.c, null in every binary that links no flan_agent.o, and
Agent.request is None there. So --two-process, flan reload and every test go on
using the socket with nothing to configure.
Three things the direct path must not quietly change. The ring's room check is
separate from its store, which was safe only while the accept loop was the sole
producer, so handle_line runs under a mutex. A delivery is still only published
-- the install is one store per function on the game thread at a frame
boundary, and doing it on the spot would be a frame running half in the old
code and half in the new. And the OCaml runtime system is released across the
call, because a delivery is a dlopen.
Measured, and the number is the point: the transport was ~50us of a 23ms
redefinition, so end to end did not move. Code generation is 19 of the 22
milliseconds. The merge's prize was never latency.
The test is a deletion, because a reply cannot say which way it came:
test_dev.ml unlinks the agent's socket file once the merged program has bound
it, and every evaluation after that still installs.
The globals section attributed a frame by its slot fingerprint, which is the
wrong cut for it: a redefined body can name entirely different globals while
binding identical locals, so the check saw no change and the new body's
reference set went into the union under the old body's frame, with the frame
numbers beside an entry saying so.
So a second fingerprint. Reach.ref_fingerprint hashes the set of globals a body
names — sorted and deduplicated, because a reference set is not ordered, where
slot indices make the slot fingerprint order-sensitive on purpose — and it
travels the path the first one already cut: %fninfo, flan_dev_frame_refsig, the
agent's snapshot, the backtrace line, Dev.globals_op. Different means the frame
is skipped by name with its reason, and the rest of the stack still contributes.
Two numbers rather than one, because they are two facts. A frame whose slots
match and whose globals do not has locals that are perfectly readable and
attribution that is not, and a combined hash would make locals refuse a frame
with nothing wrong with it. locals still checks the slot fingerprint alone.
It lives in reach.ml because expr_refs is already the walk that answers what a
body refers to, and is the walk the union itself is built from. One consequence:
emit now reaches reach, which closes a cycle through Load if cimport calls
Build.cachedir, so the header cache spells the object cache directory itself.
test_dev.ml drives the exact case — a body that binds identical locals and names
untouched where the stopped frame names pressure. With the check disabled it
fails twice: the missing refusal, and untouched appearing under frame 0.
The refusal for a frame whose body has been redefined underneath it did not
fire because four of its five hand-offs were never written. `Emit.fninfo` has
been storing `slot_fingerprint` in the last `i32` of every `%fninfo` all along;
`flan_dev.c` called that field `spare`, there was no accessor for it, the agent
never snapshotted it, the backtrace line never carried it, and `Dev.locals`
compared slot counts and nothing else. The handoff note's "every piece is
written and the refusal does not happen" was a guess, and the first step it
suggested — printing both sides of the comparison — could not have found it,
because there was no comparison.
So: `spare` becomes `slotsig` and gets `flan_dev_frame_slotsig`; the agent
snapshots it beside the slot count and puts it on the backtrace line *before*
the location, since the name is the one field that can contain a space and has
to stay last; `Dev.backtrace` parses it; `Dev.locals` compares it against
`Emit.slot_fingerprint` of the body this session holds and refuses by name when
they differ. No change to `emit.ml` — the value was already there.
The mechanism itself is right and stays. `slot_fingerprint` hashes every slot's
name together with the spelling of its type, so a rename that keeps the count
and the types — exactly the case this exists for — changes it. The count check
stays in front of it because its message is the more specific one.
The fingerprint stays off the wire. A hash is not something an editor can act
on, and the refusal says the fact in words: this frame's body was redefined
since it was entered, so its names no longer describe its values.
`test_dev.ml` gains the inverse and the control. A body that drops a `let` is
refused on the count, and `main` — untouched by the redefinition of `look` —
must still answer, which is the assertion that would catch a fingerprint that
never matched anything and made the verb useless while turning the suite green.
The half the shadow stack was built for. A slot's entry in the frame is its
address, null until the binding that fills it has run, so "not bound yet at
this point" is a null and needs no liveness analysis. The daemon compiles a
thunk that renders the types it already knows -- Tast.fn.slots, with snames
beside them -- at the addresses the stopped program supplies, and reads the
text back the way C-x C-e does. Nothing is copied out, because a value with
no header is bytes with no meaning anywhere but in the program that holds
it.
That is render.ml's walk with its root changed, which is the pointer-rooted
thunk NEXT.md said this needed, and one new arm in the backend: a cast from
one pointer type to another, which emits nothing.
Only named slots are recorded. A recorded slot escapes and stops being
promotable, and the slots that would cost most are the ones with nothing to
show -- dotimes' bound, the temporaries min and max use, the walk's own
scratch. They are refused by name rather than shown under an invented one.
Recording every slot was built and timed and is inside the noise, so the
rule stands on what it shows.
Four refusals, each by name and with its reason: a slot nobody named, a
slot the program has not reached, a type the printer has no arm for, and
two whole frames -- an evaluation's thunk, and a frame running a body that
has been redefined since, where every slot index would be a guess.
Measured, minimum of nine runs: +61% on call-heavy code over globals
against +33% for the frames alone, 0.06% of a frame at 60fps.
plan.org has specified a shadow stack in the dev column since the beginning
and nothing had ever built it. A frame is four words on the calling
function's own stack: the one it displaced, a pointer to a static
description of the function, and two words reserved for its locals. The
name and the location travel on the frame, so a backtrace needs no debug
information, no symbol table, and nothing from the platform unwinder that
plan.org deliberately does not use.
The pop is at every ret, the landing block a transfer leaves through
included. That is the half that is easy to get wrong: a pop written only on
the normal path leaves a dead frame behind every handled error, and the
test takes five breaks and resumes all of them by transfer before asking
for two frames.
(:op "backtrace") answers from a snapshot the stopped thread takes, beside
the restarts and for the same reason, and marks which frames belong to the
program and which to the evaluation the break is inside. It is refused
while the program runs.
Measured, interleaved, three pairs of binaries: 29% on 600 frames of sand,
7.6% on a benchmark that is nothing but calls -- 32us per frame of sand, a
fifth of a percent of a frame at 60fps. An array with a stack pointer was
built and timed as the alternative and is worse on both.
The 4K result cap and condition_name[128] are on the agent's socket path, which
is why the sanitizer corpus cannot reach them: a program in the sweep has no
socket and nobody on the other end of it. test_agent has both.
A 5000-byte string literal evaluated into the running program comes back as
exactly 4096 bytes ending in the ellipsis result_end puts there to say it
clamped — and it comes back through the seqlock's copy, so the cap and the new
reader are pinned by the same case. The header also shows the generation as 1,
which is the count of complete values rather than the raw counter.
A condition class of 198 characters comes back from `status` as 127 and a
terminator. Aborting out of that break is what pins the exit status at 134 now
that the loop leaves with _exit rather than exit.
SNAP_MAX, SNAP_NAMES and the dev registry's overflow guard are still read
rather than tested. Sixty-five nested restart-cases and four thousand interned
names are a lot of program to write for a clamp each, and neither is on a path
this session changed.
flan_dev_result_cap() exists so the size is asked for rather than written down
in two files: "the copy is never truncated" is only true while the agent's
buffer and the runtime's bound agree, and the agent checks that where the copy
happens.
The pipe the queue program blocks on is close-on-exec, or the child inherits
the write end and its own stdin never reaches end of file — it sat in its last
read waiting for a byte only it could send.
publish() wrote queue[head % QUEUE] without consulting tail, so the 65th module
queued between two agent/poll calls landed on the slot the game thread was
reading — twenty-four bytes of function pointers copied field by field with no
atomic near them, so the consumer could take half of one job and half of
another and call it. The comment claimed the overflow dropped the oldest
request; nothing did that.
A full ring is refused now, at the sender, before the dlopen. Dropping loses a
reload the sender was told was ok, which is the same lie more quietly; blocking
stalls the accept loop, which serves connections inline, so a program that had
stopped polling would also stop answering status and abort — the dev loop would
have no way to reach a program that had stopped listening to it. The check is
separate from the store because there is one producer: room, once seen, cannot
be taken away.
Two smaller defects in the same file:
A module with no flan_reload_install was refused and its handle dropped on the
floor. Not an exception to "nothing is ever dlclosed" — that rule is about a
module something points into, and this one installed nothing, so no cell names
it. What leaked was the handle value rather than the mapping: dlopen refcounts
by path, so re-sending the same bad file raised a count nothing could lower.
exit(134) from the break loop runs the atexit chain and the ELF destructors,
which want the loader lock the listener thread may be holding inside dlopen. A
program asked to abort would hang instead of dying. _exit, with the streams
flushed by hand at each call site. The deadlock itself is read rather than
tested; what the tests pin is that the exit status is still 134.
programs/agent-queue.flan blocks on stdin so the window is held open by the
test rather than by a timer: it takes 64 modules, refuses the 65th with a
reason, and installs 64 when it finally polls. noinstall.c's destructor prints
while the program is still running, which is the only way to see the close — at
exit the loader runs every destructor whether anything was closed or not. Both
halves fail on the old code.
