FIX.org, NEXT.md, DISCUSS.org, docs/DISCUSS.md and the session handoff at the
root are one TODO.org now: 293 entries under seven subsystem headings, each
carrying an org keyword that says where it stands. A DONE entry is a few lines
saying what was decided and what that rules out; the reasoning that would not
compress — the embedding spike and the four reports the hand-written x86
backend was built from — moved into docs/BUILT.md instead, and its entries
point there in one line.
Every entry was checked against the tree before it got a keyword, and the
prose was wrong in both directions. Things the deleted files called open were
built: the first-evaluation stall, main being redefinable, macro parameter
lists, the type-limit constants, the array constructors, the byte fills,
inc/dec, the discard's fontification, the Emacs buffers, rt_die's _exit, the
backtrace surface, and the acceptance failure that could print and still exit
zero. Things they called done were not: the backend reports' no-plan buckets
had gone stale in the other direction, the value-dependent defvar was
superseded rather than built, and macro-expansion source locations are on an
unmerged lane, so that entry is NEXT and names the branch.
Every comment that cited one of the five by name now cites a heading that
exists, in TODO.org or in docs/BUILT.md. The session reports under
docs/handoffs/ keep naming the files they worked on, because rewriting them
would falsify what those sessions did; each carries a note saying where the
content went.
The author: "I think I prefer length over len, because then I'll use len as
the variable name". One arm in check.ml, one row in the table beside it, and
every (len x) in lib, test, examples, vendor, spike, docs, web, emacs,
plan.org and NEXT.md rewritten.
Shadowing and builtin/ had already taken most of the sting out: a (defn len
...) was legal and won in its own file, and builtin/len reached past it. What
was left is that len was still a builtin — the defn earned a warning, and a
wrapper had to say builtin/ at every inner call. Now there is nothing under
the short name: len is an ordinary identifier in every position, which is
what (let [len (length xs)] ...) wants.
length takes over as shadowing's worked example rather than the feature
losing one. shadow-builtin.flan, builtin-qualified.flan, pkgs/shadowed and the
builtin/ rows in test_flan move to it and go on testing shadowing.
A call to a len nothing defines is answered where an unknown function is,
after every table and after the shadowing guard, so a program with its own len
never reaches it. The sentence is said rather than guessed at — len and length
are three edits apart and the did-you-mean's net is one — and the call is
written back out through spell_arg, as-slice's spelling lifted out of it and
now shared, so what is printed compiles.
sand.flan:33 still calls the old name and is the author's to change; until it
does, test_acceptance and test_session abort there. Both were run green
against a copy with that one line changed. FIX.org says so.
The break buffer captioned every refused restart "below this evaluation", which
is the wrong sentence at a trap: there is no transfer channel, nothing on the
list can be taken, and there is no evaluation to abandon. The terminal listing
had always said the two apart. The wire now does too — a bare ! line ahead of
the entries, because a trap with no restarts at all still has to be able to say
so, and because :abandon being nil means three different things.
The boundary was also the first entry truncation dropped: snap_push walks
innermost first, and the boundary is the outermost frame of the evaluation. A
slot and a name's worth of bytes are kept back for it.
flan_break_resume goes; nothing has called it since choices became positions,
and NEXT.md already said there was no such function. eval_boundary is cleared
between runs beside the two stacks that already were. And the note on a taken
restart stops costing a second round trip: the agent answers ok abandon for the
boundary, which is the end that knows.
Nested boundaries are tested rather than argued: two evaluations, six restarts,
and abandoning the inner leaves the outer with its own still on offer.
A C-x C-e that blew up took the session with it. The expression's break offered
either nothing at all — a bad index establishes no restart, and the program's
own are below the thunk boundary where a transfer has nowhere to land — or a
list on which every entry was refused. That left abort, and abort is _exit(134)
over a mistyped index.
So the boundary offers a restart of its own. The agent pushes a real frame
around every evaluation, after the floor is read so that it lands above it;
taking it unwinds to the thunk, flan_reload_call drops the channel it holds,
and the poll returns to the game loop. It abandons and does not undo, which is
said in the agent's line, the daemon's note, the buffer's row and the manual.
The other half was a silence. The break buffer drew every restart as takeable
and ignored the :unreachable the wire already carried, so a digit on one went
out to be refused and nothing came back. Those rows now carry the reason and
are refused where they are read, and :abandon names the position that drops the
evaluation — a position, because a program may establish a restart of that name
itself.
Not the threading, which is what the report suspected. The thunk does run on
the game thread; a thunk on a thread of its own would have had the same empty
list and the same abort.
Five fixes off the independent review, plus the author's u8 ruling.
x86 parity: the bad-index block always ran x86 (it is flan dev's
default) and now says so with an explicit --x86; the condition render
gets an assertion under the x86 backend too, beside the LLVM one, and
a user error is pinned as carrying no site on both.
ArithError's layout is now pinned: {i32 op; i64 lhs, rhs} in C against
the prelude's defstruct, read field by field through the break loop's
render, driven from the editor through a divide under a restart-case.
That also covers condition and site on LLVM.
Three refusals that were wrong: trap_site tested the prefix "err"
and so ate any site whose path began with those letters; source_line
let Sys_error from input_line escape and take the whole break reply
with it, leaking the handle; and a condition with no fields was
reported as a name no struct has. The daemon now sends its own field
count and the buffer tells the two empties apart.
Nits taken: an over-long site is dropped rather than silently
truncated into a plausible one; the caret pads with the source line's
own tabs; the headline says when it has cut the field list;
flan-cnr-layout is live again as the single spelling of that request
rather than dead beside an inlined copy.
And the ruling: a u8 renders as 97 (\a) where a person is inspecting
and stays 97 where the program is printing.
A break nested inside a trap's break — a fix candidate evaluated at a
bounds stop raising its own error — copied the outer trap's site into
its snapshot, which is a caret pointing at an unrelated line under the
inner condition's name. snap_push now consumes the global: each
snapshot owns its copy, a nested entry that set no fresh site gets
none, and the outer break keeps its own. Pinned end to end.
The fields section says why the values are missing once, above the
rows, instead of repeating the sentence per field; a row keeps its own
reason only when it has one (no printer for its type).
strip_rebind and shown_names get direct coverage, including the clean
strip a real frame almost never reaches.
The break loop used to discard the pointer it was handed, so the buffer
could name a BoundsError's fields and never show 648. Now the snapshot
stashes it, flan_agent_condition hands it back on the stopped thread,
and a daemon-built thunk — locals pointed at the condition — renders
each field. Delivered at-stop, so a resume-and-restop cannot get the
old type read over the new pointer.
The trap sites publish their loc around the hook call, the snapshot
copies it, and break answers :site with the line's text as :source —
the frame lines say where each call was; this is the only record of
the indexing itself.
Compiler temps are hidden from the locals listing rather than refused
as s4; a shadowing rebind strips its ~N except where the outer binding
is on the same list, where both keep their raw spelling.
The bug review found: [start_on] claimed [started] at the top and every
failure exit left it claimed. Under [flan dev] the constructor is the first
caller and reports to nobody, so a path nothing could bind disarmed the
program's own (agent/start ...) as well — it answered 0 with no socket, no
listener and no hooks, where before this lane the explicit form answered -1.
Success reported for nothing at all is worse than the error it replaced.
So every way out that is not a listening socket unwinds: the fd is closed, a
file the bind managed to make is unlinked, and [started] goes back to 0 so a
later start is a real attempt. Pinned by running the zero-argument fixture
with FLAN_AGENT_SOCKET pointing nowhere — constructor fails silently, main's
own call then fails loudly, "cannot listen" and exit 1.
Two arguments to (agent/start) are refused, which nothing held: the macro's
[& args] cannot say "one at most", so what says it is the expansion splicing
every argument into a function that declares one. The message names
agent/start-at and carries the expanded-from note, and that is what the
acceptance row asserts.
And the reply a delivery gets when there is no agent in the process, which
nothing held either. dev-noagent.flan parks, so it was never this case;
dev-noagent-running.flan keeps running, and the answer is a refusal naming the
socket that could not be reached — not install_note's "queued", which would
promise a poll with nothing to drain. Which leaves that note unreachable in
all three shapes rather than merely unpinned, worked through in FIX.org.
FIX.org also now says what an exported FLAN_AGENT_SOCKET would do: start_on
unlinks before it binds, so an agent-linked program started in that
environment takes the path away from whoever bound it first.
A constructor in the agent package binds FLAN_AGENT_SOCKET when it is set,
which is the daemon and nothing else — both shapes set it, before the fork in
--two-process and before the exec in the merged build. So a program under
[flan dev] that calls (agent/poll) and has no (agent/start) in it takes
redefinitions anyway, and one that does call start meets an agent that is
already listening and gets a no-op.
The window this closes was the complaint in DISCUSS.org: a program that opens
a window before starting its agent leaves the daemon waiting on a socket that
does not exist yet. Bound here, the socket exists before main whatever the
program does afterwards — so test_dev.ml's late-agent row asserts the negation
of what it used to. The delivery sent during the sleep no longer carries "the
program has not called (agent/start ...) yet", because that is no longer true
of it; what is still late, and still asserted, is the poll that installs it.
It reaches exactly as far as the linker does. Reach prunes a package nothing
calls into, so a program that mentions the agent nowhere does not link this
file and has no constructor to run: auto-start is for a program that polls and
has dropped its start call, not for one that says nothing about the agent at
all. That limit and the release-build residual are in FIX.org, along with the
daemon branch that can no longer be reached.
agent-nostart.flan is the pin, and its two numbers are the honest ones: 1
before anything could arrive, 1000 after the wait, because a listener bound
before main is still not an install.
The path was ceremony. Under [flan dev] the daemon already decides where it
wants to talk to the program and writes it into FLAN_AGENT_SOCKET, which the
C side has always honoured over whatever the source named — so the argument
was a value nothing read. Outside the daemon any path will do as long as the
program says which one it picked.
So [start] becomes a macro over two functions: no argument picks the
daemon's socket if there is one and otherwise /tmp/flan-agent-<pid>-<clock>.sock,
announced on stderr because a socket nobody can name is a socket nobody can
connect to. The explicit form stays for a program that wants a fixed path.
Extra arguments are spliced into [start-at] rather than dropped, so the arity
refusal is still the checker's, at the call site.
And the socket is removed on the way out. The bind stashes the path it bound
and registers an atexit; the two paths that leave by _exit — the break loop's
[abort] and the orphan handler — unlink it by hand, as the orphan handler
already did for its own copy of the path. Nothing else takes it away: under
the daemon it sits in a temp directory that is still never removed (FIX.org),
and outside there is no daemon at all.
test/programs/dev-loop.flan now names no socket, which puts the whole daemon
block in test_dev.ml behind the zero-argument form; agent-auto.flan is the
standalone half, reached only through the line the program printed, and it
pins the second (agent/start) as a no-op and the socket as gone at exit.
The dev runtime rendered a value twice over: [flan_dev_emit_{u64,i64,f64,str}]
into the result buffer an evaluation is read back from, and
[flan_dev_watch_emit_*] into the current watch slot, with the same four bodies
either side and the sink as the only difference. [flan_dev_result_end] and
[flan_dev_watch_end] were the same ellipsis-and-generation close, comment for
comment, over two buffers.
So the rendering takes the sink as a parameter and the eight entry points are
eight one-line calls into four statics. The exports stay eight: the compiler
emits four of them by name (Session.externs) and a program reaches the watch
four through declare-c. ABI does not collapse because the bodies did.
The escape table is now [flan_escape_char] in flan_rt.c, once: what one byte
reads as inside a quoted string, into a caller's four bytes. A table and not a
printer, because the framings are genuinely different — [flan_escape_bytes]
builds a capped slice to hand back and the dev pair streams into a buffer it
does not own the end of — and the framing is the part that is each caller's.
The dyn printer keeps its own copy, which is docs/SPIKE-DUPLICITY.md §9's one
defended repeat: it is inside the runtime that owns the storage it walks. Its
comment, and flan_rt.c's, no longer tell a reader to change the other two.
Same for the NaN rule, which was spelled four times: [flan_f64_format] is
flan_rt.c's [flan_f64_to_bytes] without the slice, and the REPL emitter and the
watch table call it rather than restating "%g, and nan unsigned".
Byte-identical, checked two ways. A session driven over the daemon's socket
before and after — every arm of the emit family, the escapes, and a string long
enough to reach the truncating close — diffs empty. And a harness linking both
trees' flan_rt.c + flan_dev.c compares 8173 renderings: every byte 0..255
through both string emitters and through flan_escape_bytes, every length across
both caps and the ellipsis either side of them, both NaN signs, both infinities,
i64 and u64 at their extremes. Identical.
Dead code, each verified by its own grep before removal. These are exported C
symbols, so a program could reach one through declare-c; the evidence is that
nothing in the tree does, including the docs that write the surface down.
flan_dev_watch_u64 (flan_dev.c) — one occurrence repo-wide, its own
definition. The i64/f64/str siblings are declare-c'd in
test/programs/dev-watch.flan and written down in emacs/MANUAL.md; this one
appears in neither, and in no other file.
flan_break_resume (flan_rt.c) — the only non-prose reference was a stale
extern in vendor/agent/flan_agent.c with no call under it. Both gone.
[flan_name_id] stays: the bounds and arithmetic conditions still hash through
it. [flan_restart_take]'s comment no longer points at a function that is not
there.
flan_dev_watch_enabled (flan_dev.c) — prototyped in flan_agent.c, never
called. [watch_on] is still read directly by the three sites that gate on it.
clang_stamp (lib/build.ml) — a [lazy] never forced; one occurrence.
marshal (lib/expand.ml) — no reference anywhere. [write], which it wrapped,
is called twice in [call], so the [let rec] group is demoted to keep it.
is_bytes (lib/js.ml) — dead within js.ml. Nothing else in that file is
touched: the JS backend is parked, not dead.
Kept on purpose: [Loc.forget_sources], documented in docs/BUILT.md as
deliberately retained, and [flan_dev_watch_num_f64], which emacs/MANUAL.md
declares as public surface.
Two comments in lib/dev.ml argued the orphan grace in terms of elisp symbols
from before the rename — [flan-dev--open], [flan-dev--connection],
[flan-dev-poll-interval], and a file called emacs/flan-dev.el. None of those
exist. Re-spelled as [flan--open], [flan--connection], [flan-poll-interval] and
emacs/flan.el, which is where they are; the reasoning is load-bearing and is
unchanged.
dune test: exit 0. 59 lines of code out, 24 lines net of the prose that says
why.
`(:op "set")` is `inspect`'s addressing with the arrow turned round: same
frame, same slot index, same path steps, same fingerprint check, and a list of
(path, expression) edits relative to what the buffer is showing. The thunk
stores and then renders the same place, so what comes back is what the program
holds afterwards rather than an echo of what was asked for.
`stopped-only` was not enough for a write. It asks whether the program is
stopped, and for a read that is the whole question — the worst a render can do
against the wrong stop is print something true of a different frame. A module
that *stores* through `flan/dev-slot` reaches its target through whatever
snapshot is on top when the store runs, so a resume and a second stop inside
the ~300ms build window lands it in the same slot index of a different stack.
Not a fault: a plausible shape, in the wrong place, silently. So the agent
grew `at-stop N` beside `stopped-only`, the generation `snap_push` already
mints for the restart machinery, checked on the game thread at the moment the
job is claimed. `stop` answers it, `inspect` carries it out on every reply, and
a write that names a stop the program has left is refused before anything is
built.
Three refusals about where rather than what, and they are in session.ml above
both backends because emit and x86 disagree about two of them. A data type's
case field has no address that does not also settle the tag. An option's
payload has none that does not settle whether there is one. A pointer would be
an address this end made up, which is the blessing the registry exists to
insist on.
`Check.expression` grew a `want` and `Check.expressions` a shared frame: the
first is why `3` into an `f32` field is an f32 three rather than "expected f32,
found i32", and the second is why two edits in one commit are not two `let`s
reading each other's storage.
Under --two-process the compiled program is a child of the daemon, and the
daemon is the only thing that ever ends it: the program is a loop with no
reason of its own to stop, and lib/dev.ml kills it from a Fun.protect finally
block on the way out. That block does not run when the daemon is SIGKILLed —
by a harness tearing down a daemon it gave up on, by a watchdog, by anybody —
and what was left behind was a program with ppid 1, sleeping, holding a socket
nothing will ever connect to. Eight of those were on the machine this was
written on, the oldest six days old, and one had been minted by this suite.
The child now arms PR_SET_PDEATHSIG when it starts its agent, gated on a
FLAN_DEV_PARENT the daemon sets and nothing else does, and closes the race
where the daemon died before the arming by comparing getppid against that pid
rather than against 1 — under a subreaper an orphan is not reparented to init.
The signal is SIGPWR and not SIGTERM, because the daemon's ordinary teardown
already sends the child SIGTERM and the child's stderr is the daemon's own; a
handler there would print "the daemon is gone" into the daemon's output on
every clean close. The handler writes one sentence, unlinks the agent socket
it bound, and exits 0, and it does nothing else because it can run on the game
thread inside stdio's lock.
A merged build arms none of this and must not. There the daemon is the
process, so a dead daemon is a dead program; its parent is whoever typed
flan dev, which is a shell or an emacs and not the session's owner. Merged
daemons do leak on this machine — four of the eight — but for a different
reason, which is that accept_loop has no idea whether its editor still exists.
That wants a mechanism that tracks the client rather than the parent, and it
is not this change.
The daemon's read end of the program's stdout pipe is also cloexec now. The
child used to inherit it, which is both a leak and the reason waiting for EOF
on that pipe could never have been the mechanism: the pipe always had a
reader. A printing program now takes SIGPIPE once its daemon is gone, which is
a second net under chatty programs and not a substitute — the test uses
dev-watch, which is silent, so that only the thing under test can end it.
The test spawns the two-process shape, sends one request so that what it kills
is demonstrably serving, reads the program's pid out of the daemon's
/proc children before killing it — it is a grandchild, so waitpid is ECHILD —
and SIGKILLs the daemon. SIGKILL and not SIGTERM: SIGTERM lets the finally
block run and the assertion would be green with this change reverted. It fails
that way, and passes with it, in well under a second.
Six refusals in the runtime called _exit(134) where every other error had
learned to park: no restart by that name, a restart taken with the wrong
arguments or with none, a defer that invoked one, a null allocator, and
free-all on something with no region. Under a merged flan dev the compiler is
in that process, so a program that named a restart nobody established took the
session down with it, which is the one thing the break loop exists to prevent.
They park now. Not through flan_break_hook, which is what bounds and
arithmetic use: that hook may answer by aiming a transfer channel, and these
six are called by emitted code that falls off the end with no channel anywhere
in the call, so a restart chosen against one would be accepted and dropped.
flan_trap_hook says the other thing instead — stop here, let everything be
read, and refuse the resume with a reason.
All six park, for two reasons rather than one. Four are guards that fire
before the operation they guard, so nothing is half done and the frame reads
like any other. The other two fire mid-transfer, with the frame's defers
possibly half run, and they park only to be looked at: stopping on a torn
unwind is strictly more than exiting before anyone can ask what tore it.
The break loop grew a per-snapshot resumable flag for it. Restarts are still
listed and still numbered, the terminal marks them untakeable and the socket
reports the same positions as unreachable, and the listener refuses a choice
with the trap's own sentence rather than the thunk-boundary one.
Standalone builds die exactly as they did: nothing installs the hook in a
program that did not import the agent, and the acceptance case for free-all
still wants exit 134 and the same message.
The review entry that asked for this named flan_exit_hook, which is normal
termination and not this at all; it is struck out with the correction.
The registry compacted whenever the table was three quarters full, and a
compaction reclaims dead entries and nothing else. A program holding more than
three quarters of the table in live blocks therefore compacted on every
allocation for the rest of its life, reclaiming nothing each time and holding
the table-wide epoch odd while it did. A listing racing that loop lost all
eight of its attempts and answered with zero rows -- "nothing is held", about a
program holding three thousand blocks, from the verb that exists to find a
leak. Measured at 199 wrong answers in 200.
The trigger now also asks whether there is an eighth of a table's worth of dead
to reclaim, which is a count four places maintain: a death, an arena's
free-all, a note written over a dead slot, and the sweep itself. That bounds
the cost from the other side too, since a sweep that runs reclaims at least 512
slots and so cannot run twice in 512 allocations.
Separately, flan_dev_reg_by_type answered a walk it could not take with zero
rows, which is the same number a program that had freed everything gets, and
stepped past slots flan_reg_snap could not copy while still calling the walk
whole. It now counts those slots and returns -1 with the count, the agent
refuses in a sentence the daemon already renders, and the snap contract says
which caller keeps it and why reg_at is allowed not to.
A note that finds no slot is still dropped -- dying because a diagnostic ran out
of room would be the diagnostic shooting the patient -- and now says so on
stderr once, quoting how many entries were dead rather than claiming the table
is all live.
test/dev_limits.c gains three modes, driven from test_reload: 3100 live blocks
read under a writer thread (1 right in 200 before, 200 after), 3000 live with
600 churned on top of them to prove the sweep still runs, and a genuinely full
table that must say so exactly once.
The writer is the game thread, in every allocation and every free; the reader
is the agent's listener, on a program that is running. Nothing stood between
them. The consequence is not a slightly wrong count: a row's type pointer and
its length mean nothing apart, and a reader that takes the new pointer with the
old length reads off the end of a string literal.
Each entry now carries the watch table's seqlock, odd while it is written, and
a reader copies the entry and re-reads the counter before believing it. The
compaction bumps a table-wide counter around itself, because it moves entries
between slots and no per-slot counter can describe that; a scan that sees that
counter move walks again. It clears the table slot by slot rather than with one
memset, since the memset would zero the counters a reader was holding.
The breakdown and the leak report stay answerable while the program runs, which
is the moment they are for. reg at does not: whether one address is still live
is exactly what a running program is changing, so it is refused the way every
break verb is refused, which is what the daemon already did on its own side.
The merged daemon now unlinks its socket on the ways out it does not
control as well as the one it does: an atexit for exit(3), which is what a
runtime trap takes, and by hand in die_now and in main's fallback, which
are _exit and skip the chain on purpose.
The client says so too. A refusal on a path that exists is a leftover, not
a daemon declining, and the raw 'Connection refused' has now misdirected
two investigations.
Separately, and it is separate: dev-repl.flan gets dev-robust.flan's
24000-tick budget. Twenty seconds of program under a two-minute test is a
second flake waiting its turn, and it is not the one fixed above -- that
one fails honestly, saying the program exited.
Three follow-ups to the registry reader.
The agent's "reg types" handler held its four row buffers as statics. The lock
argument for that was right — request_lock is held across the whole of
handle_line from both entry points — but the BSS was not: 8KB in every build
this package is linked into, including a release build of a game that imports
the agent and never writes a row. That is flan_dev.c's own argument against a
fixed table at a thirty-second of the size. Malloc'd and freed per request,
like the watch handler beside it.
A followed pointer parses as an opaque node, so the inspector refused it with
"no structure for this type" — false and unhelpful at once. It has structure,
it is drawn, and the reason you cannot step in is that the step would start
from the pointee. Said that way instead.
And the three new commands have elisp tests: the address root's wire, that a
path is refused rather than dropped, the listing's rows and totals, the
overflow warning, and a build with no registry refusing rather than showing an
empty table that reads like a program holding nothing.
