The first is CLHS 9.1.4.1: a clause that signalled the condition it handles used to re-enter itself until the stack ran out. The stale list now names a lifted clause by the function it is written in, and BUILT.md's cost line is measured against master.
The dropped (agent/start ...) note named the wrong cause rather than being unreachable: an over-long socket path fails the agent's bind while it is linked. That case is now refused before anything is built, and the comments, the TODO entry and the fixture say so.
A dev cell carries its body's signature word beside the body, every call through a cell (and every function value taken from one) compares it with the word the site was compiled for, and the session lists the stale callers by file and line on the reply. Both backends, both installers; release builds have neither the word nor the compare.
FIX.org, NEXT.md, DISCUSS.org, docs/DISCUSS.md and the session handoff at the
root are one TODO.org now: 293 entries under seven subsystem headings, each
carrying an org keyword that says where it stands. A DONE entry is a few lines
saying what was decided and what that rules out; the reasoning that would not
compress — the embedding spike and the four reports the hand-written x86
backend was built from — moved into docs/BUILT.md instead, and its entries
point there in one line.
Every entry was checked against the tree before it got a keyword, and the
prose was wrong in both directions. Things the deleted files called open were
built: the first-evaluation stall, main being redefinable, macro parameter
lists, the type-limit constants, the array constructors, the byte fills,
inc/dec, the discard's fontification, the Emacs buffers, rt_die's _exit, the
backtrace surface, and the acceptance failure that could print and still exit
zero. Things they called done were not: the backend reports' no-plan buckets
had gone stale in the other direction, the value-dependent defvar was
superseded rather than built, and macro-expansion source locations are on an
unmerged lane, so that entry is NEXT and names the branch.
Every comment that cited one of the five by name now cites a heading that
exists, in TODO.org or in docs/BUILT.md. The session reports under
docs/handoffs/ keep naming the files they worked on, because rewriting them
would falsify what those sessions did; each carries a note saying where the
content went.
Review follow-ups on the mode pass.
The regression first, because it is the one that cost something: giving
macros a kind of their own took them out of three completion tables.
flan-disassemble, flan-disassemble-ir and flan-lowering each filtered
flan--defs on kind "fn", which had been the whole truth right up until
this branch made it half of one. A macro is compiled -- defmacro is a
defn by the time anything emits code -- so all three genuinely work on
one, and only the offer had gone. One flan--compiled-kinds names both
words and the three sites read it.
Then four sentences in the FIX.org entry that were not true, which
matters more than it sounds: that file is the history somebody reads
later to find out what happened.
The corpus claim was the bad one. It said the whole corpus round-trips
with zero differing lines. It does not, and never did -- the script that
measured it bound inhibit-message around its own reporting, which in
batch means the differences were found and then swallowed. Measured
properly: 317 files, 22 files and 389 lines differing before, 20 and 373
after. Sixteen lines fixed in two files, no new difference introduced,
and 373 lines still differing that this pass never looked at.
The other three: the fns/macros dedup is required by the new rows and is
not a fix to a bug that was there before -- a macro used to appear once,
as a fn. handler-case was already a keyword. loops.flan has three
labelled loops and dotimes-range.flan has the other two.
And three gaps the review found while checking: array-fill and array-gen
were never in the keyword list, Unit was in the type rule while the
parser refuses the word, and a prelude macro's row carried a bare name
where a program's carried its parameters. A check that pulls every head
out of parse.ml and diffs it against the three lists now comes back
empty, which is what the docstring had started claiming.
flan--special was three kinds of name in one list: the forms the parser
dispatches on, the functions the compiler provides, and the words that
stand for themselves. Drawing push like let said they were the same kind
of thing. They are three lists now, each read off the file that decides
it, and the two names in the old one that are not in the language at all
-- cast and none -- are gone with it.
What else had been missed: dyn, Allocator, Vec, Map, Fn, int and float
were not types, $t was not anything, handler-case was not a keyword, a
comma was not whitespace, & was not a name character -- which a macro's
parameter list needs, now that it destructures -- imenu had no heading
for a macro and none for the four dispatch forms, and a labelled loop
indented its body under its own binding vector, which loops.flan has
four of and would have said.
Then the second half. The defs op already told the editor what every
name is, and only completion and eldoc were listening, so a macro you
defined looked exactly like a function you defined. It carried no macro
kind to listen for and could not have: a macro is a defn by the time
there is a program. So the op answers with one -- keeping the location
off the defn it drops, so M-. still goes there -- and with the type
names, read off the checker's environment, since an enum is an i32 and
an alias is gone by then.
The editor draws them by kind, after the static rules and never over
them: a program defining its own length does not get to repaint the
builtin. A hash table and a matcher rather than a regexp of every name,
rebuilt when the cache is and not when a key is pressed. With no session
the rules come off and the buffer is what it always was.
spec-memory.md's case 2, capture by value into a stack environment, and
the calling convention the author's rulings asked for.
(Fn [i32] i32) captures; {code, env}; the common case
(CFn [i32] i32) the bare address; one word; cannot capture
A local of the enclosing function that an fn names is copied into a
struct the checker synthesises, held in a slot of that function's frame,
and the value carries its address; the lifted body reads the copies back
into named slots of its own, once, at entry. So the name in the body
means what the local held at the instant the value was made --
fn-capture.flan changes the local through a pointer after the value
exists and the fn still answers with the old one.
Two types rather than a uniform environment parameter: "while it's dyn
first, static side should never have to pay the price for the existence
of the dyn side... if you fully opt out, for instance, using --no-gc
flag, then we should be operating under Odin/C semantics and never paying
any runtime costs." The environment is declared by exactly the bodies an
(Fn ...) value can reach -- a lifted literal in an Fn position, every
handler clause, and the widening thunks -- and by nothing else. An
ordinary defn emits the signature it always did; calc-me and fourteen
corpus programs were diffed to say so.
CFn, because the C carries information: a value with no environment is
the only kind that could ever cross to C, and under the --no-conditions
direction FIX.org records it becomes literally a C function pointer. It
is not that today -- a declare cannot take a function type at all -- and
crossable's refusal says so where a reader would otherwise be misled.
Nobody needs CFn: Fn accepts everything, and the commonest reason to
reach for the narrow one is that a *named* function handed to an Fn pays
a hop through the widening thunk where a CFn is a direct call.
That thunk is one small function per distinct signature widened, which
reads the bare address back out of the environment and calls it. The
cheaper trick -- the environment last, ignored by a body that never
declared it -- is legal under SysV and is a trap under wasm32's
call_indirect, which compares the signature at the call. Every indirect
call is exactly typed now.
A handler clause captures the same way and is sound with nothing left
over: its frame is popped by the body that pushed it. What is refused
there is a *store* into a captured name -- it is a copy, and writing to
it would leave the local as it was.
And the other half, which is what "non-escaping" means: a value carrying
an environment may be called, passed down and let-bound, and may not be
returned, stored, pointed at or pushed into a container. A parameter of
type Fn is treated as one, which answers "passed to something that stores
it" with no interprocedural analysis -- the store is refused inside the
callee. Everything of type CFn is clean for free, which is the second
thing having two types buys. Every refusal names case 3, the environment
the collector owns.
Two pre-existing bugs fell out on the way. A lifted fn asked for Fnval,
so `flan reload' on any function containing an fn literal died at llc
with an undefined cell; it takes Flanfn now, which is the choice a
handler clause always made. And a redefinition module now carries its
own hidden copy of every thunk it names, which is the same bug shape
caught before it shipped.
Follow-ups to the re-run flip, and all of one kind: each said something
correct about a committed re-run that still read as parked, and the flip made
it false.
flan_merged_park's note that the program stays PROGRAM_PARKED across a poll is
now true of every round but the one it leaves on — which is the round that
drains the ring with a run already taken, and the round the fix's own argument
leans on as the long one.
Program.rerun's refusal told the reader to close a window or let the program
finish. With a thunk stopped in the break loop a re-run is accepted, the state
goes running, and a second is refused by that sentence — advice about a game
loop for somebody whose thread is in a break. The C cannot see a break; this
end can, so Dev.rerun passes what it already asked the agent and the refusal
says resume or abort instead.
The five-second timeout in eval_expr and run_render_thunk reached its
break-loop sentence through the Parked arm, so a program stopped between runs
was asked whether it calls (agent/poll). Both gain the arm that names the
break. run_render_thunk's two sentences become a function first: the new one
asks the agent, and that was the body of a five-millisecond tick.
docs/BUILT.md gains the case an editor author meets — :parked nil :stopped t
with no frames running — and FIX.org records that test_dev.ml:763's refusal
assertion was racy before this and is narrowed by it.
flan_merged_rerun accepted a request under the lock and left program_state
as it found it: PROGRAM_PARKED, until the parked thread got round to waking.
describe's :parked reads that same state, so a caller that asks for a re-run
and then waits for the program to park again was liable to be answered by the
park it had just ended — the wait fell through on the old park, the next
request went out before the first run had started, and the pair of them made
one run between them; or the thread woke in between and the second was refused
as "already running". One lagging state, two symptoms, and all four of
test_dev's re-run sites could show either under load.
The window was documented over flan_merged_park as "as wide as a flush". It
stopped being that when the ring drain went in front of the park's exit: the
leaving round loads whatever was queued before it breaks, so the window was as
long as the next thing the program had to do.
The store moves to the acceptance, under the lock that made it. There is no
longer a moment in which a committed re-run reads as parked, and the park's own
store on the way back into main stays as the no-op that says where the thread
has got to. Dev.rerun reads the liveness and the break for its note before it
asks, since afterwards the answer is running by construction.
Measured on that loop driven standalone against programs/dev-rerun.flan: 9
failures in 25 runs under two busy-loop burners before, 0 in 50 under the same
load after. FIX.org, 2026-09-21, has the readers that were checked, the path
that cannot exist, and why the two-process daemon has no such window.
The break buffer captioned every refused restart "below this evaluation", which
is the wrong sentence at a trap: there is no transfer channel, nothing on the
list can be taken, and there is no evaluation to abandon. The terminal listing
had always said the two apart. The wire now does too — a bare ! line ahead of
the entries, because a trap with no restarts at all still has to be able to say
so, and because :abandon being nil means three different things.
The boundary was also the first entry truncation dropped: snap_push walks
innermost first, and the boundary is the outermost frame of the evaluation. A
slot and a name's worth of bytes are kept back for it.
flan_break_resume goes; nothing has called it since choices became positions,
and NEXT.md already said there was no such function. eval_boundary is cleared
between runs beside the two stacks that already were. And the note on a taken
restart stops costing a second round trip: the agent answers ok abandon for the
boundary, which is the end that knows.
Nested boundaries are tested rather than argued: two evaluations, six restarts,
and abandoning the inner leaves the outer with its own still on offer.
A C-x C-e that blew up took the session with it. The expression's break offered
either nothing at all — a bad index establishes no restart, and the program's
own are below the thunk boundary where a transfer has nowhere to land — or a
list on which every entry was refused. That left abort, and abort is _exit(134)
over a mistyped index.
So the boundary offers a restart of its own. The agent pushes a real frame
around every evaluation, after the floor is read so that it lands above it;
taking it unwinds to the thunk, flan_reload_call drops the channel it holds,
and the poll returns to the game loop. It abandons and does not undo, which is
said in the agent's line, the daemon's note, the buffer's row and the manual.
The other half was a silence. The break buffer drew every restart as takeable
and ignored the :unreachable the wire already carried, so a digit on one went
out to be refused and nothing came back. Those rows now carry the reason and
are refused where they are read, and :abandon names the position that drops the
evaluation — a position, because a program may establish a restart of that name
itself.
Not the threading, which is what the report suspected. The thunk does run on
the game thread; a thunk on a thread of its own would have had the same empty
list and the same abort.
The trio the author decided on 2026-09-20 is now all built: def is CL's
defparameter — its initialiser runs on every daemon re-run, unguarded, so
an edited initialiser repaints the same storage on C-c C-c plus re-run —
defonce (Clojure's name for CL's defvar, per the author) initialises once
behind the .init~once. flag, and defconst stays the image.
One parse arm reads both forms; the difference is Ast.reinit, carried to
Tast.global's grerun. Emit.startup_plan gives a def no guard flag, and
Check.check_global lifts every def initialiser — zero and literal
included — into global/<n>, so the host's startup reaches it through the
function cell and a re-evaluated def swaps it (Session's def_inits;
Emit.redefinition declares the cell for a non-sibling target). The old
defvar spelling is refused with the rename and both compiling spellings,
and every program, test, doc and editor list is swept — except sand.flan,
the author's live WIP, whose seven defvar lines are flagged in FIX.org
and keep its three dependent tests red on this branch.
The wait recognised one kind of stop and waited five seconds for all the
others, then said the program was not reaching a frame boundary — on a reply
that named the condition two fields along.
Which stop is the thunk's is decided by the agent's stop generation rather
than by the condition's name, so a break entered on the same class as the one
it was evaluated inside is still told apart from it. A (pause) is answered as
a (pause) wherever it came from, the flag having never been what made one
deliberate.
Five fixes off the independent review, plus the author's u8 ruling.
x86 parity: the bad-index block always ran x86 (it is flan dev's
default) and now says so with an explicit --x86; the condition render
gets an assertion under the x86 backend too, beside the LLVM one, and
a user error is pinned as carrying no site on both.
ArithError's layout is now pinned: {i32 op; i64 lhs, rhs} in C against
the prelude's defstruct, read field by field through the break loop's
render, driven from the editor through a divide under a restart-case.
That also covers condition and site on LLVM.
Three refusals that were wrong: trap_site tested the prefix "err"
and so ate any site whose path began with those letters; source_line
let Sys_error from input_line escape and take the whole break reply
with it, leaking the handle; and a condition with no fields was
reported as a name no struct has. The daemon now sends its own field
count and the buffer tells the two empties apart.
Nits taken: an over-long site is dropped rather than silently
truncated into a plausible one; the caret pads with the source line's
own tabs; the headline says when it has cut the field list;
flan-cnr-layout is live again as the single spelling of that request
rather than dead beside an inlined copy.
And the ruling: a u8 renders as 97 (\a) where a person is inspecting
and stays 97 where the program is printing.
The break loop used to discard the pointer it was handed, so the buffer
could name a BoundsError's fields and never show 648. Now the snapshot
stashes it, flan_agent_condition hands it back on the stopped thread,
and a daemon-built thunk — locals pointed at the condition — renders
each field. Delivered at-stop, so a resume-and-restop cannot get the
old type read over the new pointer.
The trap sites publish their loc around the hook call, the snapshot
copies it, and break answers :site with the line's text as :source —
the frame lines say where each call was; this is the only record of
the indexing itself.
Compiler temps are hidden from the locals listing rather than refused
as s4; a shadowing rebind strips its ~N except where the outer binding
is on the same list, where both keep their raw spelling.
[flan_merged_park] drained the agent's ring on one of the two flags that
wake it. [program_poll] — which an expression sets, by way of [Program.wake]
— polled and went back to sleep; [program_asked] broke out of the loop and
re-entered [flan_program_main] with the queue untouched. A plain
redefinition sets neither, so a body delivered to a parked program was still
in the ring when the run it was delivered for started, and installed at that
run's first frame boundary instead: everything main did before its first
(agent/poll) ran the body the person had already replaced, and the change
showed up one run late. A redefined main is the whole of a run, so it would
have had to be asked for twice.
Both flags drain now, and the exit drains before it leaves. The re-run is
still tested first and cannot be starved: the flag is latched at the top of
the round and nothing in the round can clear it.
The transcript row in test_dev.ml asserted the old ordering by name — two
lines out of the second run, the first of them the stale body — so it is a
line shorter now, and the absence of that line is the claim. The park-note
fixture grew a print of the redefinable body before its first poll, which is
what makes the new row able to see which body the re-run started with.
This is what the note the delivery is answered with has been promising: a
module queued against a park installs no later than the program's next run.
It now installs before that run's first frame rather than during it.
[merged_serve] waited up to ten seconds for the program to bind agent.sock
before it started [accept_loop]. The listening socket was already up, so an
editor connected fine and then heard nothing: every first request of every
session cost the whole wait when the program calls (agent/start ...) late —
sand.flan starts it after rl/init-window returns — or never.
Nothing the editor asks needs that socket. In one process a delivery is a
call into flan_agent.c, not a connect, and the two-process daemon has already
waited for the bind in [two_process] and fails if it never comes. What the
wait was for is the sentence a program with no agent deserves, and a sentence
does not have to be in front of the loop to be said. So it is a deadline the
session passes ([agent_check]) rather than a wait it does: read from the
accept loop between connections and from [serve] before each request, because
an editor holds one connection for a whole session and the loop is not
cycling while it is attached. No thread, for the reason lib/dune gives about
what the merged link does to this library's dependencies.
Delivery stays honest either way. A program that links no agent at all
refuses through [over_socket]'s ENOENT, as before. One that has the agent but
has not started it takes the module into the ring and is answered with a note
that promises the poll and not a frame: a program with no (agent/poll) in it
never installs this, and "at its next frame boundary" would be the reply that
makes a redefinition look applied when it is not. C-x C-e's five-second
timeout gets the same distinction instead of asking whether a program that
has not got to its loop yet is calling the poll in it.
And the note a parked program's delivery carries is now said once per park.
A finished program is parked, so re-evaluating while a run's output is on the
screen repeated a paragraph on every C-c C-c. The first delivery of each park
explains itself; the rest say the one line that is the claim. [rerun] clears
the flag as well as [eval] does, so a new park is a new reader.
flan.abi.require was spelled by hand in both backends, which is the one
job Mangle has. Moved; emit and x86 produce byte-identical output on the
reload path either way.
Four comments in the dyn-cast code asserted things that are not true.
The warning's location prefix now reads like every other loc-bearing
runtime diagnostic instead of inventing a shape. widen's contract says
what cast_dyn actually does with it. The thread-safety note names the
torn {ptr,len} overread rather than a duplicated line, and says why no
lock. The site table's borrowed loc pointer names what keeps it valid.
The memory op's note claimed a completeness it does not have: dyn push
and put may allocate and are deliberately silent. Said so, in the note,
in the classifier, and in FIX.org where the decision belongs.
The documented flycheck form only matched warnings, so a real error
made it say the checker returned non-zero and found nothing.
flan-clear-memory cleared one buffer where the toggle clears all.
Two comments claimed test/dyn_ops.c calls every function flan_dyn.h
declares; six are declared and never called there.
flan_dyn_stub.c's deadness is written into FIX.org for the author to
decide on. Not deleted here.
The %handler, %restart, %fninfo and %flanframe shapes were written twice:
as LLVM type strings in emit.ml and as hand-computed byte offsets in x86.ml,
with the two %fninfo initialisers spelled a third and fourth time. Emit.Rt
now holds one field list per struct and derives all four — the type string
and the getelementptr index for LLVM, the offset and the size for x86, and
the initialiser for both. The derived numbers were checked against every old
constant before the call sites moved.
The flan. prefixes were spelled in four files, including both backends
hand-writing "flan." ^ name for a DWARF linkage name instead of calling
their own helper. Mangle now holds them unquoted; each backend adds its own
sigil. The ABI markers stay apart on purpose: flan.abi.llvm and flan.abi.x86
differing is what makes the loader refuse a crossed pair.
The float-to-integer cast bounds and the division-check elision policy are
Emit.cast_range and Emit.div_checks. The second is a language decision and
had been byte-identical in both files; the first had drifted cosmetically.
emit and emit --x86 output for all 166 test/programs, at -O0 release, --dev,
--debug and --dev --debug, stdout and stderr, is byte-identical to the
pre-change compiler.
[Check.memory_sites] is the pass [Check.no_gc]'s shape: it runs over the
finished program, answers a diagnostic list, and tells nothing downstream
that it ran. Two classes on the diagnostic's kind — the collector's heap
and an allocator the program named — so the CLI, the daemon and the editor
dispatch on one field and none of them reads a message to find the class.
[--warn-memory] on check and build prints them where errors go, in the
shape flycheck parses, without moving the exit status. [(:op "memory")]
answers the same list over a session's last checked program, needing no
program on the far end. [M-x flan-check-memory] paints it two faces fainter
than an error's, cleared by an edit or by asking again.
Two of the spec's own examples turned out not to allocate, and the
precision rule outranks the enumeration: (vec-new T) passes a capacity of
literal zero to flan_vec_init, which returns before the grow, and
flan_map_init takes no block at all and says so in its own comment. The
block arrives at the first push, which is the line that is marked. The
classifier reads the capacity argument rather than the symbol, which is
what lets slurp be marked through the same entry point vec-new is silent
through. FIX.org has the rest of the evidence.
The dev runtime rendered a value twice over: [flan_dev_emit_{u64,i64,f64,str}]
into the result buffer an evaluation is read back from, and
[flan_dev_watch_emit_*] into the current watch slot, with the same four bodies
either side and the sink as the only difference. [flan_dev_result_end] and
[flan_dev_watch_end] were the same ellipsis-and-generation close, comment for
comment, over two buffers.
So the rendering takes the sink as a parameter and the eight entry points are
eight one-line calls into four statics. The exports stay eight: the compiler
emits four of them by name (Session.externs) and a program reaches the watch
four through declare-c. ABI does not collapse because the bodies did.
The escape table is now [flan_escape_char] in flan_rt.c, once: what one byte
reads as inside a quoted string, into a caller's four bytes. A table and not a
printer, because the framings are genuinely different — [flan_escape_bytes]
builds a capped slice to hand back and the dev pair streams into a buffer it
does not own the end of — and the framing is the part that is each caller's.
The dyn printer keeps its own copy, which is docs/SPIKE-DUPLICITY.md §9's one
defended repeat: it is inside the runtime that owns the storage it walks. Its
comment, and flan_rt.c's, no longer tell a reader to change the other two.
Same for the NaN rule, which was spelled four times: [flan_f64_format] is
flan_rt.c's [flan_f64_to_bytes] without the slice, and the REPL emitter and the
watch table call it rather than restating "%g, and nan unsigned".
Byte-identical, checked two ways. A session driven over the daemon's socket
before and after — every arm of the emit family, the escapes, and a string long
enough to reach the truncating close — diffs empty. And a harness linking both
trees' flan_rt.c + flan_dev.c compares 8173 renderings: every byte 0..255
through both string emitters and through flan_escape_bytes, every length across
both caps and the ellipsis either side of them, both NaN signs, both infinities,
i64 and u64 at their extremes. Identical.
Dead code, each verified by its own grep before removal. These are exported C
symbols, so a program could reach one through declare-c; the evidence is that
nothing in the tree does, including the docs that write the surface down.
flan_dev_watch_u64 (flan_dev.c) — one occurrence repo-wide, its own
definition. The i64/f64/str siblings are declare-c'd in
test/programs/dev-watch.flan and written down in emacs/MANUAL.md; this one
appears in neither, and in no other file.
flan_break_resume (flan_rt.c) — the only non-prose reference was a stale
extern in vendor/agent/flan_agent.c with no call under it. Both gone.
[flan_name_id] stays: the bounds and arithmetic conditions still hash through
it. [flan_restart_take]'s comment no longer points at a function that is not
there.
flan_dev_watch_enabled (flan_dev.c) — prototyped in flan_agent.c, never
called. [watch_on] is still read directly by the three sites that gate on it.
clang_stamp (lib/build.ml) — a [lazy] never forced; one occurrence.
marshal (lib/expand.ml) — no reference anywhere. [write], which it wrapped,
is called twice in [call], so the [let rec] group is demoted to keep it.
is_bytes (lib/js.ml) — dead within js.ml. Nothing else in that file is
touched: the JS backend is parked, not dead.
Kept on purpose: [Loc.forget_sources], documented in docs/BUILT.md as
deliberately retained, and [flan_dev_watch_num_f64], which emacs/MANUAL.md
declares as public surface.
Two comments in lib/dev.ml argued the orphan grace in terms of elisp symbols
from before the rename — [flan-dev--open], [flan-dev--connection],
[flan-dev-poll-interval], and a file called emacs/flan-dev.el. None of those
exist. Re-spelled as [flan--open], [flan--connection], [flan-poll-interval] and
emacs/flan.el, which is where they are; the reasoning is load-bearing and is
unchanged.
dune test: exit 0. 59 lines of code out, 24 lines net of the prose that says
why.
flan_merged_park called flan_dyn_root_reset, which emptied the collector's
root stack. The frames' roots had to go — main is left by longjmp, so they
name stack the next run overwrites — but the dyn globals' roots are on that
same stack, pushed once by the emitted main and never popped, and the park
took them with the frames.
The park is not a quiet state. It services evaluated thunks, a thunk
allocates, and an allocation collects. So a program with (defvar config dyn)
answered (get config :s) with its string before any thunk ran and with nil
after one that allocated past the heap's floor — a read of memory the sweep
had freed, answering nil by luck of what the freed words decoded as.
The emitted main now brackets its global pushes: flan_dyn_root_globals_begin
empties the stack, the pushes go on, flan_dyn_root_globals_end records how
many of them there are, and the park resets to that line instead of to zero.
Nothing between the two allocates, which is what keeps the globals from being
swept in the window where they are unrooted — and [begin] emptying the stack
rather than adding to it is what makes a re-entered main re-root the same
globals rather than push a second copy of each, which also closes the other
half: a re-run used to re-push roots over slots left dangling by the park.
Both emitters, because the dev loop's default backend is x86 and a fix in one
lowering is not a fix. A program with no dyn globals emits neither call and
its root stack still resets to empty, which is what an empty push list should
leave behind.
flan_dyn_root_pop now clamps at the globals rather than at zero. An
over-popping frame eating the globals is the one way that clamp could turn a
miscount into this same use-after-free.
Covered twice. test/dyn_ops.c's park mode is the runtime's half — a run, a
park with a collecting thunk in it, and another run, three times over,
asserting both that the global survives and that the frame's five hundred
objects do not. Under ASan the old reset reports heap-use-after-free in
flan_dyn_tag with the free in gc_sweep; under memcheck it reports 24 errors
and still prints the right answer, which is the shape of the bug. test_dev.ml
drives the whole daemon over its socket on both backends against
programs/dev-dyn-global.flan.
Not touched, and it wants a decision rather than a patch: a re-run re-enters
flan_program_main, which re-runs the lifted startup function, so every global
with a computed initialiser is reset by a re-run. That contradicts dev.ml's
own note and FIX.org item 1. It is independent of this — the roots are right
whether or not the values are re-initialised.
Nor is this the reload path. A defvar added by an evaluation gets its storage
from flan_dev_global (emit.ml's new_globals, x86.ml's counterpart) and there
is no flan_dyn_root_push anywhere on that path in either backend, so a dyn
global added to a live session is unrooted. That is a separate defect with a
separate fix, and nothing here makes it better or worse.
Three shapes hand a dyn word into memory nothing roots, and the check written
last round caught one of them. A (Ptr S) return was refused; (Ptr (Ptr S))
and (Ptr [S]) were not, because the predicate it asked was dyn_anywhere, which
had just been taught to stop at a pointer. Stopping there is right — a
pointer is a view of storage something else roots, and that is what lets a
(Vec (Ptr Cond)) be written — but it is the wrong question at a boundary where
nothing roots the far side at all. So there are two predicates now.
dyn_anywhere is the storage-shaped one and is unchanged; dyn_through follows
pointers and slices, and only the foreign boundary asks it. The second of
those shapes matters more than it looks: (Ptr [S]) is what shim.ml's own
advice tells people to write when C returns an aggregate.
Neither was a regression. At the commit before last there was no walk over
the externs at all and all three were accepted; what landed was one level of
a check that wanted to be recursive.
And the direction. The refusal was justified by what C hands back and then
applied to parameters as well, which made the ordinary read-only borrow
unexpressible: a foreign parameter receives the address of a *place*, and a
place is a frame slot, a global or an array inside one, every one of them
rooted with its descriptor and marked for the whole call. So (Ptr S) and [S]
as parameters are borrows and stay writable. One level down the storage is
C's again — a (Ptr (Ptr S)) parameter is an out-parameter and what C writes
into it is a pointer of C's own — so the parameter question is asked below the
outermost level and the return question is asked from the top.
Evidence, all three shapes with the check lifted, ASan: heap-use-after-free in
flan_dyn_tag, freed by gc_sweep out of flan_gc_collect, allocated by the Flan
frame that stored it. With the check back, all three refused by name. And
the allowed direction proved rather than assumed: a C read-only borrow of a
(Ptr S) and a [S] slice argument, called before and after twenty thousand
allocations, dyn fields intact and clean under ASan.
A Tast.extern carries its declare's location now, so these refusals point at
the line rather than at <unknown>:0:0.
Last, the one arm of the saturation that did not saturate: a negative array
length reached the multiplication as a small number, which is the exact shape
the cap must never be handed. Out of range in either direction saturates.
dune test --force: green, 0 failures.
x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
`(:op "set")` is `inspect`'s addressing with the arrow turned round: same
frame, same slot index, same path steps, same fingerprint check, and a list of
(path, expression) edits relative to what the buffer is showing. The thunk
stores and then renders the same place, so what comes back is what the program
holds afterwards rather than an echo of what was asked for.
`stopped-only` was not enough for a write. It asks whether the program is
stopped, and for a read that is the whole question — the worst a render can do
against the wrong stop is print something true of a different frame. A module
that *stores* through `flan/dev-slot` reaches its target through whatever
snapshot is on top when the store runs, so a resume and a second stop inside
the ~300ms build window lands it in the same slot index of a different stack.
Not a fault: a plausible shape, in the wrong place, silently. So the agent
grew `at-stop N` beside `stopped-only`, the generation `snap_push` already
mints for the restart machinery, checked on the game thread at the moment the
job is claimed. `stop` answers it, `inspect` carries it out on every reply, and
a write that names a stop the program has left is refused before anything is
built.
Three refusals about where rather than what, and they are in session.ml above
both backends because emit and x86 disagree about two of them. A data type's
case field has no address that does not also settle the tag. An option's
payload has none that does not settle whether there is one. A pointer would be
an address this end made up, which is the blessing the registry exists to
insist on.
`Check.expression` grew a `want` and `Check.expressions` a shared frame: the
first is why `3` into an `f32` field is an f32 three rather than "expected f32,
found i32", and the second is why two edits in one commit are not two `let`s
reading each other's storage.
Milestone 1 of dynamic-by-default, the runtime half: NaN-boxed values in one
machine word, a mark-sweep heap, and the operations over them.
A double is itself, which is what a language with a physics loop and a float
calculator in its corpus wants; everything else hides in the quiet-NaN space,
three tag bits and a 48-bit payload that is exactly an x86-64 user pointer.
The negative-NaN collision is answered by canonicalising every NaN on the way
in, which flan_rt.c had already decided was the right thing to print. An i64
past the payload goes on the heap rather than becoming a 48-bit integer with a
64-bit name.
The collector is mark-sweep and nothing else -- no generation, no barrier, no
free list -- because the answer to wanting it faster is to type the program.
Roots are pushed, not scanned: NaN-boxing makes a conservative guess wrong in
both directions, and flan_dev.c's frame chain is the precedent. A fixed ring
of the last sixty-four allocations is marked unconditionally, which closes the
window where an expression with two constructors in it can collect its own
first result before the compiler has rooted either.
A type mismatch traps rather than aborting, through a flan_trap exported from
flan_rt.c so it takes the same path the six existing traps take: parked for
inspection in a dev session, dead where it stands otherwise. The sentence
names the operation, both tags as words, and both values.
flan_dyn.c is its own translation unit and nothing in the release runtime
names a symbol in it, so a program with no dyn operation links no collector
and --no-gc can be file-level selection rather than an argument with the
linker.
docs/SPIKE-DYNAMIC.md carries the argument. test/dyn_ops.c drives every
operation and all twenty-four refusals from C, the way dev_limits.c does,
including a million allocations against a hundred live and the control that
says an unrooted object really is reclaimed.
flan dev takes the hand-written backend unasked now: it is what that
backend was written for, it halves the C-c C-c round trip, and nothing it
builds outlives the session. Every other command is LLVM by default and
unmoved, which is what keeps lib/x86.ml's calling convention licensed.
--debug picks LLVM on its own rather than landing on the refusal: a
redefinition module from this backend carries no line table, so the one
flag someone types to get a debugger must not choose the backend without
one. --x86 --debug together is still refused, and now says which to drop.
What the flip costs is more than the forms this backend refuses. It
pushes no shadow stack, so a session built by it stops on an error and
cannot say where -- and the agent answered that with "this program was
not built with --dev", which is false of an x86 dev host. Dev.ask
rewrites it; the IR view refuses by name instead of failing to find a
define in a .s file; and a refusal at the host build or at C-c C-c names
--llvm rather than stopping at "unsupported".
flan-daemon-args carries the flag from Emacs, spliced into the one
argument list the daemon buffer's first line is now printed from.
`arena-new` is a builtin, so it is in no program's symbol table and `defs`
never mentioned it — which made the editor answer "the running program
defines no arena-new" about a name that works. The fix is not a better
refusal: it is that the seventy-eight names the checker answers without
being told are now in the reply, under a kind of their own.
check.ml carries the table, beside the arms it describes, because a table
in another file drifts from them with nothing said. test_flan reads both
the arms and the table and fails on either having a name the other does
not, in both directions — there is no reflecting over a match, so it reads
the source.
Each entry is a signature in `signature_of_fn`'s shape and one line. The
arms that do not have one shape say what is true instead of pretending:
`?` for an argument that may be left out, `|` for the types an arm really
takes, and the checker's own predicate names for the type-directed ones.
The three user-allocator names carry a bare name and no bracket list,
because they are refused wherever they are written.
`defs` grows a fifth string for the prose, and builtins are appended last
so a completion table does not bury the names being worked on. A defn has
no docstring to put there and does not get one here: the Tast keeps none,
and that is a different piece of work.
The editor reads the kind, not a special case. `flan-doc--where` and the
xref backend both answer before their empty-location branch, because a
global's missing location and a builtin's absent one are different facts
and only one of them is about the daemon.
Three gaps in the first pass. The park flushes after the poll, which is the
only thing that gets a printing expression's line out of a fully buffered pipe
before the next run — now pinned by its position in the transcript. The globals
section under a parked break answers with everything skipped, which is a
different reply from the timeout a job nobody polled would give, so it is asked
for rather than assumed. And the render thunk's five-second sentence no longer
asks a parked program whether it is calling agent/poll.
The park waited on one flag and could do one thing, so C-x C-e on (+ 1 1) was
refused for want of a frame boundary — an expression that needs nothing from
the program, in a process holding every global the run left.
It waits on two now. A re-run leaves the park; a wake drains the agent's ring
and waits again, with the state still PARKED, which is what makes running the
thunk there exactly as safe as running it at a frame boundary: while parked
there is no concurrency to be unsafe against. The break loop is the precedent
and CL's spawned worker is deliberately not copied.
A thunk that stops now stops on a parked thread, so the restart ops refuse on
whether a break is engaged rather than on the state, and a paused expression
against the park is resumable.
[close] was the only way out of the accept loop that anybody ever took, and
an editor that is killed rather than quit never sends it. Four of the eight
orphans found on the author's machine were that.
The signal is the socket, not a new op: Emacs opens one connection for the
whole session and drops it only on paths that send [close] first, so an
editor left open overnight is an editor still attached and the grace cannot
accumulate under it. Armed only once a client has been there, so a daemon
still waiting for its first one is untouched. Parked sessions get the short
clock and live ones six times it, because a parked program is invisible and
a running one is a window somebody may be looking at.
