Retrying immediately looked like eight chances and was one: a walk that bails
at the epoch check costs almost nothing, so all eight fit inside the single
compaction they were all losing to, and the listing refused having waited for
nothing. A quarter of a millisecond between attempts -- the agent's break-loop
idiom, legal here because the waiter is the listener thread and never the game
loop -- bounds the whole refusal at two milliseconds. Measured with a writer
noting and freeing on top of three thousand live blocks: 8 right answers in 200
without the pause, 200 in 200 with it. It is not magic, and the comment says
so: a writer that spends most of its time rearranging the table still gets
refused, which is the honest answer and used to be a zero-row lie.
The two cases the last commit left unwired are wired now, and a third joins
them: a listing taken while the table really is being compacted, which nothing
covered -- the full-of-live case never compacts and the churn case is
single-threaded, so the retry itself was exercised by nothing. It asserts only
what a slower machine cannot change: never zero rows, never a count that is
neither right nor a refusal. How the rest divides is printed, not pinned.
The registry compacted whenever the table was three quarters full, and a
compaction reclaims dead entries and nothing else. A program holding more than
three quarters of the table in live blocks therefore compacted on every
allocation for the rest of its life, reclaiming nothing each time and holding
the table-wide epoch odd while it did. A listing racing that loop lost all
eight of its attempts and answered with zero rows -- "nothing is held", about a
program holding three thousand blocks, from the verb that exists to find a
leak. Measured at 199 wrong answers in 200.
The trigger now also asks whether there is an eighth of a table's worth of dead
to reclaim, which is a count four places maintain: a death, an arena's
free-all, a note written over a dead slot, and the sweep itself. That bounds
the cost from the other side too, since a sweep that runs reclaims at least 512
slots and so cannot run twice in 512 allocations.
Separately, flan_dev_reg_by_type answered a walk it could not take with zero
rows, which is the same number a program that had freed everything gets, and
stepped past slots flan_reg_snap could not copy while still calling the walk
whole. It now counts those slots and returns -1 with the count, the agent
refuses in a sentence the daemon already renders, and the snap contract says
which caller keeps it and why reg_at is allowed not to.
A note that finds no slot is still dropped -- dying because a diagnostic ran out
of room would be the diagnostic shooting the patient -- and now says so on
stderr once, quoting how many entries were dead rather than claiming the table
is all live.
test/dev_limits.c gains three modes, driven from test_reload: 3100 live blocks
read under a writer thread (1 right in 200 before, 200 after), 3000 live with
600 churned on top of them to prove the sweep still runs, and a genuinely full
table that must say so exactly once.
A program that wants to load its data once and keep it could not say so. Every
move-only global was refused where it was declared, on an argument about the
dead set being per function: two functions each freeing the same global would
be a double free nothing could see. The argument was sound and the conclusion
was too strong. It assumed a global has an owner. It does not.
Reading a move-only global is now always a borrow. Nothing may take ownership
of one, so nothing may free one, and with no owner to hand over there is no
double free left to catch. This is not a general ownership model for globals
and is not meant to grow into one: it is sound precisely because the lifetime
question that model would exist to answer has a constant answer here, the
process's. The refusal lands at the read, which is where a move would have been
recorded for a local -- passing the global to something that owns its
parameter, binding it to a local, returning it and freeing it all reach the
same place, and each is told to borrow instead, or to clone if it really wants
something of its own.
Such a global is mutable where it stands. push, put, reserve and set already
take their target through the borrow path, so a global (Vec u8) is filled and
grown in place, and the aliasing that raises is the one every Vec has:
spec-memory.md's explicit Zig/Odin contract, where a push that reallocates
invalidates a slice taken before it and the dev build's generation word traps
on the stale one. Globals get no borrow rule locals do not have, because the
hazard is not new and the trap lives on the Vec rather than on the binding.
What a move-only global may not do is carry a computed initialiser. A global's
initialiser is a link-time constant -- there is no init-at-startup path in the
LLVM backend by design, and the x86 backend that has one deliberately leaves it
out of a reload module, because re-running an initialiser wipes the live state
reloading exists to preserve. So the global starts zeroed, which for a Vec is
an empty Vec and therefore a value rather than a placeholder, and the load is
an ordinary assignment in whichever function loads it. That is also what makes
the data survive: nothing runs between one entry to main and the next, so a
re-entered main finds the global as it left it. A defconst cannot be one at
all, since a constant is not an assignable place and nothing could ever load
it; both refusals name the (defvar g (Vec u8)) that works.
The reload fixture gains a global Vec in the host and another that arrives at
run time, because that is where declaring instead of defining has teeth: a
module that defined the host's Vec would take a zeroed header of its own and
strand the block the process is still using, which a re-zeroed i64 cannot
demonstrate.
The first of the three things HANDOFF-x86-redef.md left: a function or a
defvar the running process has no symbol for. ELF cannot grow one, so the
address is asked for by string at install time -- flan_dev_cell for a cell,
flan_dev_global for a global's storage -- and parked in a slot this module
defines.
The reference side is one new [loc] case and nothing else. [Lslot] loads the
slot and answers [Reg (scratch, d)], which is exactly what [Lgot] already did
with [Got] where this has [Sym]; every site that reaches a cell already
double-loads, so no call site, no place expression and no [sym_loc] caller had
to learn a third case. [fnctx.slot] is a second predicate rather than a widened
[ext] because they answer different questions -- [ext] says "the host's, reach
it through the GOT", [slot] says "nobody's yet, reach it through a slot I
filled". It defaults to [fun _ -> None], so the whole-program path emits
byte-identical output and the survey goes on being a structural check.
flan_reload_install is now a function with a frame rather than a run of loads
and stores, because it makes calls and a call on an unaligned stack faults
inside glibc's movaps rather than anywhere a reader would look. Its shape is
emit_globals_init's, down to owning the null transfer cell no caller hands it.
A new global's declared value travels with it: flan_dev_global copies the image
onto the allocation the first time the name is interned and ignores it after,
which is where "a reload must not reset the state" lives. emit.ml folds that
value into an LLVM constant and this file has no folder, so the image is a
module-local buffer written by the initialiser lowered as ordinary code -- the
same bargain emit_globals_data already documents.
Republishing a defconst came free once the rest was there: one store of the new
constant into the host's global, which is what emit.ml does.
reload-v6.flan is new. v3's [extra] is declared zero, which calloc also gives,
so a run-time-new global whose initial value never arrived would still pass;
v6's [tuning] is 42 and the host prints 88.
test_reload.ml's x86 section now runs all four modules against the same
transcript the LLVM path is held to, and the refusal it used to assert is gone.
Load.program takes forms: it reads the import forms, resolves them with the
one resolver it always had, and parses the file with the packages' macros in
front of it. The refusal said this needed a second import resolver at the Form
level. It did not notice that the file being compiled is parsed before Load
runs too, so no shape of the feature could have left import resolution where
it was.
Names arrive qualified, as a defn's do. (mac/twice 4) is a call and (twice 4)
is an unknown name.
Stopped mid-task: dune test was never run and the acceptance wiring is
unfinished. HANDOFF-macros.md has what is left.
Neither limit had any coverage: a renderer that emits more than 4K and
a program that introduces more than 4096 run-time names are both past
anything the corpus does, so the truncation and the abort were code
that had never executed. dev_limits.c drives them directly — they are C
entry points with no Flan spelling, and flan_dev.c is compiled into
every build — one process per mode, because the name table never
shrinks and the overflow case aborts.
The cap case pins the length, the ellipsis, a byte from before the cut,
the generation moving exactly once, and the flag being cleared so a
short value after a truncated one does not inherit its ellipsis. The
registry case pins that 4096 fit and the next one stops the process
with its reason. Dropping result_full and moving the slot check by one
were both planted and watched fail.
flan_dev_global hands back the allocation it made the first time a name
was asked for, and compares the size it recorded against the size it is
asked for. Nothing exercised the comparison: v5 is v4 with extra as an
i32, loaded on top of v3, and what it does is abort the process — so it
gets a host run of its own. The message is asserted alongside the exit
status, because a process that died for some other reason is not this
guard firing and the status alone cannot tell them apart.
The mutation pass turned up one defect that did not make the suite go
red: a reader branch that forgets to advance reads the same character
for ever, and dune test waits as long as it is left to. In CI that is a
job killed by the runner with nothing named and no output to read.
watchdog.ml puts an alarm on every test binary — generous, because an
alarm that fires on a slow machine is a flake — and a five-second one
around each read in test_flan, where the budget really is small. The
first read that does not return wedges the rest, so a looping reader
costs five seconds and names the row instead of costing eight minutes
or never finishing. Both were watched: the string-escape loop now fails
in five seconds with the case named, and the per-binary backstop was
armed short and observed to fire.
Editing a defvar or a defn is a symbol the host exports. Adding one is not:
there is nothing to bind to and ELF cannot grow a symbol. runtime/flan_dev.c is
the two lookups that cover it - flan_dev_cell for a new function's cell,
flan_dev_global for a new global's storage - both idempotent, so the second
module to mention a name gets what the first one got. That is the whole point:
two modules with their own copy of a new function would each call their own,
and redefining it would update one of them.
The compiler picks per name. A name the host has is a symbol and costs one load
at a call site; a name it lacks is a registry lookup cached at install time in
a module-local slot, and costs two. The common case pays nothing for the
general one.
The redefinition unit is now a list of top-level forms rather than one
function. It has to be: v3 of the fixture adds a var and uses it from a
redefined bump, and splitting that into two loads leaves a module referring to
storage that does not exist yet. C-c C-c passes one name, C-c C-k passes a
file's worth, one path either way.
Four rules, each silent if broken. Every lookup resolves before any body is
published, or a caller reaches a function whose slots are still null - asserted
on the emitted flan_reload_install, since it cannot be race-tested.
flan_dev_global refuses a size change, which is the layout-drift rule's first
enforcement point rather than another exception to it. Nothing is ever
dlclosed, because a cell holds an address inside a module's text. And the table
is fixed capacity, because a module holds a cell's address for as long as it is
loaded and a realloc would strand it.
The test that separates this from a plausible wrong version is v4, which
redefines a name v3 introduced at run time. v3's bump is already installed and
is not rebuilt, so it picks v4 up only if its call goes through a cell both
modules found by the same name. Had v3 cached the function's address instead,
every other assertion would still pass and the transcript would read 246
instead of 432.
Sizes are spelled LLVM's way, ptrtoint getelementptr null 1, rather than by a
layout calculator in OCaml that would have to agree with LLVM's on every
target.
Two things, and either alone is useless, so they are one commit.
Emit.redefinition compiles one function into its own module against a host
that is already running. What it does *not* define is the design: a global is
external, so state survives a reload and sand's grid is not reset by editing
the code; every other function is a declare, so a redefined settle calls the
host's move-grain rather than a frozen copy; there is no main. Build.shared
puts that text through llc + ld -shared. ld, not clang, because a shared object
is allowed undefined symbols and that is the whole mechanism - and because the
driver is 50ms of a 20ms job. Measured here: llc 16ms, ld 3ms, dlopen 0.04ms.
Loading a body is not installing it, though. A call bound at link time cannot
notice a new one, so a dev build routes every Flan-to-Flan call through a cell
- a mutable global holding the address of the function that is current - and a
module publishes itself with one store. The cell load is emitted after the
arguments, so a redefinition between two calls cannot land inside one.
Three details that are not free choices. flan_reload_install is a named
function rather than an ELF constructor, because the agent has to choose when
the store happens and a constructor would do it during dlopen, mid-frame, on
whatever thread called it. A redefinition's own body is hidden, because default
visibility in a shared object is interposable and that applies to taking the
address too: plain @"flan.bump" inside the module resolves to the host's copy,
so the installer would publish the function it was replacing and the reload
would silently do nothing. And -rdynamic is what exports the cells at all, so
it and cells are one flag: Build.opts.dev, flan build --dev, the first time
opts means something semantic rather than an optimisation level.
The test is one process, because two runs would prove nothing about a swap,
and two .so paths, because dlopen caches by path and would hand back the first
handle. Every call in it goes through outer, compiled once into the host and
never rebuilt, so a changed answer can only mean its call site followed. v2
recurses through its own cell, which is the interposition case; it would print
the old body's text if it did not. helper differs between the fixtures purely
as a tripwire for a module that grew its own copy.
LLVM cannot fold the indirection - the cell is an external mutable global - and
a --dev calc-me keeps 46 indirect calls at -O2. values, machine and
sand-headless now run as dev builds in the acceptance table too; the sand hash
is the one result that would notice a call reaching the wrong function.
