The allocation registry had a recording side and half a reader. This is the
rest of the reader: point at any heap address, a breakdown by type, what is
still held, and the test that stops dev-ptr.flan's header from being read by
hand.
The recorded name, back to a type. The table records a string and has to —
the note is built where the concrete type exists and what crosses into the
runtime is bytes. What closes it is that the string is Types.to_string, which
is the source spelling, so the round trip is the language's own reader,
Parse.texpr and Check.resolve. No table of spellings is written down, so
nothing can fall behind Types.to_string, and a name that is not a type —
"pool slots" — is refused with the name quoted rather than defaulted.
The address root renders a (Ptr T) and not the pointee, which puts it through
render.ml's pointer arm: permission is asked in one place in the compiler, and
an address root and a slot root reach the same two answers by the same code.
Flan has no integer-to-pointer cast, so flan_dev_reg_addr is an extern beside
flan_agent_frame_slot, for the same reason.
One walk and two questions: a leak report is a breakdown with the dead left
out, so flan_dev_reg_by_type is one function and the agent formats it.
"At exit" is not a hook. A program killed by a signal runs no handler, which
is how a game under the editor ends, so (:op "leaks") is the authoritative
reader and can be asked at any moment including the one before the kill. The
atexit hook is for the program that returns from main, is registered from
inside flan_dev_reg_enable rather than by a file-scope destructor so that a
release build does not grow a third not-free place, and is off unless
FLAN_DEV_LEAKS is set because the acceptance table reads stderr.
The memcheck half of item 6 is deliberately not here.
The watch accumulator clears a slot lazily, on its next sample, and the reader
never compares epochs. A previous lane read that as a defect and left
watch_render_num's n=0 arm behind as dead code against the day the reader
became epoch-aware.
It should not. A stopped program takes no samples, so an epoch-aware reader
would report every slot as empty for as long as the program sat in a break
loop — and reading the numbers from the moment you stopped is the whole point
of stopping. The lazy clear is the right answer there.
What was wrong was narrower and lives in the editor: flan-watch--tick sent
`:reset t` five times a second at a program that could not answer it. The read
still goes out every tick; only the reset field drops, guarded on
flan-dev--stopped, which flan-dev.el's background poll already keeps current.
The n=0 arm is deleted rather than commented, since the only way to reach it is
the epoch check just rejected and dead code is an invitation to add one. n is
at least 1 whenever watch_render_num runs, so sum/n cannot divide by zero.
test-flan-watch.el asserts both halves with no daemon: a running tick carries
:reset, a stopped one does not, both still read the table and both leave a
reply in flight. Verified by mutation. What it cannot reach is the daemon
honouring the absent field; test_dev.ml drives a real program for that.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
flan_dev_reg_dead was reaching for the containment scan, and it is on the free
path: a dev build would have paid a 4096-entry sweep per free. A free hands back
the base address the allocator gave out, which is what the slot is keyed on, so
the question there is equality and never containment. Only free-all needs the
scan, and that runs once a frame.
The table is allocated when it is armed, not declared. A fixed array was a
quarter of a megabyte of BSS in a shipped game for a table that build never
writes; now a release build carries a null pointer and the not-taken branch.
The pointer arm binds its subject to a slot before naming it three times — the
slice arm's rule, and its reason: an inspect with a path reaches a leaf through
a bounds check, and three of those to render one pointer is the walk paying for
its own shape.
dev-ptr.flan shows both halves on a stopped stack. It was read by hand; the
test_dev.ml case that would drive it is another lane's file, and NEXT.md says so.
(Ptr Enemy) already says Enemy, at compile time, in the walk. What the renderer
lacked was any way to know whether the storage at the far end is still there —
and an allocation registry is exactly a record of which addresses it is still
true to read. So the inspector follows a live one and renders the pointee by the
same walk as anything else, and names what died at a dead one.
println does not, and the split is not squeamishness: spec-memory.md fixes what
a printed Ptr prints, a printed line belongs to the program and has to read the
same in a release build, and a release build has no registry to ask. The two
callers already differ in an emitter record; they differ in one more.
No address appears in the text. An address is not stable across two runs, so
printing one would make a rendering depend on where the heap landed — the rule
Render already follows for an allocator. What a reader wants from a dangling
pointer is what died.
registry.flan is one program read twice: a dev build answers for an address at
the heap, arena and pool tiers, and a release build answers 0 to all of it. The
arena row is the free-all Valgrind cannot see — this does not make memcheck
report it, it makes the same read answerable.
The table, and the half of the wiring that needs no type name. A struct is its
C layout with no header and no tag word, so nothing at run time can say what is
at an address — and adding a tag would break the FFI. The registry sidesteps it:
the compiler knows the type at the moment memory is asked for, so the insert is
emitted, and the dead-marking is not, because an address needs no type.
Entries are blocks rather than values and lookup is containment, which is not an
optimisation: every heap pointer a program can hold is interior. (at v i) is
v->ptr + i*size and (resolve p h) is an item in the middle of a pool. Exact hits
would answer nothing anyone can ask.
Dead entries stay until the allocator hands the address out again, which is when
the old answer stops being true. An arena's free-all marks its whole range dead
— the release memcheck is never told about. That does not make memcheck report
it; it makes the inspector able to.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
RESULT_MAX was written down twice, once in flan_dev.c and once in the agent,
with a run-time check that the two had not drifted. That second copy was a
buffer sized to be sent through a socket, and it is gone: the agent asks
flan_dev_result_cap() and allocates, which it can do because it is never the
game thread.
The bound itself stays, and the reason is the rule everything else here is
built around. result is what the game thread writes into, from a render thunk
at a frame boundary; a growable one is the frame thread calling realloc, which
is an allocation in the one place this design exists to keep allocation out of.
It would also break the seqlock, which is a protocol about torn contents and
assumes the address it copies from neither moves nor goes away underneath the
reader. Growing on the writer's side is a use-after-free the counter cannot
see.
So it is a render budget and not a wire size, and it only looked like one
because the agent had a copy of it. Removing the bound is a redesign of the
read -- probe, allocate, re-read, validate, retry -- and belongs with moving
the read to a frame boundary.
Also: flan_agent_request answers "err path too long" where the socket does,
instead of nothing. One verb table is only worth having if the two callers
cannot be told apart.
The measurement table in BUILT.md is retaken back to back across all four
shapes, because this machine drifts by more than the thing being measured. The
internal socket was ~41us of a 21ms redefinition; --two-process is unchanged
and is not slower than the merged build in any column.
The globals section attributed a frame by its slot fingerprint, which is the
wrong cut for it: a redefined body can name entirely different globals while
binding identical locals, so the check saw no change and the new body's
reference set went into the union under the old body's frame, with the frame
numbers beside an entry saying so.
So a second fingerprint. Reach.ref_fingerprint hashes the set of globals a body
names — sorted and deduplicated, because a reference set is not ordered, where
slot indices make the slot fingerprint order-sensitive on purpose — and it
travels the path the first one already cut: %fninfo, flan_dev_frame_refsig, the
agent's snapshot, the backtrace line, Dev.globals_op. Different means the frame
is skipped by name with its reason, and the rest of the stack still contributes.
Two numbers rather than one, because they are two facts. A frame whose slots
match and whose globals do not has locals that are perfectly readable and
attribution that is not, and a combined hash would make locals refuse a frame
with nothing wrong with it. locals still checks the slot fingerprint alone.
It lives in reach.ml because expr_refs is already the walk that answers what a
body refers to, and is the walk the union itself is built from. One consequence:
emit now reaches reach, which closes a cycle through Load if cimport calls
Build.cachedir, so the header cache spells the object cache directory itself.
test_dev.ml drives the exact case — a body that binds identical locals and names
untouched where the stopped frame names pressure. With the check disabled it
fails twice: the missing refusal, and untouched appearing under frame 0.
The refusal for a frame whose body has been redefined underneath it did not
fire because four of its five hand-offs were never written. `Emit.fninfo` has
been storing `slot_fingerprint` in the last `i32` of every `%fninfo` all along;
`flan_dev.c` called that field `spare`, there was no accessor for it, the agent
never snapshotted it, the backtrace line never carried it, and `Dev.locals`
compared slot counts and nothing else. The handoff note's "every piece is
written and the refusal does not happen" was a guess, and the first step it
suggested — printing both sides of the comparison — could not have found it,
because there was no comparison.
So: `spare` becomes `slotsig` and gets `flan_dev_frame_slotsig`; the agent
snapshots it beside the slot count and puts it on the backtrace line *before*
the location, since the name is the one field that can contain a space and has
to stay last; `Dev.backtrace` parses it; `Dev.locals` compares it against
`Emit.slot_fingerprint` of the body this session holds and refuses by name when
they differ. No change to `emit.ml` — the value was already there.
The mechanism itself is right and stays. `slot_fingerprint` hashes every slot's
name together with the spelling of its type, so a rename that keeps the count
and the types — exactly the case this exists for — changes it. The count check
stays in front of it because its message is the more specific one.
The fingerprint stays off the wire. A hash is not something an editor can act
on, and the refusal says the fact in words: this frame's body was redefined
since it was entered, so its names no longer describe its values.
`test_dev.ml` gains the inverse and the control. A body that drops a `let` is
refused on the count, and `main` — untouched by the redefinition of `look` —
must still answer, which is the assertion that would catch a fingerprint that
never matched anything and made the verb useless while turning the suite green.
The half the shadow stack was built for. A slot's entry in the frame is its
address, null until the binding that fills it has run, so "not bound yet at
this point" is a null and needs no liveness analysis. The daemon compiles a
thunk that renders the types it already knows -- Tast.fn.slots, with snames
beside them -- at the addresses the stopped program supplies, and reads the
text back the way C-x C-e does. Nothing is copied out, because a value with
no header is bytes with no meaning anywhere but in the program that holds
it.
That is render.ml's walk with its root changed, which is the pointer-rooted
thunk NEXT.md said this needed, and one new arm in the backend: a cast from
one pointer type to another, which emits nothing.
Only named slots are recorded. A recorded slot escapes and stops being
promotable, and the slots that would cost most are the ones with nothing to
show -- dotimes' bound, the temporaries min and max use, the walk's own
scratch. They are refused by name rather than shown under an invented one.
Recording every slot was built and timed and is inside the noise, so the
rule stands on what it shows.
Four refusals, each by name and with its reason: a slot nobody named, a
slot the program has not reached, a type the printer has no arm for, and
two whole frames -- an evaluation's thunk, and a frame running a body that
has been redefined since, where every slot index would be a guess.
Measured, minimum of nine runs: +61% on call-heavy code over globals
against +33% for the frames alone, 0.06% of a frame at 60fps.
plan.org has specified a shadow stack in the dev column since the beginning
and nothing had ever built it. A frame is four words on the calling
function's own stack: the one it displaced, a pointer to a static
description of the function, and two words reserved for its locals. The
name and the location travel on the frame, so a backtrace needs no debug
information, no symbol table, and nothing from the platform unwinder that
plan.org deliberately does not use.
The pop is at every ret, the landing block a transfer leaves through
included. That is the half that is easy to get wrong: a pop written only on
the normal path leaves a dead frame behind every handled error, and the
test takes five breaks and resumes all of them by transfer before asking
for two frames.
(:op "backtrace") answers from a snapshot the stopped thread takes, beside
the restarts and for the same reason, and marks which frames belong to the
program and which to the evaluation the break is inside. It is refused
while the program runs.
Measured, interleaved, three pairs of binaries: 29% on 600 frames of sand,
7.6% on a benchmark that is nothing but calls -- 32us per frame of sand, a
fifth of a percent of a frame at 60fps. An array with a stack pointer was
built and timed as the alternative and is worse on both.
The thunk calls flan_dev_result_begin before it evaluates anything, so an
expression that signals is stopped inside the seqlock's window — and a restart
taken from that break transfers past the thunk, so the matching end never runs.
An unpaired begin cost nothing while the counter only moved at the end. It
costs everything now: incrementing would leave the count odd for the life of
the process, every later read reporting a write in progress, and C-x C-e dead
until the program restarts.
So begin sets the low bit rather than incrementing, and end clears it by
setting rather than adding. The ordinary sequence is unchanged — 2k, 2k+1,
2k+2 — and an abandoned write is over as soon as the next evaluation starts.
What that does not fix, because one buffer cannot: an evaluation running while
another is stopped mid-render shares the buffer, so the inner value is the one
that survives. That was true before the counter was a seqlock and is not a
regression.
Also noted in NEXT.md: rt_die in flan_rt.c has the same exit-with-the-loader-
lock-held shape the break loop just lost. Not fixed with it, because rt_die is
the non-dev path too, where there is no listener to deadlock against — whether
it should be _exit always or only under --dev is a decision.
And the 4K-cap assertions clamp their own String.sub, so a short body prints a
failure instead of raising out of the test.
The 4K result cap and condition_name[128] are on the agent's socket path, which
is why the sanitizer corpus cannot reach them: a program in the sweep has no
socket and nobody on the other end of it. test_agent has both.
A 5000-byte string literal evaluated into the running program comes back as
exactly 4096 bytes ending in the ellipsis result_end puts there to say it
clamped — and it comes back through the seqlock's copy, so the cap and the new
reader are pinned by the same case. The header also shows the generation as 1,
which is the count of complete values rather than the raw counter.
A condition class of 198 characters comes back from `status` as 127 and a
terminator. Aborting out of that break is what pins the exit status at 134 now
that the loop leaves with _exit rather than exit.
SNAP_MAX, SNAP_NAMES and the dev registry's overflow guard are still read
rather than tested. Sixty-five nested restart-cases and four thousand interned
names are a lot of program to write for a clamp each, and neither is on a path
this session changed.
flan_dev_result_cap() exists so the size is asked for rather than written down
in two files: "the copy is never truncated" is only true while the agent's
buffer and the runtime's bound agree, and the agent checks that where the copy
happens.
The pipe the queue program blocks on is close-on-exec, or the child inherits
the write end and its own stdin never reaches end of file — it sat in its last
read waiting for a byte only it could send.
It read the generation, then a non-atomic length, then returned the buffer
itself — and the agent sent those bytes down a socket some time later, while
the game thread was free to be a hundred bytes into the next value. A seqlock
cannot validate a read that finishes after it returns, so the pointer was the
bug and not the ordering.
Made into a real one rather than documented down to what it guaranteed,
because what it guaranteed was nothing: the generation told the daemon a new
value had arrived and said nothing about whether the bytes it then read were
that value. Writing the honest comment would have left the daemon's only way of
reading a result unsound with a note beside it.
The counter is odd for exactly as long as a value is being written.
flan_dev_result_read copies into the caller's buffer and checks the counter
either side of the copy, retrying if it moved; a reader that loses the race
reports the last complete generation and no bytes, so a daemon polling for a
new value keeps polling rather than being shown half of one. The count handed
out is the number of complete values, so "has it moved" still means what
lib/dev.ml takes it to mean. The agent's buffer is RESULT_MAX, so the copy is
never truncated.
The race itself has no regression test. Arranging it means landing a socket
read inside a render thunk from outside the process, which is the same hook the
snapshot generation wants. What is tested is that eval still reads back the
value it rendered, through test_dev's existing cases.
C-x C-e rendered the scalars and refused the rest, which made it a calculator
rather than a REPL. The renderer is now a compile-time walk over the type,
emitting a piece at a time: structs, nested structs, fixed arrays, slices,
options, enums by name, and pointers as their shape. A raylib Color comes back
through the FFI as (rl/Color {:r 17 :g 34 :b 51 :a 68}).
Piecewise emission is what makes composites possible at all - a struct is its
fields with punctuation between them, and concatenating that in generated IR
would need an allocator the language does not have.
u64 now renders, in C, with %llu. It used to refuse because i64->bytes is
signed and it would otherwise come back as -1, but refusing a whole struct
because one field is a u64 is much worse than adding a runtime entry point.
Strings are quoted and escaped in C for the same reason: unescaped content does
not round-trip and reads as a framing bug rather than as the value it is.
An enum renders as :name, recovered from the checker's table as a chain of
comparisons, since members are erased to i32 before the backend sees them; a
value outside the declared members falls through to its number, which is what
you would want to see. A pointer is rendered and never followed - it is the
only thing that could make the walk cycle, and dereferencing one a REPL was
handed is not a safe thing to do on someone's behalf.
Three bounds, easy to conflate. depth and span bound the walk, so sand's
[100 [100 u32]] grid does not unroll into ten thousand render sites. The output
is bounded once in the runtime, since a slice renders through a loop the
compiler cannot bound, and one place enforcing it means no renderer carries a
budget.
emit.ml's cast now treats an enum as the i32 it is. Nothing in the surface
language produces that - a keyword resolves against its enum and never widens -
but the renderer needs an enum's number when it falls outside the members.
A different primitive from redefining a name. There is no name to install a
body into, so the expression is wrapped in a function with nowhere to be called
from; the module exports flan_reload_call to say "run this once", and the agent
calls it after the install - on the game thread, at a frame boundary, so an
expression that reads the program's state sees a point the program agrees is
consistent.
Nothing is marshalled back because nothing could be. A Flan value carries no
header, so no code at run time can say what it is; the compiler knows the type
and renders it there, in the thunk. That is the layout decision's bill, and it
is why the printer set is the scalars rather than everything.
The rendering does not go through stdout. Stdout belongs to the program, it is
in the hot path for anything that prints, and a dev-only feature must not put a
branch in it - so flan_rt.c is untouched and the value goes to flan_dev_result,
read back over the agent's socket. Safe without a handshake because the
generation counter is bumped last: the daemon waits for it to move rather than
assuming the program has reached a frame boundary.
u64 refuses by name, because i64->bytes is signed and anything past 2^63 would
come back negative. Everything without a derived printer refuses the same way.
A number that is quietly wrong is the failure this whole thing exists to
prevent.
An evaluation is not a declaration: the thunk is built against the program and
never spliced into it, so describe does not fill up with an eval/N for every
expression ever typed.
The test that matters is the same expression twice. The fixture increments
ticks every frame, so two evaluations must disagree - a value computed in the
compiler, or read from a copy of the program's state, would not.
Asked whether a defconst could be redefined, probed it, and got ":status ok"
for a change that did nothing at all - the module was built, delivered,
installed, and the program went on using the old value. That is the
silent-wrongness class the house rule exists to prevent, so it is now four
refusals and a fix.
A defconst's value is folded into its call sites - into an array length at
worst, which is decided before any type resolves - so it lives in the running
program's code and not only in its storage. Refused. A defenum member is the
same thing: :space is erased to an i32 literal in the caller. Refused, and
compared over declarations rather than over Tast.program, which carries no
enums at all for exactly that reason.
A defvar's initial value is deliberately not refused. Its storage holds live
state the program moved past long ago, and refusing to change the initialiser
would be refusing "edit the code, keep the sand". Same Tast.global record as a
defconst, opposite answers, told apart by gconst.
The value comparison is structural and conservative - anything it does not
recognise counts as changed. Comparing emitted text would be wrong, since
Emit.const on a string allocates a name off a per-module counter and two
different strings in two throwaway modules both come out as @".str.0".
Third: a new global's declared initial value was being dropped. flan_dev_global
callocs, so (defvar n i64 42) added at run time was silently zero. It now takes
the initial value as a blob, copies it on the allocation and ignores it
afterwards - the second half being where "a reload must not reset the program's
state" lives. In the allocation path rather than a branch at the call site, so
it cannot be got wrong at one of them.
Fourth: a change with no body to publish and no storage to allocate now answers
"nothing to install" instead of shipping an empty module. That is what the
defconst probe actually did, and it cost the program a frame's worth of reload
it did not need.
Editing a defvar or a defn is a symbol the host exports. Adding one is not:
there is nothing to bind to and ELF cannot grow a symbol. runtime/flan_dev.c is
the two lookups that cover it - flan_dev_cell for a new function's cell,
flan_dev_global for a new global's storage - both idempotent, so the second
module to mention a name gets what the first one got. That is the whole point:
two modules with their own copy of a new function would each call their own,
and redefining it would update one of them.
The compiler picks per name. A name the host has is a symbol and costs one load
at a call site; a name it lacks is a registry lookup cached at install time in
a module-local slot, and costs two. The common case pays nothing for the
general one.
The redefinition unit is now a list of top-level forms rather than one
function. It has to be: v3 of the fixture adds a var and uses it from a
redefined bump, and splitting that into two loads leaves a module referring to
storage that does not exist yet. C-c C-c passes one name, C-c C-k passes a
file's worth, one path either way.
Four rules, each silent if broken. Every lookup resolves before any body is
published, or a caller reaches a function whose slots are still null - asserted
on the emitted flan_reload_install, since it cannot be race-tested.
flan_dev_global refuses a size change, which is the layout-drift rule's first
enforcement point rather than another exception to it. Nothing is ever
dlclosed, because a cell holds an address inside a module's text. And the table
is fixed capacity, because a module holds a cell's address for as long as it is
loaded and a realloc would strand it.
The test that separates this from a plausible wrong version is v4, which
redefines a name v3 introduced at run time. v3's bump is already installed and
is not rebuilt, so it picks v4 up only if its call goes through a cell both
modules found by the same name. Had v3 cached the function's address instead,
every other assertion would still pass and the transcript would read 246
instead of 432.
Sizes are spelled LLVM's way, ptrtoint getelementptr null 1, rather than by a
layout calculator in OCaml that would have to agree with LLVM's on every
target.
