What the registry answers, and the two places a release build is not free

BUILT.md on the split that is the design — the type is emitted because only the
checker knows it, the death is not because an address needs no type — and on the
part that reads backwards: (Ptr Enemy) already said Enemy, so the registry
supplies permission rather than identification.

The honesty is in the same section rather than a footnote. A release build pays
a load and a not-taken branch per free, because flan_dev.c is in every build and
a second allocator selected by a build flag is worse than a branch. The table is
calloc'd when armed rather than declared, so nothing else is carried. The arena
free-all is answerable now and still invisible to memcheck, and those are two
different claims.

NEXT.md keeps the entry open for what was not built: an op that points at an
arbitrary address, the breakdown by type, leak attribution, the memcheck half,
and the test_dev.ml case that would drive dev-ptr.flan.
This commit is contained in:
Joseph Ferano 2026-09-13 09:25:51 +07:00
parent 6ebcea6d3a
commit c95f11ff18
2 changed files with 154 additions and 26 deletions

124
BUILT.md
View File

@ -4445,3 +4445,127 @@ everything else there is arithmetic over a container header.
failure — the region a container lived in was released, and there is no frame to go back to that would not read freed
memory — and the map path was left alone rather than converted half-way. Written down here so it is a known edge
rather than a discovery.
## An address answers with a type, and nothing grew a tag word
A Flan struct is exactly its C layout. No header, no tag word — deliberately, and it is what makes a struct free and
what makes the FFI work. The consequence is stated elsewhere in this file more than once: *a Flan value carries no
header, so nothing at run time can say what it is*, which is why a rendering is a compile-time walk over a type and
why `(watch "v" v)` cannot reach a composite without an arm in `check.ml`.
The allocation registry does not answer that question. It sidesteps it. **The allocator's caller knows the type at the
moment it asks for memory**, and the compiler is standing right there, so a dev build writes it down: base address,
extent, element size, and the type's printed spelling. Nothing about any value's layout changes, and raylib never
finds out.
### The split, which is the whole design
Two halves, and they are in different files because they need different things.
- **Recording the type is emitted.** `check.ml` builds one `flan_dev_reg_note_*` call after every operation that may
have allocated — `vec-new`, `push`, `reserve`, `clone`, `pool-new`, `insert`, `map-new`, `put`, `slurp`. Here is the
only place the concrete element type exists, so here is the only place that can name it. It is built in **every**
build, because a tree that differed by build flag would make every pass between the checker and the backend ask
which one it was looking at; `Emit`'s `Rt` arm drops the family when `dev` is off.
- **Recording that a block died is not emitted.** An address needs no type, so `flan_rt.c` calls into the table
directly from `flan_heap_proc`'s free, from a heap resize for the block it moved away from, from the arena's
`free-all`, and from `arena-destroy`. No signature in the allocator grew a type name and no ABI moved.
The drop in `Emit` happens **before** the arguments are walked, not after. A note takes the address of the container
it describes; emitting that address and then discarding the call would leave an escaped `alloca` behind, and an
escaped `alloca` is one mem2reg will not promote. So a release build's IR is the same IR it always was.
### It is armed by a constructor, and that is not fastidiousness
`@llvm.global_ctors` in the dev module, not a line at the top of `main`. A `defvar` initialiser can allocate, and it
runs before `main` does; a note that arrived before the flag was set would be a block the table never heard of, which
is a live pointer the inspector would call dead. That is the one failure mode worse than no registry at all.
### Lookup is containment, and that is not an optimisation
An entry is a **block**, not a value. Every pointer a program can hold into heap storage is interior: `(at v i)` is
`v->ptr + i*size`, and `(resolve p h)` is an item in the middle of a pool's items array. Neither is ever a base
address. A table that answered only exact hits would answer nothing anyone can ask it.
The two sides of the table therefore look different, and the difference is which thread is asking:
- `flan_dev_reg_note` and `flan_dev_reg_dead` are **probes** — a hash slot and a linear walk from it. Both are on the
writer's side, and a free hands back the same base address the allocator gave out, so equality is the whole
question there.
- `flan_dev_reg_live` and the epitaph are a **scan** of all 4096 slots. The reader is a person pressing a key, so the
cost belongs there.
- `flan_dev_reg_dead_range` is a scan too, and it is the one that runs per frame: an arena's `free-all` has no list of
what it handed out, so the region is matched against the table rather than the other way round. That is a real
per-frame cost in a dev build and it is named here rather than discovered later.
Dead entries are kept. That is the second thing the registry buys — an address that was freed still names what died —
and an entry is dropped only when the allocator hands the same address out again, which is exactly when the old answer
stopped being true. When the table fills it is compacted, dropping the dead and re-inserting the live; in a
long-running program the dead are the bulk of it.
### What it is for: permission, not identification
This is the part worth stating plainly, because the obvious reading is wrong.
`(Ptr Enemy)` **already says Enemy**, at compile time, in `Render`'s walk. The type at the far end was never the
difficulty. What was missing is *permission*: whether it is still true to read the storage there. `render.ml`'s old
comment said a pointer is never followed because "dereferencing one a REPL was handed is not a safe thing to do on
someone's behalf", and that sentence is still correct — the registry just makes the safety checkable. So the arm
became a branch:
```
("live" "(Ptr Enemy)" "<ptr (Enemy {.hp 41 .x 2})>")
("dead" "(Ptr Enemy)" "<ptr dead: was Enemy, freed at step 15>")
```
The recorded type name is therefore not what selects the renderer. It is what the *epitaph* says, and a cross-check
available to anything that wants one.
An address the registry never saw renders `<ptr>`, unchanged. That is a stack local, a global, or a pointer from C,
and the shadow stack and the static type table already answer for the first two by name.
**`println` does not follow a pointer, and will not.** `spec-memory.md` fixes what a printed `Ptr` prints, a printed
line belongs to the program and has to read the same in a release build, and a release build has no registry to ask.
The two callers of `Render` already differ in an emitter record; they now differ in a `pointers` record too, and
`check.ml` passes `None`. This is also why the epitaph carries **no address**: an address is not stable across two
runs, so printing one would make a rendering — and any test that reads one — depend on where the heap landed. It is
the rule `Render` already follows for an allocator.
### The arena hole: answerable, not reported
`test_valgrind.ml` measures a hole and this does not close it. `free-all` is retain-capacity: the offset goes to zero,
the pages stay mapped, and from `malloc`'s point of view nothing died — so memcheck is never told, and a later read of
stale bytes is a read of memory that is, as far as it knows, perfectly alive.
The registry **is** told. What that buys is that the same read is *answerable*: a pointer into a released region comes
back dead and names what used to be there. Memcheck still says nothing, and closing that half is
`VALGRIND_MAKE_MEM_UNDEFINED` in `flan_arena_proc`, which `test_valgrind.ml` names as the fix and which is still not
written. **The two must not be blurred into one claim.**
### What a release build actually carries, said honestly
"Release builds carry none of it" is the goal and it is not quite true, in exactly two places:
- **A load and a not-taken branch per free, and per `free-all`.** `flan_dev.c` is compiled into every build (see
`Build`, which says why), so `flan_rt.c` calls the dead-marking hooks unconditionally and each begins by testing a
flag only a dev build sets. The alternative is a second version of the allocator selected by a build flag, which is
worse than a branch for the reason the `Vec` header already carries its two dev words in every build: a layout or a
code path that changes with a flag is one that can disagree across the redefinition boundary silently.
- **Nothing else.** The table is `calloc`'d when it is armed, not declared as an array — a fixed 4096-entry table
would have been a quarter of a megabyte of BSS in a shipped game for something that build never writes. A release
binary carries a null pointer, a zero flag, and the declarations, which cost nothing.
The `@llvm.global_ctors` entry, the notes, and everything that reads them are dev-only, which is the same arrangement
the indirection cells and the shadow stack have.
### Tested twice, and one thing tested by hand
`programs/registry.flan` is **one program read twice** in the acceptance table. A dev build answers for an address at
the heap tier, the arena tier and the pool tier; a release build answers 0 to every question. The difference between
the two expectations *is* the assertion, and writing it as one program means nobody can change what a dev build does
without the release row noticing.
The inspector's pointer arm is **not** covered. `test/programs/dev-ptr.flan` is the program, its header carries the
two lines above, and they were read off a running session by hand — the `test_dev.ml` case that would drive it belongs
to another lane's file. `NEXT.md` says so rather than letting the verification read as automated.

56
NEXT.md
View File

@ -20,42 +20,46 @@ This matters more here than in most Lisps because the intended use is a **game l
skip a frame and carry on rather than die — exactly the case where a non-idempotent mutation bites. Write it into
`conditions.org` and `spec-conditions.md`'s prose, and into `web/index.html` beside the restart documentation.
## Queued: a dev-build allocation registry — address to type
## ~~Queued: a dev-build allocation registry~~ — **landed, in part; three of the six remain**
Raised in conversation and wanted. **What it is:** in a dev build only, every allocation records what type it was
made for. Address → type becomes a lookup.
Built. The table is in `runtime/flan_dev.c`, the note is emitted by `check.ml` and dropped by `emit.ml` in a release
build, and the inspector reads it. `BUILT.md`'s *"An address answers with a type"* is the account of it; what follows
is only what is **not** there, so that the gap is a queue entry rather than a discovery.
**Why it is possible without tagging anything.** A Flan struct is exactly its C layout with no header and no tag word
— deliberately, and it is what makes structs free and the FFI work. So "what is at this address" is normally
unanswerable at run time, and a tag cannot be added without breaking raylib. The registry sidesteps that: the
*allocator* knows the type at the moment it hands out memory, so nothing about the value's layout has to change.
Costs nothing in a release build.
**Landed:** items 1 and 2 — the inspector follows a live `(Ptr T)` and renders the pointee, and names what died at a
dead one (`<ptr dead: was Enemy, freed at step 15>`). The dead-marking covers the heap `free`, a heap resize's old
block, an arena `free-all` and `arena-destroy`.
**What it buys, in rough order of value:**
**Left, and in this order:**
- **Following a pointer.** `(Ptr T)` renders as `<ptr>` today and stops there. With a registry the inspector follows
it and renders what is actually at the other end.
- **Use-after-free that names what died.** The address is in the registry and marked dead: *"this was an Enemy, freed
at frame 412"* instead of garbage, a crash, or silence.
- Point at any heap address and get a typed rendering rather than bytes.
- **A memory breakdown by type** — how many bytes are Enemies, how many are strings. A real profiling tool for a game.
- **Leak attribution at exit**, which generalises the debug tracking allocator already queued for counting forgotten
textures.
- It closes the **arena hole Valgrind found**: releasing a region could mark everything in it dead, where today the
bytes stay quietly readable because `free-all` is retain-capacity and memcheck is never told.
- **Item 3, point at any heap address.** The lookup is there and answers for any address; nothing exposes it as an
editor op. It wants a verb beside `inspect` that takes an address and a type rather than a frame and a slot, and the
registry's own answer for the type when none is given — which needs the recorded name resolved back to a
`Types.t`, and the table records a string.
- **Item 4, a breakdown by type**, and **item 5, leak attribution at exit.** Both are a walk over the table and a
group-by; `flan_dev_reg_count` is the whole of what exists. Neither is hard and neither has a reader yet, which is
why they were left rather than half-built.
- **The memcheck half of item 6.** The registry now knows an arena's `free-all` killed everything in the region, so a
later read through a pointer into it is *answerable*. Memcheck still says nothing, because nothing told it: the
pages stay mapped and `free-all` is an integer going to zero inside one allocation. Closing that is
`VALGRIND_MAKE_MEM_UNDEFINED` in `flan_arena_proc`, which `test/test_valgrind.ml` already names. **The two must not
be blurred** — the registry answer and the memcheck answer are different tools reaching different people.
- **A test that drives the inspector's pointer arm.** `test/programs/dev-ptr.flan` is the program and its header has
the two lines a session answers with; they were read off a running session **by hand**. The case belongs beside the
other `locals`/`inspect` cases in `test_dev.ml`, which was another lane's file. `programs/registry.flan` covers the
table itself from the acceptance table, in a dev build and a release one.
**What it does not cover, and does not need to:** stack locals and globals, which the shadow stack and the static type
table already answer by name.
**On cost — settled by the author: keep it simple.** One registry insert per allocation, always on in a dev build,
no opt-out. An arena allocation is a bump pointer and an insert may well cost more than the allocation itself in a
per-frame loop, but a dev build already carries indirection cells and a shadow stack, and the author's instruction is
to build the straightforward thing and revisit only if a real program shows a problem. Do **not** build per-region
recording, range recording, or a per-allocator opt-out on speculation.
table already answer by name. A stack address is deliberately not in the table, and a pointer to one still renders
`<ptr>`.
**Note on classes:** `defclass` instances will carry shape metadata by design, so they get identification for free and
do not need the registry. This is for plain structs, `Vec`, `Map` and pool storage.
**On cost, as built.** One insert per allocation, always on in a dev build, no opt-out — the author's instruction,
followed literally. Nothing was built per-region, no range recording, no per-allocator opt-out. Revisit only if a real
program shows a problem, and `BUILT.md` names the two places a release build is not quite free.
## Picked up first, 2026-09-13
Three things, in order. The first two are one line each and unblock a real game.