What the registry answers, and the two places a release build is not free
BUILT.md on the split that is the design — the type is emitted because only the checker knows it, the death is not because an address needs no type — and on the part that reads backwards: (Ptr Enemy) already said Enemy, so the registry supplies permission rather than identification. The honesty is in the same section rather than a footnote. A release build pays a load and a not-taken branch per free, because flan_dev.c is in every build and a second allocator selected by a build flag is worse than a branch. The table is calloc'd when armed rather than declared, so nothing else is carried. The arena free-all is answerable now and still invisible to memcheck, and those are two different claims. NEXT.md keeps the entry open for what was not built: an op that points at an arbitrary address, the breakdown by type, leak attribution, the memcheck half, and the test_dev.ml case that would drive dev-ptr.flan.
This commit is contained in:
parent
6ebcea6d3a
commit
c95f11ff18
124
BUILT.md
124
BUILT.md
@ -4445,3 +4445,127 @@ everything else there is arithmetic over a container header.
|
||||
failure — the region a container lived in was released, and there is no frame to go back to that would not read freed
|
||||
memory — and the map path was left alone rather than converted half-way. Written down here so it is a known edge
|
||||
rather than a discovery.
|
||||
|
||||
|
||||
## An address answers with a type, and nothing grew a tag word
|
||||
|
||||
A Flan struct is exactly its C layout. No header, no tag word — deliberately, and it is what makes a struct free and
|
||||
what makes the FFI work. The consequence is stated elsewhere in this file more than once: *a Flan value carries no
|
||||
header, so nothing at run time can say what it is*, which is why a rendering is a compile-time walk over a type and
|
||||
why `(watch "v" v)` cannot reach a composite without an arm in `check.ml`.
|
||||
|
||||
The allocation registry does not answer that question. It sidesteps it. **The allocator's caller knows the type at the
|
||||
moment it asks for memory**, and the compiler is standing right there, so a dev build writes it down: base address,
|
||||
extent, element size, and the type's printed spelling. Nothing about any value's layout changes, and raylib never
|
||||
finds out.
|
||||
|
||||
### The split, which is the whole design
|
||||
|
||||
Two halves, and they are in different files because they need different things.
|
||||
|
||||
- **Recording the type is emitted.** `check.ml` builds one `flan_dev_reg_note_*` call after every operation that may
|
||||
have allocated — `vec-new`, `push`, `reserve`, `clone`, `pool-new`, `insert`, `map-new`, `put`, `slurp`. Here is the
|
||||
only place the concrete element type exists, so here is the only place that can name it. It is built in **every**
|
||||
build, because a tree that differed by build flag would make every pass between the checker and the backend ask
|
||||
which one it was looking at; `Emit`'s `Rt` arm drops the family when `dev` is off.
|
||||
- **Recording that a block died is not emitted.** An address needs no type, so `flan_rt.c` calls into the table
|
||||
directly from `flan_heap_proc`'s free, from a heap resize for the block it moved away from, from the arena's
|
||||
`free-all`, and from `arena-destroy`. No signature in the allocator grew a type name and no ABI moved.
|
||||
|
||||
The drop in `Emit` happens **before** the arguments are walked, not after. A note takes the address of the container
|
||||
it describes; emitting that address and then discarding the call would leave an escaped `alloca` behind, and an
|
||||
escaped `alloca` is one mem2reg will not promote. So a release build's IR is the same IR it always was.
|
||||
|
||||
### It is armed by a constructor, and that is not fastidiousness
|
||||
|
||||
`@llvm.global_ctors` in the dev module, not a line at the top of `main`. A `defvar` initialiser can allocate, and it
|
||||
runs before `main` does; a note that arrived before the flag was set would be a block the table never heard of, which
|
||||
is a live pointer the inspector would call dead. That is the one failure mode worse than no registry at all.
|
||||
|
||||
### Lookup is containment, and that is not an optimisation
|
||||
|
||||
An entry is a **block**, not a value. Every pointer a program can hold into heap storage is interior: `(at v i)` is
|
||||
`v->ptr + i*size`, and `(resolve p h)` is an item in the middle of a pool's items array. Neither is ever a base
|
||||
address. A table that answered only exact hits would answer nothing anyone can ask it.
|
||||
|
||||
The two sides of the table therefore look different, and the difference is which thread is asking:
|
||||
|
||||
- `flan_dev_reg_note` and `flan_dev_reg_dead` are **probes** — a hash slot and a linear walk from it. Both are on the
|
||||
writer's side, and a free hands back the same base address the allocator gave out, so equality is the whole
|
||||
question there.
|
||||
- `flan_dev_reg_live` and the epitaph are a **scan** of all 4096 slots. The reader is a person pressing a key, so the
|
||||
cost belongs there.
|
||||
- `flan_dev_reg_dead_range` is a scan too, and it is the one that runs per frame: an arena's `free-all` has no list of
|
||||
what it handed out, so the region is matched against the table rather than the other way round. That is a real
|
||||
per-frame cost in a dev build and it is named here rather than discovered later.
|
||||
|
||||
Dead entries are kept. That is the second thing the registry buys — an address that was freed still names what died —
|
||||
and an entry is dropped only when the allocator hands the same address out again, which is exactly when the old answer
|
||||
stopped being true. When the table fills it is compacted, dropping the dead and re-inserting the live; in a
|
||||
long-running program the dead are the bulk of it.
|
||||
|
||||
### What it is for: permission, not identification
|
||||
|
||||
This is the part worth stating plainly, because the obvious reading is wrong.
|
||||
|
||||
`(Ptr Enemy)` **already says Enemy**, at compile time, in `Render`'s walk. The type at the far end was never the
|
||||
difficulty. What was missing is *permission*: whether it is still true to read the storage there. `render.ml`'s old
|
||||
comment said a pointer is never followed because "dereferencing one a REPL was handed is not a safe thing to do on
|
||||
someone's behalf", and that sentence is still correct — the registry just makes the safety checkable. So the arm
|
||||
became a branch:
|
||||
|
||||
```
|
||||
("live" "(Ptr Enemy)" "<ptr (Enemy {.hp 41 .x 2})>")
|
||||
("dead" "(Ptr Enemy)" "<ptr dead: was Enemy, freed at step 15>")
|
||||
```
|
||||
|
||||
The recorded type name is therefore not what selects the renderer. It is what the *epitaph* says, and a cross-check
|
||||
available to anything that wants one.
|
||||
|
||||
An address the registry never saw renders `<ptr>`, unchanged. That is a stack local, a global, or a pointer from C,
|
||||
and the shadow stack and the static type table already answer for the first two by name.
|
||||
|
||||
**`println` does not follow a pointer, and will not.** `spec-memory.md` fixes what a printed `Ptr` prints, a printed
|
||||
line belongs to the program and has to read the same in a release build, and a release build has no registry to ask.
|
||||
The two callers of `Render` already differ in an emitter record; they now differ in a `pointers` record too, and
|
||||
`check.ml` passes `None`. This is also why the epitaph carries **no address**: an address is not stable across two
|
||||
runs, so printing one would make a rendering — and any test that reads one — depend on where the heap landed. It is
|
||||
the rule `Render` already follows for an allocator.
|
||||
|
||||
### The arena hole: answerable, not reported
|
||||
|
||||
`test_valgrind.ml` measures a hole and this does not close it. `free-all` is retain-capacity: the offset goes to zero,
|
||||
the pages stay mapped, and from `malloc`'s point of view nothing died — so memcheck is never told, and a later read of
|
||||
stale bytes is a read of memory that is, as far as it knows, perfectly alive.
|
||||
|
||||
The registry **is** told. What that buys is that the same read is *answerable*: a pointer into a released region comes
|
||||
back dead and names what used to be there. Memcheck still says nothing, and closing that half is
|
||||
`VALGRIND_MAKE_MEM_UNDEFINED` in `flan_arena_proc`, which `test_valgrind.ml` names as the fix and which is still not
|
||||
written. **The two must not be blurred into one claim.**
|
||||
|
||||
### What a release build actually carries, said honestly
|
||||
|
||||
"Release builds carry none of it" is the goal and it is not quite true, in exactly two places:
|
||||
|
||||
- **A load and a not-taken branch per free, and per `free-all`.** `flan_dev.c` is compiled into every build (see
|
||||
`Build`, which says why), so `flan_rt.c` calls the dead-marking hooks unconditionally and each begins by testing a
|
||||
flag only a dev build sets. The alternative is a second version of the allocator selected by a build flag, which is
|
||||
worse than a branch for the reason the `Vec` header already carries its two dev words in every build: a layout or a
|
||||
code path that changes with a flag is one that can disagree across the redefinition boundary silently.
|
||||
- **Nothing else.** The table is `calloc`'d when it is armed, not declared as an array — a fixed 4096-entry table
|
||||
would have been a quarter of a megabyte of BSS in a shipped game for something that build never writes. A release
|
||||
binary carries a null pointer, a zero flag, and the declarations, which cost nothing.
|
||||
|
||||
The `@llvm.global_ctors` entry, the notes, and everything that reads them are dev-only, which is the same arrangement
|
||||
the indirection cells and the shadow stack have.
|
||||
|
||||
### Tested twice, and one thing tested by hand
|
||||
|
||||
`programs/registry.flan` is **one program read twice** in the acceptance table. A dev build answers for an address at
|
||||
the heap tier, the arena tier and the pool tier; a release build answers 0 to every question. The difference between
|
||||
the two expectations *is* the assertion, and writing it as one program means nobody can change what a dev build does
|
||||
without the release row noticing.
|
||||
|
||||
The inspector's pointer arm is **not** covered. `test/programs/dev-ptr.flan` is the program, its header carries the
|
||||
two lines above, and they were read off a running session by hand — the `test_dev.ml` case that would drive it belongs
|
||||
to another lane's file. `NEXT.md` says so rather than letting the verification read as automated.
|
||||
|
||||
56
NEXT.md
56
NEXT.md
@ -20,42 +20,46 @@ This matters more here than in most Lisps because the intended use is a **game l
|
||||
skip a frame and carry on rather than die — exactly the case where a non-idempotent mutation bites. Write it into
|
||||
`conditions.org` and `spec-conditions.md`'s prose, and into `web/index.html` beside the restart documentation.
|
||||
|
||||
## Queued: a dev-build allocation registry — address to type
|
||||
## ~~Queued: a dev-build allocation registry~~ — **landed, in part; three of the six remain**
|
||||
|
||||
Raised in conversation and wanted. **What it is:** in a dev build only, every allocation records what type it was
|
||||
made for. Address → type becomes a lookup.
|
||||
Built. The table is in `runtime/flan_dev.c`, the note is emitted by `check.ml` and dropped by `emit.ml` in a release
|
||||
build, and the inspector reads it. `BUILT.md`'s *"An address answers with a type"* is the account of it; what follows
|
||||
is only what is **not** there, so that the gap is a queue entry rather than a discovery.
|
||||
|
||||
**Why it is possible without tagging anything.** A Flan struct is exactly its C layout with no header and no tag word
|
||||
— deliberately, and it is what makes structs free and the FFI work. So "what is at this address" is normally
|
||||
unanswerable at run time, and a tag cannot be added without breaking raylib. The registry sidesteps that: the
|
||||
*allocator* knows the type at the moment it hands out memory, so nothing about the value's layout has to change.
|
||||
Costs nothing in a release build.
|
||||
**Landed:** items 1 and 2 — the inspector follows a live `(Ptr T)` and renders the pointee, and names what died at a
|
||||
dead one (`<ptr dead: was Enemy, freed at step 15>`). The dead-marking covers the heap `free`, a heap resize's old
|
||||
block, an arena `free-all` and `arena-destroy`.
|
||||
|
||||
**What it buys, in rough order of value:**
|
||||
**Left, and in this order:**
|
||||
|
||||
- **Following a pointer.** `(Ptr T)` renders as `<ptr>` today and stops there. With a registry the inspector follows
|
||||
it and renders what is actually at the other end.
|
||||
- **Use-after-free that names what died.** The address is in the registry and marked dead: *"this was an Enemy, freed
|
||||
at frame 412"* instead of garbage, a crash, or silence.
|
||||
- Point at any heap address and get a typed rendering rather than bytes.
|
||||
- **A memory breakdown by type** — how many bytes are Enemies, how many are strings. A real profiling tool for a game.
|
||||
- **Leak attribution at exit**, which generalises the debug tracking allocator already queued for counting forgotten
|
||||
textures.
|
||||
- It closes the **arena hole Valgrind found**: releasing a region could mark everything in it dead, where today the
|
||||
bytes stay quietly readable because `free-all` is retain-capacity and memcheck is never told.
|
||||
- **Item 3, point at any heap address.** The lookup is there and answers for any address; nothing exposes it as an
|
||||
editor op. It wants a verb beside `inspect` that takes an address and a type rather than a frame and a slot, and the
|
||||
registry's own answer for the type when none is given — which needs the recorded name resolved back to a
|
||||
`Types.t`, and the table records a string.
|
||||
- **Item 4, a breakdown by type**, and **item 5, leak attribution at exit.** Both are a walk over the table and a
|
||||
group-by; `flan_dev_reg_count` is the whole of what exists. Neither is hard and neither has a reader yet, which is
|
||||
why they were left rather than half-built.
|
||||
- **The memcheck half of item 6.** The registry now knows an arena's `free-all` killed everything in the region, so a
|
||||
later read through a pointer into it is *answerable*. Memcheck still says nothing, because nothing told it: the
|
||||
pages stay mapped and `free-all` is an integer going to zero inside one allocation. Closing that is
|
||||
`VALGRIND_MAKE_MEM_UNDEFINED` in `flan_arena_proc`, which `test/test_valgrind.ml` already names. **The two must not
|
||||
be blurred** — the registry answer and the memcheck answer are different tools reaching different people.
|
||||
- **A test that drives the inspector's pointer arm.** `test/programs/dev-ptr.flan` is the program and its header has
|
||||
the two lines a session answers with; they were read off a running session **by hand**. The case belongs beside the
|
||||
other `locals`/`inspect` cases in `test_dev.ml`, which was another lane's file. `programs/registry.flan` covers the
|
||||
table itself from the acceptance table, in a dev build and a release one.
|
||||
|
||||
**What it does not cover, and does not need to:** stack locals and globals, which the shadow stack and the static type
|
||||
table already answer by name.
|
||||
|
||||
**On cost — settled by the author: keep it simple.** One registry insert per allocation, always on in a dev build,
|
||||
no opt-out. An arena allocation is a bump pointer and an insert may well cost more than the allocation itself in a
|
||||
per-frame loop, but a dev build already carries indirection cells and a shadow stack, and the author's instruction is
|
||||
to build the straightforward thing and revisit only if a real program shows a problem. Do **not** build per-region
|
||||
recording, range recording, or a per-allocator opt-out on speculation.
|
||||
table already answer by name. A stack address is deliberately not in the table, and a pointer to one still renders
|
||||
`<ptr>`.
|
||||
|
||||
**Note on classes:** `defclass` instances will carry shape metadata by design, so they get identification for free and
|
||||
do not need the registry. This is for plain structs, `Vec`, `Map` and pool storage.
|
||||
|
||||
**On cost, as built.** One insert per allocation, always on in a dev build, no opt-out — the author's instruction,
|
||||
followed literally. Nothing was built per-region, no range recording, no per-allocator opt-out. Revisit only if a real
|
||||
program shows a problem, and `BUILT.md` names the two places a release build is not quite free.
|
||||
|
||||
## Picked up first, 2026-09-13
|
||||
|
||||
Three things, in order. The first two are one line each and unblock a real game.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user