x86 built a settling right-hand side straight into its destination, so (set p (P {.a (.b p) .b (.a p)})) wrote .a and read it back for .b. assign now also asks whether the value reads storage the destination lies in, and goes through the temporary when it does. The global-initialiser entry was already closed by startup_plan's Set; the unrooted-temporary entry turns out to be a both-backends rooting gap and stays open for a decision.
FIX.org, NEXT.md, DISCUSS.org, docs/DISCUSS.md and the session handoff at the
root are one TODO.org now: 293 entries under seven subsystem headings, each
carrying an org keyword that says where it stands. A DONE entry is a few lines
saying what was decided and what that rules out; the reasoning that would not
compress — the embedding spike and the four reports the hand-written x86
backend was built from — moved into docs/BUILT.md instead, and its entries
point there in one line.
Every entry was checked against the tree before it got a keyword, and the
prose was wrong in both directions. Things the deleted files called open were
built: the first-evaluation stall, main being redefinable, macro parameter
lists, the type-limit constants, the array constructors, the byte fills,
inc/dec, the discard's fontification, the Emacs buffers, rt_die's _exit, the
backtrace surface, and the acceptance failure that could print and still exit
zero. Things they called done were not: the backend reports' no-plan buckets
had gone stale in the other direction, the value-dependent defvar was
superseded rather than built, and macro-expansion source locations are on an
unmerged lane, so that entry is NEXT and names the branch.
Every comment that cited one of the five by name now cites a heading that
exists, in TODO.org or in docs/BUILT.md. The session reports under
docs/handoffs/ keep naming the files they worked on, because rewriting them
would falsify what those sessions did; each carries a note saying where the
content went.
Review found it, and it is a silent miscompile on both backends rather
than a refusal anywhere. mangle_ty flattens a whole signature into one
hyphen-joined string, so (CFn [(Ptr i32)] i32) and (CFn [ptr i32] i32) --
the second over a struct someone called ptr -- flatten alike; keyed on
that, the second widening reused the first's thunk at the wrong arity.
The key is the types now, compared with Types.equal, and the symbol is a
counter over the thunks already minted, so nothing is derived from a
spelling. fn-thunk-share.flan is the pair, and it prints 5 and 17.
And a regression beside it: a generic whose function parameter binds the
type variable. (defn apply2 [f (Fn [$t] $t) x $t] ...) called as
(apply2 bump 1) compiled before this lane and stopped, because bind_ty
had no arm admitting a CFn argument at an Fn pattern -- and once it had
one, the call still handed one word to an instance declaring two, because
a parameter that still mentions a variable is checked with no expectation
and expect never sees the pair. Both halves: the arm, and the widening
in generic_call's catch-up pass beside the numeric one. The same gap hid
new functionality -- a CFn argument at a (CFn [$t] $t) parameter had no
arm either -- and mentions had no CFn case, so bound_exactly answered
wrong for a variable living only inside one.
The corpus missed all of it because the prelude binds $t from an earlier
argument, so the parameter is concrete before bind_ty sees it.
While here: escaping's enumeration is the *clean* set now rather than the
suspect set. It had a hole where a list like that cannot -- an (at s 0)
over a slice of Fn read as clean while the Vec, struct and pointer
spellings were refused. Unreachable today, and the header claims the
list is closed. The same inversion fixes which refusal message an index
read gets.
Three minors: Anull was defined and never constructed; the capture-dyn
message substituted a descriptor into a noun slot; and session.ml
rendered a defn's changed signature as (Fn [...] ...), which is now a
real type and not the same as (CFn [...] ...) -- it writes the parameters
and the return the way a defn writes them.
And one rounding corrected in the docs: handler-bind is not free for a
program that captures nothing. %handler grew from 24 bytes to 32, every
push writes a null into the new field, every clause gains ptr %env with
an alloca and a store, and flan_signal passes one more argument per
dispatch -- twenty changed x86 lines on loops.flan. Small, real, and
paid by every conditions program.
spec-memory.md's case 2, capture by value into a stack environment, and
the calling convention the author's rulings asked for.
(Fn [i32] i32) captures; {code, env}; the common case
(CFn [i32] i32) the bare address; one word; cannot capture
A local of the enclosing function that an fn names is copied into a
struct the checker synthesises, held in a slot of that function's frame,
and the value carries its address; the lifted body reads the copies back
into named slots of its own, once, at entry. So the name in the body
means what the local held at the instant the value was made --
fn-capture.flan changes the local through a pointer after the value
exists and the fn still answers with the old one.
Two types rather than a uniform environment parameter: "while it's dyn
first, static side should never have to pay the price for the existence
of the dyn side... if you fully opt out, for instance, using --no-gc
flag, then we should be operating under Odin/C semantics and never paying
any runtime costs." The environment is declared by exactly the bodies an
(Fn ...) value can reach -- a lifted literal in an Fn position, every
handler clause, and the widening thunks -- and by nothing else. An
ordinary defn emits the signature it always did; calc-me and fourteen
corpus programs were diffed to say so.
CFn, because the C carries information: a value with no environment is
the only kind that could ever cross to C, and under the --no-conditions
direction FIX.org records it becomes literally a C function pointer. It
is not that today -- a declare cannot take a function type at all -- and
crossable's refusal says so where a reader would otherwise be misled.
Nobody needs CFn: Fn accepts everything, and the commonest reason to
reach for the narrow one is that a *named* function handed to an Fn pays
a hop through the widening thunk where a CFn is a direct call.
That thunk is one small function per distinct signature widened, which
reads the bare address back out of the environment and calls it. The
cheaper trick -- the environment last, ignored by a body that never
declared it -- is legal under SysV and is a trap under wasm32's
call_indirect, which compares the signature at the call. Every indirect
call is exactly typed now.
A handler clause captures the same way and is sound with nothing left
over: its frame is popped by the body that pushed it. What is refused
there is a *store* into a captured name -- it is a copy, and writing to
it would leave the local as it was.
And the other half, which is what "non-escaping" means: a value carrying
an environment may be called, passed down and let-bound, and may not be
returned, stored, pointed at or pushed into a container. A parameter of
type Fn is treated as one, which answers "passed to something that stores
it" with no interprocedural analysis -- the store is refused inside the
callee. Everything of type CFn is clean for free, which is the second
thing having two types buys. Every refusal names case 3, the environment
the collector owns.
Two pre-existing bugs fell out on the way. A lifted fn asked for Fnval,
so `flan reload' on any function containing an fn literal died at llc
with an undefined cell; it takes Flanfn now, which is the choice a
handler clause always made. And a redefinition module now carries its
own hidden copy of every thunk it names, which is the same bug shape
caught before it shipped.
Two bugs, both in lib/x86.ml, both reached by signalling a condition from
inside the value being assigned.
The backend has no register wide enough to hold a struct, so it built every
aggregate in its destination, element by element as they were computed. A
transfer out of the middle left the destination in the middle: a global of
four numbers read part old and part new, and so did a local, a struct field,
an array element, a place behind a pointer. A union case was worse than the
rest, since its destination is zeroed before the fields are written.
The second one only looks like the first. An aggregate result is written
through a hidden pointer the caller supplies, and the transfer exit zeroed
that result on its way out, on the correct reasoning that the caller never
reads it. At (set g (f)) the pointer is g, so it zeroed the caller's
variable. It zeroes scalars only now — and with the first half in place
nothing reaches it, so that one is defence in depth and FIX.org says so
rather than claiming a test it does not have.
Assignment builds into a frame temporary and copies, which is the shape
emit.ml always had. The copy is paid where it buys something: [settles] says
whether lowering an expression is bound to reach the end of it, and a
right-hand side that settles is still built in place. That includes a
conversion, except the one direction that is checked — a float narrowed to an
integer — because an array of this language's literals is written
[(u32 1) (u32 2)] and refusing every cast would have taxed the commonest
aggregate there is. A let pays only inside a loop, because a slot reached
once per frame is recorded as bound after the value lands.
half-write.flan is every destination crossed with every right-hand side, run
on both backends, with a scalar row so a regression in one is not read as the
other. dev-halfwrite.flan asks the break loop the half a running program
cannot ask itself: the local, which is invisible to the program and plain to
an editor reading the frame.
The def-edit paragraph in docs/BUILT.md described this as open and is now the
claim it was waiting for. FIX.org records the two lanes the review of this one
turned up and this one does not take: an aggregate built in place can read its
own destination, which is an aliasing question rather than a transfer one; and
the temporary is a frame buffer no root table names, which matters the day a
struct may hold a dyn field.
The author: "I think I prefer length over len, because then I'll use len as
the variable name". One arm in check.ml, one row in the table beside it, and
every (len x) in lib, test, examples, vendor, spike, docs, web, emacs,
plan.org and NEXT.md rewritten.
Shadowing and builtin/ had already taken most of the sting out: a (defn len
...) was legal and won in its own file, and builtin/len reached past it. What
was left is that len was still a builtin — the defn earned a warning, and a
wrapper had to say builtin/ at every inner call. Now there is nothing under
the short name: len is an ordinary identifier in every position, which is
what (let [len (length xs)] ...) wants.
length takes over as shadowing's worked example rather than the feature
losing one. shadow-builtin.flan, builtin-qualified.flan, pkgs/shadowed and the
builtin/ rows in test_flan move to it and go on testing shadowing.
A call to a len nothing defines is answered where an unknown function is,
after every table and after the shadowing guard, so a program with its own len
never reaches it. The sentence is said rather than guessed at — len and length
are three edits apart and the did-you-mean's net is one — and the call is
written back out through spell_arg, as-slice's spelling lifted out of it and
now shared, so what is printed compiles.
sand.flan:33 still calls the old name and is the author's to change; until it
does, test_acceptance and test_session abort there. Both were run green
against a copy with that one line changed. FIX.org says so.
The author edited (def colors [4 u32] [...]) in his running game, pressed
C-c C-c, and the colours did not change — the same complaint def was built
to answer, one step further in.
The reading behind it was that a re-evaluated def is a promise about the
next re-run, so the session republished the lifted global/<n> and stopped;
nothing called it. That is wrong for the reason the form is named after: def
is Common Lisp's defparameter, and evaluating a defparameter assigns. The
difference from defvar is not "one takes effect at restart", it is "one
takes effect, the other does not touch the value at all".
So a def now does both. The storage takes the new value at the next frame
boundary, carried by the thunk a redefinition module already has — one
Set per re-evaluated def, in the same flan_reload_call the class
registrations use, run after the bodies are published and on the game
thread. And the lifted initialiser is still republished, so the next re-run
runs the edited one; dev-rerun.flan pins that half unchanged.
A brand-new def gets its initialiser run too, which needed one thing from
each backend: a lifted global/<n> asked for by name is neither a sibling nor
one of the target's own lifted clauses, so it had no cell, and a dev call
goes through a cell. Both now give an unknown one a slot of the module's
own, filled from the registry by the installer.
An initialiser that signals leaves the old value alone — the value is
computed whole before it is stored — and offers abandon-evaluation like any
other thunk. A retype is refused first, by the pass that names both types.
defonce is untouched, which is its whole contract; defconst was already the
immediate one, through consts.
Three defects, all from lifting every def initialiser, none of which the
suite caught:
A def typed fresh into a live session came up zero and stayed zero. The
image flan_dev_global copies on the allocation is the only value a new
global ever gets — the host's .init-globals never calls its initialiser —
and both backends chose that image with Tast.const_init, which a def's
lifted Call fails by construction. Emit.initial_image reads the constant
back out of the lifted body; the x86 twin had the same bug.
Changing a global between def and defonce was silently ineffective: the
guard lives in the startup function compiled into the host, which a reload
cannot republish. Session.compatible refuses both directions and says to
restart; editing the value stays allowed.
And global/<n> no longer leaks into the signature refusal when a def is
retyped — the global loop names the same fact in words a reader can act on.
flan check prints def, defonce or defconst off grerun; (defvar) with no
arguments names the shapes rather than offering (defonce ); the docs,
plan.org, runtime comments and valgrind.supp are swept; BUILT.md states
the release-build cost and the uninit caveat.
A fixed array does not decay to a slice at a call, so passing one to a
function over [$t] meant writing (slice a 0 (len a)) at every call site.
(slice a) is the whole of it now and (slice a n) is the tail from n, filled
in by check.ml into the three-argument form: same node, same static bound
checks, same runtime trap, and on a fixed array the implicit length is the
constant (len a) already folds to. Neither backend grew an arity case. A
target that is not already a name goes through a slot first, so (slice (f x))
calls f once.
at and slice also reach a string, because (bytes s) was the only route to a
byte and it is about to start copying. (at s i) is the byte, bounds-checked;
(slice s ...) at all three arities answers a string viewing the same bytes,
not a [u8], which would be a writable-looking view of storage the program
does not own.
Neither is a place, and the refusal lives in [indexed] rather than in
check_place, which is the part that matters. There are three routes to a
Pindex and they share no code: check_place, the single-index set arm that
checks its own target, and addr. Asked in check_place, the question is
answered for two of them and missed for the one a person writes — a store
into a string literal compiled, and the backends disagreed about it. So
[indexed] takes a ~place location and asks at every dimension, because
(at g 0 0) over a [[2 string]] reaches the string only at the last step.
One message, and addr gets it too, so it reads as value-versus-place rather
than as a rule about assignment.
Slicing an array a call returned is refused at every arity. The view
outlives the temporary, both backends print whatever the frame reused, and
nothing traps — which was already true of (slice (mk) 0 3) and only
survivable while nobody wrote it. (slice (mk)) is short enough to become a
habit. An array literal is not this case and stays legal.
Two backend cases. emit.ml's element_addr grew the String arm beside the
Slice one. x86.ml's index_len had answered None for a string — correct while
nothing could index one, and a skipped bounds check the moment something
could — and now reads the length word, so both check the same thing.
The INSERTIONSORT crash, all three rulings (FIX.org 2026-09-20):
- (bytes s) allocates a writable copy through the allocator surface —
context or (bytes s a), StorageExhausted with retry, a registry note in
dev builds (flan_bytes_dup, lowered like vec-new). (bytes-view s) is the
old zero-cost reinterpret, renamed, read-only by convention; every
in-repo reader swept over to it. (string b) unchanged.
- String constants were already read-only on both backends at -O0; now
pinned — bytes-copy.flan rows on LLVM/-O0/--x86, and dies_segv rows
asserting the write-through-view trap on both backends.
- A dev build installs a SIGSEGV/SIGBUS handler by the same dev-only
constructor slot that arms the registry: one line naming the address and
the innermost frame, then the trap-hook park — stopped, not dead, the
daemon serving. No agent: message and re-raise. Release builds untouched.
Pinned by trap_park over dev-segv.flan.
CLHS 4.3.6's update protocol, minus the user hook, on the dyn side's
defclass. Redefining a class used to be silent: a class is sugar for a
constructor defn, so the edit replaced a body and the instances already in
the program kept their old keys for ever.
Three pieces. A registry in flan_dyn.c holding each class's current slot
list and a generation, made only of interned kw_entry pointers so the
collector has nothing to trace in it and no root to push for it. A uint32
generation on the instance, fitted into the padding kind and mark leave in
front of len's alignment — sizeof(flan_obj) is 48 with it and was 48
without, and flan_dyn_obj_size is there so a later field that moves it
fails a test. And a registration thunk per reload, run by the agent
through flan_reload_call after the module's bodies are published: it has
to be a thunk, because the case this exists for is a class redefined and
not constructed.
Migration is lazy, at want_map, len's map arm and dyn_equal's. Slots kept
by name, gained slots nil, dropped slots gone, identity preserved, entries
rebuilt in the class's order so a migrated instance is indistinguishable
from a fresh one. Equality migrates both operands first, so it is over the
class as it is now.
The session had to stop refusing the constructor's signature change, and
does so only for a defclass and only when no compiled caller is left
behind. The checker gets there first in practice; the walk in eval holds
the reason locally rather than inheriting it.
The registry is advisory: a class instance is an open map, so a key a raw
put wrote that the class never declared is dropped by the next migration.
FIX.org says that plainly rather than pretending enforcement.
The author's revision. The name says what it writes, and the pattern is the
program's to choose: (dead-beef) is DEADBEEF, (dead-beef 0xBAADF00D) is
BA AD F0 0D. One byte-order rule covers both — a pattern's ascending bytes
are its big-endian bytes, which is how the hex literal reads left to right —
so every candidate DISCUSS.org listed is now spellable without the compiler
naming any of them.
The bare form is not a case a backend knows about: the checker writes
Tast.dead_beef_default in where the argument would have been, so
(dead-beef) and (dead-beef 0xDEADBEEF) are the same node and an acceptance
row prints both to say so.
The operand is an ordinary u32 expression, which is what the byte arm
already accepts for its byte. A literal is byte-reversed at compile time and
still reaches the loop as an immediate; a computed one is reversed at run
time, by llvm.bswap.i32 on one backend and bswap on the other, after which
the tail shifts its bytes out of the word rather than folding them. The
program runs a computed pattern over lengths 6 and 7 deliberately: that is
the case a constant-only implementation would pass by accident.
filled is untouched, and so is the fill boundary.
DISCUSS.org's sentinel-fill idea, built as two builtins because the author
asked for both: a memset with a byte the program picks, and the fixed
DE AD BE EF pattern a hex dump reads as DEADBEEF.
Both are spelled the way (zeroed) is — the value of whatever type is
expected of them — so (set grid (filled 0xFF)) fills a place and there is
no second, place-taking form beside set.
What may be filled is numbers, and structs and fixed arrays built out of
them. Everything else is refused by name: a filled dyn is a collector root
pointing at nothing, a filled Vec header frees a wild address, a filled
slice length is a bounds check that passes, and a filled bool is an i1 to
LLVM and a whole byte to x86, which is the one divergence this feature
cannot have.
The byte fill is llvm.memset / rep stosb. The four-byte pattern cannot be
a memset on either side — the intrinsic takes one repeated i8 — so it is a
counted dword loop in emit.ml and rep stosd in x86.ml, with the pattern
bytes and their little-endian word living once, in Emit. A size that is
not a multiple of four ends on DE, DE AD, or DE AD BE.
The entry point was the one call site a redefinition could not reach. A dev
build gives every Flan function an indirection cell and routes every call
through it, which is what makes C-c C-c land on the sites that already exist;
the emitted C main called the Flan-level main by symbol instead. So
flan_program_main — what M-x flan-rerun re-enters — ran the body main had at
the initial build for the life of the process, and redefining main compiled,
installed, reported ok and changed nothing anyone could see.
Both backends had the same direct call and both get the same split. LLVM loads
the cell and calls the loaded pointer; x86 does what call_flan's `Cell target
does, one load because emit_cells defines the cell in this same object.
Release builds keep the symbol, so emit and emit --x86 are byte-identical to
what they were. The cell is initialised to the body this build compiled, so
the first run is the run it always was.
What is left is an off-by-one that belongs to lib/dev.ml: a body delivered
while parked installs at the next frame boundary, which a re-run reaches
inside main, so a generation runs one re-run later than the key press. FIX.org
has the one-call close and why it is not done here.
The claim classes were built for, made against a real daemon rather than
at the session's report: dev-class.flan is compiled with one method for
one class, a second method is delivered into the live process, and the
call through the generic's cell answers with the new method's body while
the original one goes on answering. That is what a generic being exactly
one top-level name buys, and it is now pinned rather than argued.
Two things came out of writing it, neither of them about classes.
The x86 backend's redefinition module never emitted the per-type dyn
descriptors. Emit.redefinition has always emitted them, by going through
finish; the x86 twin ended at the rodata section and stopped. Nothing
had reached it, because a redefined body had to construct a struct
holding a dyn to need one, and until NoMethod there was no such struct a
compiler-written body could build. What it looks like is not a bad read
at run time but a link failure — desc_of mints a local label, the body
references it, and ld refuses the module with an undefined symbol. One
line, beside the same call in the executable path.
And the thing the test had to be written around: a dyn value answered by
eval-expr does not come back in the reply's :value at all. It renders to
the program's own stdout, which reaches a *later* reply's :output — the
dyn-global rows already read one that way and say so. So every answer
here is compared inside the expression, and what crosses the wire is a
typed 1 or 0. Left as it is; where a dyn expression's value should
surface is a question about the editor protocol, not about this lane.
The session-level test stays: it pins which names an added method
reports for installation, which is the half a daemon test cannot see.
flan.abi.require was spelled by hand in both backends, which is the one
job Mangle has. Moved; emit and x86 produce byte-identical output on the
reload path either way.
Four comments in the dyn-cast code asserted things that are not true.
The warning's location prefix now reads like every other loc-bearing
runtime diagnostic instead of inventing a shape. widen's contract says
what cast_dyn actually does with it. The thread-safety note names the
torn {ptr,len} overread rather than a duplicated line, and says why no
lock. The site table's borrowed loc pointer names what keeps it valid.
The memory op's note claimed a completeness it does not have: dyn push
and put may allocate and are deliberately silent. Said so, in the note,
in the classifier, and in FIX.org where the decision belongs.
The documented flycheck form only matched warnings, so a real error
made it say the checker returned non-zero and found nothing.
flan-clear-memory cleared one buffer where the toggle clears all.
Two comments claimed test/dyn_ops.c calls every function flan_dyn.h
declares; six are declared and never called there.
flan_dyn_stub.c's deadness is written into FIX.org for the author to
decide on. Not deleted here.
The guard flag was named .init-once.<global>, and . and - are ordinary
symbol constituents, so (defvar .init-once.x i64 7) beside a computed x
emitted the same symbol twice: the dev build died at the assembler on both
backends, and the flag's Bool was registered over the user's global in
Emit.globals so the store came out as an i1. It is .init~once.<global> now;
~ terminates a symbol in the reader, the same trick destructure~N uses.
test/programs/dev-rerun.flan carries such a global and no new printed line.
Three places said a defconst is the linker's image on one backend and a
constructor's stores on the other and a re-run reaches neither. Tast.const_init
splits on the initialiser and not on the form, so that holds only for a
constant initialiser; a computed defconst is guarded like a defvar on x86 and
refused outright by emit.ml's const. The sentences now say that, including
the divergence.
emit.ml also claimed Check.no_transfer_in_init made it impossible to leave the
guarded branch between the store and the flag. It is syntactic over the
written initialiser only: a callee can signal unhandled and take the call's
transfer edge out, leaving the flag false — which is what should happen, since
the next run retries. Read off the emitted IR for such a program.
The x86 float-Rem comment says why the dead movabs before fmod is kept, and
its mid-sentence line break is gone; math3.flan's first float-% line prints
six values, not four.
The %handler, %restart, %fninfo and %flanframe shapes were written twice:
as LLVM type strings in emit.ml and as hand-computed byte offsets in x86.ml,
with the two %fninfo initialisers spelled a third and fourth time. Emit.Rt
now holds one field list per struct and derives all four — the type string
and the getelementptr index for LLVM, the offset and the size for x86, and
the initialiser for both. The derived numbers were checked against every old
constant before the call sites moved.
The flan. prefixes were spelled in four files, including both backends
hand-writing "flan." ^ name for a DWARF linkage name instead of calling
their own helper. Mangle now holds them unquoted; each backend adds its own
sigil. The ABI markers stay apart on purpose: flan.abi.llvm and flan.abi.x86
differing is what makes the loader refuse a crossed pair.
The float-to-integer cast bounds and the division-check elision policy are
Emit.cast_range and Emit.div_checks. The second is a language decision and
had been byte-identical in both files; the first had drifted cosmetically.
emit and emit --x86 output for all 166 test/programs, at -O0 release, --dev,
--debug and --dev --debug, stdout and stderr, is byte-identical to the
pre-change compiler.
The first version of both notes claimed a call to fmod in any build with a
typed float % in it. A literal pair is folded before any call exists, which
is the same fact two paragraphs further down explaining why the corpus block
uses globals. Both now say the measurement: the calls are in the build whose
operands come through globals.
There is no SSE remainder instruction, and LLVM does not invent one: at -O0
it lowers frem to fmod or fmodf. The backend now calls those two symbols
rather than refusing the operator, which is agreement by construction rather
than a second hand-written identity that would have to get every rounding,
every signed zero and every infinity right on its own.
Rem was the only gap. emit.ml's float surface is Add, Sub, Mul, Div, Rem and
the six comparisons; x86 had everything but Rem, and its comparisons already
build LLVM's ordered predicates out of ucomis, setcc and setnp.
math3.flan grows the operator spelling beside the fmod-f32/fmod-f64 calls it
already had, through globals so the pair is not folded before either backend
sees an operator. FIX.org records the ruling the fix came from.
flan_merged_park called flan_dyn_root_reset, which emptied the collector's
root stack. The frames' roots had to go — main is left by longjmp, so they
name stack the next run overwrites — but the dyn globals' roots are on that
same stack, pushed once by the emitted main and never popped, and the park
took them with the frames.
The park is not a quiet state. It services evaluated thunks, a thunk
allocates, and an allocation collects. So a program with (defvar config dyn)
answered (get config :s) with its string before any thunk ran and with nil
after one that allocated past the heap's floor — a read of memory the sweep
had freed, answering nil by luck of what the freed words decoded as.
The emitted main now brackets its global pushes: flan_dyn_root_globals_begin
empties the stack, the pushes go on, flan_dyn_root_globals_end records how
many of them there are, and the park resets to that line instead of to zero.
Nothing between the two allocates, which is what keeps the globals from being
swept in the window where they are unrooted — and [begin] emptying the stack
rather than adding to it is what makes a re-entered main re-root the same
globals rather than push a second copy of each, which also closes the other
half: a re-run used to re-push roots over slots left dangling by the park.
Both emitters, because the dev loop's default backend is x86 and a fix in one
lowering is not a fix. A program with no dyn globals emits neither call and
its root stack still resets to empty, which is what an empty push list should
leave behind.
flan_dyn_root_pop now clamps at the globals rather than at zero. An
over-popping frame eating the globals is the one way that clamp could turn a
miscount into this same use-after-free.
Covered twice. test/dyn_ops.c's park mode is the runtime's half — a run, a
park with a collecting thunk in it, and another run, three times over,
asserting both that the global survives and that the frame's five hundred
objects do not. Under ASan the old reset reports heap-use-after-free in
flan_dyn_tag with the free in gc_sweep; under memcheck it reports 24 errors
and still prints the right answer, which is the shape of the bug. test_dev.ml
drives the whole daemon over its socket on both backends against
programs/dev-dyn-global.flan.
Not touched, and it wants a decision rather than a patch: a re-run re-enters
flan_program_main, which re-runs the lifted startup function, so every global
with a computed initialiser is reset by a re-run. That contradicts dev.ml's
own note and FIX.org item 1. It is independent of this — the roots are right
whether or not the values are re-initialised.
Nor is this the reload path. A defvar added by an evaluation gets its storage
from flan_dev_global (emit.ml's new_globals, x86.ml's counterpart) and there
is no flan_dyn_root_push anywhere on that path in either backend, so a dyn
global added to a live session is unrooted. That is a separate defect with a
separate fix, and nothing here makes it better or worse.
A (Vec T), a slice or a fixed array crossing into dyn no longer refuses; it
is a view, one word in the box, over the container's own storage. Reads box
the element on the way out; writes tag-check the dyn value's tag against the
element type on the way in and trap, by name, on a mismatch, never coercing
or silently storing.
The open question the decision left — whether the descriptor points at the
container or snapshots pointer and length beside it — is settled by kind. A
Vec view holds the address of the Vec's own header (flan_rt.c's flan_vec,
restated in flan_dyn.c under the file's standing "if either table changes,
change both" rule) and reads ptr and len live on every operation, so a push
that reallocates cannot leave it stale: flan_vec_grow overwrites that same
header in place, and there is nothing captured at the crossing for the
growth to invalidate. A slice and a fixed array cannot grow, so a flat view
snapshots data and length once; pointing it at the value's own slot instead
would be worse, since a slot's lifetime is not the slice's.
The element set is i64, f64 and bool, not everything box already handles
typed-to-dyn. A string element's dyn form is a pointer into the collector's
heap, and a typed container's storage is arena or stack memory the collector
never scans — a wider set would let a write plant a live reference nothing
ever traces, which no care at the write site closes. (Vec string) and a
typed (Map K V) keep the "does not cross into dyn yet" refusal, now for that
reason.
flan_dyn.c gains a fourth object kind, OBJ_VIEW, and flan_dyn_len/at/set_at/
push and the printer each grow one branch for it beside the existing vec
one. A view's own stale-container check is the runtime's own spelling
(flan_trap, park-and-inspect) rather than flan_rt.c's rt_die, per the
duplicity doctrine; growing a Vec through a view calls flan_rt.c's own
flan_vec_push rather than re-implementing doubling and allocator adoption a
second time. (set (at target i) x) against a dyn target — a plain dyn vec or
a view alike — was a hole in the base dyn milestone rather than something
item 3 introduced; it is wired to flan_dyn_set_at here because a view's
writes needed it to exist at all.
Both backends: emit.ml and x86.ml both already passed a Vec or a Map to a
runtime call by address rather than by value; a fixed array crossing into a
view needed the same arm added in both, for the same reason — a copy would
view the copy and never see a write to the caller's own array.
test/dyn_ops.c drives the runtime directly with a hand-built Vec header and
a plain C array, ahead of any compiler involvement: reads, writes on both
element kinds, the tag-check refusal on every element kind, the range
refusal, and the push that grows and moves a hand-built header out from
under the view watching it. test_flan.ml turns the old "does not cross into
dyn yet" refusal into acceptances for Vec/slice/array, keeps it for a string
element and for Map, and adds the element-restriction refusal by name.
test/programs/dyn-view.flan is the compiler-level survey: a Vec view mutated
through both sides including the grow-and-move case, a fixed array's and a
slice's views, a bool Vec's view, and its own two trapping modes for the
acceptance rows to run against. test_sanitize.ml carries the survey's happy
path; test_dyn.ml's new refusals are the runtime's own.
The crux was never where to put a descriptor; it was how an instance finds
one. A bare struct on the stack has no header to hang a pointer off, and
giving it one would change the layout C interop agrees on, change the stride
of an array and change what embedding a struct in another costs. So it has
none. The instance never carries a pointer to its type and the collector
never derives one from the bytes: the pairing of an address with a descriptor
is made at the *push*, by the code that put the value there and therefore
knows its static type. That is the same trick the shadow stack has always
used, and it makes the stack case the easy one rather than the impossible one.
A descriptor is the size of an instance, a count, and a table of byte offsets,
emitted once per type as private static data. Flattened, not a graph — a
struct held by value contributes its offsets shifted by where it sits, and a
fixed array contributes its element's once per element — so nesting costs
nothing at run time and there is no recursion in the marker. The offsets of a
big array would be a big table, and that is capped with a sentence rather than
half of the repeat form item 3 will bring.
Four places a value of such a type can live, and all four are rooted: a frame
slot, a global, the temporary a call's by-value return is spilled into, and
the slot a condition that is not a place is evaluated into. The last two are
new and are the ones that were not obvious. A callee roots its dyn words and
pops them in its epilogue, so between the return and the caller's store the
only copy is a register, which a collector that finds its roots by address
cannot see; the same hole was open for a Flan call answering a bare dyn and is
closed here too. And a condition crosses as a pointer into the signalling
frame while a handler allocates, which is exactly what the original refusal
said could not be made safe.
dyn_roots grows into root_plan and both backends read it, which is what the
older note about one counter deciding both ends was always for. The aggregate
temporaries are pooled by type rather than handed out in mint order: a
positional supply that drifted would pair an address with another type's
descriptor, and marking arbitrary offsets off a base is corruption where a
missed root is only a bug. Pooled, the worst a drift can do is run out.
What is still refused is a dyn no static offset can reach — inside a typed
container, in a data type's payload or a union's members where the cases
overlay, or under an Option where the payload exists only beneath the tag.
A (Ptr S) and a [S] are deliberately not on that list: neither owns storage,
and the only storage this compiler hands out for such a type is a frame slot,
a global or a fixed array in one, all of them already rooted. That is what
lets a handler clause take its (Ptr Cond) and read a dyn payload.
test/programs/dyn-struct.flan is the evidence. It runs forty thousand rows
past flan_dyn.c's one-megabyte floor, so marks and sweeps really happen, and
it holds live values through them in all four places at once. It has teeth:
with the descriptor walk stubbed out of the marker, the kept vector's length
comes back 24 instead of 628 and its first element is a stale word. Clean
under ASan and UBSan, same output at -O2, -O0 and --x86. dyn_ops.c grows an
aggregate-root mode so the runtime half can be wrong on its own, with a
header word holding a bit pattern that looks boxed and is not a dyn slot.
--no-gc still refuses, and had to be told how: a struct with a dyn field is a
collected value even when no expression in the program ever has the type dyn,
because a zeroed one still has a word the collector is asked to mark.
dune test --force: green, 0 failures across every suite.
Constraints parsing peeled a body-leading map only when its first key was
literally :where; any other keyword fell through to the body, so a typo'd
key surfaced as a baffling error from inside what was meant as a predicate
and a stray map at body start compiled away silently. Any keyword-first map
is read as a constraint map now, but only when something follows it in the
body — a single-form map body is a real dyn value and not a discarded
statement, so that case is left alone.
An empty map literal still parses as a struct literal, (P {}) still meaning
the zero struct for a real struct name — the parser has no symbol table to
tell (take {}) apart from it at that point. check.ml now catches the case
where the name turns out to be a known function instead and says so, rather
than "unknown struct take".
flan_dyn.c's tag comment still said 6 and 7 were free; keywords and maps
took 4 and a kind field under BOX_OBJ, not new top-level tags, so 5, 6 and 7
are what is actually open for the interop handle. NEXT.md and json.flan both
still pointed at test/programs/arena-edn.flan, gone since edn/read stopped
taking an allocator; both now point at what replaced it.
flan_rt.c's flan_str_eq comment claimed the empty string literal was a
hypothetical null-pointer string; it isn't, its address is an interned
symbol's. The real case the zero-length guard exists for is a zero-length
container converted to a string. check.ml's ordering refusal said a string
has no comparison at all, which stopped being true when typed = and !=
grew strings in daed039 — split the message so an equality refusal and an
ordering refusal say the right noun, and updated the pinned rejects_check
rows to match. string-eq.flan gained the row the fast path most wants
tested, a slice against the prefix it was cut from sharing a base pointer at
different lengths, plus a != row at equal length with differing bytes;
acceptance now carries the real output, captured by running the program on
all three lanes. x86.ml's xor-1 comment now names the 0/1 return contract as
a requirement flan_str_eq must hold, not an incidental fact. SPIKE-DUPLICITY
now says plainly that its equality-and-ordering argument landed in daed039
and marks its transcript as the historical state that argument was made
against. FIX.org ticks M2 queue item 5.
And the acceptance runner: the tail check that turns a nonzero failure count
into exit 1 was already there and already fired — a fresh build with one row
broken already exited 1 before anything here changed. What wasn't proven is
that every path through the file's clang/wasmtime/raylib/lldb probes still
reaches that tail rather than skipping past rows that already failed. An
at_exit guard now closes that class regardless of which path the process
leaves by, flushing stdout first so a failing run's FAIL lines survive
Unix._exit rather than being dropped from the buffer. Verified both
directions with a deliberately broken row: dune test exits nonzero and the
log still carries the FAIL line and the failure count; restored, the same
run is exit 0 with nothing printed but green summaries. The other test
binaries were checked for the same gap and none have it — each gates its
own exit on a single failures ref that the tail already reads.
Types.is_equatable splits from is_comparable: a string answers equal?
now, bytewise, but still answers no to ordered? — there is no collation
the language has picked, so < and friends keep the refusal they had.
The comparison itself is one new runtime entry point, flan_str_eq
(runtime/flan_rt.c), length-mismatch and same-pointer fast paths ahead
of the memcmp, called identically from both backends: emit.ml pulls a
string's ptr and length out of the %slice SSA value and calls it
directly in the Eq/Ne arm; x86.ml adds an arm ahead of the generic
scalar comparison that reaches it through call_native, flipping the
answer for != the same way Not already flips a bool.
test_flan.ml covers the checker side directly and through a generic
instantiated at string, including the two different ways ordered? and
equal? fail at that type. test/programs/string-eq.flan is the survey
program — same pointer, differing lengths, equal content at distinct
addresses (a literal against a fresh heap string), a difference in the
last byte, and the empty-string cases — with acceptance rows for LLVM,
-O0 and --x86 in test_acceptance.ml.
x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
The type itself, the ABI its operations call into, and the one decision the
feature could not avoid: (defn f [x y]) is one parameter or two, and which one
depends on whether y names a type.
Parse does not decide it. That lookup is the one its defn comment records being
removed for being wrong twice in one day -- the set of type names is incomplete
at parse time by construction, and macros generating definitions is what
widened the failure. So the vector is carried undecided, as Ast.pitems, and
paired in Check, after every file is loaded, every macro expanded and every
header imported. The set is complete there. It is not complete across time, and
the comment says so: a defstruct written later changes a signature with no edit
to the function.
The return slot stays mandatory and dyn is written out in it. The ambiguity
there has no syntactic resolution at all -- a capitalised head in a list is both
a type application and a struct literal -- so the third state the parameters
needed does not exist for the return type, and ret = None goes on meaning Unit.
What the feature costs, and what is taken back: a slot with no type used to be a
syntax error, so a mistyped type now reads as an extra parameter with no
diagnostic. A name within one edit of a type's gets the resolver's own
did-you-mean, and an unknown capitalised name is reported as the unknown type it
is -- not one parameter in the corpus is capitalised. A lowercase name
resembling no type is the feature working, and is the residual.
The x86 backend refuses dyn by name; both callers already name --llvm, and the
daemon takes that backend by default, so this is the first thing a user of dyn
sees. The JS dialect refuses it too, for the opposite reason -- every value
there is already dynamic and what is missing is only the lowering.
runtime/flan_dyn.h is the fixed ABI. flan_dyn_stub.c stands in until the real
collector lands and says in its header that it verifies nothing about roots.
The backend had a full dev story for everything except the one thing the
break loop reads: emit.ml pushes a shadow-stack frame on entry to every body
in a dev build and x86.ml pushed none, so an --x86 session answered backtrace,
locals, inspect and globals with "this program was not built with --dev",
which was false of it.
emit_fn now builds the same four words runtime/flan_dev.c describes -- the
previous head, a static descriptor, and a table of one address per slot, null
until the binding that fills it has run. The push is emitted at the head of
the body buffer rather than into the prologue's, because the prologue is where
the incoming registers still hold the arguments and every instruction of it
clobbers rax. The pop is one store at the epilogue label: this backend has one
epilogue and a return, the body's tail and the transfer exit all arrive there,
where emit.ml needs the same restore at five separate rets.
Two decisions are taken from emit.ml rather than reasoned out again, so that
the two dev builds answer a stopped frame identically. A function with no
named slot gets no table and reports no slots -- there the gate buys mem2reg
an alloca that does not escape, here it buys nothing but agreement, and
agreement is the only thing the break loop can check. And the descriptor's
bytes do not go through string_const: that counter is what decides whether an
expression thunk's module may be unloaded, and counting a descriptor in it
would pin every C-x C-e module's mapping for ever.
The descriptors are the one constant this backend emits that holds an address,
so they go in .data.rel.ro and not .rodata -- a relocation the loader applies
needs a section it may write, and a redefinition module is such an object.
Verified on an assembled module: R_X86_64_RELATIVE for the two pointers,
flan_frame_head through the GOT, no TEXTREL.
layout_ctx fills its globals table, which it never had to before. Reach's
ref_fingerprint asks whether a name is a global, and with an empty table no
name is one -- every frame would carry the hash of the empty set, the daemon
would recompute the real one, and the globals section would refuse a body
nobody had touched while its locals still read.
Release builds gain nothing: all of it is behind md.dev, and three corpus
programs emit byte-identical assembly before and after.
The new coverage stands up an --x86 daemon on dev-locals.flan and asks the
four questions, against the values the LLVM block above asserts of that same
program -- the claim is that the answers are the same and not merely
plausible. The existing --x86 two-process daemon gains one more: a backtrace
through a redefined body, which reports the installed body's own file, since
that descriptor travels in the module and a frame pointing at the host's would
name the body it replaced.
The x86 backend ran initialisers from .init_array and the LLVM one refused
them by name, so (defvar frame Allocator (arena-new 262144)) — which the
author kept writing — was a program on one backend and an error on the other.
A rule that holds on one backend and not the other is not a rule.
The checker lifts a computed initialiser into a function of its own and the
global's initialiser becomes the call. That is what gives it a frame, which is
the bug underneath the feature: a `let` or a `match` in an initialiser indexed
a slot array of length zero and took the x86 emitter down with an uncaught
Invalid_argument.
Both backends call the lifted initialisers from main, after flan_rt_init and
before a line of the program's own code — Odin's __$startup_runtime shape, not
a constructor, so the runtime is up and the order is the compiler's to choose.
x86 keeps .init_array for one thing only, and it is named: writing the
constant image this backend has no folder for, which is standing in for the
other backend's object image rather than for a program.
The computed globals are sorted by what they read, transitively through the
functions they call, so a global written above the one it reads works and a
ring is refused with every name in it. A reload still re-runs nothing: a new
global with a computed initialiser starts as ZII on both backends.
The refusal that lived in x86.ml is now the checker's and is narrower. Nothing
can escape an initialiser — the handler and restart stacks are empty and every
frame it pushes it also pops — so what is refused is a signal or an
invoke-restart with no handler-bind or restart-case around it, which is inert
by construction. A restart-case inside one is ordinary code, which is what
makes (defvar data (Vec u8) (slurp "level.edn")) an ordinary program.
Three refusals go with the premise they rested on: a container global with a
computed initialiser, a union member in a defvar, and a data type case in one.
A defconst is untouched and keeps all three.
One change here is not about any of that. sand.flan carried an unfinished
line — (defvar game-data (embed (with-allocator frame ))), which parses as a
declaration whose type is (embed ...) — so the checker refused the file and
`dune test` was red at the tip of dev-loop before a line of this landed,
verified by stashing this work and rebuilding. It is commented out rather than
guessed at: the arena above it is the half that works, and what the global
should read is the author's to decide.
The lo <= hi test in check_slice and slice-from-ptr's n >= 0 sat behind
--no-bounds-checks in both backends, while the comment beside each said
they could not be dropped. They are not bounds checks: hi <= len asks
whether a range fits inside a length, and lo <= hi asks whether the word
about to be written into a %slice's length field is a count at all. The
first stays behind the flag, the second is now emitted everywhere, the
way flan_vec_as_slice has always validated its own l > h in plain C.
emit.ml emits two signal blocks rather than one and i1, so an unchecked
build carries one compare. x86.ml keeps all three frame temporaries
stored outside the flag and gates only the second compare, because the
third is the length the message prints.
The IR assertion in test_acceptance now says the two slice calls are
present under --no-bounds-checks rather than absent, and the same build
is run: case 2 and case -2 of bounds.flan must still die.
