[Check.memory_sites] is the pass [Check.no_gc]'s shape: it runs over the
finished program, answers a diagnostic list, and tells nothing downstream
that it ran. Two classes on the diagnostic's kind — the collector's heap
and an allocator the program named — so the CLI, the daemon and the editor
dispatch on one field and none of them reads a message to find the class.
[--warn-memory] on check and build prints them where errors go, in the
shape flycheck parses, without moving the exit status. [(:op "memory")]
answers the same list over a session's last checked program, needing no
program on the far end. [M-x flan-check-memory] paints it two faces fainter
than an error's, cleared by an edit or by asking again.
Two of the spec's own examples turned out not to allocate, and the
precision rule outranks the enumeration: (vec-new T) passes a capacity of
literal zero to flan_vec_init, which returns before the grow, and
flan_map_init takes no block at all and says so in its own comment. The
block arrives at the first push, which is the line that is marked. The
classifier reads the capacity argument rather than the symbol, which is
what lets slurp be marked through the same entry point vec-new is silent
through. FIX.org has the rest of the evidence.
Ten test binaries share a directory and had shared nothing in it but
watchdog.ml. Everything else each one needed it wrote out again: the
failure counter and its FAIL line, the three-line report tail, a poll,
a socket connect, the wait for a [flan dev] daemon to bind, a substring
search, and the Load -> Check -> Reach.link front half of a compile.
[listening] was the clearest case. Three copies, byte for byte apart
from one comment, and two of them said in that comment that they were
kept separate because "these three files have no module between them".
That was not true when it was written: watchdog.ml was already named in
the same (modules ...) stanzas. test_support.ml is the second such
module, wired the same way, and those two sentences go with the copies
they were explaining.
test_repl.ml's [quote] was Wire.quote character for character, in a file
that already links Wire and already names Wire.quote in a comment about
what the case below it is checking. It is Wire.quote now.
One real behaviour change, and it is a fix. [connect] existed twice over
with different retries: the agent's narrowed to ECONNREFUSED with a
comment saying why -- the socket file appears at bind, a moment before
listen -- while dev's and repl's retried any Unix_error, which meant an
ENOENT or an EACCES was retried to the full timeout before raising
something the reader still had to interpret. The shared one takes the
narrow version. Every caller connects to a socket [listening] has
already seen on disk, so the race it does catch is the only one left.
The rest is left where it is, on purpose. The three output-capturing
[run]s differ in what they wrap -- a pid suffix, a sanitizer environment,
a valgrind invocation -- and are not the same function. The report tails
in test_repl, test_web and the two sweep binaries print different things
for different reasons. The per-file scratch prefixes are the feature that
keeps two suites running at once from unlinking each other's sockets, so
the shared helper takes the prefix rather than choosing one. And the
[match Sys.command "command -v clang ..."] probes stay as they are:
their skip lines are output this suite pins.
bin/main.ml has the compile pipeline written out twice more. Left alone
-- this was a test/-scoped change and bin/ should not be reaching into a
test module -- and noted in FIX.org as what it actually needs, which is
the pipeline moving into lib/.
dune test: exit 0, and its output is the same line for line once the
temp-directory hash and the millisecond counts are normalised.
(at g i j) is one Tast node carrying the whole index list, so the At
arm's guard on target.ty settled level zero and nothing after it. A
global [2 [[3 i64]]] indexed twice reached a slice's element, crossed
into dyn as a view, and printed a returned frame's contents with exit
0 (ASan: stack-use-after-scope in view_box). The arm now steps each
index the way [indexed] does and demands an array at every level;
the Field and Slice arms inherit the fix by recursing into it.
(and a b) desugared to (if a b false), so it answered the last operand
only when every operand was truthy; a falsey one came back as a bare
false, where Clojure answers the falsey operand itself. It now uses the
same expansion or got in ad0f1fb -- (let [t a] (if t b t)) against or's
(let [t a] (if t t b)) -- so the operand that decided the form is the
answer, and the test is still evaluated exactly once.
The temp binding and its if now carry the operand's own loc rather than
the whole form's, which the or fix had lost: (or (vec-new i32) v) blamed
the enclosing form at 3:13 and now points at 3:18, the operand, and and's
second operand gained the same precision.
The parse pins in test_flan.ml now tie the bound name to the temp the if
tests and the bound value to the first operand, so a desugaring that
dropped the temp and wrote the operand into the arm twice no longer
passes; and has its own pin. dyn-if-truthy.flan grows the falsey-nil and
falsey-false answers, 0 and "" as truthy operands, one- and zero-operand
forms, and a printing operand that proves both the short circuit and the
single evaluation.
One behaviour that used to compile changed: with both arms of the
desugared if now holding real values, (and dyn-value typed-bool) unifies
on the typed arm and a non-bool dyn decider traps at the strict bool
boundary -- (and (box nil) some-bool) printed false and now traps, the
mirror of what (or false (box "s")) already did on dev-loop. Recorded in
FIX.org as the author's call on how check_if should join a bool arm and a
dyn arm.
The [At] arm of [permanent_root] recursed through any indexed target, so an
element of a global SLICE answered permanent the way an element of a global
ARRAY does. An array's elements are inside the global's storage; a slice's
are ptr+len pointing wherever, which can be a frame already returned — the
program that stashes (slice local 0 2) in a global slice and views an element
compiled and segfaulted with no diagnostic. The arm now recurses only when
the target's own type is an Array.
With it, the refusal/acceptance pair in test_flan.ml (one word apart) and a
view over an element of a global array in dyn-view.flan's mode 0.
The element check now runs before the lifetime check in all three container
arms: a local (Vec string) was told to make it a global, and a global
(Vec string) is refused anyway, so the advice was a dead end.
And the four strings that claimed more than the code does. flan_dyn.h
already had the honest version — a view is exactly as stale-safe as the
thing it is a view of — so the refusal message, box's comment and FIX.org
now say that instead of promising a dyn value can never dangle; a global
[i64] cut from a dead frame still passes and still reads it (ASan:
stack-use-after-scope in view_box). The element message no longer tells a
(Vec string) that string is not the case the restriction exists for.
dyn_ops.c's hand_vec comment no longer says flan_rt.c is unlinked when it
calls two of its functions; flan_rt.c said the same thing and is fixed too.
FIX.org's arena paragraph now separates the header's lifetime (compile time,
already covered) from releasing the arena under a live view: free-all traps
cleanly on the epoch, arena-destroy is a heap-use-after-free in
view_vec_check, the same gap flan_vec_check has on the typed side.
Relocation was proved sound and stayed sound — a Vec view holding the
header's own address survives a push that grows and moves it, because
there is no snapshot to invalidate. That was never the whole of the hazard.
Refusing every container into dyn outright, before this lane, meant a
dangling view was unreachable; the moment box stopped refusing, three
routes opened at once — a view returned from the function whose frame the
Vec lived in, one stashed in a dyn global and read after that frame is
gone, and one left behind when a condition transfer unwinds it. All three
are stack-use-after-return, reachable for the first time.
The rule: a typed container crosses into dyn as a view only when its own
storage is permanent — a global's. On the dynamic side Flan follows Clojure
and Common Lisp, where holding a value can never hand you garbage; treating
a view as a bare pointer and calling the lifetime the programmer's problem
is the Odin answer, and it is the wrong trade on this side of the language.
check.ml's permanent_root walks the checked expression back to its root: a
global is permanent, a field or an array element of one is permanent at the
same fixed offset, and a slice cut directly from one at the call site
inherits it — the trace is what a slice carries, and it is lost the moment
the slice is bound to a name first, so that case is refused too rather than
guessed at. Everything else answers false: a local, a parameter, a
temporary, and anything reached through a (Ptr T), because a heap-durable
pointer and a frame's own are the same type and the checker cannot tell
them apart — admitting one admits the other, which is the whole hazard this
closes. An arena-held header turns out not to be a separate case at all: an
arena changes where a Vec's elements live, never where its own header — the
binding — lives, so it is already covered by the storage-class check above.
Both directions of the F1 escape were reproduced before the fix (a genuine
ASan stack-use-after-return, reproduced by building the pre-fix tree) and
confirmed refused at check time after it, for all three routes.
Three more findings, all in the runtime rather than the boundary:
view_vec_check, on finding a stale container, rendered the very view it had
just declared unsafe to read — which called back into the same check,
unconditionally, an infinite recursion rather than the intended trap. Fixed
by never rendering the container in the stale message at all; the sentence
names the two epochs and nothing else, which is everything a reader needs
and the one thing that was safe to read.
dyn_equal's VEC arm read x->len and x->u.v.items regardless of kind, which
for a view answers 0 and the union's other member reinterpreted as dyn
words: two views with different contents compared equal, a view and an
equal heap vec compared unequal, and a map keyed by any view collided with
every other view, silently. vecish_len and vecish_at read either shape
correctly and the arm now goes through them. obj_words gets the same
explicit OBJ_VIEW case on the same reasoning, unreachable today only
because mark_push's own gate already excludes the kind — this is the belt
next to that brace.
The three restatements of flan_vec's layout — flan_rt.c's real struct,
flan_dyn.c's mirror, and dyn_ops.c's hand-built one — had a comment
claiming a reorder would not compile or link, which was never true of a
void*-typed forward declaration. flan_vec_layout and
flan_dyn_vec_hdr_layout each report their struct's size and field offsets;
dyn_ops.c's new "layout" mode compares both against offsetof on its own
hand_vec, so a disagreement is a FAIL line in dune test instead of a
silent corruption at whichever view reads through the wrong offset next.
Also: the survey program's comment excusing a by-value parameter's view as
"value semantics, not a hole" was wrong on its own terms — a write through
such a view does reach the caller's storage, only growth diverges — but the
question is moot now: every container the program views is a global, and
the file was rewritten around that rather than patched. And an i32 element
does not cross into a view either, but the refusal used to say why in words
that were true only of a string element; it now says what i32 actually is
and what the restriction is actually for.
Rebased onto dev-loop's item-4 landing (221df5a).
The dyn if truthiness review turned up that or's answer position, unlike
and's, still traps on a non-bool dyn value: or's short-circuit sentinel
sat in the then arm of its own if, the one check_if types first, so that
sentinel decided the whole expression's type and a later non-bool dyn
answer hit the strict bool boundary and unboxed itself into a trap
rather than surviving as itself. (or nil "x") — the canonical Clojure
(or x default) idiom — crashed instead of answering "x", identical on
all three backends.
or now binds its test to a temp and answers the temp itself, exactly the
way Clojure's own or macro expands: (or a b) becomes (let [t a] (if t t
b)), not (if a true b). The temp evaluates a once and lets the answer be
a without writing it a second time as the then arm; it is the temp's own
type check_if sees first, so or hands back the actual truthy operand the
same way and always has. Verified real output, unchanged, on LLVM, -O0
and --x86, and the survey program now exercises the case its own header
used to exclude for being unsafe: a non-bool value stopping or and being
handed back as-is.
check_truthy also gets three corrections a closer look found. Its own
[loc] used to come from the enclosing if/while/not rather than from the
condition itself, so the rt call and cast it builds carried the wrong
column in an --x86 disassembly or the dev inspector whenever the
condition was not the form's first token; it now takes loc from the
scrutinee's own AST node, confirmed against a real --x86 dump. A comment
now names the precondition its exception-swallowing retry rests on: none
of check.ml's save-restore sites (barrier, in_frames, in_defer, loops,
scope) are exception-safe, which is harmless only because the retry
always either succeeds cleanly or re-raises and aborts the compile
before ctx is read again — and would stop being harmless the day some
want-sensitive elaboration on this path could succeed differently on
retry. And a bare keyword condition, which used to be checked with
want:Bool from the start and refused by the keyword arm's enum-or-refuse
case, now resolves as the dyn keyword instead and is unconditionally
truthy — a deliberate loss of that diagnostic, the author's call, pinned
in test_flan.ml so it does not regress by accident.
The two typed-refusal messages captured before this pass (a float
literal condition, an i32 while condition) are unchanged, checked again
against the same baseline. test_flan.ml's parser test for or's shape is
updated to match the new let-bound desugaring.
A (Vec T), a slice or a fixed array crossing into dyn no longer refuses; it
is a view, one word in the box, over the container's own storage. Reads box
the element on the way out; writes tag-check the dyn value's tag against the
element type on the way in and trap, by name, on a mismatch, never coercing
or silently storing.
The open question the decision left — whether the descriptor points at the
container or snapshots pointer and length beside it — is settled by kind. A
Vec view holds the address of the Vec's own header (flan_rt.c's flan_vec,
restated in flan_dyn.c under the file's standing "if either table changes,
change both" rule) and reads ptr and len live on every operation, so a push
that reallocates cannot leave it stale: flan_vec_grow overwrites that same
header in place, and there is nothing captured at the crossing for the
growth to invalidate. A slice and a fixed array cannot grow, so a flat view
snapshots data and length once; pointing it at the value's own slot instead
would be worse, since a slot's lifetime is not the slice's.
The element set is i64, f64 and bool, not everything box already handles
typed-to-dyn. A string element's dyn form is a pointer into the collector's
heap, and a typed container's storage is arena or stack memory the collector
never scans — a wider set would let a write plant a live reference nothing
ever traces, which no care at the write site closes. (Vec string) and a
typed (Map K V) keep the "does not cross into dyn yet" refusal, now for that
reason.
flan_dyn.c gains a fourth object kind, OBJ_VIEW, and flan_dyn_len/at/set_at/
push and the printer each grow one branch for it beside the existing vec
one. A view's own stale-container check is the runtime's own spelling
(flan_trap, park-and-inspect) rather than flan_rt.c's rt_die, per the
duplicity doctrine; growing a Vec through a view calls flan_rt.c's own
flan_vec_push rather than re-implementing doubling and allocator adoption a
second time. (set (at target i) x) against a dyn target — a plain dyn vec or
a view alike — was a hole in the base dyn milestone rather than something
item 3 introduced; it is wired to flan_dyn_set_at here because a view's
writes needed it to exist at all.
Both backends: emit.ml and x86.ml both already passed a Vec or a Map to a
runtime call by address rather than by value; a fixed array crossing into a
view needed the same arm added in both, for the same reason — a copy would
view the copy and never see a write to the caller's own array.
test/dyn_ops.c drives the runtime directly with a hand-built Vec header and
a plain C array, ahead of any compiler involvement: reads, writes on both
element kinds, the tag-check refusal on every element kind, the range
refusal, and the push that grows and moves a hand-built header out from
under the view watching it. test_flan.ml turns the old "does not cross into
dyn yet" refusal into acceptances for Vec/slice/array, keeps it for a string
element and for Map, and adds the element-restriction refusal by name.
test/programs/dyn-view.flan is the compiler-level survey: a Vec view mutated
through both sides including the grow-and-move case, a fixed array's and a
slice's views, a bool Vec's view, and its own two trapping modes for the
acceptance rows to run against. test_sanitize.ml carries the survey's happy
path; test_dyn.ml's new refusals are the runtime's own.
A dyn scrutinee is no longer required to already be a bool: it is tested
for truthiness, Clojure's rule, not C's or Python's — nil and false are
the only falsey values, and everything else, including 0, "", an empty
vec, an empty map and a keyword, is truthy. A typed scrutinee is
unchanged and keeps needing a strict bool.
The runtime side is one new entry point, flan_dyn_truthy
(runtime/flan_dyn.c/.h), reading the tag directly rather than unboxing —
it never traps, unlike flan_dyn_need_bool. Both backends reach it the
same generic way flan_dyn_need_bool already did: check.ml emits an
ordinary Rt call plus the existing i32-to-bool Cast, so emit.ml only
needed the LLVM declare added and x86.ml needed nothing at all.
check.ml's check_truthy is the one funnel every boolean position in the
language goes through: if's own condition, while's, and not's argument.
when and cond reach it for free because they desugar to Ast.If in
parse.ml, and so does and's condition; or's condition does too, but its
answer position is a separate story — its short-circuit sentinel is the
then arm of its own if, which check_if types before anything else, so a
non-bool dyn value reaching that position still meets the strict bool
boundary. and's sentinel sits in the else arm instead, so the real
value's type wins and and hands back the actual last operand,
Clojure-style; or does not get that for the reason above, and reordering
it is a decision for another day, not this one. shortcircuit in parse.ml
carries the note.
check_truthy checks the scrutinee with no expectation first, so a dyn
value takes the truthy path and everything else takes the strict one. A
refusal on that second path is re-checked with the old want:Bool rather
than reported from the bare check, because a bare integer or float
literal, or a bare None, answers "what type is this" differently than
"is this a bool" — check.ml's own arms only give the nicer sentence
("expected bool, found the integer literal 5", "expected bool, found
None") when asked the second way, and that sentence is preserved exactly,
letter for letter, against what a typed if already said.
test/programs/dyn-if-truthy.flan surveys every falsey and truthy case —
nil, false, true, 0, a nonzero number, an empty and nonempty string, an
empty and nonempty vec, an empty and nonempty map, a keyword — through
if, when, cond, and, or, not and while, with real output pinned in
test_acceptance.ml across LLVM, -O0 and --x86. test_flan.ml covers the
checker side directly: a typed if still takes a bare bool and still
refuses a non-bool scrutinee and a bare None with their original
messages, a dyn if/not/while/when/cond/and/or all accept a non-bool dyn
condition. test/dyn_ops.c gets a matching set of direct calls to
flan_dyn_truthy, keeping the header's own contract with the C side.
(Some nil) at an already-(Option T) want reported expect's bare-T
sentence instead of its own: checking the argument against inner's
element type routed a literal nil through expect's "wrap the type in
Option" refusal before Some's own is_nil_lit guard ever ran, and the
advice was nonsense there — the type already is one. The argument is
now checked with no want when it is syntactically nil, which is what a
bare nil resolves against on its own, so it arrives at Some's own
check still dyn and still nil.
no_fallback_slots cannot see the slot unbox_option mints: %dx/%ax are
the pool-ran-dry fallback for a temporary root_plan counted, and a
named local root_plan never counted emits neither mark. The comment
where nil-option.flan and some-nil.flan were added to that list said
otherwise; corrected to say what the check does and does not cover,
and to record that the slot was verified by reading the IR directly
instead — flan.unbox-opt's dyn slot is pushed, flan.as-dyn's is a
plain alloca correctly, since its element is always scalar there.
dyn_offsets falls through Option/Vec/Map with no arm of its own,
correct today only because Check.hidden_dyn refuses a dyn inside any
of them at every storage site first. Commented at the fallthrough,
naming hidden_dyn as the gate and the typed-container view (M2 item 3,
in review now) as the kind of change that could relax it for Vec/Map
without anything here pointing back.
Both directions of the boundary go through expect, the way every other
dyn crossing does. A dyn's tag decides which case an (Option T) becomes
on the way in; an Option's own tag decides nil or a boxed payload on the
way out. box_option/unbox_option build the same If-over-a-tag shape get
and map-remove already build for the same reason, reading an Option's
tag and payload with the raw Field access Render's structural printer
already uses — nothing new for either backend to lower. A literal
Some/None skips the runtime check entirely, since the checker already
knows which case it is.
A bare T has no None to become. The literal nil the checker can see is
refused right there, at compile time, in expect itself — the author's
decision to do both halves rather than settle for the runtime trap
alone. Everything one step removed from the syntax — a dyn that only
turns out to be nil once the program runs — reaches flan_dyn_need_i64's
existing DynType trap, unchanged; there is no dataflow in this checker
for it to be otherwise (see "Ownership tracking repealed").
(Some nil) is refused the same way: the literal at compile time, with a
message saying why nil and None would collide; a dyn that turns out to
be nil only at run time through the new flan_dyn_need_not_nil, which
traps by the same route flan_dyn_need_i64 does.
(Option (Option T)) does not cross either direction — boxing Some of an
inner None would box it as nil, indistinguishable from the outer None,
the same ambiguity (Some nil) is refused for. The type itself stays
legal on the typed side; only the crossing does not exist for it.
(Option dyn) needs no case of its own in the boundary code — the
payload is already dyn, so box_option/unbox_option treat it as the
identity — but it is not yet a value a program can hold anywhere. The
per-type-descriptor pass (M2 item 2) refuses it at every storage site
today, the same way it refuses (Vec dyn), because a struct's dyn fields
are marked by byte offsets and (Option dyn)'s payload has none. Item 4
does not lift that gate; it only makes the boundary already correct for
the day items 2/3 do.
expect grew a ctx parameter to build the fresh slot the two new
crossings need — every call site threaded through, one context
mismatch caught and fixed in check_fn's tail-expression case along the
way. var's None case grew a direct Dyn arm: None at a dyn want is nil
outright, with nothing to build.
nil-option.flan carries the crossings that succeed and ends on the
bare-T trap; some-nil.flan is (Some nil)'s run-time half, kept in its
own file the way dyn-boundary.flan is one trap per program. Both are
in no_fallback_slots and test_sanitize.ml: the new dyn temporary
unbox_option's tag test mints is rooted, and reads its Option's tag and
payload through ASan clean, --sanitize matching the unsanitized run
byte for byte.
The foreign boundary walks two questions, and they are not the same question:
one counts a dyn reachable only through a further pointer, the other counts
any dyn at all. They shared a [seen] set across the crossing between them, so
a type name marked visited on the way down was pruned from the walk below the
pointer — and the shape that hits it is a struct with a pointer to its own
name. [(defstruct Node [next (Ptr Node) x dyn])] at a [(Ptr Node)] parameter
was accepted, while exactly the same thing unrolled into two types was
refused, which is the tell. A cycle of two, [A {b (Ptr B), x dyn}] and
[B {a (Ptr A)}], went the same way.
C hung a malloc'd node off [next], put a dyn in it, and Flan allocated a
hundred thousand times: heap-use-after-free in flan_dyn_tag out of c_peek,
freed by gc_sweep out of flan_gc_collect. The checker accepted it and the
collector freed a live value, which is the one failure this boundary exists to
prevent.
The set does not travel across the crossing now. It still terminates — each
walk's own set guards its own recursion over names, and a crossing starts a
separate finite walk — and it does not start refusing a recursive shape with
no dyn under it, which has rows of its own here because that is the way a fix
like this goes wrong.
Neither recursive shape had a row. That gap is why it took four passes over
this code to surface, so both are pinned now, alongside the two acceptances
that say termination held. And the three return rows stop sharing one needle:
each names its own type, because three rows against the common half of one
sentence would all pass on a message that named the wrong shape.
dune test --force: green, 0 failures.
Three shapes hand a dyn word into memory nothing roots, and the check written
last round caught one of them. A (Ptr S) return was refused; (Ptr (Ptr S))
and (Ptr [S]) were not, because the predicate it asked was dyn_anywhere, which
had just been taught to stop at a pointer. Stopping there is right — a
pointer is a view of storage something else roots, and that is what lets a
(Vec (Ptr Cond)) be written — but it is the wrong question at a boundary where
nothing roots the far side at all. So there are two predicates now.
dyn_anywhere is the storage-shaped one and is unchanged; dyn_through follows
pointers and slices, and only the foreign boundary asks it. The second of
those shapes matters more than it looks: (Ptr [S]) is what shim.ml's own
advice tells people to write when C returns an aggregate.
Neither was a regression. At the commit before last there was no walk over
the externs at all and all three were accepted; what landed was one level of
a check that wanted to be recursive.
And the direction. The refusal was justified by what C hands back and then
applied to parameters as well, which made the ordinary read-only borrow
unexpressible: a foreign parameter receives the address of a *place*, and a
place is a frame slot, a global or an array inside one, every one of them
rooted with its descriptor and marked for the whole call. So (Ptr S) and [S]
as parameters are borrows and stay writable. One level down the storage is
C's again — a (Ptr (Ptr S)) parameter is an out-parameter and what C writes
into it is a pointer of C's own — so the parameter question is asked below the
outermost level and the return question is asked from the top.
Evidence, all three shapes with the check lifted, ASan: heap-use-after-free in
flan_dyn_tag, freed by gc_sweep out of flan_gc_collect, allocated by the Flan
frame that stored it. With the check back, all three refused by name. And
the allowed direction proved rather than assumed: a C read-only borrow of a
(Ptr S) and a [S] slice argument, called before and after twenty thousand
allocations, dyn fields intact and clean under ASan.
A Tast.extern carries its declare's location now, so these refusals point at
the line rather than at <unknown>:0:0.
Last, the one arm of the saturation that did not saturate: a negative array
length reached the multiplication as a small number, which is the exact shape
the cap must never be handed. Out of range in either direction saturates.
dune test --force: green, 0 failures.
A descriptor's symbol was the type's printed form with every character an
assembler would refuse replaced by a dot, and the table was keyed by that.
The mangle is many-to-one — a Flan name may hold -, +, *, ? and / — so row-a
and row+a were one entry, the second of them was pushed with the first's
descriptor, and the collector read at another type's offsets: past the end of
the object when the first was the larger, and never where the second's dyn
actually sat. ASan named it, a stack-buffer-overflow inside gc_mark_all. It
is the same corruption root_plan pools its temporaries to avoid, arriving
through the name rather than through the supply, which is a lesson about where
identity lives: the table is keyed by Types.to_string now, which is an
identity, and the symbol carries a counter so two types cannot collide however
they mangle. dyn-struct.flan grows the pair, held live across the churn, and
an acceptance assertion asks the emitter directly how many descriptors it
wrote under that label — two, or the two are sharing one. That assertion is
the half with teeth: whether an overread off the end of a frame slot lands on
anything is luck, and the run's own output was not red under the defect.
The cap on a flattened array's offsets was bypassable by the thing it was
meant to stop. [4611686018427387904 S] wrapped the multiplication negative,
so the test read as under the cap, the declaration was accepted, and the
emitter then sat building the offset list until something killed it. A
refusal that overflows into an acceptance is worse than no refusal. The count
saturates at one past the cap now and the message says more-than rather than a
figure that came out of a wrap.
dyn_ops.c's second assertion had no teeth: a marker never writes through a
root, so "the word at a non-dyn offset is untouched" passed under any marker
at all. What discriminates offset-driven from word-driven is a dyn word the
descriptor leaves out, holding five hundred objects, that must NOT survive —
and it is checked by adding its offset to the table and watching the line go
red.
dyn_anywhere descended through Ptr and Slice, so (Vec (Ptr Cond)) was refused
with a sentence about a dyn inside a type whose storage contains none. A
vector of pointers to condition structs is an ordinary thing to write. It
stops at a pointer now, which is the line hidden_dyn already took for a bare
(Ptr S) and the line the whole argument rests on: a pointer is a view of
storage something else roots.
Which leaves the one honest hole, and it is named at the boundary where it
opens rather than left in a comment. Storage C hands back was never rooted
and never will be, so a (Ptr S) crossing a declare with a dyn anywhere under S
is refused by name — the same sentence a bare dyn already gets there, one
level down.
dune test --force: green, 0 failures. dyn-struct.flan clean under ASan and
UBSan and identical at -O2, -O0 and --x86.
The crux was never where to put a descriptor; it was how an instance finds
one. A bare struct on the stack has no header to hang a pointer off, and
giving it one would change the layout C interop agrees on, change the stride
of an array and change what embedding a struct in another costs. So it has
none. The instance never carries a pointer to its type and the collector
never derives one from the bytes: the pairing of an address with a descriptor
is made at the *push*, by the code that put the value there and therefore
knows its static type. That is the same trick the shadow stack has always
used, and it makes the stack case the easy one rather than the impossible one.
A descriptor is the size of an instance, a count, and a table of byte offsets,
emitted once per type as private static data. Flattened, not a graph — a
struct held by value contributes its offsets shifted by where it sits, and a
fixed array contributes its element's once per element — so nesting costs
nothing at run time and there is no recursion in the marker. The offsets of a
big array would be a big table, and that is capped with a sentence rather than
half of the repeat form item 3 will bring.
Four places a value of such a type can live, and all four are rooted: a frame
slot, a global, the temporary a call's by-value return is spilled into, and
the slot a condition that is not a place is evaluated into. The last two are
new and are the ones that were not obvious. A callee roots its dyn words and
pops them in its epilogue, so between the return and the caller's store the
only copy is a register, which a collector that finds its roots by address
cannot see; the same hole was open for a Flan call answering a bare dyn and is
closed here too. And a condition crosses as a pointer into the signalling
frame while a handler allocates, which is exactly what the original refusal
said could not be made safe.
dyn_roots grows into root_plan and both backends read it, which is what the
older note about one counter deciding both ends was always for. The aggregate
temporaries are pooled by type rather than handed out in mint order: a
positional supply that drifted would pair an address with another type's
descriptor, and marking arbitrary offsets off a base is corruption where a
missed root is only a bug. Pooled, the worst a drift can do is run out.
What is still refused is a dyn no static offset can reach — inside a typed
container, in a data type's payload or a union's members where the cases
overlay, or under an Option where the payload exists only beneath the tag.
A (Ptr S) and a [S] are deliberately not on that list: neither owns storage,
and the only storage this compiler hands out for such a type is a frame slot,
a global or a fixed array in one, all of them already rooted. That is what
lets a handler clause take its (Ptr Cond) and read a dyn payload.
test/programs/dyn-struct.flan is the evidence. It runs forty thousand rows
past flan_dyn.c's one-megabyte floor, so marks and sweeps really happen, and
it holds live values through them in all four places at once. It has teeth:
with the descriptor walk stubbed out of the marker, the kept vector's length
comes back 24 instead of 628 and its first element is a stale word. Clean
under ASan and UBSan, same output at -O2, -O0 and --x86. dyn_ops.c grows an
aggregate-root mode so the runtime half can be wrong on its own, with a
header word holding a bit pattern that looks boxed and is not a dyn slot.
--no-gc still refuses, and had to be told how: a struct with a dyn field is a
collected value even when no expression in the program ever has the type dyn,
because a zeroed one still has a word the collector is asked to mark.
dune test --force: green, 0 failures across every suite.
check_struct's comment claimed a nonempty map is never mistaken for a
struct literal; is_struct_map (parse.ml) says otherwise for any size of
.field-first map, not just {} -- (g {.x 1}) on a function g still reported
bare "unknown struct g" with none of the help {} gets. Extended the
helpful message to that shape too rather than only fixing the comment,
since it is the same trap one shape over. The two shapes need different
advice, not the same one reworded: an empty map can be bound to a variable
and passed as an ordinary argument, (let [m {}] (g m)); a .field-keyed one
cannot, because expr itself refuses a bare {.field v} outside a
struct-literal position, so there is no let-binding that rescues it. The
message for that shape says so instead of repeating advice that would not
work.
Watchdog.is_dying is gone. The reviewer's own probe settled it: forcing
watchdog.ml's dying to flush_all and Unix._exit 2 directly, bypassing
at_exit entirely, gives the same exit 2 with no shared flag, no LIFO
assumption, and no window for a raise between setting a flag and exiting
to leave failures recorded and the process at exit 0. test_acceptance.exe
has no other at_exit registration (grep confirms), so nothing depends on
the watchdog path running through that chain.
test_acceptance.ml's own comment claimed calling exit inside an at_exit
handler recurses through do_at_exit. Checked directly against this
compiler (OCaml 5.2.0): it does not -- each handler gets a run-once guard
since 4.14, two stacked handlers with the inner one calling exit 7 both
ran exactly once and exited 7. Unix._exit is kept, but for the true
reason: it is not about correctness, it is about being the last word --
_exit terminates immediately and skips whatever the rest of the at_exit
chain would otherwise still do, so a handler this file grows later cannot
change the outcome underneath this one.
test_flan.ml's loose needle "keyword/value pairs" matched both the
odd-number-of-forms message and the wrong-key message; tightened to
"keyword/value pairs — found", which only the row's actual message
contains.
And {:where 1} as a defn's whole single-form dyn-map body now refuses,
where {:a 1} and {} in the same position do not -- :where is peeled
unconditionally, with no single-form exception, because that is what lets
it catch a moved closing paren leaving a stray predicate as ordinary body
code. Deliberate, and now pinned, so the next edit to this arm has to
notice it is choosing to narrow the language again rather than finding out
from a bug report.
Verified every case by compiling, including the two-handler exit(7) probe
run directly against this OCaml. dune test --force: exit 0, clean grep for
FAIL and Fatal error.
F3: the unknown-struct-vs-function message overgeneralized a one-case
parser quirk into a language rule that does not exist. (g {:a 1}) compiles
fine -- only the empty map is stolen by is_struct_map's Map [] -> true.
The message now names {} specifically and says why: it is read as the
zero-field struct literal, not "a map literal cannot be passed as an
argument".
F2: the ordering refusal named machine numbers only, when Types.is_comparable
also admits enums and enum-compare.flan orders one with (< k :mid). Both
messages -- equality and ordering -- now name every type each actually
covers.
F1 and F5, together, since both live in the same parse.ml arm: the
rest = [] carve-out that let a single-form dyn map body through
reintroduced the exact bug the earlier fix was for. (defn mx [a $t b $t]
$t {:where (ordered? $t)}) -- a :where clause with nothing after it, what
a moved closing paren produces -- no longer matched, fell through to expr,
and printed "unknown function ordered?" from inside what was meant as a
predicate. A :where map is peeled unconditionally again, whether or not
anything follows it; every other keyword, plus the empty map and any
non-keyword key, is still peeled only when the body has more after it, so
a real single-form dyn map body is still left alone. The comment
justifying the discarded-map refusal claimed a map literal has no side
effects; it does -- {:a (println "hi")} prints, confirmed by running it --
so the reasoning now names the actual problem, a value going unused, not
a false claim about purity.
Closing that F1 hole this way opened two more, both traced by rebuilding
the pre-fix parse.ml and diffing real compiler output rather than
reasoning about it: {.x 1} at a defn body's head used to get its own
message, "a bare map is not an expression", and briefly started getting
"a constraint map is keyword/value pairs" instead, because the broadened
guard no longer required a keyword and started catching the struct-field
shape too. Excluded explicitly, with a comment saying why, so expr's
dedicated diagnostic fires again. And (defn main [] i32 {} 0) and
(defn main [] i32 {"a" 1} 0) -- the empty map and the non-keyword-keyed
map, the two siblings a keyword-only guard could never have caught --
now get their own refusals alongside the keyword case, each pinned with
a rejects_check row, along with the where-with-no-body case and the two
legitimate single-form bodies that must keep compiling.
F6: the acceptance runner's tail already caught every failure on every
path through the binary -- there is no skip branch that bypasses it, and
the no-clang branch runs zero rows -- so last round's at_exit guard and
the FIX.org note both overstated what was broken. Both now say what was
actually true: the exit status was already trustworthy, the guard is
insurance against a future case leaving past the tail instead of through
it, and the other nine test binaries were already sound the same way.
The guard also had a real bug of its own: forcing exit 1 whenever
failures was nonzero would stomp the watchdog's own exit 2 if a hang
followed a few already-failed rows, since Stdlib.exit runs at_exit
handlers LIFO. watchdog.ml now flags when it is the one unwinding, and
test_acceptance.ml's guard defers to it -- shared state for a single
caller, justified by there being no other way for one at_exit handler to
know a sibling handler is already mid-exit with a code of its own to
protect.
Verified every case in this commit by compiling and, where it mattered,
running the actual program -- not by inspecting the arm and assuming.
dune test --force: exit 0, clean grep for FAIL and Fatal error.
Constraints parsing peeled a body-leading map only when its first key was
literally :where; any other keyword fell through to the body, so a typo'd
key surfaced as a baffling error from inside what was meant as a predicate
and a stray map at body start compiled away silently. Any keyword-first map
is read as a constraint map now, but only when something follows it in the
body — a single-form map body is a real dyn value and not a discarded
statement, so that case is left alone.
An empty map literal still parses as a struct literal, (P {}) still meaning
the zero struct for a real struct name — the parser has no symbol table to
tell (take {}) apart from it at that point. check.ml now catches the case
where the name turns out to be a known function instead and says so, rather
than "unknown struct take".
flan_dyn.c's tag comment still said 6 and 7 were free; keywords and maps
took 4 and a kind field under BOX_OBJ, not new top-level tags, so 5, 6 and 7
are what is actually open for the interop handle. NEXT.md and json.flan both
still pointed at test/programs/arena-edn.flan, gone since edn/read stopped
taking an allocator; both now point at what replaced it.
flan_rt.c's flan_str_eq comment claimed the empty string literal was a
hypothetical null-pointer string; it isn't, its address is an interned
symbol's. The real case the zero-length guard exists for is a zero-length
container converted to a string. check.ml's ordering refusal said a string
has no comparison at all, which stopped being true when typed = and !=
grew strings in daed039 — split the message so an equality refusal and an
ordering refusal say the right noun, and updated the pinned rejects_check
rows to match. string-eq.flan gained the row the fast path most wants
tested, a slice against the prefix it was cut from sharing a base pointer at
different lengths, plus a != row at equal length with differing bytes;
acceptance now carries the real output, captured by running the program on
all three lanes. x86.ml's xor-1 comment now names the 0/1 return contract as
a requirement flan_str_eq must hold, not an incidental fact. SPIKE-DUPLICITY
now says plainly that its equality-and-ordering argument landed in daed039
and marks its transcript as the historical state that argument was made
against. FIX.org ticks M2 queue item 5.
And the acceptance runner: the tail check that turns a nonzero failure count
into exit 1 was already there and already fired — a fresh build with one row
broken already exited 1 before anything here changed. What wasn't proven is
that every path through the file's clang/wasmtime/raylib/lldb probes still
reaches that tail rather than skipping past rows that already failed. An
at_exit guard now closes that class regardless of which path the process
leaves by, flushing stdout first so a failing run's FAIL lines survive
Unix._exit rather than being dropped from the buffer. Verified both
directions with a deliberately broken row: dune test exits nonzero and the
log still carries the FAIL line and the failure count; restored, the same
run is exit 0 with nothing printed but green summaries. The other test
binaries were checked for the same gap and none have it — each gates its
own exit on a single failures ref that the tail already reads.
Every mark it left was an addition, and addition commutes, so a backend that
ran the defers outermost-first produced byte-identical output and the row that
was supposed to be watching the order could not have told. The claim was in the
comments and not in the numbers. cleanup.flan already had the device for this —
a shift rather than a sum — so the log here is a digit trace now, and the two
frames under a catch read 12 where a wrong order reads 21.
Rewriting the trace made room for the three behaviours that worked and nothing
pinned. A return inside a clause is an ordinary return from the function that
wrote the form, because that is where a clause runs: it leaves through the
function's own exit, runs the defer registered there after the two the unwind
already ran, and leaves the handler stack empty behind it, which the bare
signal that follows in main is the check on. A defer inside a clause is refused
for the reason every nested form is refused one. And a handler-case inside a
defer works, because a defer may not start a transfer that leaves it and this
one begins and ends its own.
The program is registered with the sanitizers, where the interesting failure is
not the heap but a handler or restart frame left on a stack pointing into an
alloca that has gone — an output comparison cannot see that until something
much later calls through it. It is clean; it was also leaking sixteen bytes out
of the vector main allocates to prove the allocator context came back, which is
the test's own litter and is freed now.
docs/PORTING.md ranked handler-case as one site handler-bind covers. It still
is one site, and handler-bind still covers it, but it is no longer the closer
translation: a catch block is assumed everywhere it is written to see the
locals around it, and only the clause that runs at the form does.
The handler clauses are lifted left to right rather than by List.map, whose
order is unspecified. Each lift names itself after the count already on the
list, so an order nobody chose would number the clauses of one handler-bind
differently between builds, and those names go into a redefinition module.
The unwinding handler, which spec-conditions.md named and left unwritten while
it asked whether the thing should be a macro over the two operators that were
already here. It should. (handler-case B [(T [c] A)]) is checked as
(restart-case (handler-bind [(T [c] (invoke-restart 'R c))] B) (R [c T] A))
with R a name the form makes up for itself, which is Common Lisp's own
definition of the operator and means neither backend needed a line.
What that buys is not economy, it is the correctness of the parts nobody can
see. The defers between the signal and the form run, and the allocator a
with-allocator rebound is put back, because a transfer already does both for
every frame it leaves. The body and every clause agree on one type, because a
restart-case's body and clauses already do, and a clause that disagrees is
refused with the same message an if with disagreeing arms gets. A condition no
clause lists installs no frame that matches it and goes on outward untouched.
A clause sees the establishing function's locals, which a handler-bind clause
cannot, because a restart clause runs where it was written.
The body comes first and the clauses after it, the opposite of handler-bind's
order: one reads as something put around a body and the other as a body with
answers hung off the end of it. The restart the two halves meet over is named
after the function and numbered within it, and it has to be unique per form,
because two nested handler-cases sharing a name would have the inner frame
shadow the outer one and land a condition at the wrong place.
The refusals name handler-case rather than the machinery underneath it, which
is why check_handler_bind and the restart clauses now take the word the reader
wrote. A break loop entered under a handler-case still lists the made-up
restart, and taking it there is refused loudly rather than answered wrongly;
hiding it would mean a field in a frame layout spelled out in three files.
The survey program runs the same under LLVM, at -O0 and under --x86: normal
completion, a caught condition, one nobody listed passing through with the body
carrying on, both defers on the way out, the two nestings against handler-bind,
a clause that signals and is caught outside the form it belongs to, and a
with-allocator whose restore is on the transfer path.
Types.is_equatable splits from is_comparable: a string answers equal?
now, bytewise, but still answers no to ordered? — there is no collation
the language has picked, so < and friends keep the refusal they had.
The comparison itself is one new runtime entry point, flan_str_eq
(runtime/flan_rt.c), length-mismatch and same-pointer fast paths ahead
of the memcmp, called identically from both backends: emit.ml pulls a
string's ptr and length out of the %slice SSA value and calls it
directly in the Eq/Ne arm; x86.ml adds an arm ahead of the generic
scalar comparison that reaches it through call_native, flipping the
answer for != the same way Not already flips a bool.
test_flan.ml covers the checker side directly and through a generic
instantiated at string, including the two different ways ordered? and
equal? fail at that type. test/programs/string-eq.flan is the survey
program — same pointer, differing lengths, equal content at distinct
addresses (a literal against a fresh heap string), a difference in the
last byte, and the empty-string cases — with acceptance rows for LLVM,
-O0 and --x86 in test_acceptance.ml.
The dyn runtime gets a map object and an interned keyword, alongside the
vec it already had. {:a 1 :b s} is a map literal wherever a struct
literal isn't — the parser tells the two apart by whether the first form
in the braces is a .field symbol — and a bracket literal builds the
runtime's own vec rather than a typed array wherever a dyn is wanted, which
is what lets a map literal's values nest arrays and maps freely. get, put,
len and has-key? all learn a dyn-map arm alongside the typed-map one they
already had, and (keyword s) builds the same interned value a :foo literal
does, for a name that only exists at run time. nil is now a literal, the
dyn absence value that get answers for a key a map does not hold.
On the runtime side, flan_dyn.c gets an OBJ_MAP that shares the vec's
storage arm and doubles its accounting, a linear-scan intern table for
keywords that makes equality an identity compare, and structural map
equality by lookup rather than position. The marker traces a map's
interleaved keys and values the same way it already traced a vec.
edn/read and its callers move off the old (Option Value) union entirely:
a document is plain dyn now, sets are dyn maps to true, and arena-edn.flan
is retired along with the union it demonstrated. The acceptance suite's
edn-read and json rows were recaptured against the new shape, and a new
dyn-map.flan program exercises the map and keyword operations end to end,
including a 200k-iteration churn loop against a rooted map that runs
GC for real, across the LLVM, -O0 and x86 rows, and under the sanitizer.
Keywords are dyn everywhere an enum isn't expected, which changed what a
couple of existing checker tests actually see refused; both were updated
to the sentence the checker gives now rather than the one it used to.
The condition payload was already refused on exactly this ground; the plain
struct got the same shape past the checker. The audit's probe showed the
emitted program rooting the temporary, popping it at ret, and leaving the
field's vec reachable only through arena memory the marker never walks —
a use-after-free on a timer. Lifted with milestone 2's per-type
descriptors, alongside the condition's.
x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
The promise is that this program carries no collector, and the way to keep it is
to refuse every dyn rather than to emit a different program: a dyn value is one
the runtime allocates and the collector owns, and there is no smaller version to
fall back to. So it runs between checking and emission, answers unit or raises,
and hands the very same program on. Emit has no field to branch on and is told
nothing.
That is what makes the byte-identity claim true rather than approximate, and it
is tested by compiling three annotated programs twice and comparing the text. A
field, a mode, or a comment that mentioned the flag would break it on something
incidental, a long way from anything to do with dyn.
Every site is named, the way the global cycle refusal names the whole ring: a
reader who has to annotate their program wants the list, not the first one and
then another compile. Globals and signatures as well as body values -- the two
files it is tested against report nine sites each, and the floors are set under
that so an added line does not fail the test and a pass that named one site and
stopped would.
The four programs run at -O2 and -O0. dyn-boundary is asserted on its exit
status as well as its output, because the boundary is only interesting in that
it can fail and a test that showed it working would be testing the easy half.
The x86 survey skips them by name: a REFUSED there means a node that backend has
stopped lowering, which is a regression, and this is the opposite -- a lane that
has not started. Take a name off llvmonly when the lowering arrives and the
survey will say whether it works. 128 match, 0 differ, 0 refused.
Checked while writing these: a dyn function with an early return pops its roots
on both paths, and one with a defer pops on the transfer path too.
The type itself, the ABI its operations call into, and the one decision the
feature could not avoid: (defn f [x y]) is one parameter or two, and which one
depends on whether y names a type.
Parse does not decide it. That lookup is the one its defn comment records being
removed for being wrong twice in one day -- the set of type names is incomplete
at parse time by construction, and macros generating definitions is what
widened the failure. So the vector is carried undecided, as Ast.pitems, and
paired in Check, after every file is loaded, every macro expanded and every
header imported. The set is complete there. It is not complete across time, and
the comment says so: a defstruct written later changes a signature with no edit
to the function.
The return slot stays mandatory and dyn is written out in it. The ambiguity
there has no syntactic resolution at all -- a capitalised head in a list is both
a type application and a struct literal -- so the third state the parameters
needed does not exist for the return type, and ret = None goes on meaning Unit.
What the feature costs, and what is taken back: a slot with no type used to be a
syntax error, so a mistyped type now reads as an extra parameter with no
diagnostic. A name within one edit of a type's gets the resolver's own
did-you-mean, and an unknown capitalised name is reported as the unknown type it
is -- not one parameter in the corpus is capitalised. A lowercase name
resembling no type is the feature working, and is the residual.
The x86 backend refuses dyn by name; both callers already name --llvm, and the
daemon takes that backend by default, so this is the first thing a user of dyn
sees. The JS dialect refuses it too, for the opposite reason -- every value
there is already dynamic and what is missing is only the lowering.
runtime/flan_dyn.h is the fixed ABI. flan_dyn_stub.c stands in until the real
collector lands and says in its header that it verifies nothing about roots.
The !-means-mutates convention distinguished nothing — there is no
immutable counterpart to contrast with — so every mutating name drops
the mark: sort, sort-by, sort-bytes, swap, reverse, append, append-i64,
append-f64, encode-rune, split-next, map-remove, map-next, and the test
helpers beside them. Two could not simply shed it: map! is map-in-place,
because map is the into transform's word and means the non-mutating
thing; put! is put-at, because put is the Map builtin. The ?-means-asks
convention stays. Dated records keep the old spellings; watch.clj's
reset-spies! and the other Clojure names are not ours to rename.
The x86 backend ran initialisers from .init_array and the LLVM one refused
them by name, so (defvar frame Allocator (arena-new 262144)) — which the
author kept writing — was a program on one backend and an error on the other.
A rule that holds on one backend and not the other is not a rule.
The checker lifts a computed initialiser into a function of its own and the
global's initialiser becomes the call. That is what gives it a frame, which is
the bug underneath the feature: a `let` or a `match` in an initialiser indexed
a slot array of length zero and took the x86 emitter down with an uncaught
Invalid_argument.
Both backends call the lifted initialisers from main, after flan_rt_init and
before a line of the program's own code — Odin's __$startup_runtime shape, not
a constructor, so the runtime is up and the order is the compiler's to choose.
x86 keeps .init_array for one thing only, and it is named: writing the
constant image this backend has no folder for, which is standing in for the
other backend's object image rather than for a program.
The computed globals are sorted by what they read, transitively through the
functions they call, so a global written above the one it reads works and a
ring is refused with every name in it. A reload still re-runs nothing: a new
global with a computed initialiser starts as ZII on both backends.
The refusal that lived in x86.ml is now the checker's and is narrower. Nothing
can escape an initialiser — the handler and restart stacks are empty and every
frame it pushes it also pops — so what is refused is a signal or an
invoke-restart with no handler-bind or restart-case around it, which is inert
by construction. A restart-case inside one is ordinary code, which is what
makes (defvar data (Vec u8) (slurp "level.edn")) an ordinary program.
Three refusals go with the premise they rested on: a container global with a
computed initialiser, a union member in a defvar, and a data type case in one.
A defconst is untouched and keeps all three.
One change here is not about any of that. sand.flan carried an unfinished
line — (defvar game-data (embed (with-allocator frame ))), which parses as a
declaration whose type is (embed ...) — so the checker refused the file and
`dune test` was red at the tip of dev-loop before a line of this landed,
verified by stashing this work and rebuilding. It is commented out rather than
guessed at: the arena above it is the half that works, and what the global
should read is the author's to decide.
`arena-new` is a builtin, so it is in no program's symbol table and `defs`
never mentioned it — which made the editor answer "the running program
defines no arena-new" about a name that works. The fix is not a better
refusal: it is that the seventy-eight names the checker answers without
being told are now in the reply, under a kind of their own.
check.ml carries the table, beside the arms it describes, because a table
in another file drifts from them with nothing said. test_flan reads both
the arms and the table and fails on either having a name the other does
not, in both directions — there is no reflecting over a match, so it reads
the source.
Each entry is a signature in `signature_of_fn`'s shape and one line. The
arms that do not have one shape say what is true instead of pretending:
`?` for an argument that may be left out, `|` for the types an arm really
takes, and the checker's own predicate names for the type-directed ones.
The three user-allocator names carry a bare name and no bracket list,
because they are refused wherever they are written.
`defs` grows a fifth string for the prose, and builtins are appended last
so a completion table does not bury the names being worked on. A defn has
no docstring to put there and does not get one here: the Tast keeps none,
and that is a different piece of work.
The editor reads the kind, not a special case. `flan-doc--where` and the
xref backend both answer before their empty-location branch, because a
global's missing location and a builtin's absent one are different facts
and only one of them is about the daemon.
The members of a defenum are i32 at run time, but the reader hands the parser
an int64, so a value too large for the type arrived looking ordinary: truncated
by the x86 backend, malformed in the LLVM IR, and -- the reason this is a
correctness hole and not a nicety -- invisible to the duplicate-value rule
sitting right below it. That rule compares int64s, so (defenum E [A 0
B 4294967296]) passed it: the two differ as int64 and are both 0 as i32, and
the one check written to catch two names for one number waved through exactly
the case it exists for.
Each value is now checked where it is resolved, which is before the collision
scan runs, so the scan compares the numbers the program will actually have. A
value that does not fit is refused rather than quietly made to fit, naming the
member, its enum, and the value, with a different sentence for a value that was
written and one autoincrement walked into -- nothing in the source wrote
2147483648, so the refusal has to say where it came from before it can say it
is wrong.
The check is bound with a let rather than inlined into the cons, and that is
load-bearing: OCaml leaves :: operand order unspecified and takes the tail
first, so an inlined check would run after the recursive Int64.add and let
(defenum E [A 9223372036854775807 B]) wrap to min_int and refuse B for a number
in no one's source. Bound first, A is refused and the wrap is unreachable.
The parser is the only place this needs to happen: Parse.decl is the sole
constructor of Ast.Defenum's member values, and Load only re-qualifies the
enum's name.
Explicit-duplicate aliasing is untouched; that rule is deliberate.
The condition was checked before in_loop, outside the diff that catches a
loop giving away what the next iteration needs. Emit puts the condition in
the loop header, so it runs again every trip: a condition that frees a
local freed it once per trip. It is now diffed against the same dead set,
with its own reason. A dotimes count and a loop's initial values stay
outside: those are evaluated exactly once.
The arm was written with the others and through the same deferral, and neither
the generics row in test_flan.ml nor the paragraph in BUILT.md that enumerates
what defers had it. Its placeholder is get's, for get's reason: it answers an
(Option V), so the match around it still has to check while the key is a
variable.
FIX.org carried Addr(Pfield ...) on an Option as a hole in both backends. It is
not reachable from the language: a field access goes through struct_target,
which admits a struct or a pointer to one and refuses everything else by name
with a location, so (addr (.x o)) is refused at the field and never reaches a
place. The node that failed was one the compiler built for itself.
The refusal is pinned on the bare field and on the address of one, and FIX.org
now records the finding, including the asymmetry that stays: the x86 backend
lays out an Option's tag and value as fields and the LLVM backend does not.
Neither path is reachable, so matching them would be untestable code written to
balance a road nobody drives on.
check_main raised against Loc.unknown, so both of its refusals opened with
<unknown>:0:0. env.locs is the table of where each type was declared and a
function is not in it, so the location comes from the declaration list the
caller already holds. A main that arrived without a defn keeps the unknown
span rather than being given an invented one.
A (Vec Value) where a Value may itself hold a (Vec Value) — the recursive
dynamic value an EDN reader has to answer with when nobody hands it a target
struct type — was refused five different ways, and every one of the five gave
the same reason: the container runtime is type-erased, so it copies and
releases slots bytewise and cannot reach inside a slot. A free would release
the slots and leave every block they point at stranded.
That reason is about teardown, and it does not hold for a region. free-all
never releases an individual slot; it takes the whole arena, and every block
the elements own is in it, because they came out of it. The refusals were
over-broad, and what they were guarding was never ownership — ownership
tracking is untouched here, moves are still moves, and Types.is_move_only is
the same function it was.
So the question moved rather than disappeared. It could not stay at the type,
because can-free is a capability on an allocator value and with-allocator
rebinds a dynamic variable: which tier a (vec-new) will meet is not a property
of the place its type is written. What is decided at compile time is only
whether to ask, which is a property of the element type; the answer is a
run-time branch on the allocator, one per container and never per element,
because the alternative is a walk at release and a walk at release is the
registry of destructors the frame tier's reset exists to not have. It is
emitted at every growth and not only at the construction, because ZII means a
container can exist without ever passing through (vec-new) — a case field left
out of a literal, a global that starts zeroed — and those adopt the context on
their first push.
free on such a container is refused rather than made quietly shallow. It cannot
recurse, which is the whole premise, and releasing the outer block alone would
be "I freed it" written over a program that stranded everything inside; this
runtime refuses that collapse everywhere else. The message names free-all,
which is reachable by construction. clone stays refused for a reason the region
does not dissolve, and the old message had bundled the two failures under one
sentence: what disqualifies clone is not that it copies a header — so do at and
get, and they are fine, because they promise nothing — it is that clone
allocates a new block and promises independence, and a bytewise copy hands back
elements still pointing into the original's region.
A struct or union field is admitted only where the field's container holds
owning elements, because that container can only have been built against a
region. A field holding a plain (Vec u8) stays refused: nothing would force
that one into a region, and two copies of the aggregate would be two headers
over one heap block. vec-in-struct.flan still pins that.
The epoch already covered use after free-all, including the case this makes
reachable — an inner header copied out of an arena-held element into a local
still traps, because an Allocator is a pointer and a copied-by-value one would
carry its own epoch.
arena-value.flan builds the value by hand; arena-edn.flan reads a real document
through the tokenizer, and its reader takes no allocator and names none,
because spec-memory.md already puts the allocator in the calling convention.
arena-region.flan is the branch itself: run 0 is the (Vec (Vec i32)) control
that must not trap, and runs 1 and 2 are the two ways this dies.
The refusal was on the defvar path alone, so (defconst c U (U {.i 1}))
still came back from the emitter as "a global's value must be a
compile-time constant — this one is computed", which is true and says
nothing about unions. Both kinds of global reach the same encoder, so
both get the same message.
It is decided on the checked value rather than on the declared type now,
which is what lets the one initialiser that *is* a constant through: (U
{}) is all-bytes-zero, the same value a declaration with no value gets,
and refusing it would have been telling someone to write the thing they
had written.
The name freed up by the rename now means what C means by it: the members
overlay one storage, the size is the largest of them, the alignment the
strictest, and nothing anywhere records which one was written. It serves
two things that wanted it. Binding a C header means holding the union the
library holds and reading whichever member the library's own tag says is
live -- a tag Flan cannot see, because the rule relating them is prose in
a manual. Overlaying an f32 on a u32 to look at its bits is the other,
and it is the same read.
So that read is defined rather than refused. This is the one place in the
checker where bytes win over safety on purpose, and the alternative was
not a safer language, it was no feature: type punning *is* reading the
member that was not written. The promise is the one C's implementations
make and C's standard does not -- the layout is the target's, the bytes
are the bytes, a read is a reinterpretation of them -- and what is not
promised is anything about bytes nobody wrote, where a member wider than
the one last stored reads a tail that is indeterminate exactly as a
struct's padding is. ZII narrows that to almost nothing: a union starts
all-bytes-zero unless uninit says otherwise.
uninit on one is allowed, unlike on a defdata. The refusal there was
never about garbage; it is that a tag steers, and a tag no case names
falls past every comparison in a match into a block LLVM may treat as
unreachable. An untagged union steers nothing.
Which is also why three things are refused, each for a reason that does
not expire with a milestone. No move-only member: nothing knows which
member is live, so nothing can tear one down, and unlike the struct and
defdata refusals this is not waiting on recursive teardown -- there is no
fact for teardown to read. No bool at any depth: an i1 loaded from a byte
that is neither 0 nor 1 is a value the optimiser may assume cannot exist,
and a union is the only type that can produce one. No defdata at any
depth, for the reason uninit gives, arriving the other way round. An
Option member is fine and the walk says why: its match is a tag test and
a branch, not a chain with an unreachable tail.
Two members in one literal, a match on a union, a union map key and a
member written into a global initialiser are each refused by name.
A union is a field list whose every offset is zero, so it travels as a
Tast.structure and the checker, the emitter and the x86 backend each grow
one table rather than one shape. A value is a zeroed temporary and a
store -- Set over Pfield, which every backend already has -- so there is
no new IR node and no layout rule spelled out a second time per backend.
The LLVM type is the blob clang gives a union, the DWARF is
DW_TAG_union_type with every member at zero, and the printer names the
type and does not walk it: it cannot know which member is live, and one
of them may be a pointer.
cimport can now check what it could not. A C record holding a union
member was not recorded at all, so the defstruct beside it went unchecked
rather than checked wrongly; a named union member resolves to a defunion
now and the whole record is compared field by field. The defunion itself
is compared against the header's union as a set and not in order --
every member is at offset zero, so a permuted one is the same type and
reporting it would be a finding that is not one -- while a member the
header has and Flan lacks is reported, because that is what changes the
size. A defunion against a C struct, or a defstruct against a C union,
is reported in both directions. An anonymous union member is still
skipped, and the comment now says that the gap is on the Flan side:
there is nothing to declare.
Flan's tagged sum has been spelled defunion since it landed, which was
accurate right up until the language wanted C's untagged union as well.
Both cannot be called the same thing, and the tagged one is the one with
an alternative name that says what it is: a case, its fields, and a tag
that steers which case is live is a data type, not a union.
So the form is defdata everywhere -- the parser, the AST, the checker,
both backends, the prelude's Form, the editor's font-locking and imenu,
the docs and every .flan file in the tree. The internal vocabulary moves
with it: Tast.union is Tast.data, uname is dname, the tables the checker
and the emitter keep are datas. Leaving them would have inverted the
words permanently, with surface defunion meaning one thing and
env.unions meaning the other, which is exactly the kind of drift the
comments in those files exist to prevent. What did not move is case,
variant and vfields: a tagged sum still has cases, and it still has one
live at a time.
defunion is not kept as an alias. An alias would compile the day the
untagged form lands and mean the opposite of what it used to -- the same
silent misparse that made defn's return type mandatory, and worse,
because the reader would have no reason to look. The old spelling is a
named refusal instead, parse/defunion-renamed, which says what it is now
called and that the name is reserved for something else. It fires on the
head alone, so (defunion U [A B]) -- which would otherwise have parsed
cleanly as one field A of type B -- is refused with the rest.
A program that wants to load its data once and keep it could not say so. Every
move-only global was refused where it was declared, on an argument about the
dead set being per function: two functions each freeing the same global would
be a double free nothing could see. The argument was sound and the conclusion
was too strong. It assumed a global has an owner. It does not.
Reading a move-only global is now always a borrow. Nothing may take ownership
of one, so nothing may free one, and with no owner to hand over there is no
double free left to catch. This is not a general ownership model for globals
and is not meant to grow into one: it is sound precisely because the lifetime
question that model would exist to answer has a constant answer here, the
process's. The refusal lands at the read, which is where a move would have been
recorded for a local -- passing the global to something that owns its
parameter, binding it to a local, returning it and freeing it all reach the
same place, and each is told to borrow instead, or to clone if it really wants
something of its own.
Such a global is mutable where it stands. push, put, reserve and set already
take their target through the borrow path, so a global (Vec u8) is filled and
grown in place, and the aliasing that raises is the one every Vec has:
spec-memory.md's explicit Zig/Odin contract, where a push that reallocates
invalidates a slice taken before it and the dev build's generation word traps
on the stale one. Globals get no borrow rule locals do not have, because the
hazard is not new and the trap lives on the Vec rather than on the binding.
What a move-only global may not do is carry a computed initialiser. A global's
initialiser is a link-time constant -- there is no init-at-startup path in the
LLVM backend by design, and the x86 backend that has one deliberately leaves it
out of a reload module, because re-running an initialiser wipes the live state
reloading exists to preserve. So the global starts zeroed, which for a Vec is
an empty Vec and therefore a value rather than a placeholder, and the load is
an ordinary assignment in whichever function loads it. That is also what makes
the data survive: nothing runs between one entry to main and the next, so a
re-entered main finds the global as it left it. A defconst cannot be one at
all, since a constant is not an assignable place and nothing could ever load
it; both refusals name the (defvar g (Vec u8)) that works.
The reload fixture gains a global Vec in the host and another that arrives at
run time, because that is where declaring instead of defining has teeth: a
module that defined the host's Vec would take a zeroed header of its own and
strand the block the process is still using, which a re-zeroed i64 cannot
demonstrate.
