(defmacro do-grid [[r rows c cols] & body] ...) — positional names, a [ ]
pattern wherever an argument is a vector, and & for the tail. The reading of
the list lives in Expand, below both sides that need it: Parse turns it into
the bindings a macro body opens with, and Macro checks a call against the same
reading before expanding it, so arity and shape are refused with the call's own
location rather than with the Loc.from_macro stamp every node of an expansion
carries.
The breaking half: [args] used to bind the whole argument list and now binds
the first argument. The whole list is [& args], and every defmacro in the tree
— prelude, vendor, tests, the elisp fixtures — was migrated to it. One grammar,
not a legacy mode.
DISCUSS.org's sentinel-fill idea, built as two builtins because the author
asked for both: a memset with a byte the program picks, and the fixed
DE AD BE EF pattern a hex dump reads as DEADBEEF.
Both are spelled the way (zeroed) is — the value of whatever type is
expected of them — so (set grid (filled 0xFF)) fills a place and there is
no second, place-taking form beside set.
What may be filled is numbers, and structs and fixed arrays built out of
them. Everything else is refused by name: a filled dyn is a collector root
pointing at nothing, a filled Vec header frees a wild address, a filled
slice length is a bounds check that passes, and a filled bool is an i1 to
LLVM and a whole byte to x86, which is the one divergence this feature
cannot have.
The byte fill is llvm.memset / rep stosb. The four-byte pattern cannot be
a memset on either side — the intrinsic takes one repeated i8 — so it is a
counted dword loop in emit.ml and rep stosd in x86.ml, with the pattern
bytes and their little-endian word living once, in Emit. A size that is
not a multiple of four ends on DE, DE AD, or DE AD BE.
A bare {.field v} had its refusal in Parse.expr, before any checking, so a
defn whose return type was the only place the struct's name appeared could
not build one. The refusal moves to Check: Parse builds an Ast.Bare out of
the same struct_fields the named form uses, and check_bare reads the type
name off the expectation and hands that very list to check_struct. ZII, the
unknown-field refusal and the duplicate-field refusal are therefore not
copies of the named form's rules but the named form's rules.
Braces at a dyn want are the dyn map literal and stay exactly that. A
.field-keyed brace was never part of that spelling, and at a dyn want it is
refused by name rather than given a second meaning.
(Cell 1 2) is the other half, and it is character-for-character an ordinary
call, so only the symbol table separates them. It is decided on the last arm
of named_call, after a local of function type, a generic and the function
table -- so a defclass constructor, which is a real defn, resolves above it
and is untouched. Arity is exact: ZII is what the braces do, and a positional
list cannot say which field it left out, so it is not allowed to leave one
out. The refusal names the first field it did not reach and points at the
spelling that does mean "zero the rest".
Both are gone before any backend sees them -- Tast.Make either way -- and the
three acceptance rows print the same lines to say so.
[merged_serve] waited up to ten seconds for the program to bind agent.sock
before it started [accept_loop]. The listening socket was already up, so an
editor connected fine and then heard nothing: every first request of every
session cost the whole wait when the program calls (agent/start ...) late —
sand.flan starts it after rl/init-window returns — or never.
Nothing the editor asks needs that socket. In one process a delivery is a
call into flan_agent.c, not a connect, and the two-process daemon has already
waited for the bind in [two_process] and fails if it never comes. What the
wait was for is the sentence a program with no agent deserves, and a sentence
does not have to be in front of the loop to be said. So it is a deadline the
session passes ([agent_check]) rather than a wait it does: read from the
accept loop between connections and from [serve] before each request, because
an editor holds one connection for a whole session and the loop is not
cycling while it is attached. No thread, for the reason lib/dune gives about
what the merged link does to this library's dependencies.
Delivery stays honest either way. A program that links no agent at all
refuses through [over_socket]'s ENOENT, as before. One that has the agent but
has not started it takes the module into the ring and is answered with a note
that promises the poll and not a frame: a program with no (agent/poll) in it
never installs this, and "at its next frame boundary" would be the reply that
makes a redefinition look applied when it is not. C-x C-e's five-second
timeout gets the same distinction instead of asking whether a program that
has not got to its loop yet is calling the poll in it.
And the note a parked program's delivery carries is now said once per park.
A finished program is parked, so re-evaluating while a run's output is on the
screen repeated a paragraph on every C-c C-c. The first delivery of each park
explains itself; the rest say the one line that is the claim. [rerun] clears
the flag as well as [eval] does, so a new park is a new reader.
A let binds in sequence, so binding a method's names pairwise from the
generic's reads a name it has just bound. A generic [a b] with a method
[b a] -- a swap, which is what renaming parameters most often is -- was
handed its first argument twice and could not reach its second at all;
[b c] is the same bug one step shorter. Every argument is now copied
into a temp in the unspellable ~ namespace first and every method name
bound from a temp, uniformly rather than only for the pairs that
collide, because a rule that fires on the tangled case alone is one
nobody exercises. Both shapes are in dyn-class.flan, where the values
are what is wrong rather than the types, and across all three rows.
With it, two things the descriptor fix left behind. descriptors_asm
wrote the descriptors into .rodata and a descriptor holds the address of
its own offset table, so every one of them was a relocation in a
read-only section -- a DT_TEXTREL, which ld warns about in a PIE and
refuses in a shared object, and which was warning in the new daemon
case's own output. They go in .data.rel.ro now, in both the executable
and the reload module; readelf -d on a reload module from each backend
shows no TEXTREL. And FIX.org: the stale held line for item 6, the
fourth read site of the shape tag (say_render, not just print), the
warning that a class's qualifier is the importer's alias so a
hand-written :a/point is coupled to one import's name, and the gap
flagged for the next sweep -- marking through a descriptor an x86 reload
module emitted is still unexercised.
The claim classes were built for, made against a real daemon rather than
at the session's report: dev-class.flan is compiled with one method for
one class, a second method is delivered into the live process, and the
call through the generic's cell answers with the new method's body while
the original one goes on answering. That is what a generic being exactly
one top-level name buys, and it is now pinned rather than argued.
Two things came out of writing it, neither of them about classes.
The x86 backend's redefinition module never emitted the per-type dyn
descriptors. Emit.redefinition has always emitted them, by going through
finish; the x86 twin ended at the rodata section and stopped. Nothing
had reached it, because a redefined body had to construct a struct
holding a dyn to need one, and until NoMethod there was no such struct a
compiler-written body could build. What it looks like is not a bad read
at run time but a link failure — desc_of mints a local label, the body
references it, and ld refuses the module with an undefined symbol. One
line, beside the same call in the executable path.
And the thing the test had to be written around: a dyn value answered by
eval-expr does not come back in the reply's :value at all. It renders to
the program's own stdout, which reaches a *later* reply's :output — the
dyn-global rows already read one that way and say so. So every answer
here is compared inside the expression, and what crosses the wire is a
typed 1 or 0. Left as it is; where a dyn expression's value should
surface is a question about the editor protocol, not about this lane.
The session-level test stays: it pins which names an added method
reports for installation, which is the half a daemon test cannot see.
A defclass is a named dyn map with a shape tag, and a generic function
dispatches on it two ways: CLOS's, where the dispatch value is the class
of the first argument, and Clojure's, where a body computes it. They are
one mechanism and not two — a class dispatcher is (class-of arg0) as the
dispatch function, which is what lets a method written for the class
point and one written for the value :point be the same branch.
(defclass point [x y])
(point 3 4) ; the constructor, positional
(class-of p) ; :point, or nil for anything else
(defgeneric area [self] dyn)
(defmethod area point [p] (* (get p :x) (get p :y)))
(defmulti describe [x] dyn (get x :kind))
(defmethod describe :square [s] ...)
(defmethod describe :else [s] ...)
A slot is a key in the instance's own map, so get, put and has-key? are
how one is read and written and no operation was added for any of it.
What the class adds is the tag, and the tag lives in the object's header
rather than in a reserved entry — the queue's note said a reserved key
and this departs from it, because a key would be counted by len, walked
by the renderer and compared by equality, so every instance would answer
a length one larger than its slot count and print a key nobody wrote. A
header field cannot be reached by get or put at all, so no user key can
collide with it. It costs nothing: the map arm of flan_obj's union grows
to the size the view arm already had, and sizeof(flan_obj) is unchanged.
It needs no tracing either — the tag is an interned keyword entry, which
is immortal and is not a collector object.
The tag shows up in exactly three places: class-of answers it, equality
compares it (two instances of one class compare by their slots; an
instance and a plain map with the same entries do not, which is
Clojure's answer for a record beside a map), and both renderers print it
— #point{ :x 1 :y 2}, Clojure's own spelling.
None of the four forms reaches the checker. lib/classes.ml turns the
whole declaration list into ordinary defns at the top of build_program,
the way Shim.expand already turns a declare-c into a declare plus a
defn: a class becomes its constructor, a generic becomes one function
whose body binds the dispatch value and compares it down a chain, and a
method becomes a branch of that chain. It is a pass and not a macro
because a macro sees one form and the generic's body is not decidable
until every method is in hand — a method may be written above its
generic, below it, or arrive at a reload an hour later.
That last case is why the method bodies are inlined rather than lifted.
A generic is exactly one top-level name, so adding a method to a running
program is the ordinary redefinition of one function, through the cell
every call site already goes through. session.ml names the generic
alongside the method's own declaration name for that reason. The cost,
recorded rather than hidden: a method is not separately callable and is
not a frame of its own.
A dispatch that finds no method signals NoMethod, a prelude struct
carrying the generic's name and the dispatch value that missed. A
condition and not a trap, because a miss is something a program can be
written to answer, and handler-case around the call is the shape. Its
value field is dyn, the first condition here with one; the per-type
descriptor an item-2 struct carries is what the collector reaches it by.
No restart is established at the miss, which is BoundsError's decision
taken for BoundsError's reason.
Both backends, identically: the two new runtime entry points are
declared in emit.ml and the x86 backend needs nothing, since a dyn call
is a dyn call there. Deferred and written down in FIX.org: inheritance,
multi-argument dispatch, :before/:after/:around, named-slot
construction, unknown-slot checking, and computed dispatch values.
The refusal moves to the checker: Emit.const refused a computed defconst by
name while the x86 backend ran it through the startup function behind an
.init~once. flag, like a defvar, so the two backends disagreed about the same
program. One refusal in Check.const_defconst_init ends that, and it is the only
place that can name the way through.
The accepted set is unchanged: Tast.const_init's, which is Emit.const's and the
x86 data_sym path's, plus the integer arithmetic collect's folding pass has
already turned into an Int before the initialiser is looked at.
Emit.const's two refusals become a failwith no program reaches; emit_global's
gconst || const_init loses its left half; x86 needed no edit, since it never
classified by the form. test_flan's infers probe asks Check.expression now that
a defconst can no longer wrap an arbitrary expression.
The guard flag was named .init-once.<global>, and . and - are ordinary
symbol constituents, so (defvar .init-once.x i64 7) beside a computed x
emitted the same symbol twice: the dev build died at the assembler on both
backends, and the flag's Bool was registered over the user's global in
Emit.globals so the store came out as an i1. It is .init~once.<global> now;
~ terminates a symbol in the reader, the same trick destructure~N uses.
test/programs/dev-rerun.flan carries such a global and no new printed line.
Three places said a defconst is the linker's image on one backend and a
constructor's stores on the other and a re-run reaches neither. Tast.const_init
splits on the initialiser and not on the form, so that holds only for a
constant initialiser; a computed defconst is guarded like a defvar on x86 and
refused outright by emit.ml's const. The sentences now say that, including
the divergence.
emit.ml also claimed Check.no_transfer_in_init made it impossible to leave the
guarded branch between the store and the flag. It is syntactic over the
written initialiser only: a callee can signal unhandled and take the call's
transfer edge out, leaving the flag false — which is what should happen, since
the next run retries. Read off the emitted IR for such a program.
The x86 float-Rem comment says why the dead movabs before fmod is kept, and
its mid-sentence line break is gone; math3.flan's first float-% line prints
six values, not four.
There is no SSE remainder instruction, and LLVM does not invent one: at -O0
it lowers frem to fmod or fmodf. The backend now calls those two symbols
rather than refusing the operator, which is agreement by construction rather
than a second hand-written identity that would have to get every rounding,
every signed zero and every infinity right on its own.
Rem was the only gap. emit.ml's float surface is Add, Sub, Mul, Div, Rem and
the six comparisons; x86 had everything but Rem, and its comparisons already
build LLVM's ordered predicates out of ucomis, setcc and setnp.
math3.flan grows the operator spelling beside the fmod-f32/fmod-f64 calls it
already had, through globals so the pair is not folded before either backend
sees an operator. FIX.org records the ruling the fix came from.
flan_merged_park called flan_dyn_root_reset, which emptied the collector's
root stack. The frames' roots had to go — main is left by longjmp, so they
name stack the next run overwrites — but the dyn globals' roots are on that
same stack, pushed once by the emitted main and never popped, and the park
took them with the frames.
The park is not a quiet state. It services evaluated thunks, a thunk
allocates, and an allocation collects. So a program with (defvar config dyn)
answered (get config :s) with its string before any thunk ran and with nil
after one that allocated past the heap's floor — a read of memory the sweep
had freed, answering nil by luck of what the freed words decoded as.
The emitted main now brackets its global pushes: flan_dyn_root_globals_begin
empties the stack, the pushes go on, flan_dyn_root_globals_end records how
many of them there are, and the park resets to that line instead of to zero.
Nothing between the two allocates, which is what keeps the globals from being
swept in the window where they are unrooted — and [begin] emptying the stack
rather than adding to it is what makes a re-entered main re-root the same
globals rather than push a second copy of each, which also closes the other
half: a re-run used to re-push roots over slots left dangling by the park.
Both emitters, because the dev loop's default backend is x86 and a fix in one
lowering is not a fix. A program with no dyn globals emits neither call and
its root stack still resets to empty, which is what an empty push list should
leave behind.
flan_dyn_root_pop now clamps at the globals rather than at zero. An
over-popping frame eating the globals is the one way that clamp could turn a
miscount into this same use-after-free.
Covered twice. test/dyn_ops.c's park mode is the runtime's half — a run, a
park with a collecting thunk in it, and another run, three times over,
asserting both that the global survives and that the frame's five hundred
objects do not. Under ASan the old reset reports heap-use-after-free in
flan_dyn_tag with the free in gc_sweep; under memcheck it reports 24 errors
and still prints the right answer, which is the shape of the bug. test_dev.ml
drives the whole daemon over its socket on both backends against
programs/dev-dyn-global.flan.
Not touched, and it wants a decision rather than a patch: a re-run re-enters
flan_program_main, which re-runs the lifted startup function, so every global
with a computed initialiser is reset by a re-run. That contradicts dev.ml's
own note and FIX.org item 1. It is independent of this — the roots are right
whether or not the values are re-initialised.
Nor is this the reload path. A defvar added by an evaluation gets its storage
from flan_dev_global (emit.ml's new_globals, x86.ml's counterpart) and there
is no flan_dyn_root_push anywhere on that path in either backend, so a dyn
global added to a live session is unrooted. That is a separate defect with a
separate fix, and nothing here makes it better or worse.
(at g i j) is one Tast node carrying the whole index list, so the At
arm's guard on target.ty settled level zero and nothing after it. A
global [2 [[3 i64]]] indexed twice reached a slice's element, crossed
into dyn as a view, and printed a returned frame's contents with exit
0 (ASan: stack-use-after-scope in view_box). The arm now steps each
index the way [indexed] does and demands an array at every level;
the Field and Slice arms inherit the fix by recursing into it.
(and a b) desugared to (if a b false), so it answered the last operand
only when every operand was truthy; a falsey one came back as a bare
false, where Clojure answers the falsey operand itself. It now uses the
same expansion or got in ad0f1fb -- (let [t a] (if t b t)) against or's
(let [t a] (if t t b)) -- so the operand that decided the form is the
answer, and the test is still evaluated exactly once.
The temp binding and its if now carry the operand's own loc rather than
the whole form's, which the or fix had lost: (or (vec-new i32) v) blamed
the enclosing form at 3:13 and now points at 3:18, the operand, and and's
second operand gained the same precision.
The parse pins in test_flan.ml now tie the bound name to the temp the if
tests and the bound value to the first operand, so a desugaring that
dropped the temp and wrote the operand into the arm twice no longer
passes; and has its own pin. dyn-if-truthy.flan grows the falsey-nil and
falsey-false answers, 0 and "" as truthy operands, one- and zero-operand
forms, and a printing operand that proves both the short circuit and the
single evaluation.
One behaviour that used to compile changed: with both arms of the
desugared if now holding real values, (and dyn-value typed-bool) unifies
on the typed arm and a non-bool dyn decider traps at the strict bool
boundary -- (and (box nil) some-bool) printed false and now traps, the
mirror of what (or false (box "s")) already did on dev-loop. Recorded in
FIX.org as the author's call on how check_if should join a bool arm and a
dyn arm.
The [At] arm of [permanent_root] recursed through any indexed target, so an
element of a global SLICE answered permanent the way an element of a global
ARRAY does. An array's elements are inside the global's storage; a slice's
are ptr+len pointing wherever, which can be a frame already returned — the
program that stashes (slice local 0 2) in a global slice and views an element
compiled and segfaulted with no diagnostic. The arm now recurses only when
the target's own type is an Array.
With it, the refusal/acceptance pair in test_flan.ml (one word apart) and a
view over an element of a global array in dyn-view.flan's mode 0.
The element check now runs before the lifetime check in all three container
arms: a local (Vec string) was told to make it a global, and a global
(Vec string) is refused anyway, so the advice was a dead end.
And the four strings that claimed more than the code does. flan_dyn.h
already had the honest version — a view is exactly as stale-safe as the
thing it is a view of — so the refusal message, box's comment and FIX.org
now say that instead of promising a dyn value can never dangle; a global
[i64] cut from a dead frame still passes and still reads it (ASan:
stack-use-after-scope in view_box). The element message no longer tells a
(Vec string) that string is not the case the restriction exists for.
dyn_ops.c's hand_vec comment no longer says flan_rt.c is unlinked when it
calls two of its functions; flan_rt.c said the same thing and is fixed too.
FIX.org's arena paragraph now separates the header's lifetime (compile time,
already covered) from releasing the arena under a live view: free-all traps
cleanly on the epoch, arena-destroy is a heap-use-after-free in
view_vec_check, the same gap flan_vec_check has on the typed side.
Relocation was proved sound and stayed sound — a Vec view holding the
header's own address survives a push that grows and moves it, because
there is no snapshot to invalidate. That was never the whole of the hazard.
Refusing every container into dyn outright, before this lane, meant a
dangling view was unreachable; the moment box stopped refusing, three
routes opened at once — a view returned from the function whose frame the
Vec lived in, one stashed in a dyn global and read after that frame is
gone, and one left behind when a condition transfer unwinds it. All three
are stack-use-after-return, reachable for the first time.
The rule: a typed container crosses into dyn as a view only when its own
storage is permanent — a global's. On the dynamic side Flan follows Clojure
and Common Lisp, where holding a value can never hand you garbage; treating
a view as a bare pointer and calling the lifetime the programmer's problem
is the Odin answer, and it is the wrong trade on this side of the language.
check.ml's permanent_root walks the checked expression back to its root: a
global is permanent, a field or an array element of one is permanent at the
same fixed offset, and a slice cut directly from one at the call site
inherits it — the trace is what a slice carries, and it is lost the moment
the slice is bound to a name first, so that case is refused too rather than
guessed at. Everything else answers false: a local, a parameter, a
temporary, and anything reached through a (Ptr T), because a heap-durable
pointer and a frame's own are the same type and the checker cannot tell
them apart — admitting one admits the other, which is the whole hazard this
closes. An arena-held header turns out not to be a separate case at all: an
arena changes where a Vec's elements live, never where its own header — the
binding — lives, so it is already covered by the storage-class check above.
Both directions of the F1 escape were reproduced before the fix (a genuine
ASan stack-use-after-return, reproduced by building the pre-fix tree) and
confirmed refused at check time after it, for all three routes.
Three more findings, all in the runtime rather than the boundary:
view_vec_check, on finding a stale container, rendered the very view it had
just declared unsafe to read — which called back into the same check,
unconditionally, an infinite recursion rather than the intended trap. Fixed
by never rendering the container in the stale message at all; the sentence
names the two epochs and nothing else, which is everything a reader needs
and the one thing that was safe to read.
dyn_equal's VEC arm read x->len and x->u.v.items regardless of kind, which
for a view answers 0 and the union's other member reinterpreted as dyn
words: two views with different contents compared equal, a view and an
equal heap vec compared unequal, and a map keyed by any view collided with
every other view, silently. vecish_len and vecish_at read either shape
correctly and the arm now goes through them. obj_words gets the same
explicit OBJ_VIEW case on the same reasoning, unreachable today only
because mark_push's own gate already excludes the kind — this is the belt
next to that brace.
The three restatements of flan_vec's layout — flan_rt.c's real struct,
flan_dyn.c's mirror, and dyn_ops.c's hand-built one — had a comment
claiming a reorder would not compile or link, which was never true of a
void*-typed forward declaration. flan_vec_layout and
flan_dyn_vec_hdr_layout each report their struct's size and field offsets;
dyn_ops.c's new "layout" mode compares both against offsetof on its own
hand_vec, so a disagreement is a FAIL line in dune test instead of a
silent corruption at whichever view reads through the wrong offset next.
Also: the survey program's comment excusing a by-value parameter's view as
"value semantics, not a hole" was wrong on its own terms — a write through
such a view does reach the caller's storage, only growth diverges — but the
question is moot now: every container the program views is a global, and
the file was rewritten around that rather than patched. And an i32 element
does not cross into a view either, but the refusal used to say why in words
that were true only of a string element; it now says what i32 actually is
and what the restriction is actually for.
Rebased onto dev-loop's item-4 landing (221df5a).
The dyn if truthiness review turned up that or's answer position, unlike
and's, still traps on a non-bool dyn value: or's short-circuit sentinel
sat in the then arm of its own if, the one check_if types first, so that
sentinel decided the whole expression's type and a later non-bool dyn
answer hit the strict bool boundary and unboxed itself into a trap
rather than surviving as itself. (or nil "x") — the canonical Clojure
(or x default) idiom — crashed instead of answering "x", identical on
all three backends.
or now binds its test to a temp and answers the temp itself, exactly the
way Clojure's own or macro expands: (or a b) becomes (let [t a] (if t t
b)), not (if a true b). The temp evaluates a once and lets the answer be
a without writing it a second time as the then arm; it is the temp's own
type check_if sees first, so or hands back the actual truthy operand the
same way and always has. Verified real output, unchanged, on LLVM, -O0
and --x86, and the survey program now exercises the case its own header
used to exclude for being unsafe: a non-bool value stopping or and being
handed back as-is.
check_truthy also gets three corrections a closer look found. Its own
[loc] used to come from the enclosing if/while/not rather than from the
condition itself, so the rt call and cast it builds carried the wrong
column in an --x86 disassembly or the dev inspector whenever the
condition was not the form's first token; it now takes loc from the
scrutinee's own AST node, confirmed against a real --x86 dump. A comment
now names the precondition its exception-swallowing retry rests on: none
of check.ml's save-restore sites (barrier, in_frames, in_defer, loops,
scope) are exception-safe, which is harmless only because the retry
always either succeeds cleanly or re-raises and aborts the compile
before ctx is read again — and would stop being harmless the day some
want-sensitive elaboration on this path could succeed differently on
retry. And a bare keyword condition, which used to be checked with
want:Bool from the start and refused by the keyword arm's enum-or-refuse
case, now resolves as the dyn keyword instead and is unconditionally
truthy — a deliberate loss of that diagnostic, the author's call, pinned
in test_flan.ml so it does not regress by accident.
The two typed-refusal messages captured before this pass (a float
literal condition, an i32 while condition) are unchanged, checked again
against the same baseline. test_flan.ml's parser test for or's shape is
updated to match the new let-bound desugaring.
A (Vec T), a slice or a fixed array crossing into dyn no longer refuses; it
is a view, one word in the box, over the container's own storage. Reads box
the element on the way out; writes tag-check the dyn value's tag against the
element type on the way in and trap, by name, on a mismatch, never coercing
or silently storing.
The open question the decision left — whether the descriptor points at the
container or snapshots pointer and length beside it — is settled by kind. A
Vec view holds the address of the Vec's own header (flan_rt.c's flan_vec,
restated in flan_dyn.c under the file's standing "if either table changes,
change both" rule) and reads ptr and len live on every operation, so a push
that reallocates cannot leave it stale: flan_vec_grow overwrites that same
header in place, and there is nothing captured at the crossing for the
growth to invalidate. A slice and a fixed array cannot grow, so a flat view
snapshots data and length once; pointing it at the value's own slot instead
would be worse, since a slot's lifetime is not the slice's.
The element set is i64, f64 and bool, not everything box already handles
typed-to-dyn. A string element's dyn form is a pointer into the collector's
heap, and a typed container's storage is arena or stack memory the collector
never scans — a wider set would let a write plant a live reference nothing
ever traces, which no care at the write site closes. (Vec string) and a
typed (Map K V) keep the "does not cross into dyn yet" refusal, now for that
reason.
flan_dyn.c gains a fourth object kind, OBJ_VIEW, and flan_dyn_len/at/set_at/
push and the printer each grow one branch for it beside the existing vec
one. A view's own stale-container check is the runtime's own spelling
(flan_trap, park-and-inspect) rather than flan_rt.c's rt_die, per the
duplicity doctrine; growing a Vec through a view calls flan_rt.c's own
flan_vec_push rather than re-implementing doubling and allocator adoption a
second time. (set (at target i) x) against a dyn target — a plain dyn vec or
a view alike — was a hole in the base dyn milestone rather than something
item 3 introduced; it is wired to flan_dyn_set_at here because a view's
writes needed it to exist at all.
Both backends: emit.ml and x86.ml both already passed a Vec or a Map to a
runtime call by address rather than by value; a fixed array crossing into a
view needed the same arm added in both, for the same reason — a copy would
view the copy and never see a write to the caller's own array.
test/dyn_ops.c drives the runtime directly with a hand-built Vec header and
a plain C array, ahead of any compiler involvement: reads, writes on both
element kinds, the tag-check refusal on every element kind, the range
refusal, and the push that grows and moves a hand-built header out from
under the view watching it. test_flan.ml turns the old "does not cross into
dyn yet" refusal into acceptances for Vec/slice/array, keeps it for a string
element and for Map, and adds the element-restriction refusal by name.
test/programs/dyn-view.flan is the compiler-level survey: a Vec view mutated
through both sides including the grow-and-move case, a fixed array's and a
slice's views, a bool Vec's view, and its own two trapping modes for the
acceptance rows to run against. test_sanitize.ml carries the survey's happy
path; test_dyn.ml's new refusals are the runtime's own.
A dyn scrutinee is no longer required to already be a bool: it is tested
for truthiness, Clojure's rule, not C's or Python's — nil and false are
the only falsey values, and everything else, including 0, "", an empty
vec, an empty map and a keyword, is truthy. A typed scrutinee is
unchanged and keeps needing a strict bool.
The runtime side is one new entry point, flan_dyn_truthy
(runtime/flan_dyn.c/.h), reading the tag directly rather than unboxing —
it never traps, unlike flan_dyn_need_bool. Both backends reach it the
same generic way flan_dyn_need_bool already did: check.ml emits an
ordinary Rt call plus the existing i32-to-bool Cast, so emit.ml only
needed the LLVM declare added and x86.ml needed nothing at all.
check.ml's check_truthy is the one funnel every boolean position in the
language goes through: if's own condition, while's, and not's argument.
when and cond reach it for free because they desugar to Ast.If in
parse.ml, and so does and's condition; or's condition does too, but its
answer position is a separate story — its short-circuit sentinel is the
then arm of its own if, which check_if types before anything else, so a
non-bool dyn value reaching that position still meets the strict bool
boundary. and's sentinel sits in the else arm instead, so the real
value's type wins and and hands back the actual last operand,
Clojure-style; or does not get that for the reason above, and reordering
it is a decision for another day, not this one. shortcircuit in parse.ml
carries the note.
check_truthy checks the scrutinee with no expectation first, so a dyn
value takes the truthy path and everything else takes the strict one. A
refusal on that second path is re-checked with the old want:Bool rather
than reported from the bare check, because a bare integer or float
literal, or a bare None, answers "what type is this" differently than
"is this a bool" — check.ml's own arms only give the nicer sentence
("expected bool, found the integer literal 5", "expected bool, found
None") when asked the second way, and that sentence is preserved exactly,
letter for letter, against what a typed if already said.
test/programs/dyn-if-truthy.flan surveys every falsey and truthy case —
nil, false, true, 0, a nonzero number, an empty and nonempty string, an
empty and nonempty vec, an empty and nonempty map, a keyword — through
if, when, cond, and, or, not and while, with real output pinned in
test_acceptance.ml across LLVM, -O0 and --x86. test_flan.ml covers the
checker side directly: a typed if still takes a bare bool and still
refuses a non-bool scrutinee and a bare None with their original
messages, a dyn if/not/while/when/cond/and/or all accept a non-bool dyn
condition. test/dyn_ops.c gets a matching set of direct calls to
flan_dyn_truthy, keeping the header's own contract with the C side.
Both directions of the boundary go through expect, the way every other
dyn crossing does. A dyn's tag decides which case an (Option T) becomes
on the way in; an Option's own tag decides nil or a boxed payload on the
way out. box_option/unbox_option build the same If-over-a-tag shape get
and map-remove already build for the same reason, reading an Option's
tag and payload with the raw Field access Render's structural printer
already uses — nothing new for either backend to lower. A literal
Some/None skips the runtime check entirely, since the checker already
knows which case it is.
A bare T has no None to become. The literal nil the checker can see is
refused right there, at compile time, in expect itself — the author's
decision to do both halves rather than settle for the runtime trap
alone. Everything one step removed from the syntax — a dyn that only
turns out to be nil once the program runs — reaches flan_dyn_need_i64's
existing DynType trap, unchanged; there is no dataflow in this checker
for it to be otherwise (see "Ownership tracking repealed").
(Some nil) is refused the same way: the literal at compile time, with a
message saying why nil and None would collide; a dyn that turns out to
be nil only at run time through the new flan_dyn_need_not_nil, which
traps by the same route flan_dyn_need_i64 does.
(Option (Option T)) does not cross either direction — boxing Some of an
inner None would box it as nil, indistinguishable from the outer None,
the same ambiguity (Some nil) is refused for. The type itself stays
legal on the typed side; only the crossing does not exist for it.
(Option dyn) needs no case of its own in the boundary code — the
payload is already dyn, so box_option/unbox_option treat it as the
identity — but it is not yet a value a program can hold anywhere. The
per-type-descriptor pass (M2 item 2) refuses it at every storage site
today, the same way it refuses (Vec dyn), because a struct's dyn fields
are marked by byte offsets and (Option dyn)'s payload has none. Item 4
does not lift that gate; it only makes the boundary already correct for
the day items 2/3 do.
expect grew a ctx parameter to build the fresh slot the two new
crossings need — every call site threaded through, one context
mismatch caught and fixed in check_fn's tail-expression case along the
way. var's None case grew a direct Dyn arm: None at a dyn want is nil
outright, with nothing to build.
nil-option.flan carries the crossings that succeed and ends on the
bare-T trap; some-nil.flan is (Some nil)'s run-time half, kept in its
own file the way dyn-boundary.flan is one trap per program. Both are
in no_fallback_slots and test_sanitize.ml: the new dyn temporary
unbox_option's tag test mints is rooted, and reads its Option's tag and
payload through ASan clean, --sanitize matching the unsanitized run
byte for byte.
A descriptor's symbol was the type's printed form with every character an
assembler would refuse replaced by a dot, and the table was keyed by that.
The mangle is many-to-one — a Flan name may hold -, +, *, ? and / — so row-a
and row+a were one entry, the second of them was pushed with the first's
descriptor, and the collector read at another type's offsets: past the end of
the object when the first was the larger, and never where the second's dyn
actually sat. ASan named it, a stack-buffer-overflow inside gc_mark_all. It
is the same corruption root_plan pools its temporaries to avoid, arriving
through the name rather than through the supply, which is a lesson about where
identity lives: the table is keyed by Types.to_string now, which is an
identity, and the symbol carries a counter so two types cannot collide however
they mangle. dyn-struct.flan grows the pair, held live across the churn, and
an acceptance assertion asks the emitter directly how many descriptors it
wrote under that label — two, or the two are sharing one. That assertion is
the half with teeth: whether an overread off the end of a frame slot lands on
anything is luck, and the run's own output was not red under the defect.
The cap on a flattened array's offsets was bypassable by the thing it was
meant to stop. [4611686018427387904 S] wrapped the multiplication negative,
so the test read as under the cap, the declaration was accepted, and the
emitter then sat building the offset list until something killed it. A
refusal that overflows into an acceptance is worse than no refusal. The count
saturates at one past the cap now and the message says more-than rather than a
figure that came out of a wrap.
dyn_ops.c's second assertion had no teeth: a marker never writes through a
root, so "the word at a non-dyn offset is untouched" passed under any marker
at all. What discriminates offset-driven from word-driven is a dyn word the
descriptor leaves out, holding five hundred objects, that must NOT survive —
and it is checked by adding its offset to the table and watching the line go
red.
dyn_anywhere descended through Ptr and Slice, so (Vec (Ptr Cond)) was refused
with a sentence about a dyn inside a type whose storage contains none. A
vector of pointers to condition structs is an ordinary thing to write. It
stops at a pointer now, which is the line hidden_dyn already took for a bare
(Ptr S) and the line the whole argument rests on: a pointer is a view of
storage something else roots.
Which leaves the one honest hole, and it is named at the boundary where it
opens rather than left in a comment. Storage C hands back was never rooted
and never will be, so a (Ptr S) crossing a declare with a dyn anywhere under S
is refused by name — the same sentence a bare dyn already gets there, one
level down.
dune test --force: green, 0 failures. dyn-struct.flan clean under ASan and
UBSan and identical at -O2, -O0 and --x86.
The crux was never where to put a descriptor; it was how an instance finds
one. A bare struct on the stack has no header to hang a pointer off, and
giving it one would change the layout C interop agrees on, change the stride
of an array and change what embedding a struct in another costs. So it has
none. The instance never carries a pointer to its type and the collector
never derives one from the bytes: the pairing of an address with a descriptor
is made at the *push*, by the code that put the value there and therefore
knows its static type. That is the same trick the shadow stack has always
used, and it makes the stack case the easy one rather than the impossible one.
A descriptor is the size of an instance, a count, and a table of byte offsets,
emitted once per type as private static data. Flattened, not a graph — a
struct held by value contributes its offsets shifted by where it sits, and a
fixed array contributes its element's once per element — so nesting costs
nothing at run time and there is no recursion in the marker. The offsets of a
big array would be a big table, and that is capped with a sentence rather than
half of the repeat form item 3 will bring.
Four places a value of such a type can live, and all four are rooted: a frame
slot, a global, the temporary a call's by-value return is spilled into, and
the slot a condition that is not a place is evaluated into. The last two are
new and are the ones that were not obvious. A callee roots its dyn words and
pops them in its epilogue, so between the return and the caller's store the
only copy is a register, which a collector that finds its roots by address
cannot see; the same hole was open for a Flan call answering a bare dyn and is
closed here too. And a condition crosses as a pointer into the signalling
frame while a handler allocates, which is exactly what the original refusal
said could not be made safe.
dyn_roots grows into root_plan and both backends read it, which is what the
older note about one counter deciding both ends was always for. The aggregate
temporaries are pooled by type rather than handed out in mint order: a
positional supply that drifted would pair an address with another type's
descriptor, and marking arbitrary offsets off a base is corruption where a
missed root is only a bug. Pooled, the worst a drift can do is run out.
What is still refused is a dyn no static offset can reach — inside a typed
container, in a data type's payload or a union's members where the cases
overlay, or under an Option where the payload exists only beneath the tag.
A (Ptr S) and a [S] are deliberately not on that list: neither owns storage,
and the only storage this compiler hands out for such a type is a frame slot,
a global or a fixed array in one, all of them already rooted. That is what
lets a handler clause take its (Ptr Cond) and read a dyn payload.
test/programs/dyn-struct.flan is the evidence. It runs forty thousand rows
past flan_dyn.c's one-megabyte floor, so marks and sweeps really happen, and
it holds live values through them in all four places at once. It has teeth:
with the descriptor walk stubbed out of the marker, the kept vector's length
comes back 24 instead of 628 and its first element is a stale word. Clean
under ASan and UBSan, same output at -O2, -O0 and --x86. dyn_ops.c grows an
aggregate-root mode so the runtime half can be wrong on its own, with a
header word holding a bit pattern that looks boxed and is not a dyn slot.
--no-gc still refuses, and had to be told how: a struct with a dyn field is a
collected value even when no expression in the program ever has the type dyn,
because a zeroed one still has a word the collector is asked to mark.
dune test --force: green, 0 failures across every suite.
Three notes, all of them about a comment that was true as far as it
went. The cost sentence on defer_slot said one i64 and one compare on
a path that is already unwinding, which is the unwind's share and not
the whole bill: every function with a defer also pays a store of zero
at entry and a store of an ordinal at each defer, on the ordinary path,
whether anything transfers or not. Small, correct, and now written
down as what it is.
init-conditions.flan is in test_valgrind.ml, and only the game-data
half earns that row -- it is the one that reaches a real (free src)
over a slot slurp transferred out of. The note half is output-only:
revert the fix and its trace changes, but a wrong integer in a global
is not a memory error and memcheck stays quiet. A later edit trimming
the edn dependency would leave the row green and blind, so the header
says so.
And register_defer saves in_defer rather than clearing it. Unreachable
today, because defer_ok is false inside a defer and nothing can nest
one; written so that the flag comes back rather than being dropped on
the day that changes.
The report was that handler-case segfaults in a top-level global
initialiser. It does not, and never did. What crashes is one frame
further down, in a position that has nothing to do with startup:
(defn read-file [path string] dyn
(let [src (slurp path (heap-allocator))]
(defer (free src))
(read (as-slice src))))
slurp signals FileError, a handler further out unwinds, and the
transfer leaves this frame through its defers -- all of them. But src
was never written: the form that would have written it is the form
that transferred. free then reads whatever the stack held under that
slot, which at -O0 in a small program is zero and at -O2 is a live
pointer, which is why the same program looked like an optimiser bug
from one direction and a startup bug from the other.
return never had this. The checker splices the defers registered above
it and no others, and says so where it does it. The transfer exit took
the whole list, because it is one landing block per function and
nothing in the IR said where each defer had come into being.
So the count is kept. The first defer in a function mints an i64 slot
zeroed at the top of the body; each defer leaves a store of its own
number where it was written; and fdefers -- the transfer path's copy,
and only that copy -- tests the count before running each one. The
normal paths are untouched and still need no test. It is all in the
checker: what reaches a backend is a slot, a store and an if, so
neither emitter learned anything and the x86 one needed no frame of
its own.
test/programs/init-conditions.flan is the survey. The top half is the
part of the report that was never true: handler-case with its
condition firing and with its body completing, handler-bind, and a
restart-case, all four in a global initialiser, all four answering
what they answer anywhere. The bottom half is the part that was: a
defer below the signalling form, which must not run, beside one above
it, which must -- a fix that took the unregistered one off by taking
them all off would have traded the crash for a leak, and 302/2 is the
line that would catch it. Three acceptance rows, LLVM, -O0 and --x86.
The detector was valgrind, not ASan: reading a stack slot nobody wrote
is not ASan's bug class and it reported nothing on the broken binary,
while memcheck named the conditional jump in flan_vec_free with the
unwinding frame directly above it. The program is in both lists --
test_valgrind.ml because that is what saw it, test_sanitize.ml because
that is where the transfer exit's frames are already watched.
spec-conditions.md section 5 now says which defers a transfer runs.
The author's (defvar game-data dyn (handler-case (edn/read-file ...)
[(FileError [c] nil)])) works.
Constraints parsing peeled a body-leading map only when its first key was
literally :where; any other keyword fell through to the body, so a typo'd
key surfaced as a baffling error from inside what was meant as a predicate
and a stray map at body start compiled away silently. Any keyword-first map
is read as a constraint map now, but only when something follows it in the
body — a single-form map body is a real dyn value and not a discarded
statement, so that case is left alone.
An empty map literal still parses as a struct literal, (P {}) still meaning
the zero struct for a real struct name — the parser has no symbol table to
tell (take {}) apart from it at that point. check.ml now catches the case
where the name turns out to be a known function instead and says so, rather
than "unknown struct take".
flan_dyn.c's tag comment still said 6 and 7 were free; keywords and maps
took 4 and a kind field under BOX_OBJ, not new top-level tags, so 5, 6 and 7
are what is actually open for the interop handle. NEXT.md and json.flan both
still pointed at test/programs/arena-edn.flan, gone since edn/read stopped
taking an allocator; both now point at what replaced it.
flan_rt.c's flan_str_eq comment claimed the empty string literal was a
hypothetical null-pointer string; it isn't, its address is an interned
symbol's. The real case the zero-length guard exists for is a zero-length
container converted to a string. check.ml's ordering refusal said a string
has no comparison at all, which stopped being true when typed = and !=
grew strings in daed039 — split the message so an equality refusal and an
ordering refusal say the right noun, and updated the pinned rejects_check
rows to match. string-eq.flan gained the row the fast path most wants
tested, a slice against the prefix it was cut from sharing a base pointer at
different lengths, plus a != row at equal length with differing bytes;
acceptance now carries the real output, captured by running the program on
all three lanes. x86.ml's xor-1 comment now names the 0/1 return contract as
a requirement flan_str_eq must hold, not an incidental fact. SPIKE-DUPLICITY
now says plainly that its equality-and-ordering argument landed in daed039
and marks its transcript as the historical state that argument was made
against. FIX.org ticks M2 queue item 5.
And the acceptance runner: the tail check that turns a nonzero failure count
into exit 1 was already there and already fired — a fresh build with one row
broken already exited 1 before anything here changed. What wasn't proven is
that every path through the file's clang/wasmtime/raylib/lldb probes still
reaches that tail rather than skipping past rows that already failed. An
at_exit guard now closes that class regardless of which path the process
leaves by, flushing stdout first so a failing run's FAIL lines survive
Unix._exit rather than being dropped from the buffer. Verified both
directions with a deliberately broken row: dune test exits nonzero and the
log still carries the FAIL line and the failure count; restored, the same
run is exit 0 with nothing printed but green summaries. The other test
binaries were checked for the same gap and none have it — each gates its
own exit on a single failures ref that the tail already reads.
Every mark it left was an addition, and addition commutes, so a backend that
ran the defers outermost-first produced byte-identical output and the row that
was supposed to be watching the order could not have told. The claim was in the
comments and not in the numbers. cleanup.flan already had the device for this —
a shift rather than a sum — so the log here is a digit trace now, and the two
frames under a catch read 12 where a wrong order reads 21.
Rewriting the trace made room for the three behaviours that worked and nothing
pinned. A return inside a clause is an ordinary return from the function that
wrote the form, because that is where a clause runs: it leaves through the
function's own exit, runs the defer registered there after the two the unwind
already ran, and leaves the handler stack empty behind it, which the bare
signal that follows in main is the check on. A defer inside a clause is refused
for the reason every nested form is refused one. And a handler-case inside a
defer works, because a defer may not start a transfer that leaves it and this
one begins and ends its own.
The program is registered with the sanitizers, where the interesting failure is
not the heap but a handler or restart frame left on a stack pointing into an
alloca that has gone — an output comparison cannot see that until something
much later calls through it. It is clean; it was also leaking sixteen bytes out
of the vector main allocates to prove the allocator context came back, which is
the test's own litter and is freed now.
docs/PORTING.md ranked handler-case as one site handler-bind covers. It still
is one site, and handler-bind still covers it, but it is no longer the closer
translation: a catch block is assumed everywhere it is written to see the
locals around it, and only the clause that runs at the form does.
The handler clauses are lifted left to right rather than by List.map, whose
order is unspecified. Each lift names itself after the count already on the
list, so an order nobody chose would number the clauses of one handler-bind
differently between builds, and those names go into a redefinition module.
The unwinding handler, which spec-conditions.md named and left unwritten while
it asked whether the thing should be a macro over the two operators that were
already here. It should. (handler-case B [(T [c] A)]) is checked as
(restart-case (handler-bind [(T [c] (invoke-restart 'R c))] B) (R [c T] A))
with R a name the form makes up for itself, which is Common Lisp's own
definition of the operator and means neither backend needed a line.
What that buys is not economy, it is the correctness of the parts nobody can
see. The defers between the signal and the form run, and the allocator a
with-allocator rebound is put back, because a transfer already does both for
every frame it leaves. The body and every clause agree on one type, because a
restart-case's body and clauses already do, and a clause that disagrees is
refused with the same message an if with disagreeing arms gets. A condition no
clause lists installs no frame that matches it and goes on outward untouched.
A clause sees the establishing function's locals, which a handler-bind clause
cannot, because a restart clause runs where it was written.
The body comes first and the clauses after it, the opposite of handler-bind's
order: one reads as something put around a body and the other as a body with
answers hung off the end of it. The restart the two halves meet over is named
after the function and numbered within it, and it has to be unique per form,
because two nested handler-cases sharing a name would have the inner frame
shadow the outer one and land a condition at the wrong place.
The refusals name handler-case rather than the machinery underneath it, which
is why check_handler_bind and the restart clauses now take the word the reader
wrote. A break loop entered under a handler-case still lists the made-up
restart, and taking it there is refused loudly rather than answered wrongly;
hiding it would mean a field in a frame layout spelled out in three files.
The survey program runs the same under LLVM, at -O0 and under --x86: normal
completion, a caught condition, one nobody listed passing through with the body
carrying on, both defers on the way out, the two nestings against handler-bind,
a clause that signals and is caught outside the form it belongs to, and a
with-allocator whose restore is on the transfer path.
Types.is_equatable splits from is_comparable: a string answers equal?
now, bytewise, but still answers no to ordered? — there is no collation
the language has picked, so < and friends keep the refusal they had.
The comparison itself is one new runtime entry point, flan_str_eq
(runtime/flan_rt.c), length-mismatch and same-pointer fast paths ahead
of the memcmp, called identically from both backends: emit.ml pulls a
string's ptr and length out of the %slice SSA value and calls it
directly in the Eq/Ne arm; x86.ml adds an arm ahead of the generic
scalar comparison that reaches it through call_native, flipping the
answer for != the same way Not already flips a bool.
test_flan.ml covers the checker side directly and through a generic
instantiated at string, including the two different ways ordered? and
equal? fail at that type. test/programs/string-eq.flan is the survey
program — same pointer, differing lengths, equal content at distinct
addresses (a literal against a fresh heap string), a difference in the
last byte, and the empty-string cases — with acceptance rows for LLVM,
-O0 and --x86 in test_acceptance.ml.
The dyn runtime gets a map object and an interned keyword, alongside the
vec it already had. {:a 1 :b s} is a map literal wherever a struct
literal isn't — the parser tells the two apart by whether the first form
in the braces is a .field symbol — and a bracket literal builds the
runtime's own vec rather than a typed array wherever a dyn is wanted, which
is what lets a map literal's values nest arrays and maps freely. get, put,
len and has-key? all learn a dyn-map arm alongside the typed-map one they
already had, and (keyword s) builds the same interned value a :foo literal
does, for a name that only exists at run time. nil is now a literal, the
dyn absence value that get answers for a key a map does not hold.
On the runtime side, flan_dyn.c gets an OBJ_MAP that shares the vec's
storage arm and doubles its accounting, a linear-scan intern table for
keywords that makes equality an identity compare, and structural map
equality by lookup rather than position. The marker traces a map's
interleaved keys and values the same way it already traced a vec.
edn/read and its callers move off the old (Option Value) union entirely:
a document is plain dyn now, sets are dyn maps to true, and arena-edn.flan
is retired along with the union it demonstrated. The acceptance suite's
edn-read and json rows were recaptured against the new shape, and a new
dyn-map.flan program exercises the map and keyword operations end to end,
including a 200k-iteration churn loop against a rooted map that runs
GC for real, across the LLVM, -O0 and x86 rows, and under the sanitizer.
Keywords are dyn everywhere an enum isn't expected, which changed what a
couple of existing checker tests actually see refused; both were updated
to the sentence the checker gives now rather than the one it used to.
The louder failure had the quieter answer. A generated reader accumulates
errors on the cursor — which is what lets it be a straight line of
assignments — and the cursor is made and dropped inside the entry point, so a
stray brace in a file read at run time handed the program a zeroed struct and
said nothing at all. That is the one thing the rest of vendor:edn refuses to
do: read-file answers an Option precisely so a malformed document is
distinguishable from one that is literally nil, and the hand-written reader in
test/programs/edn.flan tests ok? and prints the reason. ReadFailed is the
derived reader being as honest, in both packages, and it sits beside
SchemaDrift because both are "the file is not what this program was built for".
Then the writing-down. docs/BUILT.md gets the section: the four things the
macro system did not have and now does, each general and none of them
mentioning EDN — a macro reading a file at the call site's path, a package
macro calling its package, one call answering several declarations, and
compile-error, which is the one piece that had to go in the compiler and the
reason it had to. The set rule the real game file decided is there too, and
what defjson shares, which is the design and not the code.
NEXT.md item 9 and PORTING.md §3.9 both close. Not as (read-edn T bytes): the
struct comes from the *file* rather than from a type declared by hand, so the
~80 lines PORTING prices for two schemas are not written at all. The competing
answer PORTING names — compile-time embedding — turned out to be the other half
rather than a competitor: the shape comes from the file at compile time and the
bytes may come from an embed beside it, which is exactly what
test/programs/edn-provide.flan does.
vendor:json depends on vendor:edn for nothing, and borrowing a shape walk
across that line would be a dependency for the sake of a resemblance. What
carries over is the shape of the answer — one walk giving a type, the
declarations that type needs and the expression that reads one; a refusal
carried in a field rather than raised; a typed one-line constructor per
collection; and a compile-error wrapped in a defn nothing calls.
What is genuinely different is four things. Strings go through string-of and
never through .text: .text is the raw interior with escapes undecoded, so a
field read off it would hold a backslash and an n where the file meant a
newline — which is the first two lines of the acceptance output and the reason
they are two. Commas and colons are tokens rather than whitespace. An object's
keys are strings, so a key has to be refused when it is not a name a program
could write, and refused again when it carries an escape: the generated reader
compares against the bytes as written, which costs no allocation per key and is
only the same question when the name is written plainly. And there are no sets,
so there is no map-key path and no fixed array — every collection is a (Vec T)
and defjson is the smaller of the two by half.
JSON has no integer type; the tokenizer draws the line at whether a number has
a fraction or an exponent, which is the only line there is, so 1 derives i64
and 1.0 derives f64. That is the file's own distinction and the honest one to
take.
@x86 matches on it and @sanitize is clean.
A defer is in the typed IR twice -- spliced into the body for the normal path,
and again in fdefers for the path a transfer leaves through -- so a dyn
temporary inside one is emitted twice. dyn_roots counted only the body's, and
the second copy went into slots nothing had rooted.
Nothing failed, and that is the whole reason this is worth a commit of its own.
dyn_tmp falls back to a plain slot rather than unbalancing the stack, so the
pushes and the pops still matched, the program ran and printed the right answer,
and the values were simply invisible. Against a stub that never collects there is
no symptom to find -- no leak, no crash, no wrong number. It would have become a
symptom the week the real collector landed, in a defer reached only on a handled
condition, which is close to the worst place to start looking.
What found it was the IR: a rooted slot is spelled %dr and the fallback %dx, and
the assertion is that no dyn program in the corpus emits one of the latter. That
is now a test over all five dyn programs, and it is the only check in the lane
that can see a missing root while there is still nothing to lose one by. When
the collector arrives it is the thing to extend rather than replace.
Also checked, both clean: flan dev --llvm builds and runs a dyn program, which
is the route the x86 refusal sends people to and would have been a link error in
the worst possible place; and the daemon's own refusal already names the flag.
The promise is that this program carries no collector, and the way to keep it is
to refuse every dyn rather than to emit a different program: a dyn value is one
the runtime allocates and the collector owns, and there is no smaller version to
fall back to. So it runs between checking and emission, answers unit or raises,
and hands the very same program on. Emit has no field to branch on and is told
nothing.
That is what makes the byte-identity claim true rather than approximate, and it
is tested by compiling three annotated programs twice and comparing the text. A
field, a mode, or a comment that mentioned the flag would break it on something
incidental, a long way from anything to do with dyn.
Every site is named, the way the global cycle refusal names the whole ring: a
reader who has to annotate their program wants the list, not the first one and
then another compile. Globals and signatures as well as body values -- the two
files it is tested against report nine sites each, and the floors are set under
that so an added line does not fail the test and a pass that named one site and
stopped would.
The four programs run at -O2 and -O0. dyn-boundary is asserted on its exit
status as well as its output, because the boundary is only interesting in that
it can fail and a test that showed it working would be testing the easy half.
The x86 survey skips them by name: a REFUSED there means a node that backend has
stopped lowering, which is a regression, and this is the opposite -- a lane that
has not started. Take a name off llvmonly when the lowering arrives and the
survey will say whether it works. 128 match, 0 differ, 0 refused.
Checked while writing these: a dyn function with an early return pops its roots
on both paths, and one with a defer pops on the transfer path too.
A precise collector has to be told where the live dyn words are, and the shadow
stack next door is the precedent for where that goes: set up in the entry block,
undone in ret, which is the one funnel all five exits pass through -- the tail,
both returns, the none arm of (some x), and the landing block a handled
condition unwinds through. A pop written only on the normal path would leave a
frame's roots on the stack after every handled error.
It differs from the shadow stack in two ways, and both are forced. It is not
gated on dev: a backtrace is a convenience and a collector that cannot find its
roots frees live values. And it is a count rather than a saved head pointer,
because the ABI offers root_pop(n) and no way to read the stack's height -- so
the number has to be known before the body is emitted, since ret runs during
emission and a tally accumulated as roots were discovered would be short at
every early return. dyn_roots works it out up front by walking the same nodes
the emission will visit, the slots are minted from that count at entry, and
dyn_tmp only hands them out. The pushes and the pops balance by construction
rather than by two walks agreeing.
Every dyn-producing call is spilled into a rooted slot the moment it exists. An
SSA value is invisible to a collector that finds roots by address, and the next
allocation could be the one that frees what it holds. Rooting all of them rather
than only those that outlive a call is conservative and is the only thing
available here: this file has no liveness and no lexical scope, the checker
having resolved both into flat slot indices long before. The cost is a stack
slot and a store per dyn value at every optimisation level, because a rooted
alloca has its address escape and mem2reg cannot promote it. That is the price
of an address-registration ABI rather than stack maps.
A function with no dyn emits nothing at all -- no push, no pop, not a pop of
zero -- which is what makes an annotated program's IR identical to what it was
before any of this existed.
Globals are rooted in main, before the startup function that fills them and
before any other push, because every pop takes the top of the stack and these
are the ones that must never be at the top. They are never popped, which is what
a global's extent means. A dyn global needed no new machinery otherwise: a call
is not a constant, so it is a computed global, and that already existed.
(vec-new dyn) is not a (Vec dyn). At milestone 1 the heterogeneous container is
the dyn runtime's own object and its type is dyn like everything else the
runtime hands back, which is what lets push, at and len on it be the dyn
operations instead of a type-erased Vec over eight-byte elements. It takes no
allocator, and the refusal says why: the storage has to be storage the collector
already knows about, where a Flan Vec's block would hold roots inside memory the
collector does not own.
len answers an i32 and at answers a dyn. The asymmetry is deliberate -- a length
is what an index loop compares against, and handing back a boxed number would
make (< i (len xs)) a dyn comparison and two allocations an iteration.
The operand-order bug, which the first test could not see because both its
operands were dyn: (+ n x) over a typed n and a dyn x threaded i64 into the
second check, expect did what an annotation site had asked for and unboxed, and
the result was a machine add of a value the runtime was never asked about -- the
program trapping on a float instead of promoting it, with nothing in the source
to say why. (+ x n) boxed correctly, so it was visible in one operand order
only. binary now takes dyn_ok from the operators that have a dyn lowering and
checks both operands on their own terms, which is safe exactly when neither
needs an expectation to check -- a literal still takes the other's type, and a
keyword still gets one, since :lo has no meaning without it.
Cast had no bool arms, so the bool boundary failed to emit; reachability hid it,
because the program that used it dropped the function. dyn does not cross to C:
it is one word and would have passed as an integer, and C has no way to ask what
the word means. A condition may not carry one either, nor hold one in a field --
a payload crosses a handler boundary and has to stay rooted across the transfer,
which is the collector's question and milestone 2's.
edn-provide.flan reads assets/edn/tileset.edn through a struct derived from it,
and its first five lines are edn-read.flan's first five character for character.
Two readers over one file agreeing is what says the derived one is right; either
alone could be self-consistently wrong. The pair memberships are the derivation
deciding in public: the set became a (Map [2 i64] bool), so [3 4] is a key and
[9 9] is not, where a version that made it 108 loose integers would have
compiled and answered differently on all four.
A tuning file beside it covers the rest of the matrix — a string, an integer, a
float, a boolean, a vector summed rather than counted, and a map inside a map
read two field loads deep — and then drift: the struct was derived from a file
with :speed and without :level, and the bytes read carry the opposite. Both are
named, and the read carries on.
The refusals write their own data file, because the data file is the test. Each
is asserted on the position it names, not on the fact of failing, and one of
them checks a line and column into a file the compiler is not reading — which
is the whole of what compile-error was added for.
Two things the tests caught. Load extends the ambient macro set rather than
replacing it, so a package reached twice handed its declarations over twice and
the module refused them as a redefinition; Macro.compile dedupes by name, which
is the rule macro_union already applies a level up. And `where` held a line and
a column at once, which the prelude's note over append-i64 says cannot be done:
i64->bytes renders into one shared static buffer, and both numbers read as the
second one.
