Joseph Ferano
c2d378957e
x86 is what flan dev takes by default and dyn is the iteration feature, so a backend that refused dyn meant the two halves of the dev loop could not be in the same program. The refusal was one arm of is_agg, and it said the true thing: it was never the representation that was missing. A dyn is uint64_t, a scalar in both calling conventions, classified by every rule this file already had; every operation on one is a Tast.Rt primitive and call_rt has always known how to make one of those. What the lane actually cost was the collector's root discipline. Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the roots for both backends now, so the pushes and the pops balance because one counter decides both ends, and the two backends root the same nodes because there is one counter and not two. A zeroed frame slot per dyn slot and per dyn-producing call, minted beside the channel and outside every scoped -- the bump allocator reclaims at the end of a statement and a slot minted in the body would be handed out again while the collector still held its address. Pushed from the body buffer, not the prologue's, because a call clobbers the registers the prologue is still spilling from. And one pop in the epilogue, which is the whole of why this backend needed no landing-pad work for it: there is exactly one epilogue, and the return, the fall-through and the transfer exit all arrive at it. emit.ml needs the same pop at five separate rets. The ABI point the dyn handoff left open for the integrator is settled by reading the other side rather than by agreeing: flan_dyn.c's mark follows a value only when the quiet-NaN prefix is set, and the zero word does not have it, so a zeroed root decodes as the double 0.0 and is never an address anything dereferences. Zero is safe for a reason. The header says so now. And one line in dev.ml that was never x86's: the merged dev host resets the condition stacks and the frame chain between runs, because main is re-entered by longjmp and pops no frame -- and it never reset the root stack, so every root a finished run pushed still named stack the next run was about to write over. That gap was an LLVM dev build's too. Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER 0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which is gone rather than empty, plus p13. dune test --force green, with --x86 acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary asserted on the same exit 134 and the same sentence on both. p13-dyn-collect.flan is the one that is not a formality. Nothing else in this repository allocates past flan_dyn.c's one-megabyte floor, so nothing else collects even once, so a program whose roots are entirely wrong passes every output test there is -- the handoff wrote that about the stub and it outlived the stub. p13 allocates several megabytes of garbage while holding live values across it: at forty times the corpus size it peaks at 4MB of RSS, which is the collector running many times over, and both backends still print the same four lines.
158 lines
8.9 KiB
Markdown
158 lines
8.9 KiB
Markdown
# dyn, milestone 1 — what was decided and what is left
|
|
|
|
The compiler half of dynamic-by-default. The runtime half is a sibling's, built
|
|
in parallel against `runtime/flan_dyn.h`, which is the fixed ABI and the thing
|
|
the two copies are diffed against.
|
|
|
|
## Two decisions that differ from the brief
|
|
|
|
**The return slot stays mandatory; `dyn` is written out in it.** The brief
|
|
expected `ret = None` to grow a third state meaning "unannotated", and listed
|
|
mechanical fallout in `load.ml`, `shim.ml` and `cimport.ml`. That fallout does
|
|
not exist, because the change was not made. The reason is in `parse.ml` beside
|
|
the `defn` case: an optional return slot has *no* syntactic resolution, since a
|
|
capitalised head in a list is both a type application and a struct literal —
|
|
`(defn f [] (Rune {.code 65}) (bar))` is the misparse that removed the old
|
|
optional slot, and it would come straight back. A parameter vector has no such
|
|
case, because every slot in it is a name or a type and never an expression. So
|
|
`ret = None` still means Unit and only `declare` and the shim produce it. One
|
|
token in the return position buys a decision the file paid for twice in one day.
|
|
|
|
**The parameter rule is resolved in `Check`, not in `parse.ml`.** The brief
|
|
asked for "a known type name is a type, anything else is another dyn param",
|
|
written where `parse.ml` argues its other misparse-closing decisions. That
|
|
lookup is exactly the one `parse.ml:904` records being removed for being wrong
|
|
twice in one day, and at parse time the set of type names is incomplete *by
|
|
construction* — macros generate definitions, packages are loaded later, C
|
|
headers are imported later. `cimport.ml` decides it: `named env n = tname n`
|
|
passes C type names through verbatim, so POSIX's `stat` and `timespec` are
|
|
lowercase Flan type names writable in parameter position, and no syntactic rule
|
|
("capitalised is a type") can be made sound.
|
|
|
|
So the vector is carried undecided as `Ast.pitem`s and paired in
|
|
`Check.pair_params`, after every file is loaded, every macro expanded and every
|
|
header imported. The argument is written at `parse.ml`'s `defn` case as asked.
|
|
|
|
## The residual the parent owns
|
|
|
|
The set of type names is complete at a point in time and **not across time**.
|
|
`(defn f [x y] ...)` is two dyn parameters until somebody writes
|
|
`(defstruct y ...)` — or imports a header that declares one — and then it is one
|
|
parameter of type `y`, with no edit to `f`. The signature changes underneath it,
|
|
and arity changes with it.
|
|
|
|
`Session.compatible` is where that is felt: it compares with `Types.equal` over
|
|
parameters and return, so a redefinition that changes dyn-ness is refused like
|
|
any other signature change (this falls out; it is pinned in `test_session.ml`).
|
|
But the *first* definition after such an edit is the one that changes, and
|
|
nothing warns.
|
|
|
|
## What the feature costs, and what was taken back
|
|
|
|
A parameter slot with no type used to be a syntax error. It is now a `dyn`
|
|
parameter, so **a mistyped type silently becomes an extra parameter** — the
|
|
arity changes with no diagnostic, which is the failure class `parse.ml` calls
|
|
the worst available. Two rules take most of it back, in `dyn_param_or_typo`:
|
|
|
|
- a name within one edit of a type's name gets the resolver's own "did you
|
|
mean", and
|
|
- an unknown **capitalised** name is reported as an unknown type. Not one
|
|
parameter in the corpus is capitalised, while `Form`, `Cursor` and `Vector2`
|
|
appear in these vectors constantly.
|
|
|
|
What is left uncovered is a lowercase name resembling no type: `(defn f [x
|
|
widget] ())` is two dyn parameters and nothing in the text says otherwise. That
|
|
is the feature working as specified.
|
|
|
|
**Sharp edge of the near-miss rule.** `near_miss` treats any two single-char
|
|
names as one edit apart, and it compares against every struct name in scope. So
|
|
a `(defstruct D ...)` anywhere in the program makes `(defn f [a d] ...)` a
|
|
refusal rather than two dyn parameters. The message is actionable — write the
|
|
type, or rename — but it is a refusal a user will meet without having done
|
|
anything wrong.
|
|
|
|
## Open ABI point for the integrator
|
|
|
|
**A rooted slot holding 0 is not a value, and the collector must skip it.**
|
|
This is written into `runtime/flan_dyn.h` beside the root functions, and it is
|
|
the one thing in that header decided by one side alone. Roots are pushed in the
|
|
function's entry block, before the code that fills them has run and possibly for
|
|
a branch that never runs, so the compiler zeroes every root slot and must mean
|
|
something by it — and 0 is the only pattern it can write without knowing the
|
|
encoding.
|
|
|
|
If the real runtime NaN-boxes and integer zero is the zero word, this is wrong
|
|
and the two sides need a different sentinel. Do not fix it on one side.
|
|
|
|
**Settled, by reading the other side.** The real runtime does NaN-box, and the
|
|
zero word is *not* the zero integer: an integer is boxed, and boxed means the
|
|
quiet-NaN prefix is set. `mark_value` in `runtime/flan_dyn.c` follows a value
|
|
only when `dyn_boxed` holds, which tests `(v & 0xFFF8000000000000) ==
|
|
0xFFF8000000000000`, and the zero word fails it. So a rooted slot holding 0
|
|
decodes as the double `0.0` — an ordinary value rather than a marker, and
|
|
crucially never an address the collector dereferences. Zero is safe, and the
|
|
header's sentence is true for a reason both sides can check rather than by the
|
|
two of them having guessed alike. No sentinel is needed and neither side
|
|
changes.
|
|
|
|
## Not in milestone 1, each refused by name with a location
|
|
|
|
- a typed container boxing into dyn (`(Vec i64)` → dyn): "not yet"; the
|
|
heterogeneous container is the runtime's own from `(vec-new dyn)`
|
|
- a dyn in a condition's payload, or in a field of one: milestone 2 — a payload
|
|
crosses a handler boundary and must stay rooted across the transfer
|
|
- a dyn crossing to C through `declare`/`declare-c`: it is one word and would
|
|
have passed as an integer with nothing on the other side able to ask what it
|
|
means. This one was **not** in the brief and is the dangerous one, because the
|
|
general "cannot cross to C" arm would have caught it with advice (`pass (Ptr
|
|
T)`) that is wrong for dyn.
|
|
- integer widths other than i64 and floats other than f64 unboxing from dyn:
|
|
the ABI carries one of each, and a `need_i64` plus a truncation would put an
|
|
implicit narrowing at the one boundary where the value's type was already
|
|
uncertain
|
|
- ~~the x86 dev backend, and~~ the JS dialect, refuse dyn entirely. The x86
|
|
backend does not any more: a dyn is one machine word in both calling
|
|
conventions and every operation on one is an ordinary `Tast.Rt` call, so what
|
|
the lane cost was the root discipline and not the arithmetic — a zeroed frame
|
|
slot per dyn local and per dyn-producing call, pushed in the body buffer at
|
|
entry, and one `flan_dyn_root_pop` in the epilogue that every return and
|
|
every transfer out of the frame already went through. `Emit.dyn_roots` is
|
|
called by both backends, which is what makes the counts agree rather than
|
|
merely both being written down
|
|
|
|
## Roots: what is and is not verified
|
|
|
|
Every dyn slot and every dyn-producing runtime call is rooted, pushed in the
|
|
entry block and popped at every `ret` — which is the funnel all five exits pass
|
|
through, the transfer landing block included. Pushes and pops balance **by
|
|
construction**: `dyn_roots` counts before emission, the slots are minted from
|
|
that count, and `dyn_tmp` only hands them out.
|
|
|
|
**The stub verifies none of this.** `flan_dyn_stub.c` mallocs and never frees,
|
|
so a program with entirely wrong root discipline passes every test that runs
|
|
against it. What is checked instead is the IR, and that check earned its keep —
|
|
it found a real hole. A defer appears twice in the typed IR, spliced into `body`
|
|
for the normal path and again in `fdefers` for the path a transfer leaves
|
|
through, so a dyn temporary inside one is emitted twice; `dyn_roots` counted
|
|
only the body's, and the second copy went into slots the collector had never
|
|
been told about.
|
|
|
|
Nothing failed, which is the point. `dyn_tmp` falls back to a plain unrooted
|
|
slot rather than unbalancing the stack, so the pushes and the pops still
|
|
matched, the program ran and printed the right answer, and four dyn values were
|
|
simply invisible. Under a stub that never collects there is no symptom at all.
|
|
|
|
The assertion that caught it is in `test_acceptance.ml`: a rooted slot is
|
|
spelled `%dr` and the fallback `%dx`, and no dyn program in the corpus may emit
|
|
the latter. When the real collector lands, that is the check to extend rather
|
|
than replace — it is the only one that can see a missing root before there is a
|
|
collector to lose one by.
|
|
|
|
Cost: a rooted alloca has its address escape through `flan_dyn_root_push`, so
|
|
mem2reg cannot promote it. Every dyn local and every dyn temporary is a real
|
|
stack slot with a real store, at every optimisation level. That is inherent to a
|
|
precise collector with an address-registration ABI rather than stack maps.
|
|
|
|
A function with no dyn emits nothing — no push, no pop, not a `pop(0)` — which
|
|
is what makes `--no-gc` byte-identity hold.
|