x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
(defn compute [] i64 (handler-bind [...] (risky))) printed 0 through LLVM
and 2 through --x86, and neither was the restart's answer. The divergence
was real and the cause was in neither backend: check_handler_bind wrote
[ignore want] and typed the form Unit, so a unit in value position was
never checked against the expectation that would have refused it. Both
lowerings then answered a caller that had no business asking -- emit.ml a
literal zeroinitializer, x86.ml whatever the body's last form had left in
the destination slot. One of those looked like a value.
Unit was the wrong answer anyway. Every use of handler-bind in value
position in this repository -- restarts.flan, cleanup.flan,
p6-transfer.flan, p10-defer-transfer.flan -- writes it as a restart-case
body, where §3 requires the body and the clauses to agree in type; making
the form unit refuses all four. So it takes with-allocator's shape, which
is the same shape for the same reason: the body's last form is the value,
threaded through [expect] like any other. handler-case, whose value is the
handler's rather than the body's, is untouched and still refused by name in
parse.ml -- that difference is the whole of what separates the two, and it
is not this one.
emit.ml returns [last] with no phi and no slot: the pad terminates at
current_pad and never at the join, so the join has one predecessor and the
body's value dominates it. x86.ml needed no change at all -- it had been
passing dst and the type through to the body all along.
p12-handler-value.flan is the shape the survey could not see, plus the
neighbours a divergence usually travels with: a clause parameter, nested
restart-cases, a defer between the signal and the restart-case, and f64
and string across the transfer. All six already agreed; the handler-bind
value was alone. @x86 MATCH 128 -> 129, DIFFER 0.
A defer is in the typed IR twice -- spliced into the body for the normal path,
and again in fdefers for the path a transfer leaves through -- so a dyn
temporary inside one is emitted twice. dyn_roots counted only the body's, and
the second copy went into slots nothing had rooted.
Nothing failed, and that is the whole reason this is worth a commit of its own.
dyn_tmp falls back to a plain slot rather than unbalancing the stack, so the
pushes and the pops still matched, the program ran and printed the right answer,
and the values were simply invisible. Against a stub that never collects there is
no symptom to find -- no leak, no crash, no wrong number. It would have become a
symptom the week the real collector landed, in a defer reached only on a handled
condition, which is close to the worst place to start looking.
What found it was the IR: a rooted slot is spelled %dr and the fallback %dx, and
the assertion is that no dyn program in the corpus emits one of the latter. That
is now a test over all five dyn programs, and it is the only check in the lane
that can see a missing root while there is still nothing to lose one by. When
the collector arrives it is the thing to extend rather than replace.
Also checked, both clean: flan dev --llvm builds and runs a dyn program, which
is the route the x86 refusal sends people to and would have been a link error in
the worst possible place; and the daemon's own refusal already names the flag.
The promise is that this program carries no collector, and the way to keep it is
to refuse every dyn rather than to emit a different program: a dyn value is one
the runtime allocates and the collector owns, and there is no smaller version to
fall back to. So it runs between checking and emission, answers unit or raises,
and hands the very same program on. Emit has no field to branch on and is told
nothing.
That is what makes the byte-identity claim true rather than approximate, and it
is tested by compiling three annotated programs twice and comparing the text. A
field, a mode, or a comment that mentioned the flag would break it on something
incidental, a long way from anything to do with dyn.
Every site is named, the way the global cycle refusal names the whole ring: a
reader who has to annotate their program wants the list, not the first one and
then another compile. Globals and signatures as well as body values -- the two
files it is tested against report nine sites each, and the floors are set under
that so an added line does not fail the test and a pass that named one site and
stopped would.
The four programs run at -O2 and -O0. dyn-boundary is asserted on its exit
status as well as its output, because the boundary is only interesting in that
it can fail and a test that showed it working would be testing the easy half.
The x86 survey skips them by name: a REFUSED there means a node that backend has
stopped lowering, which is a regression, and this is the opposite -- a lane that
has not started. Take a name off llvmonly when the lowering arrives and the
survey will say whether it works. 128 match, 0 differ, 0 refused.
Checked while writing these: a dyn function with an early return pops its roots
on both paths, and one with a defer pops on the transfer path too.
dev-chatty.flan outlives the surveys' twenty seconds by design, and unlike
dev-repl it prints while it does -- so the two backends stop at different
lines and the diff reports on scheduling rather than on lowering. It joins
dev-loop and dev-watch in the excluded-by-name list in both sweeps, with the
distinction written down.
rt_flush_out is guarded on __wasm__: the pipe it is careful about belongs to a
merged flan dev, which is only ever a native host, and wasm32 need not answer
for a descriptor mode its runtime may model differently.
And three comments that went false with the _exit: the atexit registration in
the merged entry point is no longer there for rt_die, which unlinks the socket
for itself now, so both places that said so say what it is actually left
covering.
bounds.flan covers this already and the x86 survey cannot see it: the
program picks its case out of (at args 1) and survey.sh runs everything
with no arguments, so re-gating x86's lo <= hi would have failed
nothing. This probe reaches the reversed slice on its own, through
(len args) so that neither optimiser can fold the branch and the checker
has no literal to object to. It matches under the default sweep and
under SURVEY_FLAGS=--no-bounds-checks, which is the claim.
Also records what the same reading turned up and did not fix: x86 still
reports a negative slice-from-ptr promise through flan_slice_error, so
the two backends print different sentences for it, and no corpus program
reaches that case without arguments.
`flan emit --x86` printed a three-line header and then nothing but .byte
blobs. The information was all there and none of it was written down.
Each run of bytes is now headed by the Flan form that produced it, with the
position it was written at, indented by how deeply the form nests. The
headings are queued rather than written, so a form that emits nothing does
not leave its heading on the next form's bytes; atoms queue none at all,
because a literal operand would otherwise steal the heading standing above
the imul that consumes it.
Above each function is a frame map, which is the half no disassembly
recovers: every value in this backend lives in a frame temporary, so
-0x20(%rbp) is the whole vocabulary of the listing and nothing says what it
means. It is read out of what emit_fn already keeps, so it cannot drift.
Beside it, where the arguments arrived and whether there is a hidden sret.
And the bookkeeping is named where it appears -- the transfer guard, the
bounds triple, the arithmetic guards, rep movsb, the dev indirection cell --
with each explained once in a legend at the top rather than at every site.
Always on for `emit --x86`, which exists to be read, and never for a build,
whose .s is a temp file handed to clang. spike/x86/annot.sh is the check that
this costs no byte: emit both ways, assemble both, compare every section.
342 SAME / 0 DIFFER over the corpus in default, --dev and --debug. dump.sh
now shows the annotated listing beside objdump's disassembly -- why beside
what, which is the pairing that answers the mnemonics question.
survey.sh has not been run on this; see the handoff.
@page runs web/examples/check.sh and web/examples/quotes.sh against the compiler
dune just built. @cells runs spike/x86/cells.sh, which was a real pass/fail check
-- four builds, two backends, 22 22 against 42 42 -- that nothing in the tree ran.
@checks is @page, @x86 and @cells together, and its comment argues for where the
boundary sits: everything you can run while making coffee is in, @sanitize and
@valgrind are out because folding tens of minutes in would make the umbrella the
thing nobody has time for, which is the disease rather than the cure.
All three scripts learned to resolve FLAN to an absolute path, which is what
actually stood between them and a dune rule: %{workspace_root} expands relative to
the directory the rule is written in, and every one of these scripts cd's somewhere
before using it. The first run of @page failed with twenty diffs all saying
'../bin/main.exe: No such file or directory', which is at least a failure that says
what is wrong.
docs/BUILT.md carried the same colon-spelled renderer block index.html did, from the
same sweep. Nothing checks BUILT.md, so it is corrected here by hand.
FnAddr (Fnval n) emitted the symbol, which is right for a whole-program build
and wrong the instant anything is redefined into it. It now reads the cell,
and so does every direct call, which is what emit.ml's body_of does and is the
half that matters: a redefinition is one store, and it has to reach call sites
that already exist.
What is emitted, all of it behind dev:
- one cell per function in .data, .globl, initialised to the body this build
compiled. Spelled exactly as Emit.cellname spells it, because the point of
having one here is that an LLVM-built module binds
@"flan.cell.<n>" = external global ptr against it. nm -D over the two
builds of the same program gives identical sets of 68 cell symbols.
- the cell load placed after the arguments, which emit.ml has as a
load-bearing comment: a redefinition landing between two calls must not
land in the middle of one. CallPtr stays the other way round.
- the flan_dev_reg_enable constructor, which arms the allocation registry.
Not emitted: Emit.cellptr, the deeper spelling for a name the host was never
built with. It cannot arise in a whole-program build and belongs with the
redefinition module that would introduce one.
The --x86 --dev refusal is relaxed, and the argument is that flan dev never
reaches this fork: --x86 is read only by flan build, and the daemon builds
host and modules through Build.executable / Build.shared without it. So the
flag means a host whose call sites are redefinable, and nothing claims the
module that would redefine through them exists.
Two things were needed to believe any of that. First, the corpus with --dev on
both sides: 97 MATCH, 0 DIFFER, same as without it. Before the constructor was
added that read 96/1 — registry.flan asks (live? ...) and got four zeroes,
which is the whole of what a dev host does differently besides the cells.
Second, and the corpus cannot do this one: a dev build starts with every cell
pointing at the body this build compiled, so it prints what a release build
prints whether anything reads the cell or not. spike/x86/cells.sh preloads a
shared object whose constructor dlsyms flan.cell.twice and stores a different
body there -- the one store a redefinition ends in, done from outside, no
compiler involved. Both dev builds then print the new answer for a direct call
and for a function value, and both release builds are unchanged, which is what
says the change came from the indirection and not from symbol interposition.
One thing the later lane inherits, now written in both headers rather than
left to be discovered. x86.ml licenses its own calling convention on the
grounds that a dev build is compiled entirely here and a release build
entirely by LLVM, so the two never meet in one process. A cell an LLVM-built
module can store into is the first thing that could make that false: the
conventions agree on scalars and disagree on every aggregate, so an
Emit.redefinition module dlopened into an --x86 host would be right until the
first redefined function took or returned a struct. The answer is a
redefinition emitter here, not a classifier.
Another lane landed (slice-from-ptr p n) while this backend was not looking,
and it arrived as two refusals rather than one: slice-from-ptr.flan and
bounds.flan both stopped building through --x86. Neither is a new obstacle —
a Slice _ is {ptr, i64} here exactly as it is in emit.ml, so the form is one
store of the pointer and one of the length and no new representation at all.
The half worth writing down is the check. There is nothing to compare the
length against — only the caller knows how many elements live behind that
pointer — so what is checked is that the promise is not absurd, and that test
is *signed*. check_slice's own compares are unsigned, and a negative i32
sign-extended to 64 bits is a huge unsigned value that an unsigned "hi <= len"
waves through; the result would be a slice about 2^64 long that reads as a
pass and faults somewhere else entirely.
Nothing in the corpus walks that path: every length in slice-from-ptr.flan is
a literal, and a negative literal is refused by check.ml before any code is
emitted. So spike/x86/p7-slice-from-ptr.flan takes the length as a parameter
and runs it through a restart-case, which puts the condition's low/high/length
on stdout and compares them against the LLVM build.
Four raylib fixtures fail, the same four on two consecutive runs:
images and audio, each at -O2 and -O0. They export to a hardcoded
/tmp path and get "Failed to export wave data" because /tmp is full,
and /tmp is full because of a runaway llc in another lane writing a
5.8 GB m4.o out of a 13 KB m4.ll. Nothing here writes to /tmp by a
fixed name and TMPDIR does not reach those fixtures, so item 17 says
that rather than claiming an unqualified green.
The transfer exit returned whatever the return temporary held where
emit.ml returns zero. Meaningless to a caller -- its guard sees the
channel set and never looks -- but main is a caller with no guard, and
what it finds in rax is the process exit status.
The survey compares stderr as well now, which is where every message
the new machinery produces goes: the bounds and slice errors, the
three restart refusals, the transfer failure. Each carries a location
this backend emits by hand as a .rodata label and a length in a
register, and an exit status of 134 with the wrong text beside it is
exactly the failure that reads as a match. It also walks spike/x86's
own probes.
p6-transfer.flan is the two re-propagation branches the corpus does
not reach. Every transfer in restarts.flan stops at a restart-case
inside the handler-bind's extent, so the handler frames never come off
on the transfer path; and in nested and shadowed the inner frame
offers the name, so a restart-case the transfer is not aimed at never
has to put the target back. allocators.flan already covers the third.
89 MATCH 0 DIFFER 0 refused, over test/programs and spike/x86,
comparing stdout, stderr and the exit status.
DISCUSS.md item 17 is the report.
The transfer channel was the only thing between 41 programs and the
corpus. It is there now: a guard after every Flan call, a landing pad
per restart-case, handler-bind and with-allocator, a transfer exit per
function that runs its fdefers, and check_at and check_slice, which
could not exist until the guard did.
Measured by what the programs print and what they exit with, never by
reading bytes. spike/x86/survey.sh builds every program in
test/programs both ways and diffs stdout and the exit status; it did
not exist, so it is here too, and it is the progress meter.
before 41 MATCH 1 DIFFER 41 refused by name
after 83 MATCH 0 DIFFER 0 refused by name
The one DIFFER was bounds.flan, and it was the honest answer to
"--x86 is silently a --no-bounds-checks build". It is not one any
more: check_at and check_slice signal through the channel exactly as
emit.ml's do, so a bounds violation signals, a restart-case catches
it, and an unhandled one exits 134 on both backends. The transitional
refusal that would have said so retired before it was written.
check_no_transfer is not removed, it is narrowed to the one place the
argument still holds: a global's initialiser runs from
flan..init-globals, before main and before anything can handle
anything, so a transfer out of it has nowhere to go.
Four bugs, and three of them are the shape item 16 predicted -- code
that reads correctly and answers wrong, found by output and not by
objdump:
- The body fell through into the transfer exit, so every fdefer ran
twice on a normal return. emit.ml cannot have this bug: its ret
terminates the block.
- A Vec crossed to the runtime as the address of a *copy*, so pushes
grew the copy and an in-bounds (at v 1) signalled against a length
of zero.
- ucomis sets CF, ZF and PF together for a NaN, so sete answered true
for (= x x) and the prelude's NaN test never fired: (/ 0.0 0.0)
formatted as -9223372036854775808. Flan's comparisons are LLVM's
ordered ones, so < and <= swap and =, != take a setnp beside them.
- A union read field 0 through the struct table and was refused by
name rather than laid out as a tag and a payload.
And one that could not have been found later: emit_globals_init stored
a null *into* the channel slot rather than a cell address into it,
which is a null pointer for every callee to write through. Harmless
while nothing could transfer; a fault the first time a guard loaded
through it.
The corpus sweep found it, and it found it the way item 15 said this work
fails: array-ctor.flan crashed, and the assembly around the crash read
correctly. (set (.x (at pts 0)) 1.5) went through lvalue, lvalue had no
case for At, and the fallback evaluates — so the store landed in a copy of
the element and the array kept its zeros.
emit.ml has this as addr's own At case. One line here, and the program
matches the LLVM build.
Two more programs beside the fizz: one for the internal calling
convention the fizz does not touch at all — a struct argument, a struct
return through the hidden pointer, f32 in the SSE half, eight integer
arguments so two go on the stack, and a slice by pointer — and one for
the rest of the core: a global with an initialiser, recursion, break,
continue, the bitwise family, unsigned shifts and the conversions both
ways. Both agree with LLVM.
al is now zero at every call this backend makes, including the three in
main that were reaching flan_rt_init, flan_argv and flan_exit without it.
Inert on a fixed callee; the point is that there is no exception to the
rule to remember.
x86.ml was an encoder and a frame model with nothing calling it. It now
lowers a whole Tast.program to an assembly file, and `flan build --x86`
hands that file to the same clang invocation the LLVM path uses, against
the same runtime objects. The flag is off by default; LLVM stays the
release backend and the default one.
Three programs, built both ways and compared by what they print and what
they exit with rather than by reading bytes: exit 0; a dotimes that
prints; and a fizz over a call, an if, a remainder and two string
literals. All three agree with the LLVM build.
The measurement decided the target. hist.ml over the fizz program shows
no Signal, no Handled, no RestartCase — a loop that prints does not drag
conditions in. What does is the bounds check and the allocator, and
neither is in the reachable set of a program that prints a number.
That is why there is no transfer guard here, and check_no_transfer is
what makes the omission sound rather than hopeful: if nothing reachable
can write the channel, no call can return with it set. It is a
whole-program property, so it is checked once per build and the build
stops with the node's name when it fails.
