The wrapper was called flan.dev.ctor, and that is a name Flan can reach:
(defn dev.ctor [] i32 7) mangles onto exactly it. The program compiles
and runs as a release build and fails a dev build with a redefinition
clang refuses -- loud, and only under LLVM with --dev, but mangle.ml's
own comment exists to make it impossible rather than loud. The name is
[Mangle.dev_ctor] now and starts with the dot no reader token can
produce, beside .init-globals and .init-data. The colliding program
builds and prints 7.
Two comments in survey.sh, both of them reasoning rather than
behaviour. The counts argument against a per-name -O0 list was no
argument: excluding moves the counts just as much. What actually
carries it is that dies_segv builds both programs at -O0 on both
backends and asserts more than this sweep would. And dev-segv's park
under --dev is a forever-list reason that happens to land on a program
this list already covers, not a second reason for this list.
FIX.org takes the sweep, and one thing the sweep cannot say: the two
heap cases of bytes-copy.flan leak 24 bytes through flan_bytes_dup,
measured with --leak-check=full, and both corpora are blind to it by
policy -- detect_leaks=0 on one side and --leak-check=no on the other.
The row proves the copy is in bounds and written. It says nothing about
who frees it, and a green sweep should not be read as though it did.
The two new fault-by-design programs came into the corpus with the
bytes/bytes-view lane and the survey has been comparing them at its own
optimisation level, where the store into .rodata is undefined, LLVM
deletes it and exits 0, and this backend -- which has no optimiser to
delete anything with -- executes it and takes 139. That reads as a
lowering disagreement and is not one: at -O0 the two backends agree
exactly, and test_acceptance.ml's dies_segv rows already pin that on
both of them.
So a second exclusion list beside the one for the programs that never
stop, with its own reason written down, rather than building the whole
corpus at -O0 and changing the measurement every baseline was taken
against. dev-segv would belong on it in any case: it calls agent/start,
and under --dev it parks in the break loop rather than dying.
MATCH 171, DIFFER 0, REFUSED 0, NOX86 0, SKIP 47
x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
A defer is in the typed IR twice -- spliced into the body for the normal path,
and again in fdefers for the path a transfer leaves through -- so a dyn
temporary inside one is emitted twice. dyn_roots counted only the body's, and
the second copy went into slots nothing had rooted.
Nothing failed, and that is the whole reason this is worth a commit of its own.
dyn_tmp falls back to a plain slot rather than unbalancing the stack, so the
pushes and the pops still matched, the program ran and printed the right answer,
and the values were simply invisible. Against a stub that never collects there is
no symptom to find -- no leak, no crash, no wrong number. It would have become a
symptom the week the real collector landed, in a defer reached only on a handled
condition, which is close to the worst place to start looking.
What found it was the IR: a rooted slot is spelled %dr and the fallback %dx, and
the assertion is that no dyn program in the corpus emits one of the latter. That
is now a test over all five dyn programs, and it is the only check in the lane
that can see a missing root while there is still nothing to lose one by. When
the collector arrives it is the thing to extend rather than replace.
Also checked, both clean: flan dev --llvm builds and runs a dyn program, which
is the route the x86 refusal sends people to and would have been a link error in
the worst possible place; and the daemon's own refusal already names the flag.
The promise is that this program carries no collector, and the way to keep it is
to refuse every dyn rather than to emit a different program: a dyn value is one
the runtime allocates and the collector owns, and there is no smaller version to
fall back to. So it runs between checking and emission, answers unit or raises,
and hands the very same program on. Emit has no field to branch on and is told
nothing.
That is what makes the byte-identity claim true rather than approximate, and it
is tested by compiling three annotated programs twice and comparing the text. A
field, a mode, or a comment that mentioned the flag would break it on something
incidental, a long way from anything to do with dyn.
Every site is named, the way the global cycle refusal names the whole ring: a
reader who has to annotate their program wants the list, not the first one and
then another compile. Globals and signatures as well as body values -- the two
files it is tested against report nine sites each, and the floors are set under
that so an added line does not fail the test and a pass that named one site and
stopped would.
The four programs run at -O2 and -O0. dyn-boundary is asserted on its exit
status as well as its output, because the boundary is only interesting in that
it can fail and a test that showed it working would be testing the easy half.
The x86 survey skips them by name: a REFUSED there means a node that backend has
stopped lowering, which is a regression, and this is the opposite -- a lane that
has not started. Take a name off llvmonly when the lowering arrives and the
survey will say whether it works. 128 match, 0 differ, 0 refused.
Checked while writing these: a dyn function with an early return pops its roots
on both paths, and one with a defer pops on the transfer path too.
dev-chatty.flan outlives the surveys' twenty seconds by design, and unlike
dev-repl it prints while it does -- so the two backends stop at different
lines and the diff reports on scheduling rather than on lowering. It joins
dev-loop and dev-watch in the excluded-by-name list in both sweeps, with the
distinction written down.
rt_flush_out is guarded on __wasm__: the pipe it is careful about belongs to a
merged flan dev, which is only ever a native host, and wasm32 need not answer
for a descriptor mode its runtime may model differently.
And three comments that went false with the _exit: the atexit registration in
the merged entry point is no longer there for rt_die, which unlinks the socket
for itself now, so both places that said so say what it is actually left
covering.
FnAddr (Fnval n) emitted the symbol, which is right for a whole-program build
and wrong the instant anything is redefined into it. It now reads the cell,
and so does every direct call, which is what emit.ml's body_of does and is the
half that matters: a redefinition is one store, and it has to reach call sites
that already exist.
What is emitted, all of it behind dev:
- one cell per function in .data, .globl, initialised to the body this build
compiled. Spelled exactly as Emit.cellname spells it, because the point of
having one here is that an LLVM-built module binds
@"flan.cell.<n>" = external global ptr against it. nm -D over the two
builds of the same program gives identical sets of 68 cell symbols.
- the cell load placed after the arguments, which emit.ml has as a
load-bearing comment: a redefinition landing between two calls must not
land in the middle of one. CallPtr stays the other way round.
- the flan_dev_reg_enable constructor, which arms the allocation registry.
Not emitted: Emit.cellptr, the deeper spelling for a name the host was never
built with. It cannot arise in a whole-program build and belongs with the
redefinition module that would introduce one.
The --x86 --dev refusal is relaxed, and the argument is that flan dev never
reaches this fork: --x86 is read only by flan build, and the daemon builds
host and modules through Build.executable / Build.shared without it. So the
flag means a host whose call sites are redefinable, and nothing claims the
module that would redefine through them exists.
Two things were needed to believe any of that. First, the corpus with --dev on
both sides: 97 MATCH, 0 DIFFER, same as without it. Before the constructor was
added that read 96/1 — registry.flan asks (live? ...) and got four zeroes,
which is the whole of what a dev host does differently besides the cells.
Second, and the corpus cannot do this one: a dev build starts with every cell
pointing at the body this build compiled, so it prints what a release build
prints whether anything reads the cell or not. spike/x86/cells.sh preloads a
shared object whose constructor dlsyms flan.cell.twice and stores a different
body there -- the one store a redefinition ends in, done from outside, no
compiler involved. Both dev builds then print the new answer for a direct call
and for a function value, and both release builds are unchanged, which is what
says the change came from the indirection and not from symbol interposition.
One thing the later lane inherits, now written in both headers rather than
left to be discovered. x86.ml licenses its own calling convention on the
grounds that a dev build is compiled entirely here and a release build
entirely by LLVM, so the two never meet in one process. A cell an LLVM-built
module can store into is the first thing that could make that false: the
conventions agree on scalars and disagree on every aggregate, so an
Emit.redefinition module dlopened into an --x86 host would be right until the
first redefined function took or returned a struct. The answer is a
redefinition emitter here, not a classifier.
The transfer exit returned whatever the return temporary held where
emit.ml returns zero. Meaningless to a caller -- its guard sees the
channel set and never looks -- but main is a caller with no guard, and
what it finds in rax is the process exit status.
The survey compares stderr as well now, which is where every message
the new machinery produces goes: the bounds and slice errors, the
three restart refusals, the transfer failure. Each carries a location
this backend emits by hand as a .rodata label and a length in a
register, and an exit status of 134 with the wrong text beside it is
exactly the failure that reads as a match. It also walks spike/x86's
own probes.
p6-transfer.flan is the two re-propagation branches the corpus does
not reach. Every transfer in restarts.flan stops at a restart-case
inside the handler-bind's extent, so the handler frames never come off
on the transfer path; and in nested and shadowed the inner frame
offers the name, so a restart-case the transfer is not aimed at never
has to put the target back. allocators.flan already covers the third.
89 MATCH 0 DIFFER 0 refused, over test/programs and spike/x86,
comparing stdout, stderr and the exit status.
DISCUSS.md item 17 is the report.
The transfer channel was the only thing between 41 programs and the
corpus. It is there now: a guard after every Flan call, a landing pad
per restart-case, handler-bind and with-allocator, a transfer exit per
function that runs its fdefers, and check_at and check_slice, which
could not exist until the guard did.
Measured by what the programs print and what they exit with, never by
reading bytes. spike/x86/survey.sh builds every program in
test/programs both ways and diffs stdout and the exit status; it did
not exist, so it is here too, and it is the progress meter.
before 41 MATCH 1 DIFFER 41 refused by name
after 83 MATCH 0 DIFFER 0 refused by name
The one DIFFER was bounds.flan, and it was the honest answer to
"--x86 is silently a --no-bounds-checks build". It is not one any
more: check_at and check_slice signal through the channel exactly as
emit.ml's do, so a bounds violation signals, a restart-case catches
it, and an unhandled one exits 134 on both backends. The transitional
refusal that would have said so retired before it was written.
check_no_transfer is not removed, it is narrowed to the one place the
argument still holds: a global's initialiser runs from
flan..init-globals, before main and before anything can handle
anything, so a transfer out of it has nowhere to go.
Four bugs, and three of them are the shape item 16 predicted -- code
that reads correctly and answers wrong, found by output and not by
objdump:
- The body fell through into the transfer exit, so every fdefer ran
twice on a normal return. emit.ml cannot have this bug: its ret
terminates the block.
- A Vec crossed to the runtime as the address of a *copy*, so pushes
grew the copy and an in-bounds (at v 1) signalled against a length
of zero.
- ucomis sets CF, ZF and PF together for a NaN, so sete answered true
for (= x x) and the prelude's NaN test never fired: (/ 0.0 0.0)
formatted as -9223372036854775808. Flan's comparisons are LLVM's
ordered ones, so < and <= swap and =, != take a setnp beside them.
- A union read field 0 through the struct table and was refused by
name rather than laid out as a tag and a payload.
And one that could not have been found later: emit_globals_init stored
a null *into* the channel slot rather than a cell address into it,
which is a null pointer for every callee to write through. Harmless
while nothing could transfer; a fault the first time a guard loaded
through it.
