FnAddr (Fnval n) emitted the symbol, which is right for a whole-program build
and wrong the instant anything is redefined into it. It now reads the cell,
and so does every direct call, which is what emit.ml's body_of does and is the
half that matters: a redefinition is one store, and it has to reach call sites
that already exist.
What is emitted, all of it behind dev:
- one cell per function in .data, .globl, initialised to the body this build
compiled. Spelled exactly as Emit.cellname spells it, because the point of
having one here is that an LLVM-built module binds
@"flan.cell.<n>" = external global ptr against it. nm -D over the two
builds of the same program gives identical sets of 68 cell symbols.
- the cell load placed after the arguments, which emit.ml has as a
load-bearing comment: a redefinition landing between two calls must not
land in the middle of one. CallPtr stays the other way round.
- the flan_dev_reg_enable constructor, which arms the allocation registry.
Not emitted: Emit.cellptr, the deeper spelling for a name the host was never
built with. It cannot arise in a whole-program build and belongs with the
redefinition module that would introduce one.
The --x86 --dev refusal is relaxed, and the argument is that flan dev never
reaches this fork: --x86 is read only by flan build, and the daemon builds
host and modules through Build.executable / Build.shared without it. So the
flag means a host whose call sites are redefinable, and nothing claims the
module that would redefine through them exists.
Two things were needed to believe any of that. First, the corpus with --dev on
both sides: 97 MATCH, 0 DIFFER, same as without it. Before the constructor was
added that read 96/1 — registry.flan asks (live? ...) and got four zeroes,
which is the whole of what a dev host does differently besides the cells.
Second, and the corpus cannot do this one: a dev build starts with every cell
pointing at the body this build compiled, so it prints what a release build
prints whether anything reads the cell or not. spike/x86/cells.sh preloads a
shared object whose constructor dlsyms flan.cell.twice and stores a different
body there -- the one store a redefinition ends in, done from outside, no
compiler involved. Both dev builds then print the new answer for a direct call
and for a function value, and both release builds are unchanged, which is what
says the change came from the indirection and not from symbol interposition.
One thing the later lane inherits, now written in both headers rather than
left to be discovered. x86.ml licenses its own calling convention on the
grounds that a dev build is compiled entirely here and a release build
entirely by LLVM, so the two never meet in one process. A cell an LLVM-built
module can store into is the first thing that could make that false: the
conventions agree on scalars and disagree on every aggregate, so an
Emit.redefinition module dlopened into an --x86 host would be right until the
first redefined function took or returned a struct. The answer is a
redefinition emitter here, not a classifier.
Another lane landed (slice-from-ptr p n) while this backend was not looking,
and it arrived as two refusals rather than one: slice-from-ptr.flan and
bounds.flan both stopped building through --x86. Neither is a new obstacle —
a Slice _ is {ptr, i64} here exactly as it is in emit.ml, so the form is one
store of the pointer and one of the length and no new representation at all.
The half worth writing down is the check. There is nothing to compare the
length against — only the caller knows how many elements live behind that
pointer — so what is checked is that the promise is not absurd, and that test
is *signed*. check_slice's own compares are unsigned, and a negative i32
sign-extended to 64 bits is a huge unsigned value that an unsigned "hi <= len"
waves through; the result would be a slice about 2^64 long that reads as a
pass and faults somewhere else entirely.
Nothing in the corpus walks that path: every length in slice-from-ptr.flan is
a literal, and a negative literal is refused by check.ml before any code is
emitted. So spike/x86/p7-slice-from-ptr.flan takes the length as a parameter
and runs it through a restart-case, which puts the condition's low/high/length
on stdout and compares them against the LLVM build.
Four raylib fixtures fail, the same four on two consecutive runs:
images and audio, each at -O2 and -O0. They export to a hardcoded
/tmp path and get "Failed to export wave data" because /tmp is full,
and /tmp is full because of a runaway llc in another lane writing a
5.8 GB m4.o out of a 13 KB m4.ll. Nothing here writes to /tmp by a
fixed name and TMPDIR does not reach those fixtures, so item 17 says
that rather than claiming an unqualified green.
The transfer exit returned whatever the return temporary held where
emit.ml returns zero. Meaningless to a caller -- its guard sees the
channel set and never looks -- but main is a caller with no guard, and
what it finds in rax is the process exit status.
The survey compares stderr as well now, which is where every message
the new machinery produces goes: the bounds and slice errors, the
three restart refusals, the transfer failure. Each carries a location
this backend emits by hand as a .rodata label and a length in a
register, and an exit status of 134 with the wrong text beside it is
exactly the failure that reads as a match. It also walks spike/x86's
own probes.
p6-transfer.flan is the two re-propagation branches the corpus does
not reach. Every transfer in restarts.flan stops at a restart-case
inside the handler-bind's extent, so the handler frames never come off
on the transfer path; and in nested and shadowed the inner frame
offers the name, so a restart-case the transfer is not aimed at never
has to put the target back. allocators.flan already covers the third.
89 MATCH 0 DIFFER 0 refused, over test/programs and spike/x86,
comparing stdout, stderr and the exit status.
DISCUSS.md item 17 is the report.
The transfer channel was the only thing between 41 programs and the
corpus. It is there now: a guard after every Flan call, a landing pad
per restart-case, handler-bind and with-allocator, a transfer exit per
function that runs its fdefers, and check_at and check_slice, which
could not exist until the guard did.
Measured by what the programs print and what they exit with, never by
reading bytes. spike/x86/survey.sh builds every program in
test/programs both ways and diffs stdout and the exit status; it did
not exist, so it is here too, and it is the progress meter.
before 41 MATCH 1 DIFFER 41 refused by name
after 83 MATCH 0 DIFFER 0 refused by name
The one DIFFER was bounds.flan, and it was the honest answer to
"--x86 is silently a --no-bounds-checks build". It is not one any
more: check_at and check_slice signal through the channel exactly as
emit.ml's do, so a bounds violation signals, a restart-case catches
it, and an unhandled one exits 134 on both backends. The transitional
refusal that would have said so retired before it was written.
check_no_transfer is not removed, it is narrowed to the one place the
argument still holds: a global's initialiser runs from
flan..init-globals, before main and before anything can handle
anything, so a transfer out of it has nowhere to go.
Four bugs, and three of them are the shape item 16 predicted -- code
that reads correctly and answers wrong, found by output and not by
objdump:
- The body fell through into the transfer exit, so every fdefer ran
twice on a normal return. emit.ml cannot have this bug: its ret
terminates the block.
- A Vec crossed to the runtime as the address of a *copy*, so pushes
grew the copy and an in-bounds (at v 1) signalled against a length
of zero.
- ucomis sets CF, ZF and PF together for a NaN, so sete answered true
for (= x x) and the prelude's NaN test never fired: (/ 0.0 0.0)
formatted as -9223372036854775808. Flan's comparisons are LLVM's
ordered ones, so < and <= swap and =, != take a setnp beside them.
- A union read field 0 through the struct table and was refused by
name rather than laid out as a tag and a payload.
And one that could not have been found later: emit_globals_init stored
a null *into* the channel slot rather than a cell address into it,
which is a null pointer for every callee to write through. Harmless
while nothing could transfer; a fault the first time a guard loaded
through it.
The corpus sweep found it, and it found it the way item 15 said this work
fails: array-ctor.flan crashed, and the assembly around the crash read
correctly. (set (.x (at pts 0)) 1.5) went through lvalue, lvalue had no
case for At, and the fallback evaluates — so the store landed in a copy of
the element and the array kept its zeros.
emit.ml has this as addr's own At case. One line here, and the program
matches the LLVM build.
Two more programs beside the fizz: one for the internal calling
convention the fizz does not touch at all — a struct argument, a struct
return through the hidden pointer, f32 in the SSE half, eight integer
arguments so two go on the stack, and a slice by pointer — and one for
the rest of the core: a global with an initialiser, recursion, break,
continue, the bitwise family, unsigned shifts and the conversions both
ways. Both agree with LLVM.
al is now zero at every call this backend makes, including the three in
main that were reaching flan_rt_init, flan_argv and flan_exit without it.
Inert on a fixed callee; the point is that there is no exception to the
rule to remember.
x86.ml was an encoder and a frame model with nothing calling it. It now
lowers a whole Tast.program to an assembly file, and `flan build --x86`
hands that file to the same clang invocation the LLVM path uses, against
the same runtime objects. The flag is off by default; LLVM stays the
release backend and the default one.
Three programs, built both ways and compared by what they print and what
they exit with rather than by reading bytes: exit 0; a dotimes that
prints; and a fizz over a call, an if, a remainder and two string
literals. All three agree with the LLVM build.
The measurement decided the target. hist.ml over the fizz program shows
no Signal, no Handled, no RestartCase — a loop that prints does not drag
conditions in. What does is the bounds check and the allocator, and
neither is in the reachable set of a program that prints a number.
That is why there is no transfer guard here, and check_no_transfer is
what makes the omission sound rather than hopeful: if nothing reachable
can write the channel, no call can return with it set. It is a
whole-program property, so it is checked once per build and the build
stops with the node's name when it fails.
The spike proved the shape; this makes it the feature. A generic body is
still checked abstractly once, but now it may be told what to assume:
{:where (ordered? $t)} at the head of the body, Clojure's {:pre [...]}
spelling, with five predicates - ordered?, equal?, hashable?, numeric?
and copyable?.
The syntax catch settled structurally: {K V} is still a legal return
type, and a constraint map is told from one by its leading keyword. A
keyword is not a type anywhere in the language, so the slot after the
return type is unambiguous and {K V} did not have to go.
A type variable is move-only by default, with copyable? the opt-out.
Move is the stricter rule, so assuming it can only refuse a valid
program, never admit a bad one. That is Rust's T: Copy and not Odin's
anything - Odin has no move semantics at all.
The runaway refusal no longer names a depth. It names the chain: a
generic already on the instantiation stack, asked for again at a type
built around the one it had before, is growing and will not stop.
The spike counted nodes it could do. This counts nodes a real input
actually contains, after Reach prunes, which is the question that decides
whether whole-program coverage is reachable.
The answer is worse than the spike's four buckets suggested: enum-compare
needs Str, Make, Field and Call before it prints anything, because the
prelude builds a slice to print one. And loops.flan carries Handled,
RestartCase and Signal one each -- conditions are not an advanced feature
to defer, they are in the reachable set of a program that only loops.
The same three shapes written in C and as first-class IR aggregates,
compiled by the same clang. { i8, i64 } agrees. { i8, float } does not: C
packs both halves into rax, the IR form answers in al and xmm0. And a
24-byte struct does not agree at all -- C spills through an sret pointer,
the IR form returns it in rax, rdx and rcx, and rcx is a register SysV never
uses for a return value.
That resolves the ret-big anomaly the first pass noted and moved past, and
it makes the finding stronger than it was written: the internal convention
is not the C ABI, not only undocumented in the emitted IR.
spike_call0 deleted with it -- declared, never bound, and the two unused
probes were removed for the same reason.
The verdict, as DISCUSS.md item 15. One function goes from Tast to machine
code and answers correctly, so the question is not whether it can be done.
Three findings decide the shape. Layout is already owned -- emit.ml computes
C struct layout for DWARF and is tested against LLVM's own answer -- so the
silent-drift risk item 10 feared most does not arise. The C boundary is the
easy half, because check.ml already rejects aggregates in a declare and the
shim flattens them. And the hard half was not on anyone's list: Flan calling
Flan passes aggregates by value, and LLVM's lowering of a first-class struct
is per-field rather than the C psABI -- { i8, float } comes back in al and
xmm0 where C would pack it into rax, and a %vec return takes a hidden sret
pointer that does not appear in the define line. The internal convention is
an implementation, not a document.
The audit stands on its own: overflow, shifts and evaluation order are
defined; division by zero, INT64_MIN/-1, the float cast, Uninit and
unreachable are not. Uninit is the one that bites, because poison is where
the two backends are supposed to differ.
Unloading: the shadow stack answers the running half and every dev-build
function is on it -- only the slot table is gated, not the frame. It cannot
answer the pointed-into half, which BUILT.md says is the actual reason
nothing is dlclose'd. Escaped function values need a rule the language does
not have.
SPIKE_DISASM=1 objdumps the exact buffers that ran. Kept behind a flag and
kept out of the pass/fail path: a disassembly that reads correctly beside a
function answering the wrong number is the normal outcome of hand-encoding.
Every stack movement now goes through pushv/popv and increments a depth
word on the function context. A call pads to 16 from wherever the expression
evaluator has left rsp, and asserts the parity before it emits the call.
The nested probe passes.
The stack-argument path is folded into the same counter rather than keeping
its own, because two independent notions of parity is how the bug comes back.
Three synthetic Tast functions calling C: eight integers so two go on the
stack, and a callee that does a 16-byte aligned spill and answers -1 if it
was entered with rsp misaligned. The third calls it from inside a binary
operator.
The third fails. Alignment at a call site is not a property of the prologue
-- it is a property of how much the expression evaluator has pushed, and
the evaluator spills the left operand across the right one's evaluation. A
call in that right operand runs 8 bytes off. Nothing in the arithmetic tests
could see it, because they call nothing that spills a vector register.
This is the raylib failure mode exactly, and it is left red for one commit
so the record shows the probe found it rather than agreeing with the code.
x86.ml is an instruction selector for the part of Tast that fits in one
integer register: literals, slots, let, if, arithmetic, comparison, and a
call. Everything else raises with the node that defeated it, because an
honest refusal is the measurement and a silently wrong answer would waste
the exercise.
The frontend is the real one -- Reader, Parse, Load, Check -- so what is
lowered is the same Tast.fn the LLVM backend gets. Seven arithmetic results
are compared against what the language says they should be; the disassembly
proves nothing and is not the evidence.
Nothing is wired into the build. No dune file under spike/, driven by hand
with ocamlfind and clang as spike/embed already does.
The first six probes each proved a piece. merged.sh puts them together: the
program's @main is renamed out of the way, a C main takes the main thread and
runs it there, caml_startup happens on a thread beside it, and clang links the
lot -- the emitted program object, flan_rt.c, flan_dev.c, flan_agent.c and the
whole compiler as one -output-complete-obj. It runs, and the compiler inside it
compiles the very source the program was built from.
Nothing is wired up. The two halves share an address space and do not speak.
That is the point: the question was whether they can, not what they would say.
sig.sh and symbols.sh answer the two questions the first pass got wrong or
skipped. The SIGSEGV reading in harness5.c was taken at the wrong moment --
OCaml 5 starts domains after caml_startup returns, so the disposition had to be
read from inside the runtime, and against a plain ocamlopt executable as a
control. symbols.sh is the hazard nobody looks for until the link fails: four
.c files that are compiled into two different processes today, and the OCaml
runtime, all landing in one link.
Item 12 asks five questions and says to answer them with a spike rather than a
rewrite. spike/embed/ is that spike: one script, six binaries, each one built to
fail loudly at the thing it is asking about. It is deliberately not a dune
target -- the root dune only excludes old-ocaml/, so a dune file here would land
in @default and make the spike part of the build. It drives ocamlfind and clang
by hand against the flan.cmxa dune already produces.
The probes, in the order they would kill the idea: the smallest possible link, a
C main() reaching one OCaml function; the whole compiler linked in and doing
real work; the same again with lib/dynload_stubs.c from the unmerged dlopen
branch, because that is the only C the compiler itself is built from; the game
keeping the main thread while caml_startup happens on a pthread beside it; the
SIGSEGV disposition read on both sides of caml_startup; and an 8 MiB arena
checked byte for byte across a compaction.
No result is written down yet. This is the apparatus.
