JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

The dev backend, wired — and conditions were never in the way

Browse Source 
DISCUSS.md item 16, in item 15's form.

The headline is a measurement that contradicts what everyone expected,
this report included: a program that loops and prints carries no Signal,
no Handled and no RestartCase. Zero of each, measured with hist.ml before
a line was written. What drags conditions in is the bounds check and the
allocator — check_at signals, and signalling needs the channel and the
guard — and neither is a Tast node, which is why a histogram cannot see
either and why the guess was off.

41 of the 111 programs in test/programs build through --x86 and 40 print
exactly what the LLVM build prints. The 41st is bounds.flan and it
diverges on purpose: --x86 is silently a --no-bounds-checks build, and
that is named as a divergence rather than left in a code comment.

The other 40 are refused by name. check_no_transfer is what makes the
missing guard sound instead of hopeful — if nothing reachable can write
the channel, no call can return with it set — so the build stops with the
node and the function rather than compiling forty programs wrongly.

Both bugs found along the way are the shape item 15 predicted: the crash
and the cause in different functions, and correct-reading assembly beside
a wrong answer. The frame model is the thing that did not go wrong.
This commit is contained in:
Joseph Ferano 2026-09-13 15:14:33 +07:00
parent 58b1f49cf2
commit fc4e51a106
1 changed files with 183 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

183
DISCUSS.md
View File

@ -913,3 +913,186 @@ language has, and that rule is worth having whether or not a backend is ever wri
3. **Choose option 1 or option 2 from question 3**, deliberately. Everything else follows from it.
4. **Only then**, and only if 3 says so, grow `spike/backend/x86.ml` from the node table in question 2 — floats
first, because they gate most of the prelude, and conditions last, because they are the only row with no plan.
## 16. The dev backend, wired: a whole program compiles through `x86.ml` and runs, and conditions were never in the way
Item 15's step 4, taken. `lib/x86.ml` was 502 lines of encoder and frame model with nothing calling it. It now lowers a
whole `Tast.program` to an assembly file, and `flan build --x86` hands that file to the same clang invocation the LLVM
path uses, against the same runtime objects, the same generated shim and the same linker arguments. The flag is off by
default and refused in combination with `--dev`, `--debug`, `--sanitize` and every wasm target. **LLVM stays the release
backend and the default one; nothing on the existing path changed.** `dune test --root .` is green either side.
**41 programs out of `test/programs` build through it, and 40 of them print exactly what the LLVM build prints.** The
41st is `bounds.flan`, and it diverges on purpose — see question 4.
### Question 1 — which program, and what did it actually cost
Measured with `spike/backend/hist.ml` before anything was written, over candidates, not guessed. The one picked is
`spike/x86/p3-fizz.flan` — a `dotimes`, a call, an `if`, a remainder, two string literals and `print`/`println`:
```
p3-fizz.flan: 2 reachable fns
Call 1 Do 4 If 1 Int 15 Let 2 Local 7 Prim 14
Set 1 Str 3 While 1 place/Plocal 1
prim/Add 2 prim/Bytes 3 prim/Cast 1 prim/Eq 1
prim/I64ToBytes 1 prim/Lt 1 prim/Rem 1 prim/WriteStdout 4
```
Nineteen rows, two reachable functions after `Reach` prunes, and **no `Signal`, no `Handled`, no `RestartCase`, no
`Make`, no `Field`, no allocator**. The same is true of a program that only loops and prints: zero of each.
That is worth stating flatly because the expectation going in was the opposite — that the smallest useful program
carries one of each condition node, and that printing a single value drags in `Str`, `Make`, `Field` and `Call` because
the prelude builds a slice to do it. **It does not.** Printing an integer is `Cast` → `I64ToBytes` → `WriteStdout`,
three prims and no call at all; `flan_i64_to_bytes` renders into a static buffer in `flan_rt.c` and hands back a slice.
Printing a string literal is `Str` → `Bytes` → `WriteStdout`, and `Bytes` is a non-instruction because a string and a
`[u8]` are the same two words.
**What does drag conditions in is the bounds check and the allocator, and neither is a `Tast` node.** `check_at` and
`check_slice` call `flan_bounds_error` with the transfer channel and then `guard`; `Rt flan_vec_at` takes the channel
too. So it is `m.checks` and the container runtime that make a program need the condition machinery, not looping and
not printing — and because both are *emit-time* constructs rather than IR nodes, `hist.ml` cannot see either. That is
the correction to item 15's node table: the "no plan" row is not reached by writing a loop, it is reached by writing
`(at a i)`.
### Question 2 — what runs
| program | what it is for |
|---|---|
| `spike/x86/p1-exit.flan` | `main` returns 0. The assembly path, the runtime link, a real executable. |
| `spike/x86/p2-loop-print.flan` | a `dotimes` that prints — the smallest program that does something |
| `spike/x86/p3-fizz.flan` | the measured target: a call, an `if`, `%`, two string literals |
| `spike/x86/p4-convention.flan` | the *internal calling convention*, which p3 does not touch at all |
| `spike/x86/p5-core.flan` | a global with an initialiser, recursion, `break`, `continue`, the bitwise family, unsigned shifts, both directions of every conversion |
p4 is the one that matters most, because item 15 named the internal aggregate convention as the sharpest obstacle in
the whole report. It passes a struct by value, returns a struct by value, puts an `f32` through the SSE half, calls an
eight-argument function so that two arguments go on the stack, and passes a slice — and it agrees with LLVM. **The
obstacle really did dissolve the way the header claims**: a dev build is compiled entirely here and a release build
entirely by LLVM, the two never meet in one process, so the convention is ours to pick. Every aggregate goes by
pointer, an aggregate return is a hidden pointer in the first integer register returned in `rax`, and there is no
classifier in the file. Nothing had to be discovered by disassembling clang.
Over `test/programs` (111 files):
| | n |
|---|---|
| built through `--x86` and **matched** the LLVM build's output and exit status | **40** |
| built and diverged — `bounds.flan`, by design | 1 |
| refused by name: a node this backend does not lower | 40 |
| no `main` (package and library fixtures) | 6 |
| do not compile at all (the checker-error fixtures) | 22 |
| never terminate on their own (`dev-loop`, `dev-watch`) | 2 |
So of the 81 programs that compile, have a `main` and finish, **41 went through the hand-written backend and 40 were
byte-identical in output.** That includes `edn.flan` — sixty lines of output from a hand-written EDN reader with
unions, options, nested collections and a fixed-depth balance stack.
Proved by comparing output, never by reading bytes. The script builds both ways and diffs stdout and the exit status;
`objdump` was used only after a program already had the wrong answer. Item 15 is right that this is the only honest
order.
### Question 3 — the two bugs, and both are the shape item 15 predicted
**A `(set (.x (at pts 0)) 1.5)` wrote into a copy.** `lvalue` had no case for `At`, so it fell through to "evaluate it",
and the store landed in a temporary while the array kept its zeros. `emit.ml` has this as `addr`'s own `At` case. One
line. `array-ctor.flan` found it, and it found it as a segfault several statements later.
**A discarded value was stored over the return address.** This is the better one. A form whose value is thrown away was
handed a sink, and the sink was spelled as an address — `rbp+0`. That is the saved `rbp`, and `rbp+8` is the return
address, so a non-void form written in statement position stored straight over both; a 16-byte slice did it in one
`rep movsb`. `edn.flan` crashed by jumping into `.rodata`, **several statements after the mistake and in a different
function**, and the assembly at the jump read perfectly. The sink is now compared by identity and never used as an
address: anything with a value that is handed it gets a frame temporary instead.
The second bug cannot exist on the LLVM path, and that is the general point. LLVM has no notion of "store this value
nowhere" — an unused SSA value is simply unused. Every construct this backend has that LLVM does not is a place where
a bug can live that the LLVM backend's own testing can never have covered.
Against that, the thing item 15 was most worried about did *not* happen: **nothing went wrong with the frame or the
stack alignment.** `rsp` is written exactly twice — one rounded `sub` in the prologue that covers the temporaries and
the outgoing-argument area together, and `leave` — so `rsp % 16 == 0` at every call site is a property of one
subtraction rather than an invariant every case maintains. The spike's worst bug has no door to come in by, and p4
calls an eight-argument function to prove it.
### Question 4 — where the two backends now differ, and it is three named places
Item 15's question 4 listed nine undefined cases. Three of them are now *real* divergences with a build on each side,
and they should be written down before anyone uses this for anything.
| | LLVM | here |
|---|---|---|
| **A bounds violation** | `flan_bounds_error` signals; a `restart-case` can catch it; `bounds.flan` exits 134 | **no check at all**; `bounds.flan` exits 139 |
| `(uninit)` | `poison`, and the optimiser may reason from it | whatever the stack slot held — stable garbage |
| an exhausted `match`, a `noreturn` call | `unreachable`, undefined | `ud2` — a defined SIGILL at the instruction that fell through |
**The first is the one that matters, and it is not a footnote: `--x86` is silently a `--no-bounds-checks` build.** It is
silent because it is not a decision the backend made — `check_at` signals, signalling needs the channel and the guard,
and there is no guard here, so there is no check. `bounds.flan` is the direct evidence and `edn.flan`'s 33-brackets-
against-a-32-deep-stack case is the second. Anyone reaching for this flag on a program that indexes anything should
know that the trap is gone.
The other two are improvements and cost nothing. `ud2` in particular is two bytes and turns a class of miscompile into
a crash with an address.
### Question 5 — conditions, which is still the row with no plan
**They were not reached, and converting that into a checked precondition is the most useful thing in this report.**
There is no transfer guard after a call here, no landing pad and no transfer exit. `emit.ml` emits a guard after *every*
call; this emits none. What makes that sound is a whole-program argument rather than a hope: **if nothing in the
reachable set can ever write the channel, no call can ever return with it set.** So `check_no_transfer` walks the
linked program once per build and stops it — with the node's name and the function it is in — the moment it finds a
`signal`, an `invoke-restart`, a `restart-case`, a `handler-bind`, a `with-allocator`, or the two `Rt` symbols whose
bounds check signals.
That is what the 40 refusals are:
```
27 restart-case 4 handler-bind 1 with-allocator
7 signal 1 defers on the transfer path
```
Forty programs refused by name rather than miscompiled, and one line of build output says which node and where. A
backend that quietly omitted the guard would have compiled all forty and been wrong in a way no test distinguishes
from a race.
What this does *not* do is measure what conditions cost. That is still unknown, and it is still the only row of item
15's table with nothing behind it. What is now known is the shape of the bill: the guard is per call site, the pad is
per `restart-case` activation, `fdefers` needs a second exit path that no form in `body` can reach, and **any function
with `fdefers` at all is refused today** — which is most of the prelude's file and container code, and is why the
programs that use a `Vec` are not in the 40.
### The honest no-plan bucket
Everything below is refused by name at build time, not silently wrong.
- **Conditions, entire** — the guard, the landing pad, `emit_restart_case`, `emit_with_alloc`, the transfer exit, and
`fdefers` on it. Several hundred lines of `emit.ml` reimplemented from `spec-conditions.md` rather than ported.
- **Bounds checks**, which are the same work: `check_at` and `check_slice` cannot exist without the guard.
- **`Rt` with an aggregate return**, and with it most of the container runtime; `Vec`, `Map` and `Pool` have not been
exercised at all.
- **`Fnval`'s indirection cell.** `FnAddr (Fnval n)` emits the symbol, which is correct for a whole-program build and
wrong the instant anything is redefined into it. This backend has no cells and no `--dev`; that is a deliberate
restriction and not an oversight, but it is exactly item 15's question 5 waiting where it was left.
- **`f64` → `i64` out of range**, and **`INT64_MIN / -1`**. `idiv` raises `SIGFPE` where LLVM says undefined, and
`cvttsd2si` answers the integer-indefinite value. Unchanged from item 15: these want a language decision, not a
backend.
- **Debug information.** None. `--x86` and `--debug` together are refused.
- **Code size and speed.** Not measured. Every value is in memory, every intermediate is a frame temporary, and a
block copy is `rep movsb`; that is the trade the brief asks for and nobody has put a number on it.
### The verdict
**The wiring is done and it was the easy half. What is left is conditions, and the measurement moved them from "first
obstacle" to "the only obstacle".**
The order item 15 recommended was floats first and conditions last. Floats turned out to be one afternoon's encodings
and they are done. Conditions are still last and are now the *whole* remainder: they are what stands between 41
programs and the corpus, they are what a bounds check is made of, and `check_no_transfer` is the line that says so out
loud on every build until someone writes them.
Two things are worth doing before that, and both are cheap. Decide what a bounds violation means in a build with no
handler — because "no check" is what it means today and nothing says so. And take item 15's question 4 seriously now
that there are two backends to disagree: `(uninit)` and `unreachable` already differ, deliberately, and the difference
is currently documented only in a comment in `x86.ml`.