diff --git a/DISCUSS.md b/DISCUSS.md index e1c391b..cb37870 100644 --- a/DISCUSS.md +++ b/DISCUSS.md @@ -913,3 +913,186 @@ language has, and that rule is worth having whether or not a backend is ever wri 3. **Choose option 1 or option 2 from question 3**, deliberately. Everything else follows from it. 4. **Only then**, and only if 3 says so, grow `spike/backend/x86.ml` from the node table in question 2 — floats first, because they gate most of the prelude, and conditions last, because they are the only row with no plan. + +## 16. The dev backend, wired: a whole program compiles through `x86.ml` and runs, and conditions were never in the way + +Item 15's step 4, taken. `lib/x86.ml` was 502 lines of encoder and frame model with nothing calling it. It now lowers a +whole `Tast.program` to an assembly file, and `flan build --x86` hands that file to the same clang invocation the LLVM +path uses, against the same runtime objects, the same generated shim and the same linker arguments. The flag is off by +default and refused in combination with `--dev`, `--debug`, `--sanitize` and every wasm target. **LLVM stays the release +backend and the default one; nothing on the existing path changed.** `dune test --root .` is green either side. + +**41 programs out of `test/programs` build through it, and 40 of them print exactly what the LLVM build prints.** The +41st is `bounds.flan`, and it diverges on purpose — see question 4. + +### Question 1 — which program, and what did it actually cost + +Measured with `spike/backend/hist.ml` before anything was written, over candidates, not guessed. The one picked is +`spike/x86/p3-fizz.flan` — a `dotimes`, a call, an `if`, a remainder, two string literals and `print`/`println`: + +``` +p3-fizz.flan: 2 reachable fns + Call 1 Do 4 If 1 Int 15 Let 2 Local 7 Prim 14 + Set 1 Str 3 While 1 place/Plocal 1 + prim/Add 2 prim/Bytes 3 prim/Cast 1 prim/Eq 1 + prim/I64ToBytes 1 prim/Lt 1 prim/Rem 1 prim/WriteStdout 4 +``` + +Nineteen rows, two reachable functions after `Reach` prunes, and **no `Signal`, no `Handled`, no `RestartCase`, no +`Make`, no `Field`, no allocator**. The same is true of a program that only loops and prints: zero of each. + +That is worth stating flatly because the expectation going in was the opposite — that the smallest useful program +carries one of each condition node, and that printing a single value drags in `Str`, `Make`, `Field` and `Call` because +the prelude builds a slice to do it. **It does not.** Printing an integer is `Cast` → `I64ToBytes` → `WriteStdout`, +three prims and no call at all; `flan_i64_to_bytes` renders into a static buffer in `flan_rt.c` and hands back a slice. +Printing a string literal is `Str` → `Bytes` → `WriteStdout`, and `Bytes` is a non-instruction because a string and a +`[u8]` are the same two words. + +**What does drag conditions in is the bounds check and the allocator, and neither is a `Tast` node.** `check_at` and +`check_slice` call `flan_bounds_error` with the transfer channel and then `guard`; `Rt flan_vec_at` takes the channel +too. So it is `m.checks` and the container runtime that make a program need the condition machinery, not looping and +not printing — and because both are *emit-time* constructs rather than IR nodes, `hist.ml` cannot see either. That is +the correction to item 15's node table: the "no plan" row is not reached by writing a loop, it is reached by writing +`(at a i)`. + +### Question 2 — what runs + +| program | what it is for | +|---|---| +| `spike/x86/p1-exit.flan` | `main` returns 0. The assembly path, the runtime link, a real executable. | +| `spike/x86/p2-loop-print.flan` | a `dotimes` that prints — the smallest program that does something | +| `spike/x86/p3-fizz.flan` | the measured target: a call, an `if`, `%`, two string literals | +| `spike/x86/p4-convention.flan` | the *internal calling convention*, which p3 does not touch at all | +| `spike/x86/p5-core.flan` | a global with an initialiser, recursion, `break`, `continue`, the bitwise family, unsigned shifts, both directions of every conversion | + +p4 is the one that matters most, because item 15 named the internal aggregate convention as the sharpest obstacle in +the whole report. It passes a struct by value, returns a struct by value, puts an `f32` through the SSE half, calls an +eight-argument function so that two arguments go on the stack, and passes a slice — and it agrees with LLVM. **The +obstacle really did dissolve the way the header claims**: a dev build is compiled entirely here and a release build +entirely by LLVM, the two never meet in one process, so the convention is ours to pick. Every aggregate goes by +pointer, an aggregate return is a hidden pointer in the first integer register returned in `rax`, and there is no +classifier in the file. Nothing had to be discovered by disassembling clang. + +Over `test/programs` (111 files): + +| | n | +|---|---| +| built through `--x86` and **matched** the LLVM build's output and exit status | **40** | +| built and diverged — `bounds.flan`, by design | 1 | +| refused by name: a node this backend does not lower | 40 | +| no `main` (package and library fixtures) | 6 | +| do not compile at all (the checker-error fixtures) | 22 | +| never terminate on their own (`dev-loop`, `dev-watch`) | 2 | + +So of the 81 programs that compile, have a `main` and finish, **41 went through the hand-written backend and 40 were +byte-identical in output.** That includes `edn.flan` — sixty lines of output from a hand-written EDN reader with +unions, options, nested collections and a fixed-depth balance stack. + +Proved by comparing output, never by reading bytes. The script builds both ways and diffs stdout and the exit status; +`objdump` was used only after a program already had the wrong answer. Item 15 is right that this is the only honest +order. + +### Question 3 — the two bugs, and both are the shape item 15 predicted + +**A `(set (.x (at pts 0)) 1.5)` wrote into a copy.** `lvalue` had no case for `At`, so it fell through to "evaluate it", +and the store landed in a temporary while the array kept its zeros. `emit.ml` has this as `addr`'s own `At` case. One +line. `array-ctor.flan` found it, and it found it as a segfault several statements later. + +**A discarded value was stored over the return address.** This is the better one. A form whose value is thrown away was +handed a sink, and the sink was spelled as an address — `rbp+0`. That is the saved `rbp`, and `rbp+8` is the return +address, so a non-void form written in statement position stored straight over both; a 16-byte slice did it in one +`rep movsb`. `edn.flan` crashed by jumping into `.rodata`, **several statements after the mistake and in a different +function**, and the assembly at the jump read perfectly. The sink is now compared by identity and never used as an +address: anything with a value that is handed it gets a frame temporary instead. + +The second bug cannot exist on the LLVM path, and that is the general point. LLVM has no notion of "store this value +nowhere" — an unused SSA value is simply unused. Every construct this backend has that LLVM does not is a place where +a bug can live that the LLVM backend's own testing can never have covered. + +Against that, the thing item 15 was most worried about did *not* happen: **nothing went wrong with the frame or the +stack alignment.** `rsp` is written exactly twice — one rounded `sub` in the prologue that covers the temporaries and +the outgoing-argument area together, and `leave` — so `rsp % 16 == 0` at every call site is a property of one +subtraction rather than an invariant every case maintains. The spike's worst bug has no door to come in by, and p4 +calls an eight-argument function to prove it. + +### Question 4 — where the two backends now differ, and it is three named places + +Item 15's question 4 listed nine undefined cases. Three of them are now *real* divergences with a build on each side, +and they should be written down before anyone uses this for anything. + +| | LLVM | here | +|---|---|---| +| **A bounds violation** | `flan_bounds_error` signals; a `restart-case` can catch it; `bounds.flan` exits 134 | **no check at all**; `bounds.flan` exits 139 | +| `(uninit)` | `poison`, and the optimiser may reason from it | whatever the stack slot held — stable garbage | +| an exhausted `match`, a `noreturn` call | `unreachable`, undefined | `ud2` — a defined SIGILL at the instruction that fell through | + +**The first is the one that matters, and it is not a footnote: `--x86` is silently a `--no-bounds-checks` build.** It is +silent because it is not a decision the backend made — `check_at` signals, signalling needs the channel and the guard, +and there is no guard here, so there is no check. `bounds.flan` is the direct evidence and `edn.flan`'s 33-brackets- +against-a-32-deep-stack case is the second. Anyone reaching for this flag on a program that indexes anything should +know that the trap is gone. + +The other two are improvements and cost nothing. `ud2` in particular is two bytes and turns a class of miscompile into +a crash with an address. + +### Question 5 — conditions, which is still the row with no plan + +**They were not reached, and converting that into a checked precondition is the most useful thing in this report.** + +There is no transfer guard after a call here, no landing pad and no transfer exit. `emit.ml` emits a guard after *every* +call; this emits none. What makes that sound is a whole-program argument rather than a hope: **if nothing in the +reachable set can ever write the channel, no call can ever return with it set.** So `check_no_transfer` walks the +linked program once per build and stops it — with the node's name and the function it is in — the moment it finds a +`signal`, an `invoke-restart`, a `restart-case`, a `handler-bind`, a `with-allocator`, or the two `Rt` symbols whose +bounds check signals. + +That is what the 40 refusals are: + +``` +27 restart-case 4 handler-bind 1 with-allocator + 7 signal 1 defers on the transfer path +``` + +Forty programs refused by name rather than miscompiled, and one line of build output says which node and where. A +backend that quietly omitted the guard would have compiled all forty and been wrong in a way no test distinguishes +from a race. + +What this does *not* do is measure what conditions cost. That is still unknown, and it is still the only row of item +15's table with nothing behind it. What is now known is the shape of the bill: the guard is per call site, the pad is +per `restart-case` activation, `fdefers` needs a second exit path that no form in `body` can reach, and **any function +with `fdefers` at all is refused today** — which is most of the prelude's file and container code, and is why the +programs that use a `Vec` are not in the 40. + +### The honest no-plan bucket + +Everything below is refused by name at build time, not silently wrong. + +- **Conditions, entire** — the guard, the landing pad, `emit_restart_case`, `emit_with_alloc`, the transfer exit, and + `fdefers` on it. Several hundred lines of `emit.ml` reimplemented from `spec-conditions.md` rather than ported. +- **Bounds checks**, which are the same work: `check_at` and `check_slice` cannot exist without the guard. +- **`Rt` with an aggregate return**, and with it most of the container runtime; `Vec`, `Map` and `Pool` have not been + exercised at all. +- **`Fnval`'s indirection cell.** `FnAddr (Fnval n)` emits the symbol, which is correct for a whole-program build and + wrong the instant anything is redefined into it. This backend has no cells and no `--dev`; that is a deliberate + restriction and not an oversight, but it is exactly item 15's question 5 waiting where it was left. +- **`f64` → `i64` out of range**, and **`INT64_MIN / -1`**. `idiv` raises `SIGFPE` where LLVM says undefined, and + `cvttsd2si` answers the integer-indefinite value. Unchanged from item 15: these want a language decision, not a + backend. +- **Debug information.** None. `--x86` and `--debug` together are refused. +- **Code size and speed.** Not measured. Every value is in memory, every intermediate is a frame temporary, and a + block copy is `rep movsb`; that is the trade the brief asks for and nobody has put a number on it. + +### The verdict + +**The wiring is done and it was the easy half. What is left is conditions, and the measurement moved them from "first +obstacle" to "the only obstacle".** + +The order item 15 recommended was floats first and conditions last. Floats turned out to be one afternoon's encodings +and they are done. Conditions are still last and are now the *whole* remainder: they are what stands between 41 +programs and the corpus, they are what a bounds check is made of, and `check_no_transfer` is the line that says so out +loud on every build until someone writes them. + +Two things are worth doing before that, and both are cheap. Decide what a bounds violation means in a build with no +handler — because "no check" is what it means today and nothing says so. And take item 15's question 4 seriously now +that there are two backends to disagree: `(uninit)` and `unreachable` already differ, deliberately, and the difference +is currently documented only in a comment in `x86.ml`.