The dev backend, wired — and conditions were never in the way
DISCUSS.md item 16, in item 15's form. The headline is a measurement that contradicts what everyone expected, this report included: a program that loops and prints carries no Signal, no Handled and no RestartCase. Zero of each, measured with hist.ml before a line was written. What drags conditions in is the bounds check and the allocator — check_at signals, and signalling needs the channel and the guard — and neither is a Tast node, which is why a histogram cannot see either and why the guess was off. 41 of the 111 programs in test/programs build through --x86 and 40 print exactly what the LLVM build prints. The 41st is bounds.flan and it diverges on purpose: --x86 is silently a --no-bounds-checks build, and that is named as a divergence rather than left in a code comment. The other 40 are refused by name. check_no_transfer is what makes the missing guard sound instead of hopeful — if nothing reachable can write the channel, no call can return with it set — so the build stops with the node and the function rather than compiling forty programs wrongly. Both bugs found along the way are the shape item 15 predicted: the crash and the cause in different functions, and correct-reading assembly beside a wrong answer. The frame model is the thing that did not go wrong.
This commit is contained in:
parent
58b1f49cf2
commit
fc4e51a106
183
DISCUSS.md
183
DISCUSS.md
@ -913,3 +913,186 @@ language has, and that rule is worth having whether or not a backend is ever wri
|
|||||||
3. **Choose option 1 or option 2 from question 3**, deliberately. Everything else follows from it.
|
3. **Choose option 1 or option 2 from question 3**, deliberately. Everything else follows from it.
|
||||||
4. **Only then**, and only if 3 says so, grow `spike/backend/x86.ml` from the node table in question 2 — floats
|
4. **Only then**, and only if 3 says so, grow `spike/backend/x86.ml` from the node table in question 2 — floats
|
||||||
first, because they gate most of the prelude, and conditions last, because they are the only row with no plan.
|
first, because they gate most of the prelude, and conditions last, because they are the only row with no plan.
|
||||||
|
|
||||||
|
## 16. The dev backend, wired: a whole program compiles through `x86.ml` and runs, and conditions were never in the way
|
||||||
|
|
||||||
|
Item 15's step 4, taken. `lib/x86.ml` was 502 lines of encoder and frame model with nothing calling it. It now lowers a
|
||||||
|
whole `Tast.program` to an assembly file, and `flan build --x86` hands that file to the same clang invocation the LLVM
|
||||||
|
path uses, against the same runtime objects, the same generated shim and the same linker arguments. The flag is off by
|
||||||
|
default and refused in combination with `--dev`, `--debug`, `--sanitize` and every wasm target. **LLVM stays the release
|
||||||
|
backend and the default one; nothing on the existing path changed.** `dune test --root .` is green either side.
|
||||||
|
|
||||||
|
**41 programs out of `test/programs` build through it, and 40 of them print exactly what the LLVM build prints.** The
|
||||||
|
41st is `bounds.flan`, and it diverges on purpose — see question 4.
|
||||||
|
|
||||||
|
### Question 1 — which program, and what did it actually cost
|
||||||
|
|
||||||
|
Measured with `spike/backend/hist.ml` before anything was written, over candidates, not guessed. The one picked is
|
||||||
|
`spike/x86/p3-fizz.flan` — a `dotimes`, a call, an `if`, a remainder, two string literals and `print`/`println`:
|
||||||
|
|
||||||
|
```
|
||||||
|
p3-fizz.flan: 2 reachable fns
|
||||||
|
Call 1 Do 4 If 1 Int 15 Let 2 Local 7 Prim 14
|
||||||
|
Set 1 Str 3 While 1 place/Plocal 1
|
||||||
|
prim/Add 2 prim/Bytes 3 prim/Cast 1 prim/Eq 1
|
||||||
|
prim/I64ToBytes 1 prim/Lt 1 prim/Rem 1 prim/WriteStdout 4
|
||||||
|
```
|
||||||
|
|
||||||
|
Nineteen rows, two reachable functions after `Reach` prunes, and **no `Signal`, no `Handled`, no `RestartCase`, no
|
||||||
|
`Make`, no `Field`, no allocator**. The same is true of a program that only loops and prints: zero of each.
|
||||||
|
|
||||||
|
That is worth stating flatly because the expectation going in was the opposite — that the smallest useful program
|
||||||
|
carries one of each condition node, and that printing a single value drags in `Str`, `Make`, `Field` and `Call` because
|
||||||
|
the prelude builds a slice to do it. **It does not.** Printing an integer is `Cast` → `I64ToBytes` → `WriteStdout`,
|
||||||
|
three prims and no call at all; `flan_i64_to_bytes` renders into a static buffer in `flan_rt.c` and hands back a slice.
|
||||||
|
Printing a string literal is `Str` → `Bytes` → `WriteStdout`, and `Bytes` is a non-instruction because a string and a
|
||||||
|
`[u8]` are the same two words.
|
||||||
|
|
||||||
|
**What does drag conditions in is the bounds check and the allocator, and neither is a `Tast` node.** `check_at` and
|
||||||
|
`check_slice` call `flan_bounds_error` with the transfer channel and then `guard`; `Rt flan_vec_at` takes the channel
|
||||||
|
too. So it is `m.checks` and the container runtime that make a program need the condition machinery, not looping and
|
||||||
|
not printing — and because both are *emit-time* constructs rather than IR nodes, `hist.ml` cannot see either. That is
|
||||||
|
the correction to item 15's node table: the "no plan" row is not reached by writing a loop, it is reached by writing
|
||||||
|
`(at a i)`.
|
||||||
|
|
||||||
|
### Question 2 — what runs
|
||||||
|
|
||||||
|
| program | what it is for |
|
||||||
|
|---|---|
|
||||||
|
| `spike/x86/p1-exit.flan` | `main` returns 0. The assembly path, the runtime link, a real executable. |
|
||||||
|
| `spike/x86/p2-loop-print.flan` | a `dotimes` that prints — the smallest program that does something |
|
||||||
|
| `spike/x86/p3-fizz.flan` | the measured target: a call, an `if`, `%`, two string literals |
|
||||||
|
| `spike/x86/p4-convention.flan` | the *internal calling convention*, which p3 does not touch at all |
|
||||||
|
| `spike/x86/p5-core.flan` | a global with an initialiser, recursion, `break`, `continue`, the bitwise family, unsigned shifts, both directions of every conversion |
|
||||||
|
|
||||||
|
p4 is the one that matters most, because item 15 named the internal aggregate convention as the sharpest obstacle in
|
||||||
|
the whole report. It passes a struct by value, returns a struct by value, puts an `f32` through the SSE half, calls an
|
||||||
|
eight-argument function so that two arguments go on the stack, and passes a slice — and it agrees with LLVM. **The
|
||||||
|
obstacle really did dissolve the way the header claims**: a dev build is compiled entirely here and a release build
|
||||||
|
entirely by LLVM, the two never meet in one process, so the convention is ours to pick. Every aggregate goes by
|
||||||
|
pointer, an aggregate return is a hidden pointer in the first integer register returned in `rax`, and there is no
|
||||||
|
classifier in the file. Nothing had to be discovered by disassembling clang.
|
||||||
|
|
||||||
|
Over `test/programs` (111 files):
|
||||||
|
|
||||||
|
| | n |
|
||||||
|
|---|---|
|
||||||
|
| built through `--x86` and **matched** the LLVM build's output and exit status | **40** |
|
||||||
|
| built and diverged — `bounds.flan`, by design | 1 |
|
||||||
|
| refused by name: a node this backend does not lower | 40 |
|
||||||
|
| no `main` (package and library fixtures) | 6 |
|
||||||
|
| do not compile at all (the checker-error fixtures) | 22 |
|
||||||
|
| never terminate on their own (`dev-loop`, `dev-watch`) | 2 |
|
||||||
|
|
||||||
|
So of the 81 programs that compile, have a `main` and finish, **41 went through the hand-written backend and 40 were
|
||||||
|
byte-identical in output.** That includes `edn.flan` — sixty lines of output from a hand-written EDN reader with
|
||||||
|
unions, options, nested collections and a fixed-depth balance stack.
|
||||||
|
|
||||||
|
Proved by comparing output, never by reading bytes. The script builds both ways and diffs stdout and the exit status;
|
||||||
|
`objdump` was used only after a program already had the wrong answer. Item 15 is right that this is the only honest
|
||||||
|
order.
|
||||||
|
|
||||||
|
### Question 3 — the two bugs, and both are the shape item 15 predicted
|
||||||
|
|
||||||
|
**A `(set (.x (at pts 0)) 1.5)` wrote into a copy.** `lvalue` had no case for `At`, so it fell through to "evaluate it",
|
||||||
|
and the store landed in a temporary while the array kept its zeros. `emit.ml` has this as `addr`'s own `At` case. One
|
||||||
|
line. `array-ctor.flan` found it, and it found it as a segfault several statements later.
|
||||||
|
|
||||||
|
**A discarded value was stored over the return address.** This is the better one. A form whose value is thrown away was
|
||||||
|
handed a sink, and the sink was spelled as an address — `rbp+0`. That is the saved `rbp`, and `rbp+8` is the return
|
||||||
|
address, so a non-void form written in statement position stored straight over both; a 16-byte slice did it in one
|
||||||
|
`rep movsb`. `edn.flan` crashed by jumping into `.rodata`, **several statements after the mistake and in a different
|
||||||
|
function**, and the assembly at the jump read perfectly. The sink is now compared by identity and never used as an
|
||||||
|
address: anything with a value that is handed it gets a frame temporary instead.
|
||||||
|
|
||||||
|
The second bug cannot exist on the LLVM path, and that is the general point. LLVM has no notion of "store this value
|
||||||
|
nowhere" — an unused SSA value is simply unused. Every construct this backend has that LLVM does not is a place where
|
||||||
|
a bug can live that the LLVM backend's own testing can never have covered.
|
||||||
|
|
||||||
|
Against that, the thing item 15 was most worried about did *not* happen: **nothing went wrong with the frame or the
|
||||||
|
stack alignment.** `rsp` is written exactly twice — one rounded `sub` in the prologue that covers the temporaries and
|
||||||
|
the outgoing-argument area together, and `leave` — so `rsp % 16 == 0` at every call site is a property of one
|
||||||
|
subtraction rather than an invariant every case maintains. The spike's worst bug has no door to come in by, and p4
|
||||||
|
calls an eight-argument function to prove it.
|
||||||
|
|
||||||
|
### Question 4 — where the two backends now differ, and it is three named places
|
||||||
|
|
||||||
|
Item 15's question 4 listed nine undefined cases. Three of them are now *real* divergences with a build on each side,
|
||||||
|
and they should be written down before anyone uses this for anything.
|
||||||
|
|
||||||
|
| | LLVM | here |
|
||||||
|
|---|---|---|
|
||||||
|
| **A bounds violation** | `flan_bounds_error` signals; a `restart-case` can catch it; `bounds.flan` exits 134 | **no check at all**; `bounds.flan` exits 139 |
|
||||||
|
| `(uninit)` | `poison`, and the optimiser may reason from it | whatever the stack slot held — stable garbage |
|
||||||
|
| an exhausted `match`, a `noreturn` call | `unreachable`, undefined | `ud2` — a defined SIGILL at the instruction that fell through |
|
||||||
|
|
||||||
|
**The first is the one that matters, and it is not a footnote: `--x86` is silently a `--no-bounds-checks` build.** It is
|
||||||
|
silent because it is not a decision the backend made — `check_at` signals, signalling needs the channel and the guard,
|
||||||
|
and there is no guard here, so there is no check. `bounds.flan` is the direct evidence and `edn.flan`'s 33-brackets-
|
||||||
|
against-a-32-deep-stack case is the second. Anyone reaching for this flag on a program that indexes anything should
|
||||||
|
know that the trap is gone.
|
||||||
|
|
||||||
|
The other two are improvements and cost nothing. `ud2` in particular is two bytes and turns a class of miscompile into
|
||||||
|
a crash with an address.
|
||||||
|
|
||||||
|
### Question 5 — conditions, which is still the row with no plan
|
||||||
|
|
||||||
|
**They were not reached, and converting that into a checked precondition is the most useful thing in this report.**
|
||||||
|
|
||||||
|
There is no transfer guard after a call here, no landing pad and no transfer exit. `emit.ml` emits a guard after *every*
|
||||||
|
call; this emits none. What makes that sound is a whole-program argument rather than a hope: **if nothing in the
|
||||||
|
reachable set can ever write the channel, no call can ever return with it set.** So `check_no_transfer` walks the
|
||||||
|
linked program once per build and stops it — with the node's name and the function it is in — the moment it finds a
|
||||||
|
`signal`, an `invoke-restart`, a `restart-case`, a `handler-bind`, a `with-allocator`, or the two `Rt` symbols whose
|
||||||
|
bounds check signals.
|
||||||
|
|
||||||
|
That is what the 40 refusals are:
|
||||||
|
|
||||||
|
```
|
||||||
|
27 restart-case 4 handler-bind 1 with-allocator
|
||||||
|
7 signal 1 defers on the transfer path
|
||||||
|
```
|
||||||
|
|
||||||
|
Forty programs refused by name rather than miscompiled, and one line of build output says which node and where. A
|
||||||
|
backend that quietly omitted the guard would have compiled all forty and been wrong in a way no test distinguishes
|
||||||
|
from a race.
|
||||||
|
|
||||||
|
What this does *not* do is measure what conditions cost. That is still unknown, and it is still the only row of item
|
||||||
|
15's table with nothing behind it. What is now known is the shape of the bill: the guard is per call site, the pad is
|
||||||
|
per `restart-case` activation, `fdefers` needs a second exit path that no form in `body` can reach, and **any function
|
||||||
|
with `fdefers` at all is refused today** — which is most of the prelude's file and container code, and is why the
|
||||||
|
programs that use a `Vec` are not in the 40.
|
||||||
|
|
||||||
|
### The honest no-plan bucket
|
||||||
|
|
||||||
|
Everything below is refused by name at build time, not silently wrong.
|
||||||
|
|
||||||
|
- **Conditions, entire** — the guard, the landing pad, `emit_restart_case`, `emit_with_alloc`, the transfer exit, and
|
||||||
|
`fdefers` on it. Several hundred lines of `emit.ml` reimplemented from `spec-conditions.md` rather than ported.
|
||||||
|
- **Bounds checks**, which are the same work: `check_at` and `check_slice` cannot exist without the guard.
|
||||||
|
- **`Rt` with an aggregate return**, and with it most of the container runtime; `Vec`, `Map` and `Pool` have not been
|
||||||
|
exercised at all.
|
||||||
|
- **`Fnval`'s indirection cell.** `FnAddr (Fnval n)` emits the symbol, which is correct for a whole-program build and
|
||||||
|
wrong the instant anything is redefined into it. This backend has no cells and no `--dev`; that is a deliberate
|
||||||
|
restriction and not an oversight, but it is exactly item 15's question 5 waiting where it was left.
|
||||||
|
- **`f64` → `i64` out of range**, and **`INT64_MIN / -1`**. `idiv` raises `SIGFPE` where LLVM says undefined, and
|
||||||
|
`cvttsd2si` answers the integer-indefinite value. Unchanged from item 15: these want a language decision, not a
|
||||||
|
backend.
|
||||||
|
- **Debug information.** None. `--x86` and `--debug` together are refused.
|
||||||
|
- **Code size and speed.** Not measured. Every value is in memory, every intermediate is a frame temporary, and a
|
||||||
|
block copy is `rep movsb`; that is the trade the brief asks for and nobody has put a number on it.
|
||||||
|
|
||||||
|
### The verdict
|
||||||
|
|
||||||
|
**The wiring is done and it was the easy half. What is left is conditions, and the measurement moved them from "first
|
||||||
|
obstacle" to "the only obstacle".**
|
||||||
|
|
||||||
|
The order item 15 recommended was floats first and conditions last. Floats turned out to be one afternoon's encodings
|
||||||
|
and they are done. Conditions are still last and are now the *whole* remainder: they are what stands between 41
|
||||||
|
programs and the corpus, they are what a bounds check is made of, and `check_no_transfer` is the line that says so out
|
||||||
|
loud on every build until someone writes them.
|
||||||
|
|
||||||
|
Two things are worth doing before that, and both are cheap. Decide what a bounds violation means in a build with no
|
||||||
|
handler — because "no check" is what it means today and nothing says so. And take item 15's question 4 seriously now
|
||||||
|
that there are two backends to disagree: `(uninit)` and `unreachable` already differ, deliberately, and the difference
|
||||||
|
is currently documented only in a comment in `x86.ml`.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user