flan/docs/handoffs/HANDOFF-x86-cost.md

# Handoff — what the x86 backend costs, and keeping it from rotting

Branch: `dev-loop`. This lane is item 7 of `HANDOFF-x86-rt.md`'s "What remains" — code size and speed, measured
rather than guessed — plus the closing note under it, which asks for the survey to run on its own so a refusal
cannot sit unnoticed for a month again.

Nothing in `lib/` is touched. Another lane is rewriting `lib/x86.ml` at the same time, so everything here lives in
`spike/x86` and in `test/dune`, and anything this lane finds that would want a compiler change is written down at
the bottom rather than made.

## 1. The `@x86` alias — done, and it fails when it should

`test/dune` has a fourth alias beside `@sanitize` and `@valgrind`, shaped the same way and opt-in for the same
reason: it builds every corpus program twice and runs both, which is minutes where `dune test` is forty seconds.

    dune build --root . @x86

It is a `(rule ...)` with no `(executable ...)` beside it, which is where it differs from its two neighbours. The
check already exists — `spike/x86/survey.sh` is what every handoff quotes its counts from — and a second
implementation of it in OCaml would be a second thing to drift, which is exactly the failure this alias is meant
to prevent. So the rule runs the script, and the script grew three environment variables to make that possible:

- `SURVEY_STRICT=1` turns the report into an exit status. A DIFFER is a wrong answer and a refusal by name is a
  node the backend has stopped lowering; either fails the build. NOX86 and SKIP do not — the first is usually a
  toolchain that is not installed on the machine, the second is the frontend refusing the program on both sides.
- `FLAN=<path>` hands it a compiler somebody else built. Without it the script runs `dune build` on `bin/main.exe`,
  and a dune inside a dune action waits on a lock it will never get. The rule has `bin/main.exe` in its deps
  instead, exactly as the `(tests ...)` stanza already does.
- `SURVEY_CORPUS` for where the programs are read from, which under dune is the build tree.

Standalone, with nothing set, the script is byte-for-byte the measurement it always was.

**Verified both directions.** `dune build --root . @x86` passes on a clean tree and prints the same
**97 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86 / 36 SKIP** the script prints standalone — the build tree and the
source tree agree about the corpus, which was not obvious in advance. Broken deliberately (a fake DIFFER injected
into the script, the rule narrowed to one program so the round trip was thirty seconds) it fails the dune build
with `x86 survey FAILED: 1 differ, 0 refused` and a nonzero status. Both edits reverted.

`dune test --root .` is still green and still about forty seconds. The new stanza is a rule on its own alias and
touches no `(modules ...)` list, so it cannot be pulled into `@runtest` by accident. The two
`clang: error: linker command failed` lines it prints are the `dev-robust` fixture doing its job and are called
out in `HANDOFF-x86-rt.md` §4 already.

## 2. The measurement — `spike/x86/COST.md`

`COST.md` is the deliverable and it carries the numbers and the reading of them; the raw rows are committed beside
it as `cost-corpus.tsv` and `cost-bench.tsv`. The headline, in one paragraph:

**This backend emits 3.84× the code LLVM does at `-O2`, and 1.92× what LLVM emits at `-O0`** — half the factor is
the optimiser Flan ships with and not the backend. Per program the second ratio is tight: median 2.21, quartiles
1.94 and 2.98. Speed is unmeasurable over the corpus (every program is 2.5ms of `execve`) except `recur`, which
loops ten million times and is **6× LLVM `-O0`**; on purpose-written loops the backend is 2.8× to 4.7×.

Of the five suspected costs the old handoff named, the measurement says: the frame-slot round trip on every
intermediate is most of everything and is the item worth fixing; `rep movsb` costs twenty cycles a copy and is
worth fixing cheaply; the bounds check's three temporaries cost 241 bytes and *one cycle*, so they are a code-size
item and not a speed item; the guard after every call is four instructions and invisible; and the dev build's
extra load per call site is not measurable at all. What a dev build does cost — 400× the code, because nothing may
be dropped when anything might be redefined — is `Reach`'s doing and both backends pay it.

The two scripts are how that was produced:

- `cost.sh` sweeps the corpus and prints a TSV: file size, `.text`, and **the sum of the `flan.*` defined symbols**,
  which is the program's own code with the runtime excluded. `COST_FLAGS=--dev` is the `SURVEY_FLAGS` precedent.
- `bench.sh` runs four programs in `spike/x86/bench/`, each written so that one suspected cost is most of what the
  program does, because every program in `test/programs` runs in about two and a half milliseconds of which nearly
  all is `execve`.

They live in `spike/x86/bench/`, a subdirectory, *deliberately*: `survey.sh` globs `spike/x86/*.flan`, and a
benchmark landing in the survey would move the 97 that three handoffs quote.

## 3. One trap, paid for once

Two copies of `cost.sh` sharing a scratch directory silently corrupt each other's numbers. The first release sweep
was run, killed, and restarted; `pkill` missed a child, and the survivor kept writing the same `l` and `x` files
the new run was building and timing. The result was a row for `math` carrying `dev-locals`' binaries — which is
visible only because those two happened to collide *exactly*, and a collision between two programs of merely
similar size would have produced a plausible row nobody would question.

`cost.sh` now makes a `run.$$` subdirectory with a plain `mkdir`, so a second copy cannot land in the first one's.
The sweep behind `COST.md` was re-run clean afterwards. If you extend these scripts, keep that property.

## 4. Nothing was changed in `lib/`, and nothing needed to be

No finding here wants a compiler change that this lane was not allowed to make. The costs are all in
`lib/x86.ml`'s lowering and the next lane rewriting that file will meet them; `COST.md`'s last section says which
ones are worth its attention and which are not, which was the point of measuring rather than guessing.

## 5. What was not done

- **The `--dev` sweep over the whole corpus was not finished.** It was started, ran into the `dev-*` programs
  (each of which waits out a twenty-second timeout twice), and was killed to free the machine for the clean
  release re-run. The `--dev` numbers in `COST.md` come from `bench.sh`, which measures the same thing on four
  programs and measures it better, because a corpus program's dev cost is invisible under `execve`. Re-running it
  is `COST_FLAGS=--dev spike/x86/cost.sh` and about half an hour.
- **No corpus program was disassembled.** The attribution in `COST.md` is all from the four benchmarks, which are
  small enough to read whole and were written so that each one is mostly a single suspected cost. The corpus
  outliers are explained from the `-O0` column instead — which is what that column is for, and in `bounds`' case
  it turns an alarming 11× into an ordinary 2.07×.
- **`LLVM -O0` is only reachable through `--debug`**, which also asks for DWARF. Checked and harmless: DWARF lands
  in `.debug_*` and the metric sums `.text` symbols. A plain `-O0` flag on the CLI would be a `bin/main.ml` change
  and this lane was not touching compiler sources.