Merge branch 'worktree-agent-ab7f6288f142b05df' into dev-loop

This commit is contained in:
Joseph Ferano 2026-09-14 06:55:39 +07:00
commit 2b68262249
12 changed files with 807 additions and 3 deletions

102
HANDOFF-x86-cost.md Normal file
View File

@ -0,0 +1,102 @@
# Handoff — what the x86 backend costs, and keeping it from rotting
Branch: `dev-loop`. This lane is item 7 of `HANDOFF-x86-rt.md`'s "What remains" — code size and speed, measured
rather than guessed — plus the closing note under it, which asks for the survey to run on its own so a refusal
cannot sit unnoticed for a month again.
Nothing in `lib/` is touched. Another lane is rewriting `lib/x86.ml` at the same time, so everything here lives in
`spike/x86` and in `test/dune`, and anything this lane finds that would want a compiler change is written down at
the bottom rather than made.
## 1. The `@x86` alias — done, and it fails when it should
`test/dune` has a fourth alias beside `@sanitize` and `@valgrind`, shaped the same way and opt-in for the same
reason: it builds every corpus program twice and runs both, which is minutes where `dune test` is forty seconds.
dune build --root . @x86
It is a `(rule ...)` with no `(executable ...)` beside it, which is where it differs from its two neighbours. The
check already exists — `spike/x86/survey.sh` is what every handoff quotes its counts from — and a second
implementation of it in OCaml would be a second thing to drift, which is exactly the failure this alias is meant
to prevent. So the rule runs the script, and the script grew three environment variables to make that possible:
- `SURVEY_STRICT=1` turns the report into an exit status. A DIFFER is a wrong answer and a refusal by name is a
node the backend has stopped lowering; either fails the build. NOX86 and SKIP do not — the first is usually a
toolchain that is not installed on the machine, the second is the frontend refusing the program on both sides.
- `FLAN=<path>` hands it a compiler somebody else built. Without it the script runs `dune build` on `bin/main.exe`,
and a dune inside a dune action waits on a lock it will never get. The rule has `bin/main.exe` in its deps
instead, exactly as the `(tests ...)` stanza already does.
- `SURVEY_CORPUS` for where the programs are read from, which under dune is the build tree.
Standalone, with nothing set, the script is byte-for-byte the measurement it always was.
**Verified both directions.** `dune build --root . @x86` passes on a clean tree and prints the same
**97 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86 / 36 SKIP** the script prints standalone — the build tree and the
source tree agree about the corpus, which was not obvious in advance. Broken deliberately (a fake DIFFER injected
into the script, the rule narrowed to one program so the round trip was thirty seconds) it fails the dune build
with `x86 survey FAILED: 1 differ, 0 refused` and a nonzero status. Both edits reverted.
`dune test --root .` is still green and still about forty seconds. The new stanza is a rule on its own alias and
touches no `(modules ...)` list, so it cannot be pulled into `@runtest` by accident. The two
`clang: error: linker command failed` lines it prints are the `dev-robust` fixture doing its job and are called
out in `HANDOFF-x86-rt.md` §4 already.
## 2. The measurement — `spike/x86/COST.md`
`COST.md` is the deliverable and it carries the numbers and the reading of them; the raw rows are committed beside
it as `cost-corpus.tsv` and `cost-bench.tsv`. The headline, in one paragraph:
**This backend emits 3.84× the code LLVM does at `-O2`, and 1.92× what LLVM emits at `-O0`** — half the factor is
the optimiser Flan ships with and not the backend. Per program the second ratio is tight: median 2.21, quartiles
1.94 and 2.98. Speed is unmeasurable over the corpus (every program is 2.5ms of `execve`) except `recur`, which
loops ten million times and is **6× LLVM `-O0`**; on purpose-written loops the backend is 2.8× to 4.7×.
Of the five suspected costs the old handoff named, the measurement says: the frame-slot round trip on every
intermediate is most of everything and is the item worth fixing; `rep movsb` costs twenty cycles a copy and is
worth fixing cheaply; the bounds check's three temporaries cost 241 bytes and *one cycle*, so they are a code-size
item and not a speed item; the guard after every call is four instructions and invisible; and the dev build's
extra load per call site is not measurable at all. What a dev build does cost — 400× the code, because nothing may
be dropped when anything might be redefined — is `Reach`'s doing and both backends pay it.
The two scripts are how that was produced:
- `cost.sh` sweeps the corpus and prints a TSV: file size, `.text`, and **the sum of the `flan.*` defined symbols**,
which is the program's own code with the runtime excluded. `COST_FLAGS=--dev` is the `SURVEY_FLAGS` precedent.
- `bench.sh` runs four programs in `spike/x86/bench/`, each written so that one suspected cost is most of what the
program does, because every program in `test/programs` runs in about two and a half milliseconds of which nearly
all is `execve`.
They live in `spike/x86/bench/`, a subdirectory, *deliberately*: `survey.sh` globs `spike/x86/*.flan`, and a
benchmark landing in the survey would move the 97 that three handoffs quote.
## 3. One trap, paid for once
Two copies of `cost.sh` sharing a scratch directory silently corrupt each other's numbers. The first release sweep
was run, killed, and restarted; `pkill` missed a child, and the survivor kept writing the same `l` and `x` files
the new run was building and timing. The result was a row for `math` carrying `dev-locals`' binaries — which is
visible only because those two happened to collide *exactly*, and a collision between two programs of merely
similar size would have produced a plausible row nobody would question.
`cost.sh` now makes a `run.$$` subdirectory with a plain `mkdir`, so a second copy cannot land in the first one's.
The sweep behind `COST.md` was re-run clean afterwards. If you extend these scripts, keep that property.
## 4. Nothing was changed in `lib/`, and nothing needed to be
No finding here wants a compiler change that this lane was not allowed to make. The costs are all in
`lib/x86.ml`'s lowering and the next lane rewriting that file will meet them; `COST.md`'s last section says which
ones are worth its attention and which are not, which was the point of measuring rather than guessing.
## 5. What was not done
- **The `--dev` sweep over the whole corpus was not finished.** It was started, ran into the `dev-*` programs
(each of which waits out a twenty-second timeout twice), and was killed to free the machine for the clean
release re-run. The `--dev` numbers in `COST.md` come from `bench.sh`, which measures the same thing on four
programs and measures it better, because a corpus program's dev cost is invisible under `execve`. Re-running it
is `COST_FLAGS=--dev spike/x86/cost.sh` and about half an hour.
- **No corpus program was disassembled.** The attribution in `COST.md` is all from the four benchmarks, which are
small enough to read whole and were written so that each one is mostly a single suspected cost. The corpus
outliers are explained from the `-O0` column instead — which is what that column is for, and in `bounds`' case
it turns an alarming 11× into an ordinary 2.07×.
- **`LLVM -O0` is only reachable through `--debug`**, which also asks for DWARF. Checked and harmless: DWARF lands
in `.debug_*` and the metric sums `.text` symbols. A plain `-O0` flag on the CLI would be a `bin/main.ml` change
and this lane was not touching compiler sources.

210
spike/x86/COST.md Normal file
View File

@ -0,0 +1,210 @@
# What the hand-written x86-64 backend costs
`survey.sh` has said for three handoffs that this backend agrees with LLVM on all 97 corpus programs it can build.
Item 7 of `HANDOFF-x86-rt.md` is the other half of that sentence — nobody had a number for what the agreement
costs — and it came with a list of suspects: a guard after every call, three frame temporaries per bounds check,
every intermediate in memory, `rep movsb` block copies, and an extra load per call site in a dev build. This is
the measurement. It does not change anything; two of the five suspects turn out not to matter, and the one that
matters most is not on the list.
Produced by `spike/x86/cost.sh` (the corpus, for size) and `spike/x86/bench.sh` (four purpose-written programs,
for speed). Both are documented in their own headers. Machine: 16-core x86-64, Fedora, clang as the assembler and
linker on both sides, and other work running on it throughout — which is why every time below is the *minimum* of
five or seven runs and why nothing here rests on a difference of a few percent.
The rows behind the tables are committed beside this file as `cost-corpus.tsv` and `cost-bench.tsv`, so a later
lane can recompute a ratio rather than believe one.
## What is being compared, and against what
The interesting column is not the size of the executable. A Flan binary is mostly `flan_rt.o` and libc glue, the
same object on both sides, and it drowns the signal: over the corpus the whole file is only **1.09×** bigger
through this backend and `.text` only **1.18×**, which would be a reassuring number and a meaningless one.
So the measurement is the sum of the sizes of the defined symbols the compiler *named itself* — everything called
`flan.<something>`. The runtime's C is `flan_<something>` with an underscore, so the two never collide, and a
runtime symbol has the same size on both sides (`flan_map_clone`, `0x4b3` either way), which is the check that
says the difference really is codegen and not a differently-linked runtime.
There is a third column, and it is what makes the second readable: **LLVM with `--debug`, which forces `-O0`**.
This backend has no optimiser at all, so measuring it against LLVM at `-O2` charges it for the whole of mem2reg,
inlining and constant folding. `-O0` is LLVM's instruction selection with none of that, which is the comparison
that says something about *this* backend rather than about the absence of a middle end. (`--x86 --debug` is
refused — the backend emits no DWARF — so the column exists on one side only. DWARF lands in `.debug_*` sections
and not in `.text`, checked, so it does not contaminate the symbol sums.)
## The corpus: size
Ninety-seven programs — the same set `survey.sh` matches on, minus the two that run forever. Summed over all of
them:
| | LLVM -O2 | LLVM -O0 | x86 |
|---|---|---|---|
| own code, all 97 programs | 221,608 | 444,504 | 851,638 |
| against LLVM -O2 | 1.00× | 2.01× | **3.84×** |
| against LLVM -O0 | | 1.00× | **1.92×** |
So the headline is two numbers, not one. **This backend emits 3.8× the code LLVM does at `-O2`, and half of that
factor is the optimiser Flan ships with rather than anything about the backend; against LLVM with the optimiser
off it is 1.9×.** Per program the second ratio is tight — median 2.21, quartiles 1.94 and 2.98, the whole range
1.32 to 5.44 — which is itself a finding: the cost is not a few bad nodes, it is a constant tax on everything.
The ten largest programs, which are where the bytes actually are:
| program | LLVM -O2 | LLVM -O0 | x86 | x86 / -O2 | x86 / -O0 |
|---|---|---|---|---|---|
| `strings` | 13,395 | 22,681 | 33,265 | 2.48× | 1.47× |
| `maps` | 12,833 | 19,850 | 40,445 | 3.15× | 2.04× |
| `edn` | 12,057 | 34,556 | 48,532 | 4.03× | 1.40× |
| `generics` | 10,137 | 20,427 | 39,241 | 3.87× | 1.92× |
| `slurp` | 9,744 | 14,813 | 24,656 | 2.53× | 1.66× |
| `into` | 8,277 | 12,345 | 21,320 | 2.58× | 1.73× |
| `vec` | 8,039 | 12,089 | 22,770 | 2.83× | 1.88× |
| `map-iter` | 7,211 | 10,374 | 21,715 | 3.01× | 2.09× |
| `algorithms` | 6,776 | 16,616 | 30,875 | 4.56× | 1.86× |
| `slices` | 2,393 | 9,057 | 17,753 | 7.42× | 1.96× |
And the two ends of the distribution, both of which are more interesting than the middle:
| program | LLVM -O2 | LLVM -O0 | x86 | x86 / -O2 | x86 / -O0 | why |
|---|---|---|---|---|---|---|
| `bounds` | 363 | 1,951 | 4,039 | **11.13×** | 2.07× | LLVM at `-O2` proves the indices and deletes the checks |
| `array-ctor` | 357 | 1,840 | 3,880 | 10.87× | 2.11× | the same, over a constructor's worth of stores |
| `p2-loop-print` | 283 | 106 | 577 | 2.04× | **5.44×** | a `-O0` build *smaller* than `-O2`: LLVM unrolls the five-iteration loop and `-O0` does not |
| `pkg-return` | 7,024 | 23,635 | 31,716 | 4.52× | 1.34× | mostly prelude, where `-O0` is already fat |
`bounds` is the clearest case in the table of why the `-O0` column had to exist. Eleven times is a shocking
number and it is not about this backend at all: the program's whole point is indexing, LLVM at `-O2` can see the
indices are in range and removes the check, and neither LLVM at `-O0` nor this backend can. Against the compiler
that also keeps every check, `bounds` is 2.07× — a completely ordinary row.
## Where the size goes
Every one of these is from the disassembly of the benchmark programs, which are small enough to read whole.
**Every intermediate goes through the frame, and so does every constant.** This is the big one and it is not one
feature, it is the shape of the whole backend. `(step acc 1)` in `b1-calls` compiles to:
movabs $0x1,%rax ; a 10-byte immediate ...
mov %rax,-0x30(%rbp) ; ... stored to a frame slot ...
mov -0x30(%rbp),%rsi ; ... and loaded back into the argument register
Three instructions and 24 bytes where LLVM writes `mov $1,%esi`, five. The loop bound gets the same treatment
*every iteration* — `movabs $0x1312d00` into a slot, sign-extended out of it, compared — because nothing is
hoisted. So does the loop condition: `cmp`/`setl`/`movzbq`/store a byte to the frame/reload it/`test`/`jne`,
seven instructions for what is `cmp`/`jge` anywhere else. This is most of the 2× against `-O0` and essentially
all of the difference on the programs at the bottom of the table, which have no calls, no bounds checks and no
aggregates in them at all.
**The guard after every call is four instructions and one dependent load.**
call 4009f8 <flan.step>
mov -0x18(%rbp),%r11 ; the condition frame, from its own slot
mov 0x0(%r11),%r11 ; ... dereferenced
test %r11,%r11
jne <unwind>
Roughly 25 bytes per call site. Real, cheap, and third in size behind the two above it — on a program that is
nothing *but* calls (`b1`) the whole backend is 2.8× LLVM `-O0`, and the guard is a minority of that.
**The bounds check is three frame temporaries, as suspected, and it costs code and not time.** The index is
widened, stored, reloaded, stored again, the limit goes to a third slot, and then `cmp`/`jb` — with the failure
path, its `.rodata` location string and its length, inline at the branch target. On `b2-bounds`, `flan.main` is
`0x4b1` with checks and `0x3c0` without: **241 bytes, a quarter of the function.** In time it is 112.6ms against
105.0ms over 20.5 million checked loads — **about 0.4ns, a cycle or two a check** — because the branch predicts
perfectly and the loads were going to memory anyway. That contradicts the way the handoff's list reads. The three
temporaries are a code-size item. They are not a speed item.
**`rep movsb` is real and it is the most expensive single instruction here.** A 64-byte struct copy lowers to
`lea`/`lea`/`movabs $0x40,%rcx`/`rep movsb`, and `b4-copy` runs 2 million of them in 20.8ms against LLVM `-O0`'s
7.2ms: **about 6.8ns of the difference per copy, some twenty cycles**, which is `rep movsb`'s startup cost and
almost none of it the 64 bytes. It is also the one place where the backend loses to `-O0` by a factor (2.9×) it
does not lose by on straight-line code, and the one item on the suspect list where a targeted fix — inline
16-byte moves under some size threshold — would pay for itself.
## The corpus: speed, and why there is barely any
Almost nothing. **Every program in `test/programs` runs in about 2.5 milliseconds, nearly all of it `execve` and
the dynamic loader**, and both backends produce the same 2.5 milliseconds. Best-of-five does not rescue a signal
that is not there. There is exactly one corpus program whose own code is a measurable part of its runtime, and it
is the right one:
| program | LLVM -O2 | LLVM -O0 | x86 | what it is |
|---|---|---|---|---|
| `recur` | ~1ms | 10ms | 60ms | a ten-million-iteration counting loop, written to prove `recur` is a jump |
Read that carefully, because the 25× against `-O2` is not a fact about this backend: LLVM folds the loop to its
answer and runs nothing. Against `-O0`, which also runs ten million iterations, it is **6×** — about 6ns an
iteration against 1ns, or roughly eighteen cycles for `i+1` and a compare. That is the frame-slot round trip
above, four or five times over, and it is the honest number.
## The benchmarks
Four programs in `spike/x86/bench/`, each written so that one suspected cost is most of what the program does.
They are in a subdirectory on purpose: `survey.sh` globs `spike/x86/*.flan` and a benchmark is not a case.
Times are best-of-seven, in milliseconds.
| bench | what it is | LLVM -O2 | LLVM -O0 | x86 | x86 / -O0 | own code, -O0 → x86 |
|---|---|---|---|---|---|---|
| `b1-calls` | 20M calls of a one-instruction function | 2.1 | 36.8 | 104.3 | 2.8× | 200 → 807 |
| `b2-bounds` | 20.5M bounds-checked array loads | 4.7 | 27.0 | 112.6 | 4.2× | 407 → 1390 |
| `b3-spill` | 5M iterations of a six-deep arithmetic tree | 22.2 | 29.0 | 137.3 | 4.7× | 248 → 1117 |
| `b4-copy` | 2M copies of a 64-byte struct | 2.0 | 7.2 | 20.8 | 2.9× | 348 → 877 |
`b2` with `--no-bounds-checks` on both sides: LLVM 4.5ms, x86 105.0ms — the 7.6ms the check costs over 20.5
million of them, and the 241 bytes it costs in `flan.main`, are the whole of it.
`b3`'s ratio is the one to distrust slightly: its expression ends in a `%`, which is an `idiv`, and an `idiv` is
twenty-odd cycles on every side. That is most of LLVM's own 22.2ms and a good part of its 29.0ms, so the
denominator is largely a hardware latency this backend cannot do anything about. The absolute gap — 108ms over
5 million iterations, about 21ns of extra work each — is the honest reading of that row.
`b1-calls` and `b3-spill` are the pair to read together, with the `idiv` caveat above in mind. `b1` is 20 million
calls of a one-instruction function and lands at 2.8× `-O0`; `b3` has no calls at all and pays 21ns an iteration
for six dependent arithmetic temporaries. **The backend is worse at arithmetic than it is at calling**, which is
the opposite of what the suspect list implies, and it is because a call already costs enough that four extra
instructions beside it disappear, while an add that should be one instruction costs five. `recur`, which is a
counting loop and nothing else, says the same thing on a corpus program: 6× LLVM `-O0`.
The `-O2` column in `b1` and `b4` is 2ms — the loop is gone. That is a true fact about the toolchain Flan ships
and a useless one about code generation, which is the whole reason the `-O0` column exists.
## `--dev`, which is the one axis both backends pay
`SURVEY_FLAGS=--dev` reported 97 MATCH for the lane before this one, so the comparison is available. The suspected cost was the extra
load per call site — every cross-function call going through its indirection cell. **It is not measurable.** On
`b1-calls`, 20 million calls, x86 release is 104.3ms and x86 `--dev` is 100.9ms: the same number, and the dev
build is nominally the *faster* of the two, which is what a difference below the noise floor looks like. The load
is from a `.data` cell that is in L1 after the first call and the machine was already waiting on the frame.
What a dev build actually costs is something else entirely, and both backends pay it. `b1-calls` is a program
with two functions in it:
| build | own code |
|---|---|
| LLVM, release | 82 bytes |
| x86, release | 807 bytes |
| LLVM, `--dev` | 32,714 bytes |
| x86, `--dev` | 83,018 bytes |
**A dev build emits the entire prelude**, because anything might be redefined and so nothing may be dropped. That
is four hundred times the code for this program, and it dwarfs every item on the suspect list put together. It is
also not a backend cost — LLVM pays a 400× of its own — so it is `Reach`'s business and not `x86.ml`'s. The
backend's share of it is the same ~2.5× it charges everywhere else.
## What this says to the lane rewriting `lib/x86.ml`
Ranked by what the numbers actually support, and not by the order of the list in the handoff:
1. **Keep values in registers across a single expression.** Not a register allocator — just not routing every
constant and every subexpression through a frame slot, and not re-materialising a loop bound every iteration.
This is most of the 2× against `-O0` and most of `recur`'s 6×, and it is what `b3`'s 21ns an iteration buys.
2. **Inline small aggregate copies** instead of `rep movsb`. One instruction, twenty cycles, on a copy that is
four `movdqu` pairs.
3. **`flan_dev_reg_note` in a release build** (item 2 of the old handoff's list) is worth doing and is small.
4. **The call guard is fine.** Four instructions and 25 bytes, invisible in time. Leave it.
5. **The bounds check is fine on time and fat on code.** If it is ever worth touching, it is worth touching for
the 241 bytes — hoisting the failure path out of line would get most of that back without changing a cycle.
6. **The dev call cell is free.** Whatever the redefinition emitter costs, it does not cost this.

86
spike/x86/bench.sh Executable file
View File

@ -0,0 +1,86 @@
#!/usr/bin/env bash
# The speed half of cost.sh, on programs that are long enough to time.
#
# Why this exists beside cost.sh rather than inside it: every program in
# test/programs runs in about two and a half milliseconds, of which nearly
# all is fork, exec and the dynamic loader. Best-of-five does not rescue a
# signal that is not there, and a table of 97 rows that all say "2.5ms vs
# 2.6ms" would be a measurement of execve. So the corpus answers the size
# question and these four answer the speed one, each written so that one
# suspected cost is most of what the program does.
#
# Four builds of each, and the third column is the one to read:
#
# llvm as shipped, -O2. Frequently the loop is simply gone; that is a
# true number about the toolchain and a useless one about codegen.
# llvm -O0 via --debug, which forces it. LLVM's instruction selection with
# its optimiser off -- the fair comparison for a backend that has
# no optimiser.
# x86 this backend.
# x86 nbc --no-bounds-checks, for b2, where the difference is the check.
#
# And --dev on both sides, which is the one suspected cost the two backends
# share: every cross-function call goes through an indirection cell, so it is
# a load and an indirect call where a release build has a direct one. b1 is
# where that has to show.
#
# Usage: spike/x86/bench.sh [name-substring ...]
set -u
orig=$(pwd)
here=$(cd "$(dirname "$0")" && pwd)
root=$(cd "$here/../.." && pwd)
cd "$root" || exit 1
if [ -n "${FLAN:-}" ]; then
case $FLAN in /*) flan=$FLAN;; *) flan=$orig/$FLAN;; esac
else
dune build --root . bin/main.exe 2>&1 | head -30
flan=$root/_build/default/bin/main.exe
fi
test -x "$flan" || { echo "build failed" >&2; exit 1; }
out=${COST_OUT:-${TMPDIR:-/tmp}/flan-bench.$$}
mkdir -p "$out" || exit 1
trap 'rm -rf "$out"' EXIT
REPS=${COST_REPS:-5}
best () {
min=
for i in $(seq "$REPS"); do
t0=$(date +%s%N)
timeout 120 "$1" >/dev/null 2>&1 </dev/null
t1=$(date +%s%N)
d=$(( (t1 - t0) / 1000 ))
if [ -z "$min" ] || [ "$d" -lt "$min" ]; then min=$d; fi
done
echo "$min"
}
own () {
nm --defined-only -S "$1" 2>/dev/null \
| awk 'NF==4 && ($3=="T"||$3=="t") && $4 ~ /^flan\./ {n+=strtonum("0x"$2)} END{print n+0}'
}
printf 'name\tllvm_us\to0_us\tx86_us\tllvm_nbc_us\tx86_nbc_us\tllvm_dev_us\tx86_dev_us\tllvm_own\to0_own\tx86_own\tx86_dev_own\n'
for src in "$root"/spike/x86/bench/*.flan; do
name=$(basename "$src" .flan)
if [ $# -gt 0 ]; then
want=0
for pat in "$@"; do case "$name" in *"$pat"*) want=1;; esac; done
[ $want = 1 ] || continue
fi
"$flan" build "$src" -o "$out/l" >/dev/null 2>&1 || { echo "$name: llvm build failed" >&2; continue; }
"$flan" build "$src" --debug -o "$out/d" >/dev/null 2>&1 || { echo "$name: -O0 build failed" >&2; continue; }
"$flan" build "$src" --x86 -o "$out/x" >/dev/null 2>&1 || { echo "$name: x86 build failed" >&2; continue; }
"$flan" build "$src" --no-bounds-checks -o "$out/ln" >/dev/null 2>&1
"$flan" build "$src" --x86 --no-bounds-checks -o "$out/xn" >/dev/null 2>&1
"$flan" build "$src" --dev -o "$out/lv" >/dev/null 2>&1
"$flan" build "$src" --x86 --dev -o "$out/xv" >/dev/null 2>&1
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$name" \
"$(best "$out/l")" "$(best "$out/d")" "$(best "$out/x")" \
"$(best "$out/ln")" "$(best "$out/xn")" \
"$(best "$out/lv")" "$(best "$out/xv")" \
"$(own "$out/l")" "$(own "$out/d")" "$(own "$out/x")" "$(own "$out/xv")"
done

View File

@ -0,0 +1,20 @@
;;;; A hot loop that does nothing but call.
;;;;
;;;; The suspected cost is the guard this backend emits after every call --
;;;; and, in a dev build, the load of the indirection cell before it. Neither
;;;; is visible in the corpus, where a program's whole run is process startup.
;;;; Here the call is the program: the body is one add, so whatever separates
;;;; this from LLVM at -O0 is the call sequence and not the arithmetic.
;;;;
;;;; Not in spike/x86 proper, where survey.sh would pick it up: the survey's
;;;; counts are quoted in three handoffs and a benchmark is not a case.
(defn step [a i64 b i64] i64
(+ a b))
(defn main [] i32
(let [acc (i64 0)]
(dotimes [i 20000000]
(set acc (step acc 1)))
(print acc) (println ""))
0)

View File

@ -0,0 +1,20 @@
;;;; A hot loop that does nothing but index a bounds-checked array.
;;;;
;;;; The suspected cost is three frame temporaries per check. This one has an
;;;; A/B that the others do not: --no-bounds-checks builds the same program
;;;; with the check gone, on both sides, so the difference between the two
;;;; x86 numbers is the check and nothing else, and the same difference on
;;;; the LLVM side says what the check costs when a compiler is allowed to
;;;; hoist it out of the loop.
(defvar xs [1024 i32])
(defn main [] i32
(dotimes [i 1024]
(set (at xs i) i))
(let [acc (i64 0)]
(dotimes [r 20000]
(dotimes [i 1024]
(set acc (+ acc (i64 (at xs i))))))
(print acc) (println ""))
0)

View File

@ -0,0 +1,21 @@
;;;; A hot loop of arithmetic and nothing else: no calls, no arrays, no
;;;; copies.
;;;;
;;;; The suspected cost is that every intermediate lives in a frame slot --
;;;; this backend allocates no registers, so an expression tree becomes a
;;;; chain of stores and reloads. The tree here is deliberately deep and
;;;; entirely dependent, so a register allocator would keep all of it in
;;;; registers and this backend cannot keep any of it.
(defn main [] i32
(let [acc (i64 1)]
(dotimes [i 5000000]
(let [a (+ acc 3)
b (* a 2)
c (- b 1)
d (bit-xor c 7)
e (+ d (* a 5))
f (- e (bit-and d 15))]
(set acc (+ (% f 1000003) 1))))
(print acc) (println ""))
0)

View File

@ -0,0 +1,20 @@
;;;; A hot loop of struct copies.
;;;;
;;;; The suspected cost is `rep movsb`: this backend copies an aggregate by
;;;; block-moving bytes, where LLVM either keeps the thing in registers or
;;;; emits a handful of wide moves. Eight i64 fields is 64 bytes -- big
;;;; enough that a copy is a real copy, small enough that `rep movsb` is
;;;; paying its setup cost on every one of them, which is the shape the
;;;; instruction is worst at.
(defstruct Big [a i64 b i64 c i64 d i64 e i64 f i64 g i64 h i64])
(defn main [] i32
(let [acc (i64 0)
v (Big {.a 1 .b 2 .c 3 .d 4 .e 5 .f 6 .g 7 .h 8})]
(dotimes [i 2000000]
(let [w v]
(set (.a v) (+ (.h w) 1))
(set acc (+ acc (.a w)))))
(print acc) (println ""))
0)

5
spike/x86/cost-bench.tsv Normal file
View File

@ -0,0 +1,5 @@
name llvm_us o0_us x86_us llvm_nbc_us x86_nbc_us llvm_dev_us x86_dev_us llvm_own o0_own x86_own x86_dev_own
b1-calls 2108 36764 104266 1959 103336 26827 100874 82 200 807 83018
b2-bounds 4711 27047 112567 4548 104951 4663 111295 355 407 1390 83596
b3-spill 22184 28982 137347 22772 132053 22659 132913 168 248 1117 83323
b4-copy 2043 7170 20792 1980 19559 2058 18497 140 348 877 83083
1 name llvm_us o0_us x86_us llvm_nbc_us x86_nbc_us llvm_dev_us x86_dev_us llvm_own o0_own x86_own x86_dev_own
2 b1-calls 2108 36764 104266 1959 103336 26827 100874 82 200 807 83018
3 b2-bounds 4711 27047 112567 4548 104951 4663 111295 355 407 1390 83596
4 b3-spill 22184 28982 137347 22772 132053 22659 132913 168 248 1117 83323
5 b4-copy 2043 7170 20792 1980 19559 2058 18497 140 348 877 83083

98
spike/x86/cost-corpus.tsv Normal file
View File

@ -0,0 +1,98 @@
name llvm_file llvm_text llvm_own o0_own x86_file x86_text x86_own llvm_us x86_us
agent-longname 82224 42002 79 134 82264 42450 519 -1 -1
agent-queue 82352 42306 340 567 86488 43698 1747 2491 2546
agent 82336 42434 454 798 86472 44162 2218 2609 2496
algorithms 76680 39282 6776 16616 101296 63234 30875 2499 2605
allocators 67760 33650 1297 1719 71896 37490 5130 3170 3034
array-ctor 67784 32722 357 1840 71920 36242 3880 2442 2664
bounds-condition 76416 37074 4641 6963 84648 45858 13493 2673 2469
bounds 67720 32738 363 1951 71856 36418 4039 3194 2654
break 82296 42754 819 1182 86432 44482 2549 -1 -1
bytes2 72120 37010 4576 10893 88544 52018 19654 2573 2510
cleanup 68176 33986 1583 2077 72312 36946 4583 2351 2402
conditions 68064 33586 1212 1209 68104 35346 2984 2436 2513
debug-permuted 67720 32610 261 410 67760 33794 1433 2479 2442
debug 67720 32610 261 429 67760 33794 1433 2494 2494
defer-let 67928 34018 1625 1784 72064 36754 4396 2548 2543
destructure 67792 33650 1290 4577 80120 40946 8583 2534 2523
dev-break-bounds 82440 42546 588 648 86576 43762 1830 -1 -1
dev-break 82368 42418 459 715 86504 43682 1755 -1 -1
dev-globals 82456 42162 206 534 86584 43538 1614 -1 -1
dev-inspect 82408 42610 652 704 86544 43698 1765 -1 -1
dev-locals 82336 42418 465 623 86472 43650 1721 -1 -1
dev-noagent 67720 32434 83 109 67760 32866 507 3116 2939
dev-pause 82336 42066 112 244 82376 42802 877 -1 -1
dev-ptr 86632 43954 1970 2859 90768 47490 5543 -1 -1
dev-repl 82336 42066 112 244 82376 42802 877 -1 -1
dev-robust 82336 42066 112 244 82376 42802 877 -1 -1
edn 91208 44706 12057 34556 124016 80898 48532 3218 3165
embed 67800 33330 977 2461 71936 37906 5550 2440 2650
enum-compare 67752 32562 201 581 67792 33970 1611 2607 2805
enum-convert 67752 33202 830 1522 71888 37042 4675 2737 2512
error 67744 32498 131 163 67784 32946 591 2614 2587
exhausted-unhandled 67688 33090 749 1156 67728 34850 2491 2442 2591
exhausted 76192 35778 3415 5139 80336 42674 10321 2409 2469
fn-values 68296 34226 1745 5124 80632 43026 10672 2405 2406
format 76032 36802 4413 7076 84264 45394 13038 2449 2388
frame-rollback 68440 34466 2020 3424 80768 39762 7402 2383 2394
free-all-refused 67688 32370 28 28 67728 32658 299 2418 2497
generics 81368 42786 10137 20427 114184 71602 39241 2474 2516
handles 75880 35826 3482 6289 84112 46338 13981 2520 2470
higher-order 76648 37154 4652 10329 88976 51314 18948 2413 2439
into 80256 40658 8277 12345 92584 53682 21320 2428 2412
loops 67736 33378 1016 1720 71872 37634 5269 2370 2315
machine 67960 33186 821 1925 72096 37394 5042 2515 2456
macro-unless 67720 32642 288 615 67760 34610 2243 2466 2457
macros 67688 32802 461 793 67728 35250 2896 2532 2409
map-exhausted 76152 36162 3805 5994 84392 44530 12171 2393 2506
map-iter 76096 39586 7211 10374 92520 54082 21715 2541 2531
map-stale-region 67688 33650 1307 2106 71824 36898 4537 2354 2407
maps 88504 45266 12833 19850 117216 72802 40445 3162 3082
math 67832 33986 1611 3288 72024 39362 7000 2384 2405
math2 67784 33506 1148 2862 72024 39010 6644 2300 2482
pkg-diamond 67880 32626 252 616 67928 34162 1801 2353 2227
pkg-macro-idle 67688 32338 3 3 67728 32562 210 2440 2410
pkg-macro 67800 32722 354 450 67840 33986 1626 2455 2429
pkg-return 78416 39570 7024 23635 103032 64082 31716 2432 2375
pkg-shadow 67848 32642 277 371 67888 33618 1258 2408 2236
pkg-shared 74560 34066 1676 5593 86888 45650 13283 2422 2648
pkg-unused 69704 32386 35 35 69744 34658 2298 2351 2271
pool-stale-region 67688 33282 932 1675 71824 35746 3379 2389 2466
printers 82464 42098 168 452 82504 43282 1348 -1 -1
println 76256 36098 3737 7587 88584 50114 17753 2312 2460
raylib-audio 79536 35522 2478 5610 87768 44178 11225 2574 2769
raylib-ffi 85848 39538 6245 11506 102272 55650 22537 2607 2852
raylib-font 79088 35538 2622 6260 83232 43186 10350 2661 2658
raylib-image 80152 37698 4504 8285 88384 47058 13966 2684 2946
raylib-imported 70344 33202 612 1003 74480 36658 4066 2688 2602
reach-walk 67992 33170 774 1082 68032 34802 2443 2552 2308
recur 67792 34594 2233 4070 76024 40674 8314 2468 62674
registry 72008 35330 2940 4604 80240 41010 8608 2515 2329
reload-generic 67944 32930 520 1390 67992 35442 3076 2341 2241
restarts 77200 39010 6430 9268 89528 49442 17065 2356 2304
rl-with 69704 32386 35 35 69744 34658 2298 2550 2461
sand-headless 74728 34546 2139 6326 87056 47634 15269 2783 9172
signedness 67688 32594 246 386 67728 33938 1576 2418 2411
slice-from-ptr 67752 33106 755 2549 75984 37458 5101 2421 2367
slices 68128 34802 2393 9057 88648 50114 17753 2504 2271
slurp-unhandled 67688 33522 1173 1601 67728 35298 2945 2340 2354
slurp 80288 42114 9744 14813 100808 57010 24656 2358 2408
stale-region 67688 33490 1148 1681 71824 35842 3483 2332 2286
string-of-bytes 67856 33490 981 1925 71992 36338 3817 2319 2353
strings 88912 45890 13395 22681 109432 65618 33265 2455 2391
text 72296 37122 4666 8007 84624 49378 17018 2310 2397
unions 76016 36498 4131 17714 92440 55810 23456 2323 2413
unit-main 67688 32386 33 33 67728 32674 307 2459 2377
utf8 81216 39714 7170 21515 114024 73106 40745 2563 2380
values 67720 32530 180 667 67760 34082 1719 2531 2378
vec 80080 40402 8039 12089 92408 55122 22770 2370 2389
virtual-controls-headless 70896 33666 1271 1295 75032 38962 6601 2319 3201
web-files 67736 33058 706 1000 67776 34610 2246 2321 2382
p1-exit 67688 32338 3 3 67728 32562 210 2436 2377
p2-loop-print 67688 32626 283 106 67728 32930 577 2269 2364
p3-fizz 67720 32530 169 225 67760 33282 928 2512 2359
p4-convention 67824 32786 405 1005 67864 35138 2778 2436 2332
p5-core 67792 33346 993 1507 71928 37218 4865 2346 2444
p6-transfer 68096 34354 1944 2801 72232 37618 5252 2386 2310
p7-slice-from-ptr 67840 33698 1336 1498 71984 35682 3318 2493 2392
p8-cell 67752 32514 146 270 67800 33202 847 2402 2381
1 name llvm_file llvm_text llvm_own o0_own x86_file x86_text x86_own llvm_us x86_us
2 agent-longname 82224 42002 79 134 82264 42450 519 -1 -1
3 agent-queue 82352 42306 340 567 86488 43698 1747 2491 2546
4 agent 82336 42434 454 798 86472 44162 2218 2609 2496
5 algorithms 76680 39282 6776 16616 101296 63234 30875 2499 2605
6 allocators 67760 33650 1297 1719 71896 37490 5130 3170 3034
7 array-ctor 67784 32722 357 1840 71920 36242 3880 2442 2664
8 bounds-condition 76416 37074 4641 6963 84648 45858 13493 2673 2469
9 bounds 67720 32738 363 1951 71856 36418 4039 3194 2654
10 break 82296 42754 819 1182 86432 44482 2549 -1 -1
11 bytes2 72120 37010 4576 10893 88544 52018 19654 2573 2510
12 cleanup 68176 33986 1583 2077 72312 36946 4583 2351 2402
13 conditions 68064 33586 1212 1209 68104 35346 2984 2436 2513
14 debug-permuted 67720 32610 261 410 67760 33794 1433 2479 2442
15 debug 67720 32610 261 429 67760 33794 1433 2494 2494
16 defer-let 67928 34018 1625 1784 72064 36754 4396 2548 2543
17 destructure 67792 33650 1290 4577 80120 40946 8583 2534 2523
18 dev-break-bounds 82440 42546 588 648 86576 43762 1830 -1 -1
19 dev-break 82368 42418 459 715 86504 43682 1755 -1 -1
20 dev-globals 82456 42162 206 534 86584 43538 1614 -1 -1
21 dev-inspect 82408 42610 652 704 86544 43698 1765 -1 -1
22 dev-locals 82336 42418 465 623 86472 43650 1721 -1 -1
23 dev-noagent 67720 32434 83 109 67760 32866 507 3116 2939
24 dev-pause 82336 42066 112 244 82376 42802 877 -1 -1
25 dev-ptr 86632 43954 1970 2859 90768 47490 5543 -1 -1
26 dev-repl 82336 42066 112 244 82376 42802 877 -1 -1
27 dev-robust 82336 42066 112 244 82376 42802 877 -1 -1
28 edn 91208 44706 12057 34556 124016 80898 48532 3218 3165
29 embed 67800 33330 977 2461 71936 37906 5550 2440 2650
30 enum-compare 67752 32562 201 581 67792 33970 1611 2607 2805
31 enum-convert 67752 33202 830 1522 71888 37042 4675 2737 2512
32 error 67744 32498 131 163 67784 32946 591 2614 2587
33 exhausted-unhandled 67688 33090 749 1156 67728 34850 2491 2442 2591
34 exhausted 76192 35778 3415 5139 80336 42674 10321 2409 2469
35 fn-values 68296 34226 1745 5124 80632 43026 10672 2405 2406
36 format 76032 36802 4413 7076 84264 45394 13038 2449 2388
37 frame-rollback 68440 34466 2020 3424 80768 39762 7402 2383 2394
38 free-all-refused 67688 32370 28 28 67728 32658 299 2418 2497
39 generics 81368 42786 10137 20427 114184 71602 39241 2474 2516
40 handles 75880 35826 3482 6289 84112 46338 13981 2520 2470
41 higher-order 76648 37154 4652 10329 88976 51314 18948 2413 2439
42 into 80256 40658 8277 12345 92584 53682 21320 2428 2412
43 loops 67736 33378 1016 1720 71872 37634 5269 2370 2315
44 machine 67960 33186 821 1925 72096 37394 5042 2515 2456
45 macro-unless 67720 32642 288 615 67760 34610 2243 2466 2457
46 macros 67688 32802 461 793 67728 35250 2896 2532 2409
47 map-exhausted 76152 36162 3805 5994 84392 44530 12171 2393 2506
48 map-iter 76096 39586 7211 10374 92520 54082 21715 2541 2531
49 map-stale-region 67688 33650 1307 2106 71824 36898 4537 2354 2407
50 maps 88504 45266 12833 19850 117216 72802 40445 3162 3082
51 math 67832 33986 1611 3288 72024 39362 7000 2384 2405
52 math2 67784 33506 1148 2862 72024 39010 6644 2300 2482
53 pkg-diamond 67880 32626 252 616 67928 34162 1801 2353 2227
54 pkg-macro-idle 67688 32338 3 3 67728 32562 210 2440 2410
55 pkg-macro 67800 32722 354 450 67840 33986 1626 2455 2429
56 pkg-return 78416 39570 7024 23635 103032 64082 31716 2432 2375
57 pkg-shadow 67848 32642 277 371 67888 33618 1258 2408 2236
58 pkg-shared 74560 34066 1676 5593 86888 45650 13283 2422 2648
59 pkg-unused 69704 32386 35 35 69744 34658 2298 2351 2271
60 pool-stale-region 67688 33282 932 1675 71824 35746 3379 2389 2466
61 printers 82464 42098 168 452 82504 43282 1348 -1 -1
62 println 76256 36098 3737 7587 88584 50114 17753 2312 2460
63 raylib-audio 79536 35522 2478 5610 87768 44178 11225 2574 2769
64 raylib-ffi 85848 39538 6245 11506 102272 55650 22537 2607 2852
65 raylib-font 79088 35538 2622 6260 83232 43186 10350 2661 2658
66 raylib-image 80152 37698 4504 8285 88384 47058 13966 2684 2946
67 raylib-imported 70344 33202 612 1003 74480 36658 4066 2688 2602
68 reach-walk 67992 33170 774 1082 68032 34802 2443 2552 2308
69 recur 67792 34594 2233 4070 76024 40674 8314 2468 62674
70 registry 72008 35330 2940 4604 80240 41010 8608 2515 2329
71 reload-generic 67944 32930 520 1390 67992 35442 3076 2341 2241
72 restarts 77200 39010 6430 9268 89528 49442 17065 2356 2304
73 rl-with 69704 32386 35 35 69744 34658 2298 2550 2461
74 sand-headless 74728 34546 2139 6326 87056 47634 15269 2783 9172
75 signedness 67688 32594 246 386 67728 33938 1576 2418 2411
76 slice-from-ptr 67752 33106 755 2549 75984 37458 5101 2421 2367
77 slices 68128 34802 2393 9057 88648 50114 17753 2504 2271
78 slurp-unhandled 67688 33522 1173 1601 67728 35298 2945 2340 2354
79 slurp 80288 42114 9744 14813 100808 57010 24656 2358 2408
80 stale-region 67688 33490 1148 1681 71824 35842 3483 2332 2286
81 string-of-bytes 67856 33490 981 1925 71992 36338 3817 2319 2353
82 strings 88912 45890 13395 22681 109432 65618 33265 2455 2391
83 text 72296 37122 4666 8007 84624 49378 17018 2310 2397
84 unions 76016 36498 4131 17714 92440 55810 23456 2323 2413
85 unit-main 67688 32386 33 33 67728 32674 307 2459 2377
86 utf8 81216 39714 7170 21515 114024 73106 40745 2563 2380
87 values 67720 32530 180 667 67760 34082 1719 2531 2378
88 vec 80080 40402 8039 12089 92408 55122 22770 2370 2389
89 virtual-controls-headless 70896 33666 1271 1295 75032 38962 6601 2319 3201
90 web-files 67736 33058 706 1000 67776 34610 2246 2321 2382
91 p1-exit 67688 32338 3 3 67728 32562 210 2436 2377
92 p2-loop-print 67688 32626 283 106 67728 32930 577 2269 2364
93 p3-fizz 67720 32530 169 225 67760 33282 928 2512 2359
94 p4-convention 67824 32786 405 1005 67864 35138 2778 2436 2332
95 p5-core 67792 33346 993 1507 71928 37218 4865 2346 2444
96 p6-transfer 68096 34354 1944 2801 72232 37618 5252 2386 2310
97 p7-slice-from-ptr 67840 33698 1336 1498 71984 35682 3318 2493 2392
98 p8-cell 67752 32514 146 270 67800 33202 847 2402 2381

135
spike/x86/cost.sh Executable file
View File

@ -0,0 +1,135 @@
#!/usr/bin/env bash
# What does the hand-written backend cost, against LLVM, on the same programs?
#
# survey.sh answers "does it agree". This answers "what does agreeing cost",
# which is item 7 of HANDOFF-x86-rt.md and the one thing about this backend
# nobody had a number for. It builds each program the same two ways the
# survey does, and for each records three sizes and a time:
#
# file the whole executable on disk. Mostly runtime and libc glue, and
# the least interesting of the three -- it is here because it is
# the number anybody looks at first, and it should be visible how
# much of it is noise.
# text the .text section, from `size -A`. Still contains flan_rt.o,
# which is the same object on both sides.
# own the sum of the sizes of the defined symbols named `flan.<name>`
# -- the program's *own* code and nothing else. The runtime's C is
# `flan_<name>` with an underscore, so the two do not collide, and
# spot-checking a runtime symbol on both sides (flan_map_clone,
# 0x4b3 either way) says the runtime really is byte-identical and
# the difference in `own` is all codegen.
#
# The `own` column is the measurement; the other two are context.
#
# A fourth build, LLVM with --debug, is the reference that makes the number
# readable. --debug forces -O0, so it is LLVM's codegen with its optimiser
# switched off -- the closest thing available to what this backend is doing,
# which has no optimiser at all. Without it every ratio silently blames the
# backend for the whole of mem2reg and inlining. (--x86 --debug is refused,
# so the column exists on one side only, and that is the point of it.)
#
# Time is best-of-N, not a mean: a mean measures the other tenants of the
# machine. Even so, a corpus program is mostly process startup -- these are
# milliseconds -- so read the time column only where it is tens of
# milliseconds or more, and read the rest as size.
#
# Usage: spike/x86/cost.sh [name-substring ...] -> a TSV on stdout
# COST_FLAGS=--dev extra flags, given to both sides, as SURVEY_FLAGS is
# COST_REPS=5 timing repetitions
# COST_O0=0 skip the LLVM -O0 reference column
set -u
orig=$(pwd)
here=$(cd "$(dirname "$0")" && pwd)
root=$(cd "$here/../.." && pwd)
cd "$root" || exit 1
if [ -n "${FLAN:-}" ]; then
case $FLAN in /*) flan=$FLAN;; *) flan=$orig/$FLAN;; esac
else
dune build --root . bin/main.exe 2>&1 | head -30
flan=$root/_build/default/bin/main.exe
fi
test -x "$flan" || { echo "build failed" >&2; exit 1; }
# Not mktemp under /tmp by default: this writes a few hundred executables of
# a megabyte or two, and a full /tmp on this machine has already frozen one
# session. The guard is cheap and a wedged run is not.
# A directory of this run's own, made with a plain mkdir so that a second
# copy of this script cannot land in the first one's: two runs sharing a
# scratch directory overwrite each other's `l` and `x` between the build and
# the timing, and the result is a row of numbers that belong to two different
# programs. That happened once here and the numbers looked entirely ordinary.
work=${COST_OUT:-${TMPDIR:-/tmp}/flan-cost}
mkdir -p "$work" || exit 1
out=$work/run.$$
mkdir "$out" || exit 1
trap 'rm -rf "$out"' EXIT
free=$(df -Pk "$out" | awk 'NR==2 {print $4}')
[ "$free" -gt 2000000 ] || { echo "less than 2GB free at $out" >&2; exit 1; }
forever="dev-loop dev-watch"
REPS=${COST_REPS:-5}
read -r -a extra <<<"${COST_FLAGS:-}"
o0=${COST_O0:-1}
[ -z "${COST_FLAGS:-}" ] || o0=0
# The sum of the defined text symbols the compiler itself named. `nm -S`
# prints value, size, type, name; a symbol with no size is not printed with
# four fields at all, which is why the guard is on NF.
own () {
nm --defined-only -S "$1" 2>/dev/null \
| awk 'NF==4 && ($3=="T"||$3=="t") && $4 ~ /^flan\./ {n+=strtonum("0x"$2)} END{print n+0}'
}
text () { size -A "$1" 2>/dev/null | awk '$1==".text" {print $2}'; }
# Best of REPS, in whole microseconds. A program that fails on one run and
# not another would make this meaningless, so the exit status of the first
# run is remembered and a run that disagrees with it poisons the row as -1.
# A program that hits the timeout is not timed at all: several of the corpus
# programs are agents or daemons that sit waiting for something that is not
# there, and five repetitions of a twenty-second wait, twice, is most of an
# afternoon spent measuring `timeout`.
best () {
exe=$1; min=; rc0=
for i in $(seq "$REPS"); do
t0=$(date +%s%N)
( cd "$out" && timeout 20 "$exe" >/dev/null 2>&1 </dev/null )
rc=$?
t1=$(date +%s%N)
[ -n "$rc0" ] || rc0=$rc
[ "$rc" = "$rc0" ] || { echo "-1"; return; }
d=$(( (t1 - t0) / 1000 ))
[ "$d" -lt 15000000 ] || { echo "-1"; return; }
if [ -z "$min" ] || [ "$d" -lt "$min" ]; then min=$d; fi
done
echo "$min"
}
printf 'name\tllvm_file\tllvm_text\tllvm_own\to0_own\tx86_file\tx86_text\tx86_own\tllvm_us\tx86_us\n'
for src in "$root"/test/programs/*.flan "$root"/spike/x86/*.flan; do
name=$(basename "$src" .flan)
if [ $# -gt 0 ]; then
want=0
for pat in "$@"; do case "$name" in *"$pat"*) want=1;; esac; done
[ $want = 1 ] || continue
fi
case " $forever " in *" $name "*) continue;; esac
rm -f "$out/l" "$out/x" "$out/d"
"$flan" build "$src" "${extra[@]}" -o "$out/l" >/dev/null 2>&1 || continue
# No main is a link failure, and it leaves nothing behind to measure.
test -x "$out/l" || continue
"$flan" build "$src" --x86 "${extra[@]}" -o "$out/x" >/dev/null 2>&1 || continue
test -x "$out/x" || continue
d0=0
if [ "$o0" = 1 ] && "$flan" build "$src" --debug -o "$out/d" >/dev/null 2>&1; then
d0=$(own "$out/d")
fi
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$name" \
"$(stat -c %s "$out/l")" "$(text "$out/l")" "$(own "$out/l")" "$d0" \
"$(stat -c %s "$out/x")" "$(text "$out/x")" "$(own "$out/x")" \
"$(best "$out/l")" "$(best "$out/x")"
done

View File

@ -32,14 +32,33 @@
#
# Usage: spike/x86/survey.sh [name-substring ...]
set -u
orig=$(pwd)
here=$(cd "$(dirname "$0")" && pwd)
root=$(cd "$here/../.." && pwd)
cd "$root" || exit 1
dune build --root . bin/main.exe 2>&1 | head -30
flan=$root/_build/default/bin/main.exe
# FLAN is how the dune @x86 alias hands this script a compiler that dune has
# already built. Building it here instead would be a second dune inside the
# first one's lock, which does not run at all; and the alias has bin/main.exe
# in its deps precisely so it does not have to. Standalone -- the way the
# baseline in every handoff was measured -- nothing sets it and the build
# below is what it always was.
if [ -n "${FLAN:-}" ]; then
# Resolved against the directory this was invoked from, not against $root:
# dune spells its deps relative to the dune file, and the cd above has
# already happened by the time this is read.
case $FLAN in /*) flan=$FLAN;; *) flan=$orig/$FLAN;; esac
else
dune build --root . bin/main.exe 2>&1 | head -30
flan=$root/_build/default/bin/main.exe
fi
test -x "$flan" || { echo "build failed"; exit 1; }
# Where the corpus is read from. Under dune the script runs from the build
# tree, where test/programs is present but spike/x86 is not, so the alias
# points this at the source tree and gets both.
corpus=${SURVEY_CORPUS:-$root}
out=$(mktemp -d); trap 'rm -rf "$out"' EXIT
# The two that run until something stops them. Not a failure and not a match;
@ -58,7 +77,7 @@ read -r -a extra <<<"${SURVEY_FLAGS:-}"
declare -a match=() differ=() refused=() nox86=() skip=()
for src in "$root"/test/programs/*.flan "$root"/spike/x86/*.flan; do
for src in "$corpus"/test/programs/*.flan "$corpus"/spike/x86/*.flan; do
name=$(basename "$src" .flan)
if [ $# -gt 0 ]; then
want=0
@ -127,3 +146,19 @@ if [ "${#skip[@]}" != 0 ] && [ "${SURVEY_QUIET:-}" != 1 ]; then
printf '%s\n' "${skip[@]}" | sed 's/^[^:]*://' | sort | uniq -c \
| sed 's/^/ /'
fi
# Strict mode, for the @x86 alias: the counts above are a report, and a report
# nobody reads is how two refusals from another lane's new primitive sat in
# the tree for a month. A DIFFER is a wrong answer and a refusal by name is a
# node this backend has stopped lowering; either is a failure. NOX86 and SKIP
# are not: the first is usually a toolchain that is not installed here, and
# the second is the frontend refusing the program on both sides.
if [ "${SURVEY_STRICT:-}" = 1 ]; then
if [ "${#differ[@]}" != 0 ] || [ "${#refused[@]}" != 0 ]; then
echo
echo "x86 survey FAILED: ${#differ[@]} differ, ${#refused[@]} refused"
exit 1
fi
echo
echo "x86 survey ok: ${#match[@]} match"
fi

View File

@ -169,3 +169,55 @@
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*))
(action (run ./test_valgrind.exe)))
; The corpus a fourth time, through the hand-written x86-64 backend, compared
; against LLVM on what each program prints and what it exits with. Its own
; alias for the same reason the two above have one -- it builds every program
; twice and runs both, which is a couple of minutes against `dune test`'s
; seconds -- but the reason it exists at all is different. @sanitize and
; @valgrind ask whether the runtime is sound. This one asks whether the
; second backend still lowers the language: it refuses by name rather than
; miscompiling, so when another lane adds a primitive the backend says so
; loudly, and nothing was listening. Two such refusals sat in the tree for a
; month. Now they fail a build somebody can run.
;
; dune build --root . @x86
;
; A rule with no executable beside it, unlike @sanitize and @valgrind: the
; check already exists as spike/x86/survey.sh, which is what every handoff
; quotes its counts from, and a second implementation in OCaml would be a
; second thing to drift. SURVEY_STRICT=1 turns its report into an exit
; status. FLAN is passed because the script otherwise runs `dune build` on
; the compiler, and a dune inside a dune action waits on a lock it cannot
; get; main.exe is in the deps instead. SURVEY_QUIET keeps the skip
; breakdown out of a passing build's log.
(rule
(alias x86)
(deps
(file %{workspace_root}/spike/x86/survey.sh)
(glob_files %{workspace_root}/spike/x86/*.flan)
(file %{workspace_root}/bin/main.exe)
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
(glob_files programs/assets/*)
; A glob per package directory, because dune's glob does not descend.
(glob_files programs/pkgs/shape/*)
(glob_files programs/pkgs/area/*)
(glob_files programs/pkgs/draw/*)
(glob_files programs/pkgs/ring-a/*)
(glob_files programs/pkgs/ring-b/*)
(glob_files programs/pkgs/ring-c/*)
(glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*))
(action
(setenv SURVEY_STRICT 1
(setenv SURVEY_QUIET 1
(setenv FLAN %{workspace_root}/bin/main.exe
(run bash %{workspace_root}/spike/x86/survey.sh))))))