# Handoff — `flan emit --x86` annotated, so its output can be read Branch `dev-loop`, from `f772162`. The x86 backend exists so that a dev build is ours end to end and so that a reader can be told *why* each instruction is there. Until now `flan emit --x86` printed a three-line file header and then nothing but `.byte` blobs: no source form, no frame key, no statement of the calling convention, nothing naming the bookkeeping. Reading it meant `as` plus `objdump`, and even that answers only *what*. **It is annotated now.** Every run of bytes is headed by the Flan form that produced it; every function carries a frame map and a statement of how its arguments arrived; and every piece of bookkeeping the compiler adds has a name where it appears and an explanation once, in a legend at the top of the file. ## What was built | file | what | |---|---| | `lib/loc.ml` | `Loc.snippet` — the text of a span, on one line, whitespace collapsed, or `None` where there is no readable source. The squiggle's other half: the squiggle points at a form in its own file, this quotes it somewhere the file is not | | `lib/x86.ml` | the annotation machinery: a queue of pending comments on `buf`, the per-form hook in `lower`, `frame_map`, the bookkeeping `note`s, and the legend | | `bin/main.ml` | `emit --x86` annotates; `--no-annotate` is the bare spelling | | `spike/x86/annot.sh` | **new** — emits every program in the corpus both ways, assembles both, and compares every section of the two objects byte for byte, in all three of default, `--dev` and `--debug` | | `spike/x86/dump.sh` | its x86 section now shows the annotated listing *and* the disassembly of the object that listing assembles to — why beside what | `lib/build.ml` was not touched. A `--x86` build emits exactly the assembly it emitted before. ## The two decisions **Annotation is always on for `flan emit --x86` and always off for a `--x86` build.** `flan emit` exists to be read by a person, and there is no reason to make a reader ask for the thing the command is for. A build's `.s` is a temporary file handed straight to clang and read by nobody, so leaving it untouched costs nothing and buys something: `survey.sh`'s 103 MATCH stays a statement about the same text it has always been about, rather than about text this lane rewrote. `--no-annotate` exists for exactly one consumer, `annot.sh`. **None of this belongs to the LLVM path.** `flan emit` already prints IR that names its values, carries `!dbg` on every instruction and a `!DILocalVariable` per slot. The problem being fixed here is one this backend has and LLVM does not. ## Mnemonics: not done, deliberately The brief allowed a trailing mnemonic per line as a bonus and said not to write a disassembler for it. No disassembler was written — but the honest reason is not that one would be needed. The encoder knows the mnemonic at the moment it emits the bytes; threading a text trace through every one of its entry points, with memory-operand formatting to match, is a larger change than the request and would touch every encoding function in the file, which is the part of it the survey is a structural check on. What replaces it is cheaper and arguably better: `dump.sh` now prints the annotated `.s` and `objdump`'s disassembly of the same object side by side. The disassembly says what the instructions are; the listing says why they exist. Neither answers the other's question, and having both is what the four-way comparison was for. ## How the annotation is attached, and why that shape The bytes accumulate in `buf.pend` and flush as one `.byte` directive. A comment therefore cannot simply be written when a form starts lowering — the form may emit nothing, and a heading left standing would be read as belonging to whatever came next. So headings are **queued, not written**. `annote` puts one on `buf.ann` with a serial; the next byte written flushes the pending directive and then the queue, so the comment lands immediately above the bytes it is about; and `unannote` withdraws whatever a form queued and never spent. The serials are monotonic, so "was mine written?" is one integer comparison against the highest serial ever written. The hook is in `lower`, beside `dwline`, and for the same reason `dwline` is there: the recursion that lowers a nested call also lowers its arguments, so a heading queued in `lower` spans exactly the bytes that form and everything inside it emit. The margin moves with the nesting, so an argument's code steps in and the call's steps back out and the shape of the expression is visible without reading a word. **Atoms are not annotated**, and this is the single detail the acceptance test turns on. A literal, a local or a global would steal its parent's heading: `(* n 2)` lowers as a load, a load and an `imul`, and if the two operands each queued a heading of their own then the line standing above the `imul` would name the literal `2`. Skipping them leaves `(* n 2)` queued until the first byte and spanning the whole run. Read the sample below: the line above the `imul` is the one that had to be right. A form the checker invented carries `Loc.unknown` and is not annotated either — there is no source text to quote, and it inherits the heading of the form that contains it, which is where it really came from. A form at the same position as the last heading written is skipped, which is what stops a macro from printing its call site once per form of its expansion. A function whose source is not readable from here — the prelude — gets its frame map, no form headings, and one line saying so, rather than a column of bare positions. ## The frame map This is the half that matters most and the one no amount of disassembly recovers. LLVM's output names its values; this backend's cannot, because every value it has is a bump-allocated frame temporary and a temporary has no name. So `-0x20(%rbp)` is the whole vocabulary of the listing and the map is its key. Everything in it is read out of state `emit_fn` already keeps — `f.slots` *is* what the prologue stores through, `fn.snames` is what the source called each slot — so it cannot drift from the code it describes. The boundary between named slots and temporaries is captured as `fixed` right after the fixed allocations and before the body is lowered; `maxframe` would be the wrong number, because that is the high-water mark of the temporaries rather than where they start. What is deliberately *not* described is any individual temporary. `scoped` reclaims them and a later form reuses the bytes, so naming an offset that holds something else half the time is worse than saying where the region begins — the same call the DWARF above it makes about locals, and for the same reason. ## The bookkeeping, named Five things the compiler adds that no form asked for, each named where it appears and explained once in the file legend rather than at every site: the transfer guard after every call to Flan code, the bounds check and its signalling slow path, the arithmetic guard and the float-to-integer range check, `rep movsb` for every aggregate copy, and the indirection cell a `--dev` build calls through. The prologue, the epilogue, the transfer exit and C's `main` carry a prose block each. ## A worked sample `spike/x86/dump.sh small.flan twice`, on ``` (defn twice [n i64] i64 (* n 2)) ``` LLVM at `-O2`: ``` flan.twice: movq %rdi, -8(%rsp) leaq (%rdi,%rdi), %rax retq ``` and the same function out of this backend: ``` # ──────────────────────────────────────────────────────────────────── # "flan.twice" (defn twice [n i64] i64 small.flan:1:7 # # Arguments: n from rdi. # Returns i64 in rax. # The transfer channel arrives last of all, from rsi. It is a pointer to the # cell a callee writes its target into, and reading it is what every guard # below does. # # The frame is 0x30 bytes below rbp. No call in it passes an argument on the # stack. # # -0x8 n i64 parameter 1, from rdi # -0x10 ptr the transfer channel this frame passes on # -0x18 i64 the return value the epilogue loads # Everything below -0x18 is a temporary. They are bump-allocated and # reclaimed at the end of the form that made them, so a later form reuses # the bytes and no one offset down there means one thing for long. # ──────────────────────────────────────────────────────────────────── .globl "flan.twice" .type "flan.twice", @function "flan.twice": # The prologue: save rbp, take the frame in one sub, and spill every incoming # register into its slot. rsp is written here and by leave and nowhere else, # so rsp % 16 == 0 at every call site below is a property of that one rounded # sub rather than an invariant each case has to keep. .byte 0x55,0x48,0x89,0xe5,0x48,0x81,0xec,0x30,... # (* n 2) small.flan:2:3 .byte 0x48,0x8b,0x85,0xf8,0xff,0xff,0xff,0x48,0x89,0x85,0xe0,... # The epilogue, and every return and every transfer out of this frame arrives # here, so the frame is torn down once. .Lret1048: .byte 0x48,0x8b,0x85,0xe8,0xff,0xff,0xff,0xc9,0xc3 .size "flan.twice", . - "flan.twice" ``` Three LLVM instructions against thirteen, and the listing now says where the difference went: a real frame rather than a red zone, because rsp is written twice per function and never at a call site; a transfer channel parameter that LLVM's `-O2` dropped as dead and this backend spills because a dev build does not optimise; and both operands of the multiply through frame temporaries at `-0x20` and `-0x28`, because every intermediate in this backend is a frame temporary. None of that is visible in the disassembly and all of it is visible here. A denser one, from `bounds.flan` — the four instructions after every call, and the bounds triple: ``` # (at args 1) bounds.flan:11:35 .byte 0x48,0xb8,0x01,... # The bounds check. One unsigned compare catches a negative index as # well as an oversized one, and the not-taken branch is the whole of # the fast path. .byte 0x48,0x63,0x85,0xa4,...,0x48,0x39,0xc8,0x0f,0x82 .long .Linb1078 - . - 4 # Out of bounds: the location string, the operands, and this frame's # channel, then flan_bounds_error, which signals .byte 0x48,0x8d,0x3d .long .Lk1079 - . - 4 ... call flan_bounds_error # The transfer guard, after every call to Flan code: load this # frame's channel, load through it, test, and branch if it is set — # a callee that transferred left a target there and the value in rax # means nothing. .byte 0x4c,0x8b,0x9d,0xd0,...,0x0f,0x85 .long .Lxfer1077 - . - 4 # ud2, where emit.ml writes unreachable. Nothing answered the signal, # so the runtime already died inside that call and nothing falls # through to here. .byte 0x0f,0x0b ``` ## Verification **Byte identity — `spike/x86/annot.sh`, 342 SAME / 0 DIFFER, measured on this code.** Every program in `test/programs` and in `spike/x86`, emitted both ways, assembled both ways, and every section of the two objects compared byte for byte — in the default build, in `--dev`, and in `--debug`. The `--debug` case is the sharp one: a `.debug_line` row is an address expressed as a label, and annotation issues no labels and consumes no `uniq`, precisely so that those cannot move. The 84 SKIPs are programs that do not compile at all — the reject corpus and the generic-milestone ones — and are the same set the survey skips. **`dune test --root .` — 232 checks, 0 failures**, run against this code. `dune build @page` and `dune build @cells` both exit 0; `@cells` reports `x86 --dev: 22 22` and `x86 : 42 42`, which is the check that the indirection cells and the ABI marker still come out where they were. **`spike/x86/survey.sh` has NOT been run on this work.** Two runs were started and both were invalidated by racing with a `dune build` that replaced `bin/main.exe` underneath them; a third was killed on instruction before it finished. **Whoever picks this up must not assume the 103 MATCH / 0 DIFFER / 0 REFUSED baseline still holds — it is the one check that matters and it is outstanding.** Run it detached and with the compiler pinned, so it cannot race a rebuild: ``` FLAN=_build/default/bin/main.exe setsid timeout 2400 spike/x86/survey.sh > log 2>&1