flan/docs/handoffs/HANDOFF-x86-debug.md

# Handoff — dropping the registry notes, and debug information for `--x86`

Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`. Started from `957ba07` and rebased onto `eacf7c4`
when the arithmetic-condition lane landed; the rebase was clean, and everything below was re-measured after
it. Items **2** and **6** of `HANDOFF-x86-rt.md` §6. Both landed.

## The finding that changes the brief, read this first

**`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line
tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is
true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and
GAS builds its line table from `dwarf2_emit_insn`, which runs only out of `md_assemble`. No instruction is
ever assembled, so nothing flushes the pending `.loc`.

What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location
counter has reached by then. Measured on `as` 2.44:

```
	.file	1 "foo.flan"
main:	.loc 1 10 1
	.byte	0x55
	.byte	0x48,0x89,0xe5
	.loc	1 11 1
	.byte	0xb8,0x07,0x00,0x00,0x00
	.loc	1 12 1
	.byte	0xc9
	.byte	0xc3
```

```
foo.flan     10      0x4     x      <- should be 0x0
foo.flan     11      0x9     x      <- should be 0x4
foo.flan      -      0xb            <- line 12 has no row at all
```

Every row is one statement late and the last statement of every function gets none. Interposing labels
between the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting
the `.loc` directives back by one — the last-statement hole is a correctness gap, not just fragility, and the
behaviour being exploited is undocumented.

So `.debug_line` is written out here by hand, as `.byte` / `.uleb128` / `.quad` in a `.debug_line` section,
which is the same thing this backend already does for instructions.

**`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag
positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out
*exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct
advances. So this lane emits them, in a `--debug` build — see below.

## What landed

### Item 2 — a release build stops calling a registry that is switched off

`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test
for test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never
a `Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so
the body is `()` and `dst` is untouched.

`emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the
container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are
not touched until `call_rt`, so the guard is already early enough.

Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`:

| program | `--x86` release | `--x86 --dev` | LLVM release |
|---|---|---|---|
| `vec.flan` | 46 → **1** | 46 | 1 |
| `maps.flan` | 55 → **1** | 55 | — |
| `registry.flan` | 36 → **1** | 36 | — |

The one that remains in every release build is not emitted code: it is inside the runtime's own
`flan_dev_reg_note_vec` in `flan_rt.c`. LLVM's release build has exactly the same one, so the two backends
agree.

### Item 6 — `--x86 --debug` works, at parity with LLVM bar locals

`lib/build.ml`'s refusal list no longer names `--debug`; only `--sanitize` and wasm are left there.
`X86.program` takes `~debug` the way it already took `~dev`.

In `lib/x86.ml`:

| piece | what |
|---|---|
| the `Debug information` section, above `fnctx` | `dwrow` / `dwsub` / `dwarf`, and `dwfile`. Carries the header comment explaining the `.loc` finding above |
| `fnctx.dw : dwarf option` | `None` in every build but a `--debug` one, which is what makes the release path structurally unchanged |
| `dwline`, beside `new_label` | the one hook. A label and a row, deduplicated on (file, line, column) *and* on the byte counter — see below |
| `lower` | calls `dwline f e.Tast.loc` and nothing else |
| `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte |
| `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) |
| `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end |
| `cfi_after_push` / `cfi_after_mov` / `cfi_after_leave`, above `emit_fn` | the whole frame model in five directives, at the three sites that share a prologue: `emit_fn`, `emit_main`, `emit_globals_init` |

Two decisions worth knowing about because they are not obvious:

**The line program is written in the dumb form.** Every row is a full `DW_LNE_set_address` on a label, then
`set_file` / `set_column` / `advance_line` as needed, then `DW_LNS_copy` — eleven bytes and up per row, where
a special opcode would be one. The alternative is `DW_LNS_advance_pc` with a `.uleb128` of a difference of two
labels, which asks the assembler to resolve a value whose *size* changes the offsets after it. That does work,
and its failure would look exactly like a DWARF bug. A debug build can afford the bytes.

**The `.file 1 "..."` directive at the top is load-bearing and is not about our own table.** Without it,
clang's integrated assembler generates a compile unit of *its own* over the `.s`, whose rows land at the
`call` mnemonics — the only real instructions this backend emits — and those addresses are inside functions
our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an
empty line table and nothing else. `-gdwarf-4` is added alongside it in `build.ml` so that empty stub is a
version 4 header; at the default version it is a DWARF 5 header whose file table `readelf` calls corrupt, and
a warning on a cross-check is a warning you learn to ignore.

**Rows are deduplicated on the byte counter as well as the position.** `lower` recurses, so an outer form and
the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the
outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one
address takes the last. `buf.n` already existed and was written but never read; this is its first reader.

## Verification — a real debugger, on the corpus's own debug fixture

`test/programs/debug.flan` built `--x86 --debug`, under `gdb` 16.x, breaking by file and line:

```
$ flan build test/programs/debug.flan --x86 --debug -o dbg
$ gdb -batch -x gdb5.cmd ./dbg
Breakpoint 1 at 0x400689: file debug.flan, line 19.

Breakpoint 1, flan.tick () at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick () at debug.flan:19
#1  0x00000000004007b5 in flan.main () at debug.flan:25
#2  0x0000000000400bea in main ()
Line 19 of "debug.flan" starts at address 0x400689 <flan.tick+97> and ends at 0x4006af <flan.tick+135>.
Current source file is debug.flan
Compilation directory is .../test/programs
Located in .../test/programs/debug.flan
Contains 30 lines.
Source language is c.
Producer is flan (x86-64).
Compiled with DWARF 4 debugging format.
```

The same script against an **LLVM** `--debug` build of the same program, which is the parity this is measured
against:

```
Breakpoint 1 at 0x400644: file debug.flan, line 19.

Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
#1  0x00000000004006cf in flan.main () at debug.flan:25
#2  0x0000000000400806 in main ()
Producer is flan.
Compiled with DWARF 5 debugging format.
```

Identical bar `c=0x7fffffffd8f0, n=41` — which is the locals work, and is the honest summary of what is
missing.

**`break tick` fails on both backends**, with `Function "tick" not defined`. That is not a gap this lane
opened: `emit.ml` writes `name: "tick", linkageName: "flan.tick"`, gdb takes the linkage name as the search
name because `flan.tick` is not a mangled C++ name, and an LLVM `--debug` build behaves the same way. The
project's own source-level debugging case runs under **lldb**, which does not have this problem. `break
debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from
it; if this is worth fixing it should be fixed in both places at once.

**lldb**, which is the debugger this project's own source-level case runs under, does better still — it
resolves `flan.tick` by name, shows the source inline and gives a column:

```
$ lldb -b -o "breakpoint set --name flan.tick" -o run -o bt ./dbg
* thread #1, stop reason = breakpoint 1.1
    frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3
   17  	(defn tick [c (Ptr Cell)  n i32] i32
-> 18  	  (let [bump (+ n 1)]
    	  ^
   19  	    (set (.heat c) (+ (.heat c) 1.5))
(lldb) bt
  * frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3
    frame #1: 0x00000000004007b5 dbg`flan.main at debug.flan:25:28
    frame #2: 0x0000000000400bea dbg`main + 41
    frame #3: 0x00007ffff7cb9575 libc.so.6`__libc_start_call_main + 117
```

That transcript is now a test rather than a transcript: `test/test_acceptance.ml`'s lldb block has an `--x86`
arm beside its `--dev` one, asserting the breakpoint resolves and both Flan frames name `debug.flan`, and that
the program still prints what it printed. It claims no `frame variable`, which is the gap.

**Unwinding out of the C runtime works**, which is the case that matters for this project's error paths and
which the brief did not ask for but should have. `bounds.flan` built `--x86 --debug`, breaking on the C symbol:

```
Breakpoint 1, flan_bounds_error (loc=0x40a880 "test/programs/bounds.flan:11:44", ...) at flan_rt.c:534
#0  flan_bounds_error (...) at flan_rt.c:534
#1  0x00000000004006be in flan.main () at bounds.flan:11
#2  0x000000000040162e in main ()
```

**The compile unit's range is exact.** `DW_AT_low_pc` 0x400628 is `flan.tick`, the first function;
`DW_AT_high_pc` 0x5cf puts the end at 0x400bf7, and `nm -S` says `main` — the last thing this object puts in
`.text` — ends at 0x400bc1 + 0x36 = 0x400bf7. So `.Ldwtext` / `.Ldwtext_end` bracket the four functions this
unit emitted and nothing else.

`--x86 --dev --debug` together, which is a reachable combination and is exercised by nothing else, builds,
runs, and gives a line table and an `.eh_frame` that `readelf` reads with no warnings — 280 FDEs, ours among
the runtime's. That is the one combination where the cells' `.data`, the `.eh_frame` and the debug sections
all have to be ordered against each other.

Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`,
`bounds-condition`, `algorithms`, `handles`, `registry`, `destructure`, `edn`, twelve in all — every one
builds with `--x86 --debug`, runs, and produces `.debug_line` that `readelf --debug-dump=decodedline` reads
with **zero warnings**. `edn` and `generics` are the large ones, at 40 and 36 subprograms. `generics` is the
multi-file case: its directory table has one entry and its file table two, `generics.flan` and `<prelude>`.
`<prelude>` has no path on disk and a debugger simply finds no source for it, which is the truth.

## Baseline, measured on this tree

| | before (`957ba07`) | after, on `957ba07` | after, rebased onto `eacf7c4` |
|---|---|---|---|
| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | **99 / 0 / 0 / 0**, skips 28 + 6 + 2 | **101 / 0 / 0 / 0**, skips 28 + 6 + 2 |
| `spike/x86/cells.sh` | 4/4 ok | 4/4 ok | **4/4 ok** |
| `dune test --root .` | exit 0 | exit 0 | **exit 0**, run without a pipe; acceptance reports **232 checks, 0 failures** |

The MATCH count went from 99 to 101 across the rebase and neither is this lane's doing: the
arithmetic-condition lane added two probes to `spike/x86`. The number that matters is that DIFFER, REFUSED and
NOX86 are all zero on both sides and the skip breakdown did not move.

The survey is protected structurally as well as measured: every piece of this lane's work on the release path
is behind `debug`, so `--x86` without `--debug` emits the assembly it emitted before — `fnctx.dw` is `None`,
`dwline` does nothing, the `.Ldwtext` label is not written and neither are the `.cfi` directives.

Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you
`tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way
while the run still exits 0.

## What was not reached, and what the next lane should do with it

1. **Locals and types.** Not attempted, and the reason is structural rather than a lack of time. `emit.ml`
   writes a `!DILocalVariable` per slot because every slot there is an `alloca` that `llvm.dbg.declare`
   points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: `alloc`
   knows its `rbp`-relative offset, but `scoped` reclaims temporaries and a later expression reuses the bytes,
   so the *lifetime* is not modelled. A `DW_TAG_variable` with a `DW_OP_fbreg` would be right at some
   addresses and confidently wrong at others. Note that the named slots — `fn.Tast.snames` — are the *slots*
   and not the temporaries, and those do live for the whole function, so a narrower version of this is
   reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with
   `DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the
   part `emit.ml` spends most of its debug lines on.
2. **`.cfi` is emitted, but only in a `--debug` build.** Five directives at three sites — `emit_fn`,
   `emit_main`, `emit_globals_init`, which share a prologue. `readelf --debug-dump=frames` gives the exact
   advances. `emit_main` gets no closing rule because it has no epilogue: it leaves through `flan_exit` and
   the `ud2` after that is unreachable, so the `rbp` rule holds to the last byte.

   gdb did not need it — its prologue analyser already unwound out of `flan_bounds_error`, because
   `push rbp; mov rbp,rsp; sub rsp,N` is the pattern it recognises — so this is a description stated rather
   than guessed, plus a correct unwind from the very first byte of a function, before the `push`.

   **The open decision is whether to make it unconditional.** The description is correct in every build and a
   release build is where a crash would most want it. What stopped it here is only that nothing measures the
   `.eh_frame` it would add, and another lane is measuring backend cost right now. One `if` in three places.
3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's
   bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all
   (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded
   body will show raw addresses.
4. **`emit_main` and `flan..init-globals` have no rows, and this is a gap rather than a wrong answer.**
   Both sit inside the compile unit's `low_pc` / `high_pc` range and neither has a line-table sequence, so
   the worry is that a debugger picks a nearby row out of a *neighbouring* function's sequence and reports a
   confident wrong Flan line. It does not. Checked, breaking inside the initialiser of `dev-globals.flan`
   built `--x86 --debug`:

   ```
   Breakpoint 1, 0x0000000000400cec in flan..init-globals ()
   #0  0x0000000000400cec in flan..init-globals ()
   #1  0x00007ffff7cb96a4 in __libc_start_main_impl () from /lib64/libc.so.6
   #2  0x00000000004006c5 in _start ()
   No line number information available.
   ```

   The frame is named from the ELF symbol, the line is honestly absent, and the unwind out to libc is the
   `.cfi` working. So this is worth doing and is not urgent: `emit_globals_init` does lower expressions that
   carry locations, and giving it a `dwsub` the way `emit_fn` has one is the whole of it — it already takes
   `~cfi`, so `?dw` goes in beside it.
5. **`debug-permuted.flan` has no `--x86` arm.** The new acceptance case covers `debug.flan` only. The
   permuted fixture exists to catch a member offset that does not follow the declaration, and that is a
   *types* claim — there are no type DIEs here to get wrong, so it would test nothing today. It becomes the
   right test the moment item 1 above is attempted, and it is already written.

## Open questions for the author

- **`break tick` naming.** Is the `name` / `linkageName` split worth keeping, given it costs `break <fn>` in
  gdb on *both* backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may
  be deliberate. It was left alone rather than diverged from.
- **Should `--x86 --debug` imply `--dev`?** It does not today, and they are independent axes, which seems
  right — but a debug build you cannot redefine into is half of what the dev loop wants.