flan/HANDOFF-x86-debug.md

# Handoff — dropping the registry notes, and debug information for `--x86`

Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`, from `957ba07`. Items **2** and **6** of
`HANDOFF-x86-rt.md` §6. Both landed.

## The finding that changes the brief, read this first

**`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line
tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is
true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and
GAS builds its line table from `dwarf2_emit_insn`, which runs only out of `md_assemble`. No instruction is
ever assembled, so nothing flushes the pending `.loc`.

What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location
counter has reached by then. Measured on `as` 2.44:

```
	.file	1 "foo.flan"
main:	.loc 1 10 1
	.byte	0x55
	.byte	0x48,0x89,0xe5
	.loc	1 11 1
	.byte	0xb8,0x07,0x00,0x00,0x00
	.loc	1 12 1
	.byte	0xc9
	.byte	0xc3
```

```
foo.flan     10      0x4     x      <- should be 0x0
foo.flan     11      0x9     x      <- should be 0x4
foo.flan      -      0xb            <- line 12 has no row at all
```

Every row is one statement late and the last statement of every function gets none. Interposing labels
between the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting
the `.loc` directives back by one — the last-statement hole is a correctness gap, not just fragility, and the
behaviour being exploited is undocumented.

So `.debug_line` is written out here by hand, as `.byte` / `.uleb128` / `.quad` in a `.debug_line` section,
which is the same thing this backend already does for instructions.

**`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag
positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out
*exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct
advances. Nothing here emits them yet (see "What was not reached"), but the next lane should know it can.

## What landed

### Item 2 — a release build stops calling a registry that is switched off

`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test
for test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never
a `Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so
the body is `()` and `dst` is untouched.

`emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the
container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are
not touched until `call_rt`, so the guard is already early enough.

Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`:

| program | `--x86` release | `--x86 --dev` | LLVM release |
|---|---|---|---|
| `vec.flan` | 46 → **1** | 46 | 1 |
| `maps.flan` | 55 → **1** | 55 | — |
| `registry.flan` | 36 → **1** | 36 | — |

The one that remains in every release build is not emitted code: it is inside the runtime's own
`flan_dev_reg_note_vec` in `flan_rt.c`. LLVM's release build has exactly the same one, so the two backends
agree.

### Item 6 — `--x86 --debug` works, at parity with LLVM bar locals

`lib/build.ml`'s refusal list no longer names `--debug`; only `--sanitize` and wasm are left there.
`X86.program` takes `~debug` the way it already took `~dev`.

In `lib/x86.ml`:

| piece | what |
|---|---|
| the `Debug information` section, above `fnctx` | `dwrow` / `dwsub` / `dwarf`, and `dwfile`. Carries the header comment explaining the `.loc` finding above |
| `fnctx.dw : dwarf option` | `None` in every build but a `--debug` one, which is what makes the release path structurally unchanged |
| `dwline`, beside `new_label` | the one hook. A label and a row, deduplicated on (file, line, column) *and* on the byte counter — see below |
| `lower` | calls `dwline f e.Tast.loc` and nothing else |
| `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte |
| `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) |
| `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end |

Two decisions worth knowing about because they are not obvious:

**The line program is written in the dumb form.** Every row is a full `DW_LNE_set_address` on a label, then
`set_file` / `set_column` / `advance_line` as needed, then `DW_LNS_copy` — eleven bytes and up per row, where
a special opcode would be one. The alternative is `DW_LNS_advance_pc` with a `.uleb128` of a difference of two
labels, which asks the assembler to resolve a value whose *size* changes the offsets after it. That does work,
and its failure would look exactly like a DWARF bug. A debug build can afford the bytes.

**The `.file 1 "..."` directive at the top is load-bearing and is not about our own table.** Without it,
clang's integrated assembler generates a compile unit of *its own* over the `.s`, whose rows land at the
`call` mnemonics — the only real instructions this backend emits — and those addresses are inside functions
our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an
empty line table and nothing else. `-gdwarf-4` is added alongside it in `build.ml` so that empty stub is a
version 4 header; at the default version it is a DWARF 5 header whose file table `readelf` calls corrupt, and
a warning on a cross-check is a warning you learn to ignore.

**Rows are deduplicated on the byte counter as well as the position.** `lower` recurses, so an outer form and
the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the
outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one
address takes the last. `buf.n` already existed and was written but never read; this is its first reader.

## Verification — a real debugger, on the corpus's own debug fixture

`test/programs/debug.flan` built `--x86 --debug`, under `gdb` 16.x, breaking by file and line:

```
$ flan build test/programs/debug.flan --x86 --debug -o dbg
$ gdb -batch -x gdb5.cmd ./dbg
Breakpoint 1 at 0x400689: file debug.flan, line 19.

Breakpoint 1, flan.tick () at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick () at debug.flan:19
#1  0x00000000004007b5 in flan.main () at debug.flan:25
#2  0x0000000000400bea in main ()
Line 19 of "debug.flan" starts at address 0x400689 <flan.tick+97> and ends at 0x4006af <flan.tick+135>.
Current source file is debug.flan
Compilation directory is .../test/programs
Located in .../test/programs/debug.flan
Contains 30 lines.
Source language is c.
Producer is flan (x86-64).
Compiled with DWARF 4 debugging format.
```

The same script against an **LLVM** `--debug` build of the same program, which is the parity this is measured
against:

```
Breakpoint 1 at 0x400644: file debug.flan, line 19.

Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
#1  0x00000000004006cf in flan.main () at debug.flan:25
#2  0x0000000000400806 in main ()
Producer is flan.
Compiled with DWARF 5 debugging format.
```

Identical bar `c=0x7fffffffd8f0, n=41` — which is the locals work, and is the honest summary of what is
missing.

**`break tick` fails on both backends**, with `Function "tick" not defined`. That is not a gap this lane
opened: `emit.ml` writes `name: "tick", linkageName: "flan.tick"`, gdb takes the linkage name as the search
name because `flan.tick` is not a mangled C++ name, and an LLVM `--debug` build behaves the same way. The
project's own source-level debugging case runs under **lldb**, which does not have this problem. `break
debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from
it; if this is worth fixing it should be fixed in both places at once.

Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`,
`bounds-condition`, `algorithms`, `handles`, `registry` — all build with `--x86 --debug`, run, and produce
`.debug_line` that `readelf --debug-dump=decodedline` reads with **zero warnings**. `generics` is the
multi-file case: its directory table has one entry and its file table two, `generics.flan` and `<prelude>`.
`<prelude>` has no path on disk and a debugger simply finds no source for it, which is the truth.

## Baseline, measured on this tree

| | before | after |
|---|---|---|
| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | (see below) |
| `spike/x86/cells.sh` | 4/4 ok | (see below) |
| `dune test --root .` | exit 0 | (see below) |

Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you
`tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way
while the run still exits 0.

## What was not reached, and what the next lane should do with it

1. **Locals and types.** Not attempted, and the reason is structural rather than a lack of time. `emit.ml`
   writes a `!DILocalVariable` per slot because every slot there is an `alloca` that `llvm.dbg.declare`
   points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: `alloc`
   knows its `rbp`-relative offset, but `scoped` reclaims temporaries and a later expression reuses the bytes,
   so the *lifetime* is not modelled. A `DW_TAG_variable` with a `DW_OP_fbreg` would be right at some
   addresses and confidently wrong at others. Note that the named slots — `fn.Tast.snames` — are the *slots*
   and not the temporaries, and those do live for the whole function, so a narrower version of this is
   reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with
   `DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the
   part `emit.ml` spends most of its debug lines on.
2. **`.cfi`, and it is now known to be cheap.** Nothing here emits it, so gdb unwinds by its prologue
   analyser — which works, because `push rbp; mov rbp,rsp; sub rsp,N` is the canonical pattern and this
   backend's header guarantees `rsp` is written exactly twice. The backtraces above are that analyser
   working. But the experiment at the top of this file says `.cfi` against `.byte` output is exact, and the
   frame model makes the content textbook: CFA is `rsp+16` after the `push`, `rbp`-based after the `mov`, and
   unchanged for the whole body. It would want `.cfi_startproc` / `.cfi_endproc` in `emit_fn` and three
   directives in the prologue. Worth doing; it is what makes a backtrace survive an unwind from inside the
   runtime's C.
3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's
   bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all
   (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded
   body will show raw addresses.
4. **`emit_main` and `flan..init-globals` have no rows.** They are inside the compile unit's `low_pc` /
   `high_pc` range, which is honest — a debugger finds no line for an address in them and says so — but a
   backtrace through the globals' initialiser names nothing. `emit_globals_init` does lower expressions that
   carry locations, so this is a small piece of work: give it a `dwsub` the way `emit_fn` has one.
5. **Nothing tests this in `dune test`.** The corpus's source-level debugging case (`test/programs/debug.flan`
   and `debug-permuted.flan`) runs under lldb against the LLVM backend. An `--x86` arm of it would be the
   right regression test and does not exist; the verification above is a transcript in a handoff, which rots.

## Open questions for the author

- **`break tick` naming.** Is the `name` / `linkageName` split worth keeping, given it costs `break <fn>` in
  gdb on *both* backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may
  be deliberate. It was left alone rather than diverged from.
- **Should `--x86 --debug` imply `--dev`?** It does not today, and they are independent axes, which seems
  right — but a debug build you cannot redefine into is half of what the dev loop wants.