flan/HANDOFF-x86-debug.md

Handoff — dropping the registry notes, and debug information for --x86

Branch dev-loop, worktree agent-a123a91f32f4c9779, from 957ba07. Items 2 and 6 of HANDOFF-x86-rt.md §6. Both landed.

The finding that changes the brief, read this first

.loc does not work in this backend's output, and no amount of care makes it work. The brief called line tables "by far the least work, since as will build .debug_line for you from .loc directives". That is true of a backend that emits mnemonics. This one emits .byte blobs — the header says so and says why — and GAS builds its line table from dwarf2_emit_insn, which runs only out of md_assemble. No instruction is ever assembled, so nothing flushes the pending .loc.

What GAS does instead is flush the pending .loc when it sees the next .loc, at whatever the location counter has reached by then. Measured on as 2.44:

	.file	1 "foo.flan"
main:	.loc 1 10 1
	.byte	0x55
	.byte	0x48,0x89,0xe5
	.loc	1 11 1
	.byte	0xb8,0x07,0x00,0x00,0x00
	.loc	1 12 1
	.byte	0xc9
	.byte	0xc3

foo.flan     10      0x4     x      <- should be 0x0
foo.flan     11      0x9     x      <- should be 0x4
foo.flan      -      0xb            <- line 12 has no row at all

Every row is one statement late and the last statement of every function gets none. Interposing labels between the .loc and the .byte does not help; it was tested. Do not try to correct this by shifting the .loc directives back by one — the last-statement hole is a correctness gap, not just fragility, and the behaviour being exploited is undocumented.

So .debug_line is written out here by hand, as .byte / .uleb128 / .quad in a .debug_line section, which is the same thing this backend already does for instructions.

.cfi is the opposite finding and it is good news. GAS's CFI machinery computes its advances from frag positions and not from md_assemble, so .cfi_def_cfa_offset and friends interleaved with .byte come out exactly right — measured, readelf --debug-dump=frames on a .byte-only function gives the correct advances. Nothing here emits them yet (see "What was not reached"), but the next lane should know it can.

What landed

Item 2 — a release build stops calling a registry that is switched off

lib/x86.ml, prim's Tast.Rt dispatch, one arm above the general one, mirroring lib/emit.ml:1918 test for test including the strict > 17: bare flan_dev_reg_note is the runtime's own C entry point and is never a Tast.Rt; what check.ml builds is the _vec, _map and _pool wrappers. The node's type is Unit, so the body is () and dst is untouched.

emit.ml is careful to drop the note before the arguments are walked, because emitting the address of the container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are not touched until call_rt, so the guard is already early enough.

Measured by disassembly, objdump -d | grep -c 'call.*flan_dev_reg_note':

program	--x86 release	--x86 --dev	LLVM release
vec.flan	46 → 1	46	1
maps.flan	55 → 1	55	—
registry.flan	36 → 1	36	—

The one that remains in every release build is not emitted code: it is inside the runtime's own flan_dev_reg_note_vec in flan_rt.c. LLVM's release build has exactly the same one, so the two backends agree.

Item 6 — --x86 --debug works, at parity with LLVM bar locals

lib/build.ml's refusal list no longer names --debug; only --sanitize and wasm are left there. X86.program takes ~debug the way it already took ~dev.

In lib/x86.ml:

piece	what
the Debug information section, above fnctx	dwrow / dwsub / dwarf, and dwfile. Carries the header comment explaining the .loc finding above
fnctx.dw : dwarf option	None in every build but a --debug one, which is what makes the release path structurally unchanged
dwline, beside new_label	the one hook. A label and a row, deduplicated on (file, line, column) and on the byte counter — see below
lower	calls dwline f e.Tast.loc and nothing else
emit_fn ?dw	opens the subprogram, seeds it with a row at the function symbol on the defn's line, closes it with a label one past the last byte
emit_dwarf	.debug_abbrev (two abbreviations), .debug_info (one CU, one DW_TAG_subprogram per function), .debug_line (header, directory table, file table, program)
program ~debug	a numbered .file at the top, .Ldwtext / .Ldwtext_end around the text, and the sections at the end

Two decisions worth knowing about because they are not obvious:

The line program is written in the dumb form. Every row is a full DW_LNE_set_address on a label, then set_file / set_column / advance_line as needed, then DW_LNS_copy — eleven bytes and up per row, where a special opcode would be one. The alternative is DW_LNS_advance_pc with a .uleb128 of a difference of two labels, which asks the assembler to resolve a value whose size changes the offsets after it. That does work, and its failure would look exactly like a DWARF bug. A debug build can afford the bytes.

The .file 1 "..." directive at the top is load-bearing and is not about our own table. Without it, clang's integrated assembler generates a compile unit of its own over the .s, whose rows land at the call mnemonics — the only real instructions this backend emits — and those addresses are inside functions our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an empty line table and nothing else. -gdwarf-4 is added alongside it in build.ml so that empty stub is a version 4 header; at the default version it is a DWARF 5 header whose file table readelf calls corrupt, and a warning on a cross-check is a warning you learn to ignore.

Rows are deduplicated on the byte counter as well as the position. lower recurses, so an outer form and the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one address takes the last. buf.n already existed and was written but never read; this is its first reader.

Verification — a real debugger, on the corpus's own debug fixture

test/programs/debug.flan built --x86 --debug, under gdb 16.x, breaking by file and line:

$ flan build test/programs/debug.flan --x86 --debug -o dbg
$ gdb -batch -x gdb5.cmd ./dbg
Breakpoint 1 at 0x400689: file debug.flan, line 19.

Breakpoint 1, flan.tick () at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick () at debug.flan:19
#1  0x00000000004007b5 in flan.main () at debug.flan:25
#2  0x0000000000400bea in main ()
Line 19 of "debug.flan" starts at address 0x400689 <flan.tick+97> and ends at 0x4006af <flan.tick+135>.
Current source file is debug.flan
Compilation directory is .../test/programs
Located in .../test/programs/debug.flan
Contains 30 lines.
Source language is c.
Producer is flan (x86-64).
Compiled with DWARF 4 debugging format.

The same script against an LLVM --debug build of the same program, which is the parity this is measured against:

Breakpoint 1 at 0x400644: file debug.flan, line 19.

Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
19	    (set (.heat c) (+ (.heat c) 1.5))
#0  flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
#1  0x00000000004006cf in flan.main () at debug.flan:25
#2  0x0000000000400806 in main ()
Producer is flan.
Compiled with DWARF 5 debugging format.

Identical bar c=0x7fffffffd8f0, n=41 — which is the locals work, and is the honest summary of what is missing.

break tick fails on both backends, with Function "tick" not defined. That is not a gap this lane opened: emit.ml writes name: "tick", linkageName: "flan.tick", gdb takes the linkage name as the search name because flan.tick is not a mangled C++ name, and an LLVM --debug build behaves the same way. The project's own source-level debugging case runs under lldb, which does not have this problem. break debug.flan:19 and break flan.tick both work everywhere. Matching emit.ml was chosen over diverging from it; if this is worth fixing it should be fixed in both places at once.

Cross-checked on a spread of the corpus — debug, generics, vec, maps, strings, conditions, bounds-condition, algorithms, handles, registry — all build with --x86 --debug, run, and produce .debug_line that readelf --debug-dump=decodedline reads with zero warnings. generics is the multi-file case: its directory table has one entry and its file table two, generics.flan and <prelude>. <prelude> has no path on disk and a debugger simply finds no source for it, which is the truth.

Baseline, measured on this tree

	before	after
spike/x86/survey.sh	99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2	(see below)
spike/x86/cells.sh	4/4 ok	(see below)
dune test --root .	exit 0	(see below)

Run dune test --root . without a pipe — HANDOFF-x86-redef.md is emphatic and right: piping gives you tail's exit status, and the dev-robust fixture puts ld and clang failure text in the output either way while the run still exits 0.

What was not reached, and what the next lane should do with it

1. Locals and types. Not attempted, and the reason is structural rather than a lack of time. emit.ml writes a !DILocalVariable per slot because every slot there is an alloca that llvm.dbg.declare points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: alloc knows its rbp-relative offset, but scoped reclaims temporaries and a later expression reuses the bytes, so the lifetime is not modelled. A DW_TAG_variable with a DW_OP_fbreg would be right at some addresses and confidently wrong at others. Note that the named slots — fn.Tast.snames — are the slots and not the temporaries, and those do live for the whole function, so a narrower version of this is reachable: DW_TAG_formal_parameter and DW_TAG_variable for snames-named slots only, with DW_AT_frame_base as DW_OP_call_frame_cfa or DW_OP_reg6. That needs a type-DIE emitter, which is the part emit.ml spends most of its debug lines on.
2. .cfi, and it is now known to be cheap. Nothing here emits it, so gdb unwinds by its prologue analyser — which works, because push rbp; mov rbp,rsp; sub rsp,N is the canonical pattern and this backend's header guarantees rsp is written exactly twice. The backtraces above are that analyser working. But the experiment at the top of this file says .cfi against .byte output is exact, and the frame model makes the content textbook: CFA is rsp+16 after the push, rbp-based after the mov, and unchanged for the whole body. It would want .cfi_startproc / .cfi_endproc in emit_fn and three directives in the prologue. Worth doing; it is what makes a backtrace survive an unwind from inside the runtime's C.
3. X86.redefinition emits no debug information. It is passed no dwarf, so a redefinition module's bodies have no lines. Nothing reaches that yet — flan dev does not build --x86 at all (HANDOFF-x86-redef.md §"What remains" item 3) — but when it does, a break loop stopping in a reloaded body will show raw addresses.
4. emit_main and flan..init-globals have no rows. They are inside the compile unit's low_pc / high_pc range, which is honest — a debugger finds no line for an address in them and says so — but a backtrace through the globals' initialiser names nothing. emit_globals_init does lower expressions that carry locations, so this is a small piece of work: give it a dwsub the way emit_fn has one.
5. Nothing tests this in dune test. The corpus's source-level debugging case (test/programs/debug.flan and debug-permuted.flan) runs under lldb against the LLVM backend. An --x86 arm of it would be the right regression test and does not exist; the verification above is a transcript in a handoff, which rots.

Open questions for the author

• break tick naming. Is the name / linkageName split worth keeping, given it costs break <fn> in gdb on both backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may be deliberate. It was left alone rather than diverged from.
• Should --x86 --debug imply --dev? It does not today, and they are independent axes, which seems right — but a debug build you cannot redefine into is half of what the dev loop wants.