From 853bc35be73ae4c82a948fcc34fcf85f27b61680 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:14:32 +0700 Subject: [PATCH 1/7] A release build stops calling into a registry that is switched off emit.ml:1918 drops the allocation registry's notes in a release build -- the checker builds a Tast.Rt it cannot know is unwanted, because it does not know whether this is a dev build. The x86 backend had no counterpart and emitted the calls for real: correct, since the registry answers nothing when it is disabled, but one call per container operation into a function that returns immediately. The guard is emit.ml's byte for byte, strict > 17 included: bare flan_dev_reg_note is the runtime's own C entry point and is never a Tast.Rt, and what check.ml builds is the _vec, _map and _pool wrappers. emit.ml drops the note before the arguments are walked so that taking the address of the container does not leave an escaped alloca behind; here the arguments are not touched until call_rt, so answering () is already early enough. Measured, call sites of flan_dev_reg_note in the disassembly: vec.flan 46 -> 1 (--dev: 46) maps.flan 55 -> 1 (--dev: 55) registry.flan 36 -> 1 (--dev: 36) The one left in each is not emitted code -- it is inside the runtime's own flan_dev_reg_note_vec. An LLVM release build of vec.flan has the same one, so the two backends now agree. HANDOFF-x86-debug.md is the stub for item 6, which is next, and it leads with the finding that changes that item's plan: .loc does not work against a backend that emits .byte blobs, so the line table has to be written out by hand. --- HANDOFF-x86-debug.md | 84 ++++++++++++++++++++++++++++++++++++++++++++ lib/x86.ml | 17 +++++++++ 2 files changed, 101 insertions(+) create mode 100644 HANDOFF-x86-debug.md diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md new file mode 100644 index 0000000..c88e42e --- /dev/null +++ b/HANDOFF-x86-debug.md @@ -0,0 +1,84 @@ +# Handoff — dropping the registry notes, and debug information for `--x86` + +Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`, from `957ba07`. Items **2** and **6** of +`HANDOFF-x86-rt.md` §6. + +## The finding that changes the brief, read this first + +**`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line +tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is +true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and +GAS builds its line table from `dwarf2_emit_insn`, which only runs out of `md_assemble`. No instruction is +ever assembled, so nothing flushes the pending `.loc`. + +What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location +counter has reached by then. Measured on `as` 2.44: + +``` + .file 1 "foo.flan" +main: .loc 1 10 1 + .byte 0x55 + .byte 0x48,0x89,0xe5 + .loc 1 11 1 + .byte 0xb8,0x07,0x00,0x00,0x00 + .loc 1 12 1 + .byte 0xc9 + .byte 0xc3 +``` + +``` +foo.flan 10 0x4 x <- should be 0x0 +foo.flan 11 0x9 x <- should be 0x4 +foo.flan - 0xb <- line 12 has no row at all +``` + +Every row is one statement late and the last statement in the function gets none. Interposing labels between +the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting the `.loc` +directives back by one statement — the last-statement hole is a correctness gap, not just fragility, and the +behaviour being exploited is undocumented. + +So the line table is written out by hand here, as raw `.byte` / `.uleb128` / `.quad` in `.debug_line`, which is +the same thing this backend already does for instructions. Scope grew accordingly; see the plan below. + +## Plan, in the order it is being done + +1. **Item 2 — `flan_dev_reg_note` dropped in a release build.** Done; see below. +2. A hand-written `.debug_line`, plus the minimal `.debug_info` / `.debug_abbrev` CU that makes a debugger + *find* it. These are one step and not two: `readelf` will read a bare `.debug_line`, but gdb reaches a line + table only by walking `.debug_info` and following `DW_AT_stmt_list`. +3. `DW_TAG_subprogram` DIEs, so `break` and `list` scope by function. Note that gdb already names frames from + the ELF symbol table, which this backend emits via `.globl`, so this buys less than it looks like. +4. `.cfi` — only if it can be placed correctly against `.byte` output. If it cannot, that is said plainly and + gdb's prologue analyser is left to recognise `push rbp; mov rbp,rsp; sub rsp,N`, which is canonical. +5. Locals and types, only if everything above is solid and committed. + +## Item 2 — done + +`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test for +test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never a +`Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so the +body is `()` and `dst` is untouched. + +`emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the +container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are +not touched until `call_rt`, so the guard is already early enough. + +Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`: + +| program | `--x86` release | `--x86 --dev` | LLVM release | +|---|---|---|---| +| `vec.flan` | 46 → **1** | 46 | 1 | +| `maps.flan` | 55 → **1** | 55 | — | +| `registry.flan` | 36 → **1** | 36 | — | + +The one that remains in every release build is not emitted code: it is inside the runtime's own +`flan_dev_reg_note_vec` in `flan_rt.c`, which calls `flan_dev_reg_note`. LLVM's release build has exactly the +same one, so the two backends now agree. + +## Baseline + +Measured on this tree, not recalled. (Filled in below as it is measured.) + +## Open questions + +(none yet) diff --git a/lib/x86.ml b/lib/x86.ml index f525e6a..43a8eb2 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -2245,6 +2245,23 @@ and prim f (e : Tast.expr) (p : Tast.prim) (args : Tast.expr list) dst = let l = lvalue f a in addr_into f ~reg:rax l; store_int f.b ~src:rax ~mm:(lmem f dst ~scratch:r11) ~size:8 + (* The allocation registry's notes are the one runtime family a release + build drops on the floor, and [emit.ml:1918] drops it with this same + guard — the checker builds a [Tast.Rt] it does not know is unwanted, + because it does not know whether this is a dev build and does not have + to. [emit.ml] is careful to drop it before the arguments are walked, so + that taking the address of the container being described does not leave + an escaped alloca for mem2reg to refuse; here the arguments are not + touched until [call_rt], so answering [()] is already early enough. The + test is [emit.ml]'s byte for byte, strict [>] included: bare + [flan_dev_reg_note] is the runtime's own entry point and is never a + [Tast.Rt]; what [check.ml] builds is the [_vec], [_map] and [_pool] + wrappers, each of which is longer than the prefix. The node's type is + [Unit], so there is nothing to store and [dst] is untouched. *) + | Tast.Rt sym, _ + when (not f.md.Emit.dev) + && String.length sym > 17 + && String.equal (String.sub sym 0 17) "flan_dev_reg_note" -> () | Tast.Rt sym, _ -> call_rt f ~sym ~args ~rty:t dst | Tast.Cast target, [ a ] -> cast f a target dst | _ -> unsupported "primitive with %d arguments" (List.length args) From 201cd87bc90627fcb9b6786b10b7b4e2993f1ed8 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:23:52 +0700 Subject: [PATCH 2/7] Debug information for --x86, written out as bytes --x86 --debug was refused in build.ml for want of DWARF. It emits it now: a compile unit, a subprogram per function, and a line table, so gdb breaks by Flan file and line and a backtrace names Flan source. The reason it is written as data rather than as .loc directives is the finding worth carrying forward, and HANDOFF-x86-debug.md leads with it. GAS builds its line table out of dwarf2_emit_insn, which runs only when an instruction is assembled, and this backend assembles none -- everything is a .byte blob. A pending .loc therefore sits until the next .loc and is flushed at whatever the location counter has reached by then: every row comes out one statement late and the last statement of every function gets no row at all. Measured on GAS 2.44, and interposing labels does not help. So .debug_line is emitted here the way the instructions are. Every row is a full DW_LNE_set_address on a label rather than an advance_pc with a computed delta, because a delta would be a difference of two labels inside a .uleb128 -- a value whose size changes the offsets after it, and whose failure would look exactly like a DWARF bug. The .file 1 directive at the top of the assembly is not about our table at all: without it clang's integrated assembler generates a compile unit of its own over the .s, whose rows land at the call mnemonics, and those addresses are inside the functions we already describe. -gdwarf-4 goes with it so the empty stub it leaves behind parses. Rows are deduplicated on the byte counter as well as on the position. lower recurses, so an outer form and the inner one that emits its first byte both ask for a row at the same address; keeping the first is smaller and is the better answer, since a debugger takes the last of a run. buf.n already existed and was written but never read. This is its first reader. No locals and no types, deliberately. A slot here is a bump-allocated frame temporary whose offset is known but whose lifetime is not modelled -- scoped reclaims temporaries and a later expression reuses the bytes -- so a DW_TAG_variable would be right at some addresses and confidently wrong at others. That is the call build.ml already makes about wasm32's member offsets. Verified under gdb against test/programs/debug.flan, and against an LLVM --debug build of the same program. Identical bar the parameter values, which is the locals work. Transcript in the handoff. --- HANDOFF-x86-debug.md | 188 +++++++++++++++++--- lib/build.ml | 28 ++- lib/x86.ml | 414 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 591 insertions(+), 39 deletions(-) diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md index c88e42e..f511d04 100644 --- a/HANDOFF-x86-debug.md +++ b/HANDOFF-x86-debug.md @@ -1,14 +1,14 @@ # Handoff — dropping the registry notes, and debug information for `--x86` Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`, from `957ba07`. Items **2** and **6** of -`HANDOFF-x86-rt.md` §6. +`HANDOFF-x86-rt.md` §6. Both landed. ## The finding that changes the brief, read this first **`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and -GAS builds its line table from `dwarf2_emit_insn`, which only runs out of `md_assemble`. No instruction is +GAS builds its line table from `dwarf2_emit_insn`, which runs only out of `md_assemble`. No instruction is ever assembled, so nothing flushes the pending `.loc`. What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location @@ -32,32 +32,27 @@ foo.flan 11 0x9 x <- should be 0x4 foo.flan - 0xb <- line 12 has no row at all ``` -Every row is one statement late and the last statement in the function gets none. Interposing labels between -the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting the `.loc` -directives back by one statement — the last-statement hole is a correctness gap, not just fragility, and the +Every row is one statement late and the last statement of every function gets none. Interposing labels +between the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting +the `.loc` directives back by one — the last-statement hole is a correctness gap, not just fragility, and the behaviour being exploited is undocumented. -So the line table is written out by hand here, as raw `.byte` / `.uleb128` / `.quad` in `.debug_line`, which is -the same thing this backend already does for instructions. Scope grew accordingly; see the plan below. +So `.debug_line` is written out here by hand, as `.byte` / `.uleb128` / `.quad` in a `.debug_line` section, +which is the same thing this backend already does for instructions. -## Plan, in the order it is being done +**`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag +positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out +*exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct +advances. Nothing here emits them yet (see "What was not reached"), but the next lane should know it can. -1. **Item 2 — `flan_dev_reg_note` dropped in a release build.** Done; see below. -2. A hand-written `.debug_line`, plus the minimal `.debug_info` / `.debug_abbrev` CU that makes a debugger - *find* it. These are one step and not two: `readelf` will read a bare `.debug_line`, but gdb reaches a line - table only by walking `.debug_info` and following `DW_AT_stmt_list`. -3. `DW_TAG_subprogram` DIEs, so `break` and `list` scope by function. Note that gdb already names frames from - the ELF symbol table, which this backend emits via `.globl`, so this buys less than it looks like. -4. `.cfi` — only if it can be placed correctly against `.byte` output. If it cannot, that is said plainly and - gdb's prologue analyser is left to recognise `push rbp; mov rbp,rsp; sub rsp,N`, which is canonical. -5. Locals and types, only if everything above is solid and committed. +## What landed -## Item 2 — done +### Item 2 — a release build stops calling a registry that is switched off -`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test for -test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never a -`Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so the -body is `()` and `dst` is untouched. +`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test +for test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never +a `Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so +the body is `()` and `dst` is untouched. `emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are @@ -72,13 +67,150 @@ Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`: | `registry.flan` | 36 → **1** | 36 | — | The one that remains in every release build is not emitted code: it is inside the runtime's own -`flan_dev_reg_note_vec` in `flan_rt.c`, which calls `flan_dev_reg_note`. LLVM's release build has exactly the -same one, so the two backends now agree. +`flan_dev_reg_note_vec` in `flan_rt.c`. LLVM's release build has exactly the same one, so the two backends +agree. -## Baseline +### Item 6 — `--x86 --debug` works, at parity with LLVM bar locals -Measured on this tree, not recalled. (Filled in below as it is measured.) +`lib/build.ml`'s refusal list no longer names `--debug`; only `--sanitize` and wasm are left there. +`X86.program` takes `~debug` the way it already took `~dev`. -## Open questions +In `lib/x86.ml`: -(none yet) +| piece | what | +|---|---| +| the `Debug information` section, above `fnctx` | `dwrow` / `dwsub` / `dwarf`, and `dwfile`. Carries the header comment explaining the `.loc` finding above | +| `fnctx.dw : dwarf option` | `None` in every build but a `--debug` one, which is what makes the release path structurally unchanged | +| `dwline`, beside `new_label` | the one hook. A label and a row, deduplicated on (file, line, column) *and* on the byte counter — see below | +| `lower` | calls `dwline f e.Tast.loc` and nothing else | +| `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte | +| `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) | +| `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end | + +Two decisions worth knowing about because they are not obvious: + +**The line program is written in the dumb form.** Every row is a full `DW_LNE_set_address` on a label, then +`set_file` / `set_column` / `advance_line` as needed, then `DW_LNS_copy` — eleven bytes and up per row, where +a special opcode would be one. The alternative is `DW_LNS_advance_pc` with a `.uleb128` of a difference of two +labels, which asks the assembler to resolve a value whose *size* changes the offsets after it. That does work, +and its failure would look exactly like a DWARF bug. A debug build can afford the bytes. + +**The `.file 1 "..."` directive at the top is load-bearing and is not about our own table.** Without it, +clang's integrated assembler generates a compile unit of *its own* over the `.s`, whose rows land at the +`call` mnemonics — the only real instructions this backend emits — and those addresses are inside functions +our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an +empty line table and nothing else. `-gdwarf-4` is added alongside it in `build.ml` so that empty stub is a +version 4 header; at the default version it is a DWARF 5 header whose file table `readelf` calls corrupt, and +a warning on a cross-check is a warning you learn to ignore. + +**Rows are deduplicated on the byte counter as well as the position.** `lower` recurses, so an outer form and +the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the +outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one +address takes the last. `buf.n` already existed and was written but never read; this is its first reader. + +## Verification — a real debugger, on the corpus's own debug fixture + +`test/programs/debug.flan` built `--x86 --debug`, under `gdb` 16.x, breaking by file and line: + +``` +$ flan build test/programs/debug.flan --x86 --debug -o dbg +$ gdb -batch -x gdb5.cmd ./dbg +Breakpoint 1 at 0x400689: file debug.flan, line 19. + +Breakpoint 1, flan.tick () at debug.flan:19 +19 (set (.heat c) (+ (.heat c) 1.5)) +#0 flan.tick () at debug.flan:19 +#1 0x00000000004007b5 in flan.main () at debug.flan:25 +#2 0x0000000000400bea in main () +Line 19 of "debug.flan" starts at address 0x400689 and ends at 0x4006af . +Current source file is debug.flan +Compilation directory is .../test/programs +Located in .../test/programs/debug.flan +Contains 30 lines. +Source language is c. +Producer is flan (x86-64). +Compiled with DWARF 4 debugging format. +``` + +The same script against an **LLVM** `--debug` build of the same program, which is the parity this is measured +against: + +``` +Breakpoint 1 at 0x400644: file debug.flan, line 19. + +Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19 +19 (set (.heat c) (+ (.heat c) 1.5)) +#0 flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19 +#1 0x00000000004006cf in flan.main () at debug.flan:25 +#2 0x0000000000400806 in main () +Producer is flan. +Compiled with DWARF 5 debugging format. +``` + +Identical bar `c=0x7fffffffd8f0, n=41` — which is the locals work, and is the honest summary of what is +missing. + +**`break tick` fails on both backends**, with `Function "tick" not defined`. That is not a gap this lane +opened: `emit.ml` writes `name: "tick", linkageName: "flan.tick"`, gdb takes the linkage name as the search +name because `flan.tick` is not a mangled C++ name, and an LLVM `--debug` build behaves the same way. The +project's own source-level debugging case runs under **lldb**, which does not have this problem. `break +debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from +it; if this is worth fixing it should be fixed in both places at once. + +Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`, +`bounds-condition`, `algorithms`, `handles`, `registry` — all build with `--x86 --debug`, run, and produce +`.debug_line` that `readelf --debug-dump=decodedline` reads with **zero warnings**. `generics` is the +multi-file case: its directory table has one entry and its file table two, `generics.flan` and ``. +`` has no path on disk and a debugger simply finds no source for it, which is the truth. + +## Baseline, measured on this tree + +| | before | after | +|---|---|---| +| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | (see below) | +| `spike/x86/cells.sh` | 4/4 ok | (see below) | +| `dune test --root .` | exit 0 | (see below) | + +Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you +`tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way +while the run still exits 0. + +## What was not reached, and what the next lane should do with it + +1. **Locals and types.** Not attempted, and the reason is structural rather than a lack of time. `emit.ml` + writes a `!DILocalVariable` per slot because every slot there is an `alloca` that `llvm.dbg.declare` + points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: `alloc` + knows its `rbp`-relative offset, but `scoped` reclaims temporaries and a later expression reuses the bytes, + so the *lifetime* is not modelled. A `DW_TAG_variable` with a `DW_OP_fbreg` would be right at some + addresses and confidently wrong at others. Note that the named slots — `fn.Tast.snames` — are the *slots* + and not the temporaries, and those do live for the whole function, so a narrower version of this is + reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with + `DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the + part `emit.ml` spends most of its debug lines on. +2. **`.cfi`, and it is now known to be cheap.** Nothing here emits it, so gdb unwinds by its prologue + analyser — which works, because `push rbp; mov rbp,rsp; sub rsp,N` is the canonical pattern and this + backend's header guarantees `rsp` is written exactly twice. The backtraces above are that analyser + working. But the experiment at the top of this file says `.cfi` against `.byte` output is exact, and the + frame model makes the content textbook: CFA is `rsp+16` after the `push`, `rbp`-based after the `mov`, and + unchanged for the whole body. It would want `.cfi_startproc` / `.cfi_endproc` in `emit_fn` and three + directives in the prologue. Worth doing; it is what makes a backtrace survive an unwind from inside the + runtime's C. +3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's + bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all + (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded + body will show raw addresses. +4. **`emit_main` and `flan..init-globals` have no rows.** They are inside the compile unit's `low_pc` / + `high_pc` range, which is honest — a debugger finds no line for an address in them and says so — but a + backtrace through the globals' initialiser names nothing. `emit_globals_init` does lower expressions that + carry locations, so this is a small piece of work: give it a `dwsub` the way `emit_fn` has one. +5. **Nothing tests this in `dune test`.** The corpus's source-level debugging case (`test/programs/debug.flan` + and `debug-permuted.flan`) runs under lldb against the LLVM backend. An `--x86` arm of it would be the + right regression test and does not exist; the verification above is a transcript in a handoff, which rots. + +## Open questions for the author + +- **`break tick` naming.** Is the `name` / `linkageName` split worth keeping, given it costs `break ` in + gdb on *both* backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may + be deliberate. It was left alone rather than diverged from. +- **Should `--x86 --debug` imply `--dev`?** It does not today, and they are independent axes, which seems + right — but a debug build you cannot redefine into is half of what the dev loop wants. diff --git a/lib/build.ml b/lib/build.ml index e4f62ac..e863805 100644 --- a/lib/build.ml +++ b/lib/build.ml @@ -756,11 +756,20 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) [flan reload] and [flan dev] both build host and module through LLVM — and the fix when something does is to emit the module through this backend too, not to grow a classifier. *) - if opts.x86 && (wasm_target opts || opts.debug || opts.sanitize) + (* [--debug] used to be in this list too. It is not any more: [x86.ml] emits + a compile unit, a subprogram per function and a line table, all written + out as bytes because [.loc] cannot work against a file whose instructions + are [.byte] blobs -- see that file's own debug-information section. What + it does *not* emit is locals and types, for the reason given there: a + slot is a bump-allocated frame temporary whose lifetime this backend does + not model, so there is nothing honest for a [DW_TAG_variable] to point + at. So `--x86 --debug` gives a backtrace that names Flan files, functions + and lines, and `print x` says the name is not in the current context. *) + if opts.x86 && (wasm_target opts || opts.sanitize) then failwith - "--x86 is the native dev backend on its own: it emits no DWARF and \ - there is no sanitizer pass over hand-written assembly"; + "--x86 is the native dev backend on its own: there is no sanitizer pass \ + over hand-written assembly"; let dir = workdir () in (* The one fork in this function. The x86 backend hands clang an assembly file where LLVM hands it IR text; clang takes either on its command line, @@ -771,7 +780,8 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) (Filename.basename out ^ if opts.x86 then ".s" else ".ll") in write ll - (if opts.x86 then X86.program ~checks:opts.checks ~dev:opts.dev p + (if opts.x86 then + X86.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug p else Emit.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug ~pnames ~sanitize:opts.sanitize p); @@ -820,6 +830,16 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) the .ll, which is the only place the Flan half of the program gets instrumented at all. *) @ cflags opts + (* The .s carries a hand-written DWARF 4 compile unit. The numbered + [.file] directive in it stops clang's integrated assembler from + generating one of its own, but the directive still leaves an empty + line table behind, and at the default version that stub is a DWARF 5 + header whose file table readelf reports as corrupt. Asking for 4 + makes the stub parse, so a readelf cross-check of the real table is + not read past a warning. It reaches only this command, which + assembles the .s and links; the C objects were compiled by + [compile_c] and are already done. *) + @ (if opts.x86 && opts.debug then [ "-gdwarf-4" ] else []) @ (if opts.dev then [ "-rdynamic" ] else []) @ tflags @ (if web_target opts then web_link_flags ~out else []) diff --git a/lib/x86.ml b/lib/x86.ml index 43a8eb2..61fc65c 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -408,6 +408,90 @@ let gsym n = asm_sym ("flan." ^ n) Byte-for-byte or the link fails and the piece served nothing. *) let csym n = asm_sym ("flan.cell." ^ n) +(* ── Debug information ───────────────────────────────────────────────── *) + +(* DWARF, written out as bytes, for the same reason the instructions are — and + the reason is worth stating first because it is the one thing about this + backend that makes debug information cost more here than it does anywhere + else. + + {b [.loc] does not work against an assembly file that has no instructions + in it.} GAS builds its line table from [dwarf2_emit_insn], which runs only + when an instruction is assembled, and this file assembles none: everything + is a [.byte] blob. A pending [.loc] therefore sits until the *next* [.loc] + and is flushed at whatever the location counter has reached by then, so + every row comes out one statement late and the last statement of every + function gets no row at all. Measured on GAS 2.44 and reproduced with + labels interposed, which do not help. So [.debug_line] is emitted here as + data, which is in any case the only spelling consistent with the rest of + the file. + + DWARF 4 rather than 5. Version 4's file and directory tables are + NUL-terminated strings and a terminator byte where 5 form-codes them, and + nothing above needs a version 5 feature. A compile unit declares its own + version, so a v4 unit sitting beside the v5 ones clang gives the runtime's + C is not a conflict — each is read on its own terms. + + What is described is deliberately shallow: the compile unit, one subprogram + per function, and the line table. {b No locals and no types.} [emit.ml] + writes a [!DILocalVariable] per slot because every slot there is an + [alloca] that [llvm.dbg.declare] can point at and LLVM computes the frame + offset; here a slot is a bump-allocated frame temporary whose offset this + file knows but whose *lifetime* it does not model — [scoped] reclaims + temporaries and a later expression reuses the bytes. Naming an offset that + holds something else half the time is worse than naming nothing, so this + emits nothing rather than a confident wrong answer. That is the same call + [build.ml] makes about wasm32's member offsets. *) + +(* One row of the line table: the label whose address it is, and the position + it names. Addresses are labels rather than numbers because the assembler + places the function and this file does not. *) +type dwrow = { rlbl : string; rfile : int; rline : int; rcol : int } + +type dwsub = { + sname : string; (* what a debugger calls the frame *) + ssym : string; (* the symbol the linker sees *) + sfile : int; + sline : int; + send : string; (* a label one past the function's last byte *) + mutable srows : dwrow list; (* newest first *) +} + +type dwarf = { + dfiles : (string, int) Hashtbl.t; + mutable dpaths : string list; (* newest first *) + mutable dsubs : dwsub list; (* newest first *) + mutable dcur : dwsub option; (* the function being lowered *) + mutable dlast : (int * int * int) option; (* the last row's position *) + (* [buf.n] as it stood when the last row was made. [lower] recurses, so an + outer form and the inner one that emits the first byte of it both ask for + a row at the same address; without this the table carries a run of rows + that a debugger resolves by taking the last, which is the innermost form + rather than the statement. Keeping the first is both smaller and the + better answer. Reset to -1 per function, because the body's byte counter + starts again at 0 and the entry row is at the prologue's 0. *) + mutable dlastn : int; +} + +let new_dwarf () = + { dfiles = Hashtbl.create 8; dpaths = []; dsubs = []; dcur = None; + dlast = None; dlastn = -1 } + +(* File indices are 1-based and handed out in first-seen order, which is the + order the file table is written in below. A program spans more than one + file whenever the prelude or a macro contributed a form, and the checker's + own invented nodes carry [Loc.unknown], whose file is [] — that + one never reaches here, because a row with line 0 is attributed to the + enclosing function's file instead. *) +let dwfile dw path = + match Hashtbl.find_opt dw.dfiles path with + | Some n -> n + | None -> + let n = List.length dw.dpaths + 1 in + Hashtbl.replace dw.dfiles path n; + dw.dpaths <- path :: dw.dpaths; + n + (* ── Function context ────────────────────────────────────────────────── *) type fnctx = { @@ -451,6 +535,11 @@ type fnctx = { module answers true for the host's cells, globals and bodies, and those go through the GOT -- see [modrm_got]. *) ext : string -> bool; + (* The line table under construction, in a [--debug] build. [None] is every + other build, and then [dwline] below is the only thing that looks at it + and does nothing — so a release build's output is byte-identical to what + it was before debug information existed. *) + dw : dwarf option; } (* Module-wide rather than per-function. Two functions each holding an [if] @@ -462,6 +551,36 @@ let uniq = ref 0 let new_label _f tag = incr uniq; Printf.sprintf ".L%s%d" tag !uniq +(* A line-table row at the point the output has reached, unless the last row + already named this position. [lower] calls this for every expression, so + the dedup is what keeps a statement made of a dozen nodes on one line from + costing a dozen rows; the column is part of the key, so two forms on one + line are still told apart. + + Line 0 is [Loc.unknown] — a node the checker invented rather than one + anyone wrote. It is attributed to the enclosing function's own line, for + [emit.ml]'s reason at its [at_loc]: a zero line in DWARF means "no line", + and a debugger given one steps over the whole construct. *) +let dwline f (loc : Loc.t) = + match f.dw with + | None -> () + | Some dw -> + (match dw.dcur with + | None -> () + | Some s -> + let file, line, col = + if loc.Loc.line = 0 then (s.sfile, s.sline, 1) + else (dwfile dw loc.Loc.file, loc.Loc.line, loc.Loc.col) + in + if dw.dlast <> Some (file, line, col) && f.b.n > dw.dlastn then begin + let l = new_label f "dl" in + lbl f.b l; + dw.dlast <- Some (file, line, col); + dw.dlastn <- f.b.n; + s.srows <- + { rlbl = l; rfile = file; rline = line; rcol = col } :: s.srows + end) + (* Bump-allocate a frame temporary and answer its rbp-relative offset. The offset is negative, so the running total is rounded *up* to the alignment; rbp is 16-aligned, so that is the alignment the value actually gets. *) @@ -962,6 +1081,12 @@ let emit_args f (args : arg list) = let sink = Lf 0 let rec lower f (e : Tast.expr) (dst : loc) : unit = + (* The one hook the line table needs, and it is here rather than at + statement granularity on purpose: the same recursion that lowers a nested + call lowers its arguments, and a row per expression is what makes a + backtrace through an argument name the argument rather than the call. It + is inert in every build but a [--debug] one. *) + dwline f e.Tast.loc; if dst == sink && not (is_void e.Tast.ty) then scoped f (fun () -> let o = tmp f e.Tast.ty in @@ -2365,7 +2490,7 @@ let incoming_of ~sret (params : Types.t list) = sret_at, ps, next_int () let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) - ?(hidden = false) (fn : Tast.fn) : string * string = + ?(hidden = false) ?dw (fn : Tast.fn) : string * string = let b = create () in let nslots = Array.length fn.Tast.slots in let f = @@ -2374,7 +2499,35 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) xfer_off = 0; sret_off = 0; retval = 0; frame = 0; maxframe = 0; outgoing = 0; loops = []; pads = []; xfer_lbl = ""; unwound = false; - rodata = Buffer.create 64; externs; fns; ext } + rodata = Buffer.create 64; externs; fns; ext; dw } + in + (* The subprogram this function's rows hang off. Its first row is the + function symbol itself, at the line the [defn] was written on, so the + entry has a position before the prologue has run; every row after it + comes out of [dwline] as the body is lowered. [dlast] is not primed with + it, which is deliberate — the first form of the body sits at a different + column even when it is on the same line, so it gets a row of its own, and + two rows are what let a debugger put a breakpoint after the prologue + rather than on it. *) + let sub = + match dw with + | None -> None + | Some d -> + let line = + if fn.Tast.floc.Loc.line = 0 then 1 else fn.Tast.floc.Loc.line + in + let file = dwfile d fn.Tast.floc.Loc.file in + let s = + { sname = fn.Tast.name; ssym = fsym fn.Tast.name; sfile = file; + sline = line; send = new_label f "fe"; + srows = [ { rlbl = fsym fn.Tast.name; rfile = file; rline = line; + rcol = 1 } ] } + in + d.dcur <- Some s; + d.dlast <- None; + d.dlastn <- -1; + d.dsubs <- s :: d.dsubs; + Some s in (* The header's own frame model: every slot and every temporary is bump-allocated below rbp, and the high-water mark is what the prologue @@ -2567,6 +2720,14 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) Buffer.add_string out (sym ^ ":\n"); Buffer.add_string out (Buffer.contents pb.out); Buffer.add_string out (Buffer.contents f.b.out); + (* One past the last byte, which is what [DW_AT_high_pc] and the line + table's closing [DW_LNE_end_sequence] both want. [.size]'s [. - sym] says + the same thing but is an expression rather than a symbol, and + [DW_FORM_addr] takes a symbol. *) + (match sub with + | Some s -> Buffer.add_string out (s.send ^ ":\n") + | None -> ()); + (match dw with Some d -> d.dcur <- None | None -> ()); Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" sym sym); Buffer.contents out, Buffer.contents f.rodata @@ -2645,7 +2806,8 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = fret = Types.Unit; slots = [||]; xfer_off = 0; sret_off = 0; retval = 0; frame = 0; maxframe = 0; outgoing = 0; loops = []; pads = []; xfer_lbl = ""; unwound = false; - rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false) } + rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false); + dw = None } in (* Two slots, not one: [xfer_off] holds the *pointer* every call passes on, and [cell] is what it points at. Storing a null into [xfer_off] itself — @@ -2773,8 +2935,210 @@ let emit_cells (p : Tast.program) = p.Tast.fns; Buffer.contents out +(* ── The DWARF sections ──────────────────────────────────────────────── *) + +(* A path inside an assembler string literal. Flan source paths contain + neither of these two characters in practice, but a path is user input and + a stray backslash would end the directive rather than the string. *) +let asm_str s = + let b = Buffer.create (String.length s + 8) in + String.iter + (fun c -> + if c = '"' || c = '\\' then Buffer.add_char b '\\'; + Buffer.add_char b c) + s; + Buffer.contents b + +(* Absolute, because the file table below gives every entry directory index 0 + and a debugger then reads the name as written. [emit.ml]'s [dfile] splits + the same absolute path into a [!DIFile]'s filename and directory; DWARF is + happy with either and one string is fewer moving parts. *) +let abspath p = + if Filename.is_relative p then Filename.concat (Sys.getcwd ()) p else p + +(* [DW_LNE_set_address] on a label: the escape opcode, a length of 9, the + sub-opcode, and eight bytes of address the assembler relocates. + + Every row gets one of these rather than a [DW_LNS_advance_pc] with a + computed delta, and the reason is that a delta would be a difference of two + labels inside a [.uleb128], which asks the assembler to resolve a value + whose size affects the values after it. That does work, but it is exactly + the kind of thing whose failure looks like a DWARF bug. Rows are eleven + bytes each here and that is a debug build's business. *) +let dw_set_address b lbl = + Buffer.add_string b (Printf.sprintf "\t.byte\t0, 9, 2\n\t.quad\t%s\n" lbl) + +(* The line-number program: one sequence per function, closed with an + end_sequence at the label one past its last byte. The registers are reset + at the head of every sequence, which is why [line] starts at 1 and [file] + at 1 in each. Rows within a sequence must be address-ordered, and they are + for free: [dwline] appends them in the order the bytes are emitted. *) +let dw_line_program (dw : dwarf) = + let b = Buffer.create 4096 in + List.iter + (fun s -> + let line = ref 1 and file = ref 1 and col = ref 0 in + List.iter + (fun r -> + dw_set_address b r.rlbl; + if r.rfile <> !file then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t4\n\t.uleb128 %d\n" r.rfile); + file := r.rfile + end; + if r.rcol <> !col then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t5\n\t.uleb128 %d\n" r.rcol); + col := r.rcol + end; + if r.rline <> !line then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t3\n\t.sleb128 %d\n" (r.rline - !line)); + line := r.rline + end; + Buffer.add_string b "\t.byte\t1\n") + (List.rev s.srows); + dw_set_address b s.send; + Buffer.add_string b "\t.byte\t0, 1, 1\n") + (List.rev dw.dsubs); + Buffer.contents b + +(* The three sections. [tbeg] and [tend] bracket everything this object puts + in [.text] — the compile unit claims that whole range, including the C + [main] shim and the globals' initialiser, neither of which has any rows. + That is honest: they are code this unit produced, and a debugger that finds + no line for an address inside them says so. *) +let emit_dwarf (dw : dwarf) ~cufile ~tbeg ~tend = + let out = Buffer.create 8192 in + (* The compile unit's directory, and the directory table the file entries + index into. [emit.ml]'s [dfile] splits every path into a basename and a + directory the same way, and the reason to match it is what a debugger + prints: a frame reads [debug.flan:19] rather than the eighty characters + of an absolute path. Directory index 0 is the compile unit's own + directory, so a file sitting beside the one that named the unit -- which + is most of them -- costs no entry at all. *) + let cudir = Filename.dirname (abspath cufile) in + let dirs = ref [] in + let dirix d = + if String.equal d cudir then 0 + else + match List.assoc_opt d !dirs with + | Some n -> n + | None -> + let n = List.length !dirs + 1 in + dirs := !dirs @ [ (d, n) ]; + n + in + let files = + List.map + (fun p -> + let a = abspath p in + (Filename.basename a, dirix (Filename.dirname a))) + (List.rev dw.dpaths) + in + Buffer.add_string out + "\n# Debug information. Written out as data rather than left to the\n\ + # assembler's .loc, which cannot work here: GAS builds its line table\n\ + # when it assembles an instruction, and this file assembles none.\n"; + (* .debug_abbrev. Two abbreviations, because two shapes of DIE are all that + is described. *) + Buffer.add_string out + "\t.section\t.debug_abbrev,\"\",@progbits\n\ + .Ldwabbrev:\n\ + \t.uleb128 1\n\ + \t.uleb128 0x11\t\t# DW_TAG_compile_unit\n\ + \t.byte\t1\t\t# has children\n\ + \t.uleb128 0x25\n\t.uleb128 0x08\t# DW_AT_producer DW_FORM_string\n\ + \t.uleb128 0x13\n\t.uleb128 0x05\t# DW_AT_language DW_FORM_data2\n\ + \t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\ + \t.uleb128 0x1b\n\t.uleb128 0x08\t# DW_AT_comp_dir DW_FORM_string\n\ + \t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\ + \t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\ + \t.uleb128 0x10\n\t.uleb128 0x17\t# DW_AT_stmt_list DW_FORM_sec_offset\n\ + \t.byte\t0, 0\n\ + \t.uleb128 2\n\ + \t.uleb128 0x2e\t\t# DW_TAG_subprogram\n\ + \t.byte\t0\t\t# no children\n\ + \t.uleb128 0x3f\n\t.uleb128 0x19\t# DW_AT_external DW_FORM_flag_present\n\ + \t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\ + \t.uleb128 0x6e\n\t.uleb128 0x08\t# DW_AT_linkage_name DW_FORM_string\n\ + \t.uleb128 0x3a\n\t.uleb128 0x0f\t# DW_AT_decl_file DW_FORM_udata\n\ + \t.uleb128 0x3b\n\t.uleb128 0x0f\t# DW_AT_decl_line DW_FORM_udata\n\ + \t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\ + \t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\ + \t.byte\t0, 0\n\ + \t.byte\t0\n"; + (* .debug_info. DW_LANG_C99 for [emit.ml]'s reason: it is less a claim about + the source language than the truth about the data model, and it is what + makes a debugger's own struct printing correct against this layout. *) + Buffer.add_string out + (Printf.sprintf + "\t.section\t.debug_info,\"\",@progbits\n\ + .Ldwinfo:\n\ + \t.long\t.Ldwinfo_end - .Ldwinfo_ver\n\ + .Ldwinfo_ver:\n\ + \t.short\t4\n\ + \t.long\t.Ldwabbrev\n\ + \t.byte\t8\n\ + \t.uleb128 1\n\ + \t.asciz\t\"flan (x86-64)\"\n\ + \t.short\t0x000c\t\t# DW_LANG_C99\n\ + \t.asciz\t\"%s\"\n\ + \t.asciz\t\"%s\"\n\ + \t.quad\t%s\n\ + \t.quad\t%s - %s\n\ + \t.long\t.Ldwline\n" + (asm_str (Filename.basename (abspath cufile))) + (asm_str cudir) + tbeg tend tbeg); + List.iter + (fun s -> + Buffer.add_string out + (Printf.sprintf + "\t.uleb128 2\n\t.asciz\t\"%s\"\n\t.asciz\t\"%s\"\n\ + \t.uleb128 %d\n\t.uleb128 %d\n\t.quad\t%s\n\t.quad\t%s - %s\n" + (asm_str s.sname) + (asm_str ("flan." ^ s.sname)) + s.sfile s.sline s.ssym s.send s.ssym)) + (List.rev dw.dsubs); + Buffer.add_string out "\t.byte\t0\n.Ldwinfo_end:\n"; + (* .debug_line. The standard opcode lengths are the standard ones; changing + them would change nothing, since every row below is spelled out of the + three opcodes this emitter uses and never out of a special opcode. *) + Buffer.add_string out + "\t.section\t.debug_line,\"\",@progbits\n\ + .Ldwline:\n\ + \t.long\t.Ldwline_end - .Ldwline_ver\n\ + .Ldwline_ver:\n\ + \t.short\t4\n\ + \t.long\t.Ldwline_prog - .Ldwline_hdr\n\ + .Ldwline_hdr:\n\ + \t.byte\t1\t\t# minimum_instruction_length\n\ + \t.byte\t1\t\t# maximum_operations_per_instruction\n\ + \t.byte\t1\t\t# default_is_stmt\n\ + \t.byte\t0xfb\t\t# line_base = -5\n\ + \t.byte\t14\t\t# line_range\n\ + \t.byte\t13\t\t# opcode_base\n\ + \t.byte\t0,1,1,1,1,0,0,0,1,0,0,1\n"; + List.iter + (fun (d, _) -> + Buffer.add_string out (Printf.sprintf "\t.asciz\t\"%s\"\n" (asm_str d))) + !dirs; + Buffer.add_string out "\t.byte\t0\t\t# end of the directory table\n"; + List.iter + (fun (name, dir) -> + Buffer.add_string out + (Printf.sprintf "\t.asciz\t\"%s\"\n\t.uleb128 %d\n\t.uleb128 0\n\ + \t.uleb128 0\n" + (asm_str name) dir)) + files; + Buffer.add_string out "\t.byte\t0\n.Ldwline_prog:\n"; + Buffer.add_string out (dw_line_program dw); + Buffer.add_string out ".Ldwline_end:\n"; + Buffer.contents out + (* A whole program as one assembly file. *) -let program ~checks ?(dev = false) (p : Tast.program) : string = +let program ~checks ?(dev = false) ?(debug = false) (p : Tast.program) : string = check_no_transfer p; let md = layout_ctx ~checks ~dev p in let externs = Hashtbl.create 16 in @@ -2783,15 +3147,41 @@ let program ~checks ?(dev = false) (p : Tast.program) : string = p.Tast.externs; let fns = Hashtbl.create 64 in List.iter (fun (fn : Tast.fn) -> Hashtbl.replace fns fn.Tast.name ()) p.Tast.fns; + let dw = if debug then Some (new_dwarf ()) else None in + (* The file every diagnostic in this unit is really about: the first + function anyone actually wrote. [emit.ml]'s [new_dbg] picks it the same + way and for the same reason -- the prelude contributes functions too, and + naming the prelude as the compile unit would be true and useless. *) + let cufile = + match + List.find_opt (fun (f : Tast.fn) -> f.Tast.floc.Loc.line > 0) p.Tast.fns + with + | Some f -> f.Tast.floc.Loc.file + | None -> "" + in let text = Buffer.create 65536 and rodata = Buffer.create 4096 in Buffer.add_string text "# Generated by flan's x86-64 backend (the dev one). The instructions are\n\ # .byte blobs so that every byte offset stays exactly known; the few\n\ - # fields that need a relocation are assembler expressions.\n\ - \t.text\n\n"; + # fields that need a relocation are assembler expressions.\n"; + (* A numbered [.file] is what stops clang's integrated assembler from + generating a compile unit of its *own* over this file -- one that names + the .s, and whose rows land at the [call] mnemonics, which are the only + real instructions here. Those addresses are inside the functions this + unit already describes, so two units would claim them. Measured: with the + directive, the assembler emits an empty line table and nothing else. + Harmless in a release build, where it is simply never emitted. *) + if debug then + Buffer.add_string text + (Printf.sprintf "\t.file\t1 \"%s\"\n" (asm_str (abspath cufile))); + (* The label the compile unit's range starts at, and it is emitted only in a + debug build so that a release build's assembly is byte-for-byte what it + was before any of this existed. *) + Buffer.add_string text + (if debug then "\t.text\n.Ldwtext:\n\n" else "\t.text\n\n"); List.iter (fun (fn : Tast.fn) -> - let t, r = emit_fn md ~externs ~fns fn in + let t, r = emit_fn md ~externs ~fns ?dw fn in Buffer.add_string text t; Buffer.add_string rodata r) p.Tast.fns; @@ -2805,6 +3195,11 @@ let program ~checks ?(dev = false) (p : Tast.program) : string = which is what [reload_host.c] is. Refusing here made a --x86 host for the reload tests impossible to build. *) | None -> ()); + (* The end of everything this object puts in .text, and therefore the end of + the compile unit's range. It has to be written while .text is still the + current section, which is why it is here and not beside the sections + below. *) + if debug then Buffer.add_string text ".Ldwtext_end:\n"; let out = Buffer.create 65536 in Buffer.add_buffer out text; (* The globals' initialiser runs before main, through the same constructor @@ -2825,6 +3220,11 @@ let program ~checks ?(dev = false) (p : Tast.program) : string = Buffer.add_string out (emit_globals_data md p.Tast.globals); Buffer.add_string out "\n\t.section\t.rodata\n"; Buffer.add_buffer out rodata; + (match dw with + | Some d -> + Buffer.add_string out + (emit_dwarf d ~cufile ~tbeg:".Ldwtext" ~tend:".Ldwtext_end") + | None -> ()); Buffer.add_string out "\n\t.section\t.note.GNU-stack,\"\",@progbits\n"; Buffer.contents out From c9c23c5079a929ed839a809b43fbcfc0d3d4593c Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:27:36 +0700 Subject: [PATCH 3/7] Call frame information, which is the thing the assembler gets right The debug-information section added in the last commit is about what GAS gets wrong against a file with no instructions in it: .loc is flushed when an instruction is assembled, and this backend assembles none. CFI is the opposite case and worth recording beside it. Its advances come from frag positions, so .cfi_def_cfa_offset interleaved with .byte comes out exact -- measured, readelf --debug-dump=frames on a .byte-only function gives the right advances. The content is a constant, and that is the header's claim about the frame model paying for itself. rsp is written exactly twice, so: on entry the CFA is rsp+8; push rbp makes it rsp+16 with the saved rbp at cfa-16; mov rsp,rbp moves the rule onto rbp and it stays there for the whole body; after leave rsp is rbp+8 and the CFA is rsp+8 again. Five directives, three sites -- emit_fn, emit_main and emit_globals_init, which have the same prologue. emit_main has no closing rule because it has no epilogue: it leaves through flan_exit and the ud2 after that is unreachable, so the rbp rule holds to the last byte, which is what a backtrace out of anything main called wants. gdb did not need this -- its prologue analyser already unwound out of flan_bounds_error into flan.main with a line number, because push rbp; mov rbp,rsp; sub rsp,N is the pattern it recognises. It is here because the description is now stated rather than guessed, and because a break at the very first byte of a function -- before the push -- now unwinds from a rule rather than from a heuristic. Gated on --debug so a release build's assembly stays byte-for-byte what it was. That is conservative rather than principled: the description is correct in every build and a release build is where a crash would most want it. What stops it being unconditional is only that nothing measures the .eh_frame it would add, and another lane is measuring backend cost right now. --- lib/x86.ml | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/lib/x86.ml b/lib/x86.ml index 61fc65c..814903c 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -2451,6 +2451,36 @@ and cast f (a : Tast.expr) (target : Types.t) dst = cvttf2si f.b ~f64:(f64_of src_t) ~dst:rax ~src:xmm0; store_loc f ~reg:rax dst dst_t +(* ── Call frame information ──────────────────────────────────────────── *) + +(* The whole frame model, in five directives. + + [.cfi] is the one thing the assembler gets right against a file with no + instructions in it, and it is worth saying so beside the debug-information + section above, which is about the thing it gets wrong: CFI advances are + computed from frag positions, where the line table's are computed from + having assembled an instruction. Measured — a [.byte]-only function comes + out of [readelf --debug-dump=frames] with exact advances. + + The content is a constant because of the header's own claim that [rsp] is + written exactly twice. On entry the CFA is [rsp+8]; [push rbp] makes it + [rsp+16] and puts the saved [rbp] at [cfa-16]; [mov rsp, rbp] moves the + rule onto [rbp], where it stays for the whole body, because the only other + write to [rsp] is the [leave]. After that [rsp] is [rbp+8] again and the + CFA is [rsp+8]. Register 6 is [rbp] and 7 is [rsp] in DWARF's numbering; + the return address is column 16 and the CIE already says it is at + [cfa-8]. + + Emitted only in a [--debug] build, so that a release build's assembly stays + byte-for-byte what it was. That is a conservative call rather than a + principled one: this description is correct in every build, and a release + build is where an unwind through a crash would most want it. What stops it + from being unconditional today is only that nothing measures the [.eh_frame] + it would add. *) +let cfi_after_push b = text b "\t.cfi_def_cfa_offset 16\n\t.cfi_offset 6, -16\n" +let cfi_after_mov b = text b "\t.cfi_def_cfa_register 6\n" +let cfi_after_leave b = text b "\t.cfi_def_cfa 7, 8\n" + (* ── A function ──────────────────────────────────────────────────────── *) (* The frame is rounded to 16 and reserves the outgoing-argument area in the @@ -2509,6 +2539,7 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) column even when it is on the same line, so it gets a row of its own, and two rows are what let a debugger put a breakpoint after the prologue rather than on it. *) + let cfi = match dw with None -> false | Some _ -> true in let sub = match dw with | None -> None @@ -2641,7 +2672,9 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) (* The prologue, now that the frame size is known. *) let pb = create () in push_r pb rbp; + if cfi then cfi_after_push pb; mov_rr pb ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov pb; let n = frame_bytes f in if n > 0 then sub_imm pb ~dst:rsp n; (match sret_at with @@ -2703,6 +2736,7 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) load_scalar f ~reg:(if is_float fn.Tast.ret then xmm0 else rax) ~off:f.retval fn.Tast.ret; leave f.b; + if cfi then cfi_after_leave f.b; ret f.b; flush pb; flush f.b; @@ -2718,6 +2752,7 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) if hidden then Buffer.add_string out (Printf.sprintf "\t.hidden\t%s\n" sym); Buffer.add_string out (Printf.sprintf "\t.type\t%s, @function\n" sym); Buffer.add_string out (sym ^ ":\n"); + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents pb.out); Buffer.add_string out (Buffer.contents f.b.out); (* One past the last byte, which is what [DW_AT_high_pc] and the line @@ -2728,6 +2763,7 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) | Some s -> Buffer.add_string out (s.send ^ ":\n") | None -> ()); (match dw with Some d -> d.dcur <- None | None -> ()); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" sym sym); Buffer.contents out, Buffer.contents f.rodata @@ -2738,10 +2774,12 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) the loader put them in, the program's own end of the transfer channel is a null cell on this frame, and the exit goes through [flan_exit] because stdout is a FILE* and something has to flush it. *) -let emit_main (md : Emit.m) (fn : Tast.fn) = +let emit_main ?(cfi = false) (md : Emit.m) (fn : Tast.fn) = let b = create () in push_r b rbp; + if cfi then cfi_after_push b; mov_rr b ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov b; sub_imm b ~dst:rsp 48; (* [al] is zero at every call this backend makes, variadic or not — see [call_native]. Setting it here too costs two bytes and keeps the rule @@ -2771,7 +2809,13 @@ let emit_main (md : Emit.m) (fn : Tast.fn) = ignore md; let out = Buffer.create 256 in Buffer.add_string out "\t.globl\tmain\n\t.type\tmain, @function\nmain:\n"; + (* No [.cfi_def_cfa 7, 8] to close with, because this one has no epilogue: + it leaves through [flan_exit] and the [ud2] after that is unreachable. + The rbp rule therefore holds to the last byte, which is what a backtrace + out of anything [main] called needs. *) + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents b.out); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out "\t.size\tmain, . - main\n\n"; Buffer.contents out @@ -2799,7 +2843,8 @@ let emit_globals_data (md : Emit.m) (globals : Tast.global list) = let init_sym = "\"flan..init-globals\"" -let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = +let emit_globals_init ?(cfi = false) (md : Emit.m) ~externs ~fns + (globals : Tast.global list) = let b = create () in let f = { b; md; fnname = ""; retlbl = new_label () "ginit"; @@ -2830,7 +2875,9 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = end; let pb = create () in push_r pb rbp; + if cfi then cfi_after_push pb; mov_rr pb ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov pb; let n = frame_bytes f in if n > 0 then sub_imm pb ~dst:rsp n; (* No caller hands this one a channel, so it gets a null cell of its own and @@ -2841,13 +2888,16 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = store_int pb ~src:rax ~mm:(Frame f.xfer_off) ~size:8; lbl f.b f.retlbl; leave f.b; + if cfi then cfi_after_leave f.b; ret f.b; flush pb; flush f.b; let out = Buffer.create 512 in Buffer.add_string out (Printf.sprintf "\t.type\t%s, @function\n%s:\n" init_sym init_sym); + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents pb.out); Buffer.add_string out (Buffer.contents f.b.out); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" init_sym init_sym); Buffer.contents out, Buffer.contents f.rodata @@ -3185,11 +3235,11 @@ let program ~checks ?(dev = false) ?(debug = false) (p : Tast.program) : string Buffer.add_string text t; Buffer.add_string rodata r) p.Tast.fns; - let ginit, gr = emit_globals_init md ~externs ~fns p.Tast.globals in + let ginit, gr = emit_globals_init ~cfi:debug md ~externs ~fns p.Tast.globals in Buffer.add_string text ginit; Buffer.add_string rodata gr; (match List.find_opt (fun (f : Tast.fn) -> f.Tast.name = "main") p.Tast.fns with - | Some fn -> Buffer.add_string text (emit_main md fn) + | Some fn -> Buffer.add_string text (emit_main ~cfi:debug md fn) (* No [main] is not an error, and [emit.ml] treats it the same way: a program can be linked against a C host that brings its own entry point, which is what [reload_host.c] is. Refusing here made a --x86 host for the From 78368811c084450dc66cab17fb77bdb7c8742ba7 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:30:42 +0700 Subject: [PATCH 4/7] The x86 backend's DWARF is checked by a debugger, not by a transcript test_acceptance.ml's lldb case is the only part of the suite that says a person can debug a Flan program: a breakpoint on a Flan name, a backtrace with .flan files and lines, and locals with their own types. It ran against the LLVM backend only. An --x86 arm, with a narrower claim: the breakpoint and the backtrace, and that the program still prints what it printed. No frame variable, because x86.ml emits no DW_TAG_variable -- a slot there is a bump-allocated frame temporary whose lifetime the backend does not model. That gap between the two backends is now recorded in the place it will be read. Worth pinning rather than leaving to a handoff, because everything it exercises is bytes x86.ml wrote by hand -- a line program, a compile unit, an abbreviation table -- and a wrong byte in any of them is silent. debug_compile grows an ~x86 flag beside the ~dev one it already had. --- test/test_acceptance.ml | 49 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 45 insertions(+), 4 deletions(-) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 5455c9e..bd6be1b 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -3105,11 +3105,12 @@ ERR@7 unexpected token: not the kind the caller was reading else print_endline "acceptance: the DWARF verifier cases skipped (no opt)"; (* A debug build and a release build must still be the same program. *) - let debug_compile ?(dev = false) path = + let debug_compile ?(dev = false) ?(x86 = false) path = let exe = Filename.concat scratch ("flan-dbg-" ^ Filename.remove_extension (Filename.basename path) - ^ if dev then "-dev" else "") + ^ (if dev then "-dev" else "") + ^ if x86 then "-x86" else "") in let l = Load.program ~file:path (Reader.read_file path) in let p = Check.program l.Load.decls in @@ -3125,7 +3126,7 @@ ERR@7 unexpected token: not the kind the caller was reading in let p, csrcs, lflags = Reach.link ~dev l p in ignore - (Build.executable ~opts:{ Build.default with debug = true; dev } + (Build.executable ~opts:{ Build.default with debug = true; dev; x86 } ~csrcs ~lflags ~pnames p ~out:exe); exe in @@ -3258,7 +3259,47 @@ ERR@7 unexpected token: not the kind the caller was reading n; print_endline text end) - [ "flan.tick"; "flan.main at debug.flan:" ] + [ "flan.tick"; "flan.main at debug.flan:" ]; + + (* And the same program through the hand-written x86-64 backend, which + emits its own DWARF rather than handing LLVM metadata. + + The claim is narrower than the one above and deliberately so: a + breakpoint resolved on a Flan name, and a backtrace whose frames name + a .flan file and a line. No [frame variable], because [x86.ml] emits + no [DW_TAG_variable] -- a slot there is a bump-allocated frame + temporary whose lifetime the backend does not model, and a name + attached to an offset something else reuses would be a lie. That is + the gap between the two backends and this is where it is recorded. + + Worth pinning rather than leaving to a handoff's transcript, because + everything this exercises is bytes [x86.ml] wrote by hand -- a line + program, a compile unit and an abbreviation table -- and a wrong byte + in any of them is silent. The program still printing the same four + lines is checked too, since debug information that breaks the build + it describes has helped nobody. *) + let exe = debug_compile ~x86:true "programs/debug.flan" in + let code, text = run exe None in + if text <> expected || code <> 0 then begin + incr failures; + Printf.printf + "FAIL an --x86 --debug build of debug.flan runs the same\n\ + \ got: %S (exit %d)\n wanted: %S\n" + text code expected + end; + let _, text = + lldb_run exe [ "breakpoint set --name flan.tick"; "run"; "bt" ] + in + List.iter + (fun n -> + if not (contains text n) then begin + incr failures; + Printf.printf + "FAIL lldb: --x86 --debug names Flan files and lines\n\ + \ wanted %S\n" n; + print_endline text + end) + [ "flan.tick"; "at debug.flan:"; "flan.main at debug.flan:" ] end else print_endline "acceptance: lldb cases skipped (no lldb on PATH)"; From c8464abc1805e3f219a97bc01f71b162a73ebd03 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:32:26 +0700 Subject: [PATCH 5/7] The handoff, with every number in it measured on this tree The baseline rows, the twelve-program DWARF sweep re-run against the code that was actually committed rather than a superseded version of it, the lldb transcript, the unwind out of flan_bounds_error, and the compile unit's range checked against nm rather than asserted. --- HANDOFF-x86-debug.md | 87 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 69 insertions(+), 18 deletions(-) diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md index f511d04..aa579e9 100644 --- a/HANDOFF-x86-debug.md +++ b/HANDOFF-x86-debug.md @@ -43,7 +43,7 @@ which is the same thing this backend already does for instructions. **`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out *exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct -advances. Nothing here emits them yet (see "What was not reached"), but the next lane should know it can. +advances. So this lane emits them, in a `--debug` build — see below. ## What landed @@ -86,6 +86,7 @@ In `lib/x86.ml`: | `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte | | `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) | | `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end | +| `cfi_after_push` / `cfi_after_mov` / `cfi_after_leave`, above `emit_fn` | the whole frame model in five directives, at the three sites that share a prologue: `emit_fn`, `emit_main`, `emit_globals_init` | Two decisions worth knowing about because they are not obvious: @@ -157,19 +158,64 @@ project's own source-level debugging case runs under **lldb**, which does not ha debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from it; if this is worth fixing it should be fixed in both places at once. +**lldb**, which is the debugger this project's own source-level case runs under, does better still — it +resolves `flan.tick` by name, shows the source inline and gives a column: + +``` +$ lldb -b -o "breakpoint set --name flan.tick" -o run -o bt ./dbg +* thread #1, stop reason = breakpoint 1.1 + frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3 + 17 (defn tick [c (Ptr Cell) n i32] i32 +-> 18 (let [bump (+ n 1)] + ^ + 19 (set (.heat c) (+ (.heat c) 1.5)) +(lldb) bt + * frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3 + frame #1: 0x00000000004007b5 dbg`flan.main at debug.flan:25:28 + frame #2: 0x0000000000400bea dbg`main + 41 + frame #3: 0x00007ffff7cb9575 libc.so.6`__libc_start_call_main + 117 +``` + +That transcript is now a test rather than a transcript: `test/test_acceptance.ml`'s lldb block has an `--x86` +arm beside its `--dev` one, asserting the breakpoint resolves and both Flan frames name `debug.flan`, and that +the program still prints what it printed. It claims no `frame variable`, which is the gap. + +**Unwinding out of the C runtime works**, which is the case that matters for this project's error paths and +which the brief did not ask for but should have. `bounds.flan` built `--x86 --debug`, breaking on the C symbol: + +``` +Breakpoint 1, flan_bounds_error (loc=0x40a880 "test/programs/bounds.flan:11:44", ...) at flan_rt.c:534 +#0 flan_bounds_error (...) at flan_rt.c:534 +#1 0x00000000004006be in flan.main () at bounds.flan:11 +#2 0x000000000040162e in main () +``` + +**The compile unit's range is exact.** `DW_AT_low_pc` 0x400628 is `flan.tick`, the first function; +`DW_AT_high_pc` 0x5cf puts the end at 0x400bf7, and `nm -S` says `main` — the last thing this object puts in +`.text` — ends at 0x400bc1 + 0x36 = 0x400bf7. So `.Ldwtext` / `.Ldwtext_end` bracket the four functions this +unit emitted and nothing else. + +`--x86 --dev --debug` together, which is a reachable combination and was never exercised by anything else, +builds, runs and gives a line table `readelf` reads with no warnings. + Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`, -`bounds-condition`, `algorithms`, `handles`, `registry` — all build with `--x86 --debug`, run, and produce -`.debug_line` that `readelf --debug-dump=decodedline` reads with **zero warnings**. `generics` is the +`bounds-condition`, `algorithms`, `handles`, `registry`, `destructure`, `edn`, twelve in all — every one +builds with `--x86 --debug`, runs, and produces `.debug_line` that `readelf --debug-dump=decodedline` reads +with **zero warnings**. `edn` and `generics` are the large ones, at 40 and 36 subprograms. `generics` is the multi-file case: its directory table has one entry and its file table two, `generics.flan` and ``. `` has no path on disk and a debugger simply finds no source for it, which is the truth. ## Baseline, measured on this tree -| | before | after | +| | before (`957ba07`) | after | |---|---|---| -| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | (see below) | -| `spike/x86/cells.sh` | 4/4 ok | (see below) | -| `dune test --root .` | exit 0 | (see below) | +| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | **99 / 0 / 0 / 0**, skips **28 + 6 + 2** — identical | +| `spike/x86/cells.sh` | 4/4 ok | **4/4 ok** | +| `dune test --root .` | exit 0 | **exit 0**, run without a pipe; acceptance reports 232 checks, 0 failures | + +The survey is protected structurally as well as measured: every piece of this lane's work on the release path +is behind `debug`, so `--x86` without `--debug` emits the assembly it emitted before — `fnctx.dw` is `None`, +`dwline` does nothing, the `.Ldwtext` label is not written and neither are the `.cfi` directives. Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you `tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way @@ -187,14 +233,18 @@ while the run still exits 0. reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with `DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the part `emit.ml` spends most of its debug lines on. -2. **`.cfi`, and it is now known to be cheap.** Nothing here emits it, so gdb unwinds by its prologue - analyser — which works, because `push rbp; mov rbp,rsp; sub rsp,N` is the canonical pattern and this - backend's header guarantees `rsp` is written exactly twice. The backtraces above are that analyser - working. But the experiment at the top of this file says `.cfi` against `.byte` output is exact, and the - frame model makes the content textbook: CFA is `rsp+16` after the `push`, `rbp`-based after the `mov`, and - unchanged for the whole body. It would want `.cfi_startproc` / `.cfi_endproc` in `emit_fn` and three - directives in the prologue. Worth doing; it is what makes a backtrace survive an unwind from inside the - runtime's C. +2. **`.cfi` is emitted, but only in a `--debug` build.** Five directives at three sites — `emit_fn`, + `emit_main`, `emit_globals_init`, which share a prologue. `readelf --debug-dump=frames` gives the exact + advances. `emit_main` gets no closing rule because it has no epilogue: it leaves through `flan_exit` and + the `ud2` after that is unreachable, so the `rbp` rule holds to the last byte. + + gdb did not need it — its prologue analyser already unwound out of `flan_bounds_error`, because + `push rbp; mov rbp,rsp; sub rsp,N` is the pattern it recognises — so this is a description stated rather + than guessed, plus a correct unwind from the very first byte of a function, before the `push`. + + **The open decision is whether to make it unconditional.** The description is correct in every build and a + release build is where a crash would most want it. What stopped it here is only that nothing measures the + `.eh_frame` it would add, and another lane is measuring backend cost right now. One `if` in three places. 3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded @@ -203,9 +253,10 @@ while the run still exits 0. `high_pc` range, which is honest — a debugger finds no line for an address in them and says so — but a backtrace through the globals' initialiser names nothing. `emit_globals_init` does lower expressions that carry locations, so this is a small piece of work: give it a `dwsub` the way `emit_fn` has one. -5. **Nothing tests this in `dune test`.** The corpus's source-level debugging case (`test/programs/debug.flan` - and `debug-permuted.flan`) runs under lldb against the LLVM backend. An `--x86` arm of it would be the - right regression test and does not exist; the verification above is a transcript in a handoff, which rots. +5. **`debug-permuted.flan` has no `--x86` arm.** The new acceptance case covers `debug.flan` only. The + permuted fixture exists to catch a member offset that does not follow the declaration, and that is a + *types* claim — there are no type DIEs here to get wrong, so it would test nothing today. It becomes the + right test the moment item 1 above is attempted, and it is already written. ## Open questions for the author From ae1b113ad41f7a30adf3e245d91bcd0b9f6f0902 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:42:37 +0700 Subject: [PATCH 6/7] Re-measured after the rebase onto the arithmetic-condition lane 101 MATCH rather than 99, and neither number is this lane's doing -- that lane added two probes to spike/x86. Zero DIFFER, zero REFUSED, zero NOX86 on both sides, the skip breakdown unmoved, cells 4/4, dune test exit 0 with 232 checks and no failures. --- HANDOFF-x86-debug.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md index aa579e9..2c665c4 100644 --- a/HANDOFF-x86-debug.md +++ b/HANDOFF-x86-debug.md @@ -1,7 +1,8 @@ # Handoff — dropping the registry notes, and debug information for `--x86` -Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`, from `957ba07`. Items **2** and **6** of -`HANDOFF-x86-rt.md` §6. Both landed. +Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`. Started from `957ba07` and rebased onto `eacf7c4` +when the arithmetic-condition lane landed; the rebase was clean, and everything below was re-measured after +it. Items **2** and **6** of `HANDOFF-x86-rt.md` §6. Both landed. ## The finding that changes the brief, read this first @@ -207,11 +208,15 @@ multi-file case: its directory table has one entry and its file table two, `gene ## Baseline, measured on this tree -| | before (`957ba07`) | after | -|---|---|---| -| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | **99 / 0 / 0 / 0**, skips **28 + 6 + 2** — identical | -| `spike/x86/cells.sh` | 4/4 ok | **4/4 ok** | -| `dune test --root .` | exit 0 | **exit 0**, run without a pipe; acceptance reports 232 checks, 0 failures | +| | before (`957ba07`) | after, on `957ba07` | after, rebased onto `eacf7c4` | +|---|---|---|---| +| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | **99 / 0 / 0 / 0**, skips 28 + 6 + 2 | **101 / 0 / 0 / 0**, skips 28 + 6 + 2 | +| `spike/x86/cells.sh` | 4/4 ok | 4/4 ok | **4/4 ok** | +| `dune test --root .` | exit 0 | exit 0 | **exit 0**, run without a pipe; acceptance reports **232 checks, 0 failures** | + +The MATCH count went from 99 to 101 across the rebase and neither is this lane's doing: the +arithmetic-condition lane added two probes to `spike/x86`. The number that matters is that DIFFER, REFUSED and +NOX86 are all zero on both sides and the skip breakdown did not move. The survey is protected structurally as well as measured: every piece of this lane's work on the release path is behind `debug`, so `--x86` without `--debug` emits the assembly it emitted before — `fnctx.dw` is `None`, From a9c5cf0c26190d2c6f00c3b1d29c302948381985 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 23:43:41 +0700 Subject: [PATCH 7/7] A frame with no line does not get the wrong line flan..init-globals and the C main shim sit inside the compile unit's range and have no line-table sequence. The worry is that a debugger picks a row out of a neighbouring function's sequence and reports a confident wrong Flan line; it does not. Checked by breaking inside dev-globals.flan's initialiser: the frame is named from the ELF symbol, the line is honestly absent, and the unwind out to libc is the .cfi working. So that handoff item is a gap and not a wrong answer, which is worth the distinction because the two deserve different urgency. --- HANDOFF-x86-debug.md | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md index 2c665c4..ae23b53 100644 --- a/HANDOFF-x86-debug.md +++ b/HANDOFF-x86-debug.md @@ -196,8 +196,10 @@ Breakpoint 1, flan_bounds_error (loc=0x40a880 "test/programs/bounds.flan:11:44", `.text` — ends at 0x400bc1 + 0x36 = 0x400bf7. So `.Ldwtext` / `.Ldwtext_end` bracket the four functions this unit emitted and nothing else. -`--x86 --dev --debug` together, which is a reachable combination and was never exercised by anything else, -builds, runs and gives a line table `readelf` reads with no warnings. +`--x86 --dev --debug` together, which is a reachable combination and is exercised by nothing else, builds, +runs, and gives a line table and an `.eh_frame` that `readelf` reads with no warnings — 280 FDEs, ours among +the runtime's. That is the one combination where the cells' `.data`, the `.eh_frame` and the debug sections +all have to be ordered against each other. Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`, `bounds-condition`, `algorithms`, `handles`, `registry`, `destructure`, `edn`, twelve in all — every one @@ -254,10 +256,24 @@ while the run still exits 0. bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded body will show raw addresses. -4. **`emit_main` and `flan..init-globals` have no rows.** They are inside the compile unit's `low_pc` / - `high_pc` range, which is honest — a debugger finds no line for an address in them and says so — but a - backtrace through the globals' initialiser names nothing. `emit_globals_init` does lower expressions that - carry locations, so this is a small piece of work: give it a `dwsub` the way `emit_fn` has one. +4. **`emit_main` and `flan..init-globals` have no rows, and this is a gap rather than a wrong answer.** + Both sit inside the compile unit's `low_pc` / `high_pc` range and neither has a line-table sequence, so + the worry is that a debugger picks a nearby row out of a *neighbouring* function's sequence and reports a + confident wrong Flan line. It does not. Checked, breaking inside the initialiser of `dev-globals.flan` + built `--x86 --debug`: + + ``` + Breakpoint 1, 0x0000000000400cec in flan..init-globals () + #0 0x0000000000400cec in flan..init-globals () + #1 0x00007ffff7cb96a4 in __libc_start_main_impl () from /lib64/libc.so.6 + #2 0x00000000004006c5 in _start () + No line number information available. + ``` + + The frame is named from the ELF symbol, the line is honestly absent, and the unwind out to libc is the + `.cfi` working. So this is worth doing and is not urgent: `emit_globals_init` does lower expressions that + carry locations, and giving it a `dwsub` the way `emit_fn` has one is the whole of it — it already takes + `~cfi`, so `?dw` goes in beside it. 5. **`debug-permuted.flan` has no `--x86` arm.** The new acceptance case covers `debug.flan` only. The permuted fixture exists to catch a member offset that does not follow the declaration, and that is a *types* claim — there are no type DIEs here to get wrong, so it would test nothing today. It becomes the