diff --git a/HANDOFF-x86-debug.md b/HANDOFF-x86-debug.md new file mode 100644 index 0000000..ae23b53 --- /dev/null +++ b/HANDOFF-x86-debug.md @@ -0,0 +1,288 @@ +# Handoff — dropping the registry notes, and debug information for `--x86` + +Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`. Started from `957ba07` and rebased onto `eacf7c4` +when the arithmetic-condition lane landed; the rebase was clean, and everything below was re-measured after +it. Items **2** and **6** of `HANDOFF-x86-rt.md` §6. Both landed. + +## The finding that changes the brief, read this first + +**`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line +tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is +true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and +GAS builds its line table from `dwarf2_emit_insn`, which runs only out of `md_assemble`. No instruction is +ever assembled, so nothing flushes the pending `.loc`. + +What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location +counter has reached by then. Measured on `as` 2.44: + +``` + .file 1 "foo.flan" +main: .loc 1 10 1 + .byte 0x55 + .byte 0x48,0x89,0xe5 + .loc 1 11 1 + .byte 0xb8,0x07,0x00,0x00,0x00 + .loc 1 12 1 + .byte 0xc9 + .byte 0xc3 +``` + +``` +foo.flan 10 0x4 x <- should be 0x0 +foo.flan 11 0x9 x <- should be 0x4 +foo.flan - 0xb <- line 12 has no row at all +``` + +Every row is one statement late and the last statement of every function gets none. Interposing labels +between the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting +the `.loc` directives back by one — the last-statement hole is a correctness gap, not just fragility, and the +behaviour being exploited is undocumented. + +So `.debug_line` is written out here by hand, as `.byte` / `.uleb128` / `.quad` in a `.debug_line` section, +which is the same thing this backend already does for instructions. + +**`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag +positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out +*exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct +advances. So this lane emits them, in a `--debug` build — see below. + +## What landed + +### Item 2 — a release build stops calling a registry that is switched off + +`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test +for test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never +a `Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so +the body is `()` and `dst` is untouched. + +`emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the +container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are +not touched until `call_rt`, so the guard is already early enough. + +Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`: + +| program | `--x86` release | `--x86 --dev` | LLVM release | +|---|---|---|---| +| `vec.flan` | 46 → **1** | 46 | 1 | +| `maps.flan` | 55 → **1** | 55 | — | +| `registry.flan` | 36 → **1** | 36 | — | + +The one that remains in every release build is not emitted code: it is inside the runtime's own +`flan_dev_reg_note_vec` in `flan_rt.c`. LLVM's release build has exactly the same one, so the two backends +agree. + +### Item 6 — `--x86 --debug` works, at parity with LLVM bar locals + +`lib/build.ml`'s refusal list no longer names `--debug`; only `--sanitize` and wasm are left there. +`X86.program` takes `~debug` the way it already took `~dev`. + +In `lib/x86.ml`: + +| piece | what | +|---|---| +| the `Debug information` section, above `fnctx` | `dwrow` / `dwsub` / `dwarf`, and `dwfile`. Carries the header comment explaining the `.loc` finding above | +| `fnctx.dw : dwarf option` | `None` in every build but a `--debug` one, which is what makes the release path structurally unchanged | +| `dwline`, beside `new_label` | the one hook. A label and a row, deduplicated on (file, line, column) *and* on the byte counter — see below | +| `lower` | calls `dwline f e.Tast.loc` and nothing else | +| `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte | +| `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) | +| `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end | +| `cfi_after_push` / `cfi_after_mov` / `cfi_after_leave`, above `emit_fn` | the whole frame model in five directives, at the three sites that share a prologue: `emit_fn`, `emit_main`, `emit_globals_init` | + +Two decisions worth knowing about because they are not obvious: + +**The line program is written in the dumb form.** Every row is a full `DW_LNE_set_address` on a label, then +`set_file` / `set_column` / `advance_line` as needed, then `DW_LNS_copy` — eleven bytes and up per row, where +a special opcode would be one. The alternative is `DW_LNS_advance_pc` with a `.uleb128` of a difference of two +labels, which asks the assembler to resolve a value whose *size* changes the offsets after it. That does work, +and its failure would look exactly like a DWARF bug. A debug build can afford the bytes. + +**The `.file 1 "..."` directive at the top is load-bearing and is not about our own table.** Without it, +clang's integrated assembler generates a compile unit of *its own* over the `.s`, whose rows land at the +`call` mnemonics — the only real instructions this backend emits — and those addresses are inside functions +our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an +empty line table and nothing else. `-gdwarf-4` is added alongside it in `build.ml` so that empty stub is a +version 4 header; at the default version it is a DWARF 5 header whose file table `readelf` calls corrupt, and +a warning on a cross-check is a warning you learn to ignore. + +**Rows are deduplicated on the byte counter as well as the position.** `lower` recurses, so an outer form and +the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the +outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one +address takes the last. `buf.n` already existed and was written but never read; this is its first reader. + +## Verification — a real debugger, on the corpus's own debug fixture + +`test/programs/debug.flan` built `--x86 --debug`, under `gdb` 16.x, breaking by file and line: + +``` +$ flan build test/programs/debug.flan --x86 --debug -o dbg +$ gdb -batch -x gdb5.cmd ./dbg +Breakpoint 1 at 0x400689: file debug.flan, line 19. + +Breakpoint 1, flan.tick () at debug.flan:19 +19 (set (.heat c) (+ (.heat c) 1.5)) +#0 flan.tick () at debug.flan:19 +#1 0x00000000004007b5 in flan.main () at debug.flan:25 +#2 0x0000000000400bea in main () +Line 19 of "debug.flan" starts at address 0x400689 and ends at 0x4006af . +Current source file is debug.flan +Compilation directory is .../test/programs +Located in .../test/programs/debug.flan +Contains 30 lines. +Source language is c. +Producer is flan (x86-64). +Compiled with DWARF 4 debugging format. +``` + +The same script against an **LLVM** `--debug` build of the same program, which is the parity this is measured +against: + +``` +Breakpoint 1 at 0x400644: file debug.flan, line 19. + +Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19 +19 (set (.heat c) (+ (.heat c) 1.5)) +#0 flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19 +#1 0x00000000004006cf in flan.main () at debug.flan:25 +#2 0x0000000000400806 in main () +Producer is flan. +Compiled with DWARF 5 debugging format. +``` + +Identical bar `c=0x7fffffffd8f0, n=41` — which is the locals work, and is the honest summary of what is +missing. + +**`break tick` fails on both backends**, with `Function "tick" not defined`. That is not a gap this lane +opened: `emit.ml` writes `name: "tick", linkageName: "flan.tick"`, gdb takes the linkage name as the search +name because `flan.tick` is not a mangled C++ name, and an LLVM `--debug` build behaves the same way. The +project's own source-level debugging case runs under **lldb**, which does not have this problem. `break +debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from +it; if this is worth fixing it should be fixed in both places at once. + +**lldb**, which is the debugger this project's own source-level case runs under, does better still — it +resolves `flan.tick` by name, shows the source inline and gives a column: + +``` +$ lldb -b -o "breakpoint set --name flan.tick" -o run -o bt ./dbg +* thread #1, stop reason = breakpoint 1.1 + frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3 + 17 (defn tick [c (Ptr Cell) n i32] i32 +-> 18 (let [bump (+ n 1)] + ^ + 19 (set (.heat c) (+ (.heat c) 1.5)) +(lldb) bt + * frame #0: 0x0000000000400647 dbg`flan.tick at debug.flan:18:3 + frame #1: 0x00000000004007b5 dbg`flan.main at debug.flan:25:28 + frame #2: 0x0000000000400bea dbg`main + 41 + frame #3: 0x00007ffff7cb9575 libc.so.6`__libc_start_call_main + 117 +``` + +That transcript is now a test rather than a transcript: `test/test_acceptance.ml`'s lldb block has an `--x86` +arm beside its `--dev` one, asserting the breakpoint resolves and both Flan frames name `debug.flan`, and that +the program still prints what it printed. It claims no `frame variable`, which is the gap. + +**Unwinding out of the C runtime works**, which is the case that matters for this project's error paths and +which the brief did not ask for but should have. `bounds.flan` built `--x86 --debug`, breaking on the C symbol: + +``` +Breakpoint 1, flan_bounds_error (loc=0x40a880 "test/programs/bounds.flan:11:44", ...) at flan_rt.c:534 +#0 flan_bounds_error (...) at flan_rt.c:534 +#1 0x00000000004006be in flan.main () at bounds.flan:11 +#2 0x000000000040162e in main () +``` + +**The compile unit's range is exact.** `DW_AT_low_pc` 0x400628 is `flan.tick`, the first function; +`DW_AT_high_pc` 0x5cf puts the end at 0x400bf7, and `nm -S` says `main` — the last thing this object puts in +`.text` — ends at 0x400bc1 + 0x36 = 0x400bf7. So `.Ldwtext` / `.Ldwtext_end` bracket the four functions this +unit emitted and nothing else. + +`--x86 --dev --debug` together, which is a reachable combination and is exercised by nothing else, builds, +runs, and gives a line table and an `.eh_frame` that `readelf` reads with no warnings — 280 FDEs, ours among +the runtime's. That is the one combination where the cells' `.data`, the `.eh_frame` and the debug sections +all have to be ordered against each other. + +Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`, +`bounds-condition`, `algorithms`, `handles`, `registry`, `destructure`, `edn`, twelve in all — every one +builds with `--x86 --debug`, runs, and produces `.debug_line` that `readelf --debug-dump=decodedline` reads +with **zero warnings**. `edn` and `generics` are the large ones, at 40 and 36 subprograms. `generics` is the +multi-file case: its directory table has one entry and its file table two, `generics.flan` and ``. +`` has no path on disk and a debugger simply finds no source for it, which is the truth. + +## Baseline, measured on this tree + +| | before (`957ba07`) | after, on `957ba07` | after, rebased onto `eacf7c4` | +|---|---|---|---| +| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | **99 / 0 / 0 / 0**, skips 28 + 6 + 2 | **101 / 0 / 0 / 0**, skips 28 + 6 + 2 | +| `spike/x86/cells.sh` | 4/4 ok | 4/4 ok | **4/4 ok** | +| `dune test --root .` | exit 0 | exit 0 | **exit 0**, run without a pipe; acceptance reports **232 checks, 0 failures** | + +The MATCH count went from 99 to 101 across the rebase and neither is this lane's doing: the +arithmetic-condition lane added two probes to `spike/x86`. The number that matters is that DIFFER, REFUSED and +NOX86 are all zero on both sides and the skip breakdown did not move. + +The survey is protected structurally as well as measured: every piece of this lane's work on the release path +is behind `debug`, so `--x86` without `--debug` emits the assembly it emitted before — `fnctx.dw` is `None`, +`dwline` does nothing, the `.Ldwtext` label is not written and neither are the `.cfi` directives. + +Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you +`tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way +while the run still exits 0. + +## What was not reached, and what the next lane should do with it + +1. **Locals and types.** Not attempted, and the reason is structural rather than a lack of time. `emit.ml` + writes a `!DILocalVariable` per slot because every slot there is an `alloca` that `llvm.dbg.declare` + points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: `alloc` + knows its `rbp`-relative offset, but `scoped` reclaims temporaries and a later expression reuses the bytes, + so the *lifetime* is not modelled. A `DW_TAG_variable` with a `DW_OP_fbreg` would be right at some + addresses and confidently wrong at others. Note that the named slots — `fn.Tast.snames` — are the *slots* + and not the temporaries, and those do live for the whole function, so a narrower version of this is + reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with + `DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the + part `emit.ml` spends most of its debug lines on. +2. **`.cfi` is emitted, but only in a `--debug` build.** Five directives at three sites — `emit_fn`, + `emit_main`, `emit_globals_init`, which share a prologue. `readelf --debug-dump=frames` gives the exact + advances. `emit_main` gets no closing rule because it has no epilogue: it leaves through `flan_exit` and + the `ud2` after that is unreachable, so the `rbp` rule holds to the last byte. + + gdb did not need it — its prologue analyser already unwound out of `flan_bounds_error`, because + `push rbp; mov rbp,rsp; sub rsp,N` is the pattern it recognises — so this is a description stated rather + than guessed, plus a correct unwind from the very first byte of a function, before the `push`. + + **The open decision is whether to make it unconditional.** The description is correct in every build and a + release build is where a crash would most want it. What stopped it here is only that nothing measures the + `.eh_frame` it would add, and another lane is measuring backend cost right now. One `if` in three places. +3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's + bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all + (`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded + body will show raw addresses. +4. **`emit_main` and `flan..init-globals` have no rows, and this is a gap rather than a wrong answer.** + Both sit inside the compile unit's `low_pc` / `high_pc` range and neither has a line-table sequence, so + the worry is that a debugger picks a nearby row out of a *neighbouring* function's sequence and reports a + confident wrong Flan line. It does not. Checked, breaking inside the initialiser of `dev-globals.flan` + built `--x86 --debug`: + + ``` + Breakpoint 1, 0x0000000000400cec in flan..init-globals () + #0 0x0000000000400cec in flan..init-globals () + #1 0x00007ffff7cb96a4 in __libc_start_main_impl () from /lib64/libc.so.6 + #2 0x00000000004006c5 in _start () + No line number information available. + ``` + + The frame is named from the ELF symbol, the line is honestly absent, and the unwind out to libc is the + `.cfi` working. So this is worth doing and is not urgent: `emit_globals_init` does lower expressions that + carry locations, and giving it a `dwsub` the way `emit_fn` has one is the whole of it — it already takes + `~cfi`, so `?dw` goes in beside it. +5. **`debug-permuted.flan` has no `--x86` arm.** The new acceptance case covers `debug.flan` only. The + permuted fixture exists to catch a member offset that does not follow the declaration, and that is a + *types* claim — there are no type DIEs here to get wrong, so it would test nothing today. It becomes the + right test the moment item 1 above is attempted, and it is already written. + +## Open questions for the author + +- **`break tick` naming.** Is the `name` / `linkageName` split worth keeping, given it costs `break ` in + gdb on *both* backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may + be deliberate. It was left alone rather than diverged from. +- **Should `--x86 --debug` imply `--dev`?** It does not today, and they are independent axes, which seems + right — but a debug build you cannot redefine into is half of what the dev loop wants. diff --git a/lib/build.ml b/lib/build.ml index f9d35e5..52efa58 100644 --- a/lib/build.ml +++ b/lib/build.ml @@ -756,11 +756,20 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) [flan reload] and [flan dev] both build host and module through LLVM — and the fix when something does is to emit the module through this backend too, not to grow a classifier. *) - if opts.x86 && (wasm_target opts || opts.debug || opts.sanitize) + (* [--debug] used to be in this list too. It is not any more: [x86.ml] emits + a compile unit, a subprogram per function and a line table, all written + out as bytes because [.loc] cannot work against a file whose instructions + are [.byte] blobs -- see that file's own debug-information section. What + it does *not* emit is locals and types, for the reason given there: a + slot is a bump-allocated frame temporary whose lifetime this backend does + not model, so there is nothing honest for a [DW_TAG_variable] to point + at. So `--x86 --debug` gives a backtrace that names Flan files, functions + and lines, and `print x` says the name is not in the current context. *) + if opts.x86 && (wasm_target opts || opts.sanitize) then failwith - "--x86 is the native dev backend on its own: it emits no DWARF and \ - there is no sanitizer pass over hand-written assembly"; + "--x86 is the native dev backend on its own: there is no sanitizer pass \ + over hand-written assembly"; let dir = workdir () in (* The one fork in this function. The x86 backend hands clang an assembly file where LLVM hands it IR text; clang takes either on its command line, @@ -771,7 +780,8 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) (Filename.basename out ^ if opts.x86 then ".s" else ".ll") in write ll - (if opts.x86 then X86.program ~checks:opts.checks ~dev:opts.dev p + (if opts.x86 then + X86.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug p else Emit.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug ~pnames ~sanitize:opts.sanitize p); @@ -820,6 +830,16 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = []) the .ll, which is the only place the Flan half of the program gets instrumented at all. *) @ cflags opts + (* The .s carries a hand-written DWARF 4 compile unit. The numbered + [.file] directive in it stops clang's integrated assembler from + generating one of its own, but the directive still leaves an empty + line table behind, and at the default version that stub is a DWARF 5 + header whose file table readelf reports as corrupt. Asking for 4 + makes the stub parse, so a readelf cross-check of the real table is + not read past a warning. It reaches only this command, which + assembles the .s and links; the C objects were compiled by + [compile_c] and are already done. *) + @ (if opts.x86 && opts.debug then [ "-gdwarf-4" ] else []) @ (if opts.dev then [ "-rdynamic" ] else []) @ tflags @ (if web_target opts then web_link_flags ~out else []) diff --git a/lib/x86.ml b/lib/x86.ml index f525e6a..814903c 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -408,6 +408,90 @@ let gsym n = asm_sym ("flan." ^ n) Byte-for-byte or the link fails and the piece served nothing. *) let csym n = asm_sym ("flan.cell." ^ n) +(* ── Debug information ───────────────────────────────────────────────── *) + +(* DWARF, written out as bytes, for the same reason the instructions are — and + the reason is worth stating first because it is the one thing about this + backend that makes debug information cost more here than it does anywhere + else. + + {b [.loc] does not work against an assembly file that has no instructions + in it.} GAS builds its line table from [dwarf2_emit_insn], which runs only + when an instruction is assembled, and this file assembles none: everything + is a [.byte] blob. A pending [.loc] therefore sits until the *next* [.loc] + and is flushed at whatever the location counter has reached by then, so + every row comes out one statement late and the last statement of every + function gets no row at all. Measured on GAS 2.44 and reproduced with + labels interposed, which do not help. So [.debug_line] is emitted here as + data, which is in any case the only spelling consistent with the rest of + the file. + + DWARF 4 rather than 5. Version 4's file and directory tables are + NUL-terminated strings and a terminator byte where 5 form-codes them, and + nothing above needs a version 5 feature. A compile unit declares its own + version, so a v4 unit sitting beside the v5 ones clang gives the runtime's + C is not a conflict — each is read on its own terms. + + What is described is deliberately shallow: the compile unit, one subprogram + per function, and the line table. {b No locals and no types.} [emit.ml] + writes a [!DILocalVariable] per slot because every slot there is an + [alloca] that [llvm.dbg.declare] can point at and LLVM computes the frame + offset; here a slot is a bump-allocated frame temporary whose offset this + file knows but whose *lifetime* it does not model — [scoped] reclaims + temporaries and a later expression reuses the bytes. Naming an offset that + holds something else half the time is worse than naming nothing, so this + emits nothing rather than a confident wrong answer. That is the same call + [build.ml] makes about wasm32's member offsets. *) + +(* One row of the line table: the label whose address it is, and the position + it names. Addresses are labels rather than numbers because the assembler + places the function and this file does not. *) +type dwrow = { rlbl : string; rfile : int; rline : int; rcol : int } + +type dwsub = { + sname : string; (* what a debugger calls the frame *) + ssym : string; (* the symbol the linker sees *) + sfile : int; + sline : int; + send : string; (* a label one past the function's last byte *) + mutable srows : dwrow list; (* newest first *) +} + +type dwarf = { + dfiles : (string, int) Hashtbl.t; + mutable dpaths : string list; (* newest first *) + mutable dsubs : dwsub list; (* newest first *) + mutable dcur : dwsub option; (* the function being lowered *) + mutable dlast : (int * int * int) option; (* the last row's position *) + (* [buf.n] as it stood when the last row was made. [lower] recurses, so an + outer form and the inner one that emits the first byte of it both ask for + a row at the same address; without this the table carries a run of rows + that a debugger resolves by taking the last, which is the innermost form + rather than the statement. Keeping the first is both smaller and the + better answer. Reset to -1 per function, because the body's byte counter + starts again at 0 and the entry row is at the prologue's 0. *) + mutable dlastn : int; +} + +let new_dwarf () = + { dfiles = Hashtbl.create 8; dpaths = []; dsubs = []; dcur = None; + dlast = None; dlastn = -1 } + +(* File indices are 1-based and handed out in first-seen order, which is the + order the file table is written in below. A program spans more than one + file whenever the prelude or a macro contributed a form, and the checker's + own invented nodes carry [Loc.unknown], whose file is [] — that + one never reaches here, because a row with line 0 is attributed to the + enclosing function's file instead. *) +let dwfile dw path = + match Hashtbl.find_opt dw.dfiles path with + | Some n -> n + | None -> + let n = List.length dw.dpaths + 1 in + Hashtbl.replace dw.dfiles path n; + dw.dpaths <- path :: dw.dpaths; + n + (* ── Function context ────────────────────────────────────────────────── *) type fnctx = { @@ -451,6 +535,11 @@ type fnctx = { module answers true for the host's cells, globals and bodies, and those go through the GOT -- see [modrm_got]. *) ext : string -> bool; + (* The line table under construction, in a [--debug] build. [None] is every + other build, and then [dwline] below is the only thing that looks at it + and does nothing — so a release build's output is byte-identical to what + it was before debug information existed. *) + dw : dwarf option; } (* Module-wide rather than per-function. Two functions each holding an [if] @@ -462,6 +551,36 @@ let uniq = ref 0 let new_label _f tag = incr uniq; Printf.sprintf ".L%s%d" tag !uniq +(* A line-table row at the point the output has reached, unless the last row + already named this position. [lower] calls this for every expression, so + the dedup is what keeps a statement made of a dozen nodes on one line from + costing a dozen rows; the column is part of the key, so two forms on one + line are still told apart. + + Line 0 is [Loc.unknown] — a node the checker invented rather than one + anyone wrote. It is attributed to the enclosing function's own line, for + [emit.ml]'s reason at its [at_loc]: a zero line in DWARF means "no line", + and a debugger given one steps over the whole construct. *) +let dwline f (loc : Loc.t) = + match f.dw with + | None -> () + | Some dw -> + (match dw.dcur with + | None -> () + | Some s -> + let file, line, col = + if loc.Loc.line = 0 then (s.sfile, s.sline, 1) + else (dwfile dw loc.Loc.file, loc.Loc.line, loc.Loc.col) + in + if dw.dlast <> Some (file, line, col) && f.b.n > dw.dlastn then begin + let l = new_label f "dl" in + lbl f.b l; + dw.dlast <- Some (file, line, col); + dw.dlastn <- f.b.n; + s.srows <- + { rlbl = l; rfile = file; rline = line; rcol = col } :: s.srows + end) + (* Bump-allocate a frame temporary and answer its rbp-relative offset. The offset is negative, so the running total is rounded *up* to the alignment; rbp is 16-aligned, so that is the alignment the value actually gets. *) @@ -962,6 +1081,12 @@ let emit_args f (args : arg list) = let sink = Lf 0 let rec lower f (e : Tast.expr) (dst : loc) : unit = + (* The one hook the line table needs, and it is here rather than at + statement granularity on purpose: the same recursion that lowers a nested + call lowers its arguments, and a row per expression is what makes a + backtrace through an argument name the argument rather than the call. It + is inert in every build but a [--debug] one. *) + dwline f e.Tast.loc; if dst == sink && not (is_void e.Tast.ty) then scoped f (fun () -> let o = tmp f e.Tast.ty in @@ -2245,6 +2370,23 @@ and prim f (e : Tast.expr) (p : Tast.prim) (args : Tast.expr list) dst = let l = lvalue f a in addr_into f ~reg:rax l; store_int f.b ~src:rax ~mm:(lmem f dst ~scratch:r11) ~size:8 + (* The allocation registry's notes are the one runtime family a release + build drops on the floor, and [emit.ml:1918] drops it with this same + guard — the checker builds a [Tast.Rt] it does not know is unwanted, + because it does not know whether this is a dev build and does not have + to. [emit.ml] is careful to drop it before the arguments are walked, so + that taking the address of the container being described does not leave + an escaped alloca for mem2reg to refuse; here the arguments are not + touched until [call_rt], so answering [()] is already early enough. The + test is [emit.ml]'s byte for byte, strict [>] included: bare + [flan_dev_reg_note] is the runtime's own entry point and is never a + [Tast.Rt]; what [check.ml] builds is the [_vec], [_map] and [_pool] + wrappers, each of which is longer than the prefix. The node's type is + [Unit], so there is nothing to store and [dst] is untouched. *) + | Tast.Rt sym, _ + when (not f.md.Emit.dev) + && String.length sym > 17 + && String.equal (String.sub sym 0 17) "flan_dev_reg_note" -> () | Tast.Rt sym, _ -> call_rt f ~sym ~args ~rty:t dst | Tast.Cast target, [ a ] -> cast f a target dst | _ -> unsupported "primitive with %d arguments" (List.length args) @@ -2309,6 +2451,36 @@ and cast f (a : Tast.expr) (target : Types.t) dst = cvttf2si f.b ~f64:(f64_of src_t) ~dst:rax ~src:xmm0; store_loc f ~reg:rax dst dst_t +(* ── Call frame information ──────────────────────────────────────────── *) + +(* The whole frame model, in five directives. + + [.cfi] is the one thing the assembler gets right against a file with no + instructions in it, and it is worth saying so beside the debug-information + section above, which is about the thing it gets wrong: CFI advances are + computed from frag positions, where the line table's are computed from + having assembled an instruction. Measured — a [.byte]-only function comes + out of [readelf --debug-dump=frames] with exact advances. + + The content is a constant because of the header's own claim that [rsp] is + written exactly twice. On entry the CFA is [rsp+8]; [push rbp] makes it + [rsp+16] and puts the saved [rbp] at [cfa-16]; [mov rsp, rbp] moves the + rule onto [rbp], where it stays for the whole body, because the only other + write to [rsp] is the [leave]. After that [rsp] is [rbp+8] again and the + CFA is [rsp+8]. Register 6 is [rbp] and 7 is [rsp] in DWARF's numbering; + the return address is column 16 and the CIE already says it is at + [cfa-8]. + + Emitted only in a [--debug] build, so that a release build's assembly stays + byte-for-byte what it was. That is a conservative call rather than a + principled one: this description is correct in every build, and a release + build is where an unwind through a crash would most want it. What stops it + from being unconditional today is only that nothing measures the [.eh_frame] + it would add. *) +let cfi_after_push b = text b "\t.cfi_def_cfa_offset 16\n\t.cfi_offset 6, -16\n" +let cfi_after_mov b = text b "\t.cfi_def_cfa_register 6\n" +let cfi_after_leave b = text b "\t.cfi_def_cfa 7, 8\n" + (* ── A function ──────────────────────────────────────────────────────── *) (* The frame is rounded to 16 and reserves the outgoing-argument area in the @@ -2348,7 +2520,7 @@ let incoming_of ~sret (params : Types.t list) = sret_at, ps, next_int () let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) - ?(hidden = false) (fn : Tast.fn) : string * string = + ?(hidden = false) ?dw (fn : Tast.fn) : string * string = let b = create () in let nslots = Array.length fn.Tast.slots in let f = @@ -2357,7 +2529,36 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) xfer_off = 0; sret_off = 0; retval = 0; frame = 0; maxframe = 0; outgoing = 0; loops = []; pads = []; xfer_lbl = ""; unwound = false; - rodata = Buffer.create 64; externs; fns; ext } + rodata = Buffer.create 64; externs; fns; ext; dw } + in + (* The subprogram this function's rows hang off. Its first row is the + function symbol itself, at the line the [defn] was written on, so the + entry has a position before the prologue has run; every row after it + comes out of [dwline] as the body is lowered. [dlast] is not primed with + it, which is deliberate — the first form of the body sits at a different + column even when it is on the same line, so it gets a row of its own, and + two rows are what let a debugger put a breakpoint after the prologue + rather than on it. *) + let cfi = match dw with None -> false | Some _ -> true in + let sub = + match dw with + | None -> None + | Some d -> + let line = + if fn.Tast.floc.Loc.line = 0 then 1 else fn.Tast.floc.Loc.line + in + let file = dwfile d fn.Tast.floc.Loc.file in + let s = + { sname = fn.Tast.name; ssym = fsym fn.Tast.name; sfile = file; + sline = line; send = new_label f "fe"; + srows = [ { rlbl = fsym fn.Tast.name; rfile = file; rline = line; + rcol = 1 } ] } + in + d.dcur <- Some s; + d.dlast <- None; + d.dlastn <- -1; + d.dsubs <- s :: d.dsubs; + Some s in (* The header's own frame model: every slot and every temporary is bump-allocated below rbp, and the high-water mark is what the prologue @@ -2471,7 +2672,9 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) (* The prologue, now that the frame size is known. *) let pb = create () in push_r pb rbp; + if cfi then cfi_after_push pb; mov_rr pb ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov pb; let n = frame_bytes f in if n > 0 then sub_imm pb ~dst:rsp n; (match sret_at with @@ -2533,6 +2736,7 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) load_scalar f ~reg:(if is_float fn.Tast.ret then xmm0 else rax) ~off:f.retval fn.Tast.ret; leave f.b; + if cfi then cfi_after_leave f.b; ret f.b; flush pb; flush f.b; @@ -2548,8 +2752,18 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) if hidden then Buffer.add_string out (Printf.sprintf "\t.hidden\t%s\n" sym); Buffer.add_string out (Printf.sprintf "\t.type\t%s, @function\n" sym); Buffer.add_string out (sym ^ ":\n"); + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents pb.out); Buffer.add_string out (Buffer.contents f.b.out); + (* One past the last byte, which is what [DW_AT_high_pc] and the line + table's closing [DW_LNE_end_sequence] both want. [.size]'s [. - sym] says + the same thing but is an expression rather than a symbol, and + [DW_FORM_addr] takes a symbol. *) + (match sub with + | Some s -> Buffer.add_string out (s.send ^ ":\n") + | None -> ()); + (match dw with Some d -> d.dcur <- None | None -> ()); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" sym sym); Buffer.contents out, Buffer.contents f.rodata @@ -2560,10 +2774,12 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false) the loader put them in, the program's own end of the transfer channel is a null cell on this frame, and the exit goes through [flan_exit] because stdout is a FILE* and something has to flush it. *) -let emit_main (md : Emit.m) (fn : Tast.fn) = +let emit_main ?(cfi = false) (md : Emit.m) (fn : Tast.fn) = let b = create () in push_r b rbp; + if cfi then cfi_after_push b; mov_rr b ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov b; sub_imm b ~dst:rsp 48; (* [al] is zero at every call this backend makes, variadic or not — see [call_native]. Setting it here too costs two bytes and keeps the rule @@ -2593,7 +2809,13 @@ let emit_main (md : Emit.m) (fn : Tast.fn) = ignore md; let out = Buffer.create 256 in Buffer.add_string out "\t.globl\tmain\n\t.type\tmain, @function\nmain:\n"; + (* No [.cfi_def_cfa 7, 8] to close with, because this one has no epilogue: + it leaves through [flan_exit] and the [ud2] after that is unreachable. + The rbp rule therefore holds to the last byte, which is what a backtrace + out of anything [main] called needs. *) + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents b.out); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out "\t.size\tmain, . - main\n\n"; Buffer.contents out @@ -2621,14 +2843,16 @@ let emit_globals_data (md : Emit.m) (globals : Tast.global list) = let init_sym = "\"flan..init-globals\"" -let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = +let emit_globals_init ?(cfi = false) (md : Emit.m) ~externs ~fns + (globals : Tast.global list) = let b = create () in let f = { b; md; fnname = ""; retlbl = new_label () "ginit"; fret = Types.Unit; slots = [||]; xfer_off = 0; sret_off = 0; retval = 0; frame = 0; maxframe = 0; outgoing = 0; loops = []; pads = []; xfer_lbl = ""; unwound = false; - rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false) } + rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false); + dw = None } in (* Two slots, not one: [xfer_off] holds the *pointer* every call passes on, and [cell] is what it points at. Storing a null into [xfer_off] itself — @@ -2651,7 +2875,9 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = end; let pb = create () in push_r pb rbp; + if cfi then cfi_after_push pb; mov_rr pb ~dst:rbp ~src:rsp; + if cfi then cfi_after_mov pb; let n = frame_bytes f in if n > 0 then sub_imm pb ~dst:rsp n; (* No caller hands this one a channel, so it gets a null cell of its own and @@ -2662,13 +2888,16 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) = store_int pb ~src:rax ~mm:(Frame f.xfer_off) ~size:8; lbl f.b f.retlbl; leave f.b; + if cfi then cfi_after_leave f.b; ret f.b; flush pb; flush f.b; let out = Buffer.create 512 in Buffer.add_string out (Printf.sprintf "\t.type\t%s, @function\n%s:\n" init_sym init_sym); + if cfi then Buffer.add_string out "\t.cfi_startproc\n"; Buffer.add_string out (Buffer.contents pb.out); Buffer.add_string out (Buffer.contents f.b.out); + if cfi then Buffer.add_string out "\t.cfi_endproc\n"; Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" init_sym init_sym); Buffer.contents out, Buffer.contents f.rodata @@ -2756,8 +2985,210 @@ let emit_cells (p : Tast.program) = p.Tast.fns; Buffer.contents out +(* ── The DWARF sections ──────────────────────────────────────────────── *) + +(* A path inside an assembler string literal. Flan source paths contain + neither of these two characters in practice, but a path is user input and + a stray backslash would end the directive rather than the string. *) +let asm_str s = + let b = Buffer.create (String.length s + 8) in + String.iter + (fun c -> + if c = '"' || c = '\\' then Buffer.add_char b '\\'; + Buffer.add_char b c) + s; + Buffer.contents b + +(* Absolute, because the file table below gives every entry directory index 0 + and a debugger then reads the name as written. [emit.ml]'s [dfile] splits + the same absolute path into a [!DIFile]'s filename and directory; DWARF is + happy with either and one string is fewer moving parts. *) +let abspath p = + if Filename.is_relative p then Filename.concat (Sys.getcwd ()) p else p + +(* [DW_LNE_set_address] on a label: the escape opcode, a length of 9, the + sub-opcode, and eight bytes of address the assembler relocates. + + Every row gets one of these rather than a [DW_LNS_advance_pc] with a + computed delta, and the reason is that a delta would be a difference of two + labels inside a [.uleb128], which asks the assembler to resolve a value + whose size affects the values after it. That does work, but it is exactly + the kind of thing whose failure looks like a DWARF bug. Rows are eleven + bytes each here and that is a debug build's business. *) +let dw_set_address b lbl = + Buffer.add_string b (Printf.sprintf "\t.byte\t0, 9, 2\n\t.quad\t%s\n" lbl) + +(* The line-number program: one sequence per function, closed with an + end_sequence at the label one past its last byte. The registers are reset + at the head of every sequence, which is why [line] starts at 1 and [file] + at 1 in each. Rows within a sequence must be address-ordered, and they are + for free: [dwline] appends them in the order the bytes are emitted. *) +let dw_line_program (dw : dwarf) = + let b = Buffer.create 4096 in + List.iter + (fun s -> + let line = ref 1 and file = ref 1 and col = ref 0 in + List.iter + (fun r -> + dw_set_address b r.rlbl; + if r.rfile <> !file then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t4\n\t.uleb128 %d\n" r.rfile); + file := r.rfile + end; + if r.rcol <> !col then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t5\n\t.uleb128 %d\n" r.rcol); + col := r.rcol + end; + if r.rline <> !line then begin + Buffer.add_string b + (Printf.sprintf "\t.byte\t3\n\t.sleb128 %d\n" (r.rline - !line)); + line := r.rline + end; + Buffer.add_string b "\t.byte\t1\n") + (List.rev s.srows); + dw_set_address b s.send; + Buffer.add_string b "\t.byte\t0, 1, 1\n") + (List.rev dw.dsubs); + Buffer.contents b + +(* The three sections. [tbeg] and [tend] bracket everything this object puts + in [.text] — the compile unit claims that whole range, including the C + [main] shim and the globals' initialiser, neither of which has any rows. + That is honest: they are code this unit produced, and a debugger that finds + no line for an address inside them says so. *) +let emit_dwarf (dw : dwarf) ~cufile ~tbeg ~tend = + let out = Buffer.create 8192 in + (* The compile unit's directory, and the directory table the file entries + index into. [emit.ml]'s [dfile] splits every path into a basename and a + directory the same way, and the reason to match it is what a debugger + prints: a frame reads [debug.flan:19] rather than the eighty characters + of an absolute path. Directory index 0 is the compile unit's own + directory, so a file sitting beside the one that named the unit -- which + is most of them -- costs no entry at all. *) + let cudir = Filename.dirname (abspath cufile) in + let dirs = ref [] in + let dirix d = + if String.equal d cudir then 0 + else + match List.assoc_opt d !dirs with + | Some n -> n + | None -> + let n = List.length !dirs + 1 in + dirs := !dirs @ [ (d, n) ]; + n + in + let files = + List.map + (fun p -> + let a = abspath p in + (Filename.basename a, dirix (Filename.dirname a))) + (List.rev dw.dpaths) + in + Buffer.add_string out + "\n# Debug information. Written out as data rather than left to the\n\ + # assembler's .loc, which cannot work here: GAS builds its line table\n\ + # when it assembles an instruction, and this file assembles none.\n"; + (* .debug_abbrev. Two abbreviations, because two shapes of DIE are all that + is described. *) + Buffer.add_string out + "\t.section\t.debug_abbrev,\"\",@progbits\n\ + .Ldwabbrev:\n\ + \t.uleb128 1\n\ + \t.uleb128 0x11\t\t# DW_TAG_compile_unit\n\ + \t.byte\t1\t\t# has children\n\ + \t.uleb128 0x25\n\t.uleb128 0x08\t# DW_AT_producer DW_FORM_string\n\ + \t.uleb128 0x13\n\t.uleb128 0x05\t# DW_AT_language DW_FORM_data2\n\ + \t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\ + \t.uleb128 0x1b\n\t.uleb128 0x08\t# DW_AT_comp_dir DW_FORM_string\n\ + \t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\ + \t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\ + \t.uleb128 0x10\n\t.uleb128 0x17\t# DW_AT_stmt_list DW_FORM_sec_offset\n\ + \t.byte\t0, 0\n\ + \t.uleb128 2\n\ + \t.uleb128 0x2e\t\t# DW_TAG_subprogram\n\ + \t.byte\t0\t\t# no children\n\ + \t.uleb128 0x3f\n\t.uleb128 0x19\t# DW_AT_external DW_FORM_flag_present\n\ + \t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\ + \t.uleb128 0x6e\n\t.uleb128 0x08\t# DW_AT_linkage_name DW_FORM_string\n\ + \t.uleb128 0x3a\n\t.uleb128 0x0f\t# DW_AT_decl_file DW_FORM_udata\n\ + \t.uleb128 0x3b\n\t.uleb128 0x0f\t# DW_AT_decl_line DW_FORM_udata\n\ + \t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\ + \t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\ + \t.byte\t0, 0\n\ + \t.byte\t0\n"; + (* .debug_info. DW_LANG_C99 for [emit.ml]'s reason: it is less a claim about + the source language than the truth about the data model, and it is what + makes a debugger's own struct printing correct against this layout. *) + Buffer.add_string out + (Printf.sprintf + "\t.section\t.debug_info,\"\",@progbits\n\ + .Ldwinfo:\n\ + \t.long\t.Ldwinfo_end - .Ldwinfo_ver\n\ + .Ldwinfo_ver:\n\ + \t.short\t4\n\ + \t.long\t.Ldwabbrev\n\ + \t.byte\t8\n\ + \t.uleb128 1\n\ + \t.asciz\t\"flan (x86-64)\"\n\ + \t.short\t0x000c\t\t# DW_LANG_C99\n\ + \t.asciz\t\"%s\"\n\ + \t.asciz\t\"%s\"\n\ + \t.quad\t%s\n\ + \t.quad\t%s - %s\n\ + \t.long\t.Ldwline\n" + (asm_str (Filename.basename (abspath cufile))) + (asm_str cudir) + tbeg tend tbeg); + List.iter + (fun s -> + Buffer.add_string out + (Printf.sprintf + "\t.uleb128 2\n\t.asciz\t\"%s\"\n\t.asciz\t\"%s\"\n\ + \t.uleb128 %d\n\t.uleb128 %d\n\t.quad\t%s\n\t.quad\t%s - %s\n" + (asm_str s.sname) + (asm_str ("flan." ^ s.sname)) + s.sfile s.sline s.ssym s.send s.ssym)) + (List.rev dw.dsubs); + Buffer.add_string out "\t.byte\t0\n.Ldwinfo_end:\n"; + (* .debug_line. The standard opcode lengths are the standard ones; changing + them would change nothing, since every row below is spelled out of the + three opcodes this emitter uses and never out of a special opcode. *) + Buffer.add_string out + "\t.section\t.debug_line,\"\",@progbits\n\ + .Ldwline:\n\ + \t.long\t.Ldwline_end - .Ldwline_ver\n\ + .Ldwline_ver:\n\ + \t.short\t4\n\ + \t.long\t.Ldwline_prog - .Ldwline_hdr\n\ + .Ldwline_hdr:\n\ + \t.byte\t1\t\t# minimum_instruction_length\n\ + \t.byte\t1\t\t# maximum_operations_per_instruction\n\ + \t.byte\t1\t\t# default_is_stmt\n\ + \t.byte\t0xfb\t\t# line_base = -5\n\ + \t.byte\t14\t\t# line_range\n\ + \t.byte\t13\t\t# opcode_base\n\ + \t.byte\t0,1,1,1,1,0,0,0,1,0,0,1\n"; + List.iter + (fun (d, _) -> + Buffer.add_string out (Printf.sprintf "\t.asciz\t\"%s\"\n" (asm_str d))) + !dirs; + Buffer.add_string out "\t.byte\t0\t\t# end of the directory table\n"; + List.iter + (fun (name, dir) -> + Buffer.add_string out + (Printf.sprintf "\t.asciz\t\"%s\"\n\t.uleb128 %d\n\t.uleb128 0\n\ + \t.uleb128 0\n" + (asm_str name) dir)) + files; + Buffer.add_string out "\t.byte\t0\n.Ldwline_prog:\n"; + Buffer.add_string out (dw_line_program dw); + Buffer.add_string out ".Ldwline_end:\n"; + Buffer.contents out + (* A whole program as one assembly file. *) -let program ~checks ?(dev = false) (p : Tast.program) : string = +let program ~checks ?(dev = false) ?(debug = false) (p : Tast.program) : string = check_no_transfer p; let md = layout_ctx ~checks ~dev p in let externs = Hashtbl.create 16 in @@ -2766,28 +3197,59 @@ let program ~checks ?(dev = false) (p : Tast.program) : string = p.Tast.externs; let fns = Hashtbl.create 64 in List.iter (fun (fn : Tast.fn) -> Hashtbl.replace fns fn.Tast.name ()) p.Tast.fns; + let dw = if debug then Some (new_dwarf ()) else None in + (* The file every diagnostic in this unit is really about: the first + function anyone actually wrote. [emit.ml]'s [new_dbg] picks it the same + way and for the same reason -- the prelude contributes functions too, and + naming the prelude as the compile unit would be true and useless. *) + let cufile = + match + List.find_opt (fun (f : Tast.fn) -> f.Tast.floc.Loc.line > 0) p.Tast.fns + with + | Some f -> f.Tast.floc.Loc.file + | None -> "" + in let text = Buffer.create 65536 and rodata = Buffer.create 4096 in Buffer.add_string text "# Generated by flan's x86-64 backend (the dev one). The instructions are\n\ # .byte blobs so that every byte offset stays exactly known; the few\n\ - # fields that need a relocation are assembler expressions.\n\ - \t.text\n\n"; + # fields that need a relocation are assembler expressions.\n"; + (* A numbered [.file] is what stops clang's integrated assembler from + generating a compile unit of its *own* over this file -- one that names + the .s, and whose rows land at the [call] mnemonics, which are the only + real instructions here. Those addresses are inside the functions this + unit already describes, so two units would claim them. Measured: with the + directive, the assembler emits an empty line table and nothing else. + Harmless in a release build, where it is simply never emitted. *) + if debug then + Buffer.add_string text + (Printf.sprintf "\t.file\t1 \"%s\"\n" (asm_str (abspath cufile))); + (* The label the compile unit's range starts at, and it is emitted only in a + debug build so that a release build's assembly is byte-for-byte what it + was before any of this existed. *) + Buffer.add_string text + (if debug then "\t.text\n.Ldwtext:\n\n" else "\t.text\n\n"); List.iter (fun (fn : Tast.fn) -> - let t, r = emit_fn md ~externs ~fns fn in + let t, r = emit_fn md ~externs ~fns ?dw fn in Buffer.add_string text t; Buffer.add_string rodata r) p.Tast.fns; - let ginit, gr = emit_globals_init md ~externs ~fns p.Tast.globals in + let ginit, gr = emit_globals_init ~cfi:debug md ~externs ~fns p.Tast.globals in Buffer.add_string text ginit; Buffer.add_string rodata gr; (match List.find_opt (fun (f : Tast.fn) -> f.Tast.name = "main") p.Tast.fns with - | Some fn -> Buffer.add_string text (emit_main md fn) + | Some fn -> Buffer.add_string text (emit_main ~cfi:debug md fn) (* No [main] is not an error, and [emit.ml] treats it the same way: a program can be linked against a C host that brings its own entry point, which is what [reload_host.c] is. Refusing here made a --x86 host for the reload tests impossible to build. *) | None -> ()); + (* The end of everything this object puts in .text, and therefore the end of + the compile unit's range. It has to be written while .text is still the + current section, which is why it is here and not beside the sections + below. *) + if debug then Buffer.add_string text ".Ldwtext_end:\n"; let out = Buffer.create 65536 in Buffer.add_buffer out text; (* The globals' initialiser runs before main, through the same constructor @@ -2808,6 +3270,11 @@ let program ~checks ?(dev = false) (p : Tast.program) : string = Buffer.add_string out (emit_globals_data md p.Tast.globals); Buffer.add_string out "\n\t.section\t.rodata\n"; Buffer.add_buffer out rodata; + (match dw with + | Some d -> + Buffer.add_string out + (emit_dwarf d ~cufile ~tbeg:".Ldwtext" ~tend:".Ldwtext_end") + | None -> ()); Buffer.add_string out "\n\t.section\t.note.GNU-stack,\"\",@progbits\n"; Buffer.contents out diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 1ea61d6..8888c31 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -3201,11 +3201,12 @@ ERR@7 unexpected token: not the kind the caller was reading else print_endline "acceptance: the DWARF verifier cases skipped (no opt)"; (* A debug build and a release build must still be the same program. *) - let debug_compile ?(dev = false) path = + let debug_compile ?(dev = false) ?(x86 = false) path = let exe = Filename.concat scratch ("flan-dbg-" ^ Filename.remove_extension (Filename.basename path) - ^ if dev then "-dev" else "") + ^ (if dev then "-dev" else "") + ^ if x86 then "-x86" else "") in let l = Load.program ~file:path (Reader.read_file path) in let p = Check.program l.Load.decls in @@ -3221,7 +3222,7 @@ ERR@7 unexpected token: not the kind the caller was reading in let p, csrcs, lflags = Reach.link ~dev l p in ignore - (Build.executable ~opts:{ Build.default with debug = true; dev } + (Build.executable ~opts:{ Build.default with debug = true; dev; x86 } ~csrcs ~lflags ~pnames p ~out:exe); exe in @@ -3354,7 +3355,47 @@ ERR@7 unexpected token: not the kind the caller was reading n; print_endline text end) - [ "flan.tick"; "flan.main at debug.flan:" ] + [ "flan.tick"; "flan.main at debug.flan:" ]; + + (* And the same program through the hand-written x86-64 backend, which + emits its own DWARF rather than handing LLVM metadata. + + The claim is narrower than the one above and deliberately so: a + breakpoint resolved on a Flan name, and a backtrace whose frames name + a .flan file and a line. No [frame variable], because [x86.ml] emits + no [DW_TAG_variable] -- a slot there is a bump-allocated frame + temporary whose lifetime the backend does not model, and a name + attached to an offset something else reuses would be a lie. That is + the gap between the two backends and this is where it is recorded. + + Worth pinning rather than leaving to a handoff's transcript, because + everything this exercises is bytes [x86.ml] wrote by hand -- a line + program, a compile unit and an abbreviation table -- and a wrong byte + in any of them is silent. The program still printing the same four + lines is checked too, since debug information that breaks the build + it describes has helped nobody. *) + let exe = debug_compile ~x86:true "programs/debug.flan" in + let code, text = run exe None in + if text <> expected || code <> 0 then begin + incr failures; + Printf.printf + "FAIL an --x86 --debug build of debug.flan runs the same\n\ + \ got: %S (exit %d)\n wanted: %S\n" + text code expected + end; + let _, text = + lldb_run exe [ "breakpoint set --name flan.tick"; "run"; "bt" ] + in + List.iter + (fun n -> + if not (contains text n) then begin + incr failures; + Printf.printf + "FAIL lldb: --x86 --debug names Flan files and lines\n\ + \ wanted %S\n" n; + print_endline text + end) + [ "flan.tick"; "at debug.flan:"; "flan.main at debug.flan:" ] end else print_endline "acceptance: lldb cases skipped (no lldb on PATH)";