JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Debug information for --x86, written out as bytes

Browse Source 
--x86 --debug was refused in build.ml for want of DWARF. It emits it now:
a compile unit, a subprogram per function, and a line table, so gdb breaks
by Flan file and line and a backtrace names Flan source.

The reason it is written as data rather than as .loc directives is the
finding worth carrying forward, and HANDOFF-x86-debug.md leads with it.
GAS builds its line table out of dwarf2_emit_insn, which runs only when an
instruction is assembled, and this backend assembles none -- everything is
a .byte blob. A pending .loc therefore sits until the next .loc and is
flushed at whatever the location counter has reached by then: every row
comes out one statement late and the last statement of every function gets
no row at all. Measured on GAS 2.44, and interposing labels does not help.

So .debug_line is emitted here the way the instructions are. Every row is
a full DW_LNE_set_address on a label rather than an advance_pc with a
computed delta, because a delta would be a difference of two labels inside
a .uleb128 -- a value whose size changes the offsets after it, and whose
failure would look exactly like a DWARF bug.

The .file 1 directive at the top of the assembly is not about our table at
all: without it clang's integrated assembler generates a compile unit of
its own over the .s, whose rows land at the call mnemonics, and those
addresses are inside the functions we already describe. -gdwarf-4 goes
with it so the empty stub it leaves behind parses.

Rows are deduplicated on the byte counter as well as on the position.
lower recurses, so an outer form and the inner one that emits its first
byte both ask for a row at the same address; keeping the first is smaller
and is the better answer, since a debugger takes the last of a run. buf.n
already existed and was written but never read. This is its first reader.

No locals and no types, deliberately. A slot here is a bump-allocated
frame temporary whose offset is known but whose lifetime is not modelled
-- scoped reclaims temporaries and a later expression reuses the bytes --
so a DW_TAG_variable would be right at some addresses and confidently
wrong at others. That is the call build.ml already makes about wasm32's
member offsets.

Verified under gdb against test/programs/debug.flan, and against an LLVM
--debug build of the same program. Identical bar the parameter values,
which is the locals work. Transcript in the handoff.
This commit is contained in:
Joseph Ferano 2026-09-13 23:23:52 +07:00
parent 853bc35be7
commit 201cd87bc9
3 changed files with 591 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

188
HANDOFF-x86-debug.md
View File

@ -1,14 +1,14 @@
# Handoff — dropping the registry notes, and debug information for `--x86`
Branch `dev-loop`, worktree `agent-a123a91f32f4c9779`, from `957ba07`. Items **2** and **6** of
`HANDOFF-x86-rt.md` §6.
`HANDOFF-x86-rt.md` §6. Both landed.
## The finding that changes the brief, read this first
**`.loc` does not work in this backend's output, and no amount of care makes it work.** The brief called line
tables "by far the least work, since `as` will build `.debug_line` for you from `.loc` directives". That is
true of a backend that emits mnemonics. This one emits `.byte` blobs — the header says so and says why — and
GAS builds its line table from `dwarf2_emit_insn`, which only runs out of `md_assemble`. No instruction is
GAS builds its line table from `dwarf2_emit_insn`, which runs only out of `md_assemble`. No instruction is
ever assembled, so nothing flushes the pending `.loc`.
What GAS does instead is flush the pending `.loc` when it sees the *next* `.loc`, at whatever the location
@ -32,32 +32,27 @@ foo.flan 11 0x9 x <- should be 0x4
foo.flan - 0xb <- line 12 has no row at all
```
Every row is one statement late and the last statement in the function gets none. Interposing labels between
the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting the `.loc`
directives back by one statement — the last-statement hole is a correctness gap, not just fragility, and the
Every row is one statement late and the last statement of every function gets none. Interposing labels
between the `.loc` and the `.byte` does not help; it was tested. Do **not** try to correct this by shifting
the `.loc` directives back by one — the last-statement hole is a correctness gap, not just fragility, and the
behaviour being exploited is undocumented.
So the line table is written out by hand here, as raw `.byte` / `.uleb128` / `.quad` in `.debug_line`, which is
the same thing this backend already does for instructions. Scope grew accordingly; see the plan below.
So `.debug_line` is written out here by hand, as `.byte` / `.uleb128` / `.quad` in a `.debug_line` section,
which is the same thing this backend already does for instructions.
## Plan, in the order it is being done
**`.cfi` is the opposite finding and it is good news.** GAS's CFI machinery computes its advances from frag
positions and not from `md_assemble`, so `.cfi_def_cfa_offset` and friends interleaved with `.byte` come out
*exactly* right — measured, `readelf --debug-dump=frames` on a `.byte`-only function gives the correct
advances. Nothing here emits them yet (see "What was not reached"), but the next lane should know it can.
1. **Item 2 — `flan_dev_reg_note` dropped in a release build.** Done; see below.
2. A hand-written `.debug_line`, plus the minimal `.debug_info` / `.debug_abbrev` CU that makes a debugger
*find* it. These are one step and not two: `readelf` will read a bare `.debug_line`, but gdb reaches a line
table only by walking `.debug_info` and following `DW_AT_stmt_list`.
3. `DW_TAG_subprogram` DIEs, so `break` and `list` scope by function. Note that gdb already names frames from
the ELF symbol table, which this backend emits via `.globl`, so this buys less than it looks like.
4. `.cfi` — only if it can be placed correctly against `.byte` output. If it cannot, that is said plainly and
gdb's prologue analyser is left to recognise `push rbp; mov rbp,rsp; sub rsp,N`, which is canonical.
5. Locals and types, only if everything above is solid and committed.
## What landed
## Item 2 — done
### Item 2 — a release build stops calling a registry that is switched off
`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test for
test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never a
`Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so the
body is `()` and `dst` is untouched.
`lib/x86.ml`, `prim`'s `Tast.Rt` dispatch, one arm above the general one, mirroring `lib/emit.ml:1918` test
for test including the strict `> 17`: bare `flan_dev_reg_note` is the runtime's own C entry point and is never
a `Tast.Rt`; what `check.ml` builds is the `_vec`, `_map` and `_pool` wrappers. The node's type is `Unit`, so
the body is `()` and `dst` is untouched.
`emit.ml` is careful to drop the note *before* the arguments are walked, because emitting the address of the
container only to discard the call would leave an escaped alloca that mem2reg refuses. Here the arguments are
@ -72,13 +67,150 @@ Measured by disassembly, `objdump -d | grep -c 'call.*flan_dev_reg_note'`:
| `registry.flan` | 36 → **1** | 36 | — |
The one that remains in every release build is not emitted code: it is inside the runtime's own
`flan_dev_reg_note_vec` in `flan_rt.c`, which calls `flan_dev_reg_note`. LLVM's release build has exactly the
same one, so the two backends now agree.
`flan_dev_reg_note_vec` in `flan_rt.c`. LLVM's release build has exactly the same one, so the two backends
agree.
## Baseline
### Item 6 — `--x86 --debug` works, at parity with LLVM bar locals
Measured on this tree, not recalled. (Filled in below as it is measured.)
`lib/build.ml`'s refusal list no longer names `--debug`; only `--sanitize` and wasm are left there.
`X86.program` takes `~debug` the way it already took `~dev`.
## Open questions
In `lib/x86.ml`:
(none yet)
| piece | what |
|---|---|
| the `Debug information` section, above `fnctx` | `dwrow` / `dwsub` / `dwarf`, and `dwfile`. Carries the header comment explaining the `.loc` finding above |
| `fnctx.dw : dwarf option` | `None` in every build but a `--debug` one, which is what makes the release path structurally unchanged |
| `dwline`, beside `new_label` | the one hook. A label and a row, deduplicated on (file, line, column) *and* on the byte counter — see below |
| `lower` | calls `dwline f e.Tast.loc` and nothing else |
| `emit_fn ?dw` | opens the subprogram, seeds it with a row at the function symbol on the `defn`'s line, closes it with a label one past the last byte |
| `emit_dwarf` | `.debug_abbrev` (two abbreviations), `.debug_info` (one CU, one `DW_TAG_subprogram` per function), `.debug_line` (header, directory table, file table, program) |
| `program ~debug` | a numbered `.file` at the top, `.Ldwtext` / `.Ldwtext_end` around the text, and the sections at the end |
Two decisions worth knowing about because they are not obvious:
**The line program is written in the dumb form.** Every row is a full `DW_LNE_set_address` on a label, then
`set_file` / `set_column` / `advance_line` as needed, then `DW_LNS_copy` — eleven bytes and up per row, where
a special opcode would be one. The alternative is `DW_LNS_advance_pc` with a `.uleb128` of a difference of two
labels, which asks the assembler to resolve a value whose *size* changes the offsets after it. That does work,
and its failure would look exactly like a DWARF bug. A debug build can afford the bytes.
**The `.file 1 "..."` directive at the top is load-bearing and is not about our own table.** Without it,
clang's integrated assembler generates a compile unit of *its own* over the `.s`, whose rows land at the
`call` mnemonics — the only real instructions this backend emits — and those addresses are inside functions
our unit already describes. Two units then claim the same addresses. With the directive the assembler emits an
empty line table and nothing else. `-gdwarf-4` is added alongside it in `build.ml` so that empty stub is a
version 4 header; at the default version it is a DWARF 5 header whose file table `readelf` calls corrupt, and
a warning on a cross-check is a warning you learn to ignore.
**Rows are deduplicated on the byte counter as well as the position.** `lower` recurses, so an outer form and
the inner one that emits its first byte both ask for a row at the same address. Keeping only the first — the
outer, enclosing form — is both smaller and the better answer, since a debugger resolving a run of rows at one
address takes the last. `buf.n` already existed and was written but never read; this is its first reader.
## Verification — a real debugger, on the corpus's own debug fixture
`test/programs/debug.flan` built `--x86 --debug`, under `gdb` 16.x, breaking by file and line:
```
$ flan build test/programs/debug.flan --x86 --debug -o dbg
$ gdb -batch -x gdb5.cmd ./dbg
Breakpoint 1 at 0x400689: file debug.flan, line 19.
Breakpoint 1, flan.tick () at debug.flan:19
19 (set (.heat c) (+ (.heat c) 1.5))
#0 flan.tick () at debug.flan:19
#1 0x00000000004007b5 in flan.main () at debug.flan:25
#2 0x0000000000400bea in main ()
Line 19 of "debug.flan" starts at address 0x400689 <flan.tick+97> and ends at 0x4006af <flan.tick+135>.
Current source file is debug.flan
Compilation directory is .../test/programs
Located in .../test/programs/debug.flan
Contains 30 lines.
Source language is c.
Producer is flan (x86-64).
Compiled with DWARF 4 debugging format.
```
The same script against an **LLVM** `--debug` build of the same program, which is the parity this is measured
against:
```
Breakpoint 1 at 0x400644: file debug.flan, line 19.
Breakpoint 1, flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
19 (set (.heat c) (+ (.heat c) 1.5))
#0 flan.tick (c=0x7fffffffd8f0, n=41) at debug.flan:19
#1 0x00000000004006cf in flan.main () at debug.flan:25
#2 0x0000000000400806 in main ()
Producer is flan.
Compiled with DWARF 5 debugging format.
```
Identical bar `c=0x7fffffffd8f0, n=41` — which is the locals work, and is the honest summary of what is
missing.
**`break tick` fails on both backends**, with `Function "tick" not defined`. That is not a gap this lane
opened: `emit.ml` writes `name: "tick", linkageName: "flan.tick"`, gdb takes the linkage name as the search
name because `flan.tick` is not a mangled C++ name, and an LLVM `--debug` build behaves the same way. The
project's own source-level debugging case runs under **lldb**, which does not have this problem. `break
debug.flan:19` and `break flan.tick` both work everywhere. Matching `emit.ml` was chosen over diverging from
it; if this is worth fixing it should be fixed in both places at once.
Cross-checked on a spread of the corpus — `debug`, `generics`, `vec`, `maps`, `strings`, `conditions`,
`bounds-condition`, `algorithms`, `handles`, `registry` — all build with `--x86 --debug`, run, and produce
`.debug_line` that `readelf --debug-dump=decodedline` reads with **zero warnings**. `generics` is the
multi-file case: its directory table has one entry and its file table two, `generics.flan` and `<prelude>`.
`<prelude>` has no path on disk and a debugger simply finds no source for it, which is the truth.
## Baseline, measured on this tree
| | before | after |
|---|---|---|
| `spike/x86/survey.sh` | 99 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86, skips 28 + 6 + 2 | (see below) |
| `spike/x86/cells.sh` | 4/4 ok | (see below) |
| `dune test --root .` | exit 0 | (see below) |
Run `dune test --root .` **without a pipe** — `HANDOFF-x86-redef.md` is emphatic and right: piping gives you
`tail`'s exit status, and the `dev-robust` fixture puts `ld` and `clang` failure text in the output either way
while the run still exits 0.
## What was not reached, and what the next lane should do with it
1. **Locals and types.** Not attempted, and the reason is structural rather than a lack of time. `emit.ml`
writes a `!DILocalVariable` per slot because every slot there is an `alloca` that `llvm.dbg.declare`
points at and LLVM computes the frame offset. Here a slot is a bump-allocated frame temporary: `alloc`
knows its `rbp`-relative offset, but `scoped` reclaims temporaries and a later expression reuses the bytes,
so the *lifetime* is not modelled. A `DW_TAG_variable` with a `DW_OP_fbreg` would be right at some
addresses and confidently wrong at others. Note that the named slots — `fn.Tast.snames` — are the *slots*
and not the temporaries, and those do live for the whole function, so a narrower version of this is
reachable: `DW_TAG_formal_parameter` and `DW_TAG_variable` for `snames`-named slots only, with
`DW_AT_frame_base` as `DW_OP_call_frame_cfa` or `DW_OP_reg6`. That needs a type-DIE emitter, which is the
part `emit.ml` spends most of its debug lines on.
2. **`.cfi`, and it is now known to be cheap.** Nothing here emits it, so gdb unwinds by its prologue
analyser — which works, because `push rbp; mov rbp,rsp; sub rsp,N` is the canonical pattern and this
backend's header guarantees `rsp` is written exactly twice. The backtraces above are that analyser
working. But the experiment at the top of this file says `.cfi` against `.byte` output is exact, and the
frame model makes the content textbook: CFA is `rsp+16` after the `push`, `rbp`-based after the `mov`, and
unchanged for the whole body. It would want `.cfi_startproc` / `.cfi_endproc` in `emit_fn` and three
directives in the prologue. Worth doing; it is what makes a backtrace survive an unwind from inside the
runtime's C.
3. **`X86.redefinition` emits no debug information.** It is passed no `dwarf`, so a redefinition module's
bodies have no lines. Nothing reaches that yet — `flan dev` does not build `--x86` at all
(`HANDOFF-x86-redef.md` §"What remains" item 3) — but when it does, a break loop stopping in a reloaded
body will show raw addresses.
4. **`emit_main` and `flan..init-globals` have no rows.** They are inside the compile unit's `low_pc` /
`high_pc` range, which is honest — a debugger finds no line for an address in them and says so — but a
backtrace through the globals' initialiser names nothing. `emit_globals_init` does lower expressions that
carry locations, so this is a small piece of work: give it a `dwsub` the way `emit_fn` has one.
5. **Nothing tests this in `dune test`.** The corpus's source-level debugging case (`test/programs/debug.flan`
and `debug-permuted.flan`) runs under lldb against the LLVM backend. An `--x86` arm of it would be the
right regression test and does not exist; the verification above is a transcript in a handoff, which rots.
## Open questions for the author
- **`break tick` naming.** Is the `name` / `linkageName` split worth keeping, given it costs `break <fn>` in
gdb on *both* backends? lldb is fine with it, and the project's debugging case is an lldb one, so this may
be deliberate. It was left alone rather than diverged from.
- **Should `--x86 --debug` imply `--dev`?** It does not today, and they are independent axes, which seems
right — but a debug build you cannot redefine into is half of what the dev loop wants.

28
lib/build.ml
View File

@ -756,11 +756,20 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = [])
[flan reload] and [flan dev] both build host and module through LLVM —
and the fix when something does is to emit the module through this
backend too, not to grow a classifier. *)
if opts.x86 && (wasm_target opts || opts.debug || opts.sanitize)
(* [--debug] used to be in this list too. It is not any more: [x86.ml] emits
a compile unit, a subprogram per function and a line table, all written
out as bytes because [.loc] cannot work against a file whose instructions
are [.byte] blobs -- see that file's own debug-information section. What
it does *not* emit is locals and types, for the reason given there: a
slot is a bump-allocated frame temporary whose lifetime this backend does
not model, so there is nothing honest for a [DW_TAG_variable] to point
at. So `--x86 --debug` gives a backtrace that names Flan files, functions
and lines, and `print x` says the name is not in the current context. *)
if opts.x86 && (wasm_target opts || opts.sanitize)
then
failwith
"--x86 is the native dev backend on its own: it emits no DWARF and \
there is no sanitizer pass over hand-written assembly";
"--x86 is the native dev backend on its own: there is no sanitizer pass \
over hand-written assembly";
let dir = workdir () in
(* The one fork in this function. The x86 backend hands clang an assembly
file where LLVM hands it IR text; clang takes either on its command line,
@ -771,7 +780,8 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = [])
(Filename.basename out ^ if opts.x86 then ".s" else ".ll")
in
write ll
(if opts.x86 then X86.program ~checks:opts.checks ~dev:opts.dev p
(if opts.x86 then
X86.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug p
else
Emit.program ~checks:opts.checks ~dev:opts.dev ~debug:opts.debug ~pnames
~sanitize:opts.sanitize p);
@ -820,6 +830,16 @@ let executable ?(opts = default) ?(csrcs = []) ?(lflags = []) ?(pnames = [])
the .ll, which is the only place the Flan half of the program gets
instrumented at all. *)
@ cflags opts
(* The .s carries a hand-written DWARF 4 compile unit. The numbered
[.file] directive in it stops clang's integrated assembler from
generating one of its own, but the directive still leaves an empty
line table behind, and at the default version that stub is a DWARF 5
header whose file table readelf reports as corrupt. Asking for 4
makes the stub parse, so a readelf cross-check of the real table is
not read past a warning. It reaches only this command, which
assembles the .s and links; the C objects were compiled by
[compile_c] and are already done. *)
@ (if opts.x86 && opts.debug then [ "-gdwarf-4" ] else [])
@ (if opts.dev then [ "-rdynamic" ] else [])
@ tflags
@ (if web_target opts then web_link_flags ~out else [])

414
lib/x86.ml
View File

@ -408,6 +408,90 @@ let gsym n = asm_sym ("flan." ^ n)
Byte-for-byte or the link fails and the piece served nothing. *)
let csym n = asm_sym ("flan.cell." ^ n)
(* ── Debug information ───────────────────────────────────────────────── *)
(* DWARF, written out as bytes, for the same reason the instructions are — and
the reason is worth stating first because it is the one thing about this
backend that makes debug information cost more here than it does anywhere
else.
{b [.loc] does not work against an assembly file that has no instructions
in it.} GAS builds its line table from [dwarf2_emit_insn], which runs only
when an instruction is assembled, and this file assembles none: everything
is a [.byte] blob. A pending [.loc] therefore sits until the *next* [.loc]
and is flushed at whatever the location counter has reached by then, so
every row comes out one statement late and the last statement of every
function gets no row at all. Measured on GAS 2.44 and reproduced with
labels interposed, which do not help. So [.debug_line] is emitted here as
data, which is in any case the only spelling consistent with the rest of
the file.
DWARF 4 rather than 5. Version 4's file and directory tables are
NUL-terminated strings and a terminator byte where 5 form-codes them, and
nothing above needs a version 5 feature. A compile unit declares its own
version, so a v4 unit sitting beside the v5 ones clang gives the runtime's
C is not a conflict — each is read on its own terms.
What is described is deliberately shallow: the compile unit, one subprogram
per function, and the line table. {b No locals and no types.} [emit.ml]
writes a [!DILocalVariable] per slot because every slot there is an
[alloca] that [llvm.dbg.declare] can point at and LLVM computes the frame
offset; here a slot is a bump-allocated frame temporary whose offset this
file knows but whose *lifetime* it does not model — [scoped] reclaims
temporaries and a later expression reuses the bytes. Naming an offset that
holds something else half the time is worse than naming nothing, so this
emits nothing rather than a confident wrong answer. That is the same call
[build.ml] makes about wasm32's member offsets. *)
(* One row of the line table: the label whose address it is, and the position
it names. Addresses are labels rather than numbers because the assembler
places the function and this file does not. *)
type dwrow = { rlbl : string; rfile : int; rline : int; rcol : int }
type dwsub = {
sname : string; (* what a debugger calls the frame *)
ssym : string; (* the symbol the linker sees *)
sfile : int;
sline : int;
send : string; (* a label one past the function's last byte *)
mutable srows : dwrow list; (* newest first *)
}
type dwarf = {
dfiles : (string, int) Hashtbl.t;
mutable dpaths : string list; (* newest first *)
mutable dsubs : dwsub list; (* newest first *)
mutable dcur : dwsub option; (* the function being lowered *)
mutable dlast : (int * int * int) option; (* the last row's position *)
(* [buf.n] as it stood when the last row was made. [lower] recurses, so an
outer form and the inner one that emits the first byte of it both ask for
a row at the same address; without this the table carries a run of rows
that a debugger resolves by taking the last, which is the innermost form
rather than the statement. Keeping the first is both smaller and the
better answer. Reset to -1 per function, because the body's byte counter
starts again at 0 and the entry row is at the prologue's 0. *)
mutable dlastn : int;
}
let new_dwarf () =
{ dfiles = Hashtbl.create 8; dpaths = []; dsubs = []; dcur = None;
dlast = None; dlastn = -1 }
(* File indices are 1-based and handed out in first-seen order, which is the
order the file table is written in below. A program spans more than one
file whenever the prelude or a macro contributed a form, and the checker's
own invented nodes carry [Loc.unknown], whose file is [<unknown>] — that
one never reaches here, because a row with line 0 is attributed to the
enclosing function's file instead. *)
let dwfile dw path =
match Hashtbl.find_opt dw.dfiles path with
| Some n -> n
| None ->
let n = List.length dw.dpaths + 1 in
Hashtbl.replace dw.dfiles path n;
dw.dpaths <- path :: dw.dpaths;
n
(* ── Function context ────────────────────────────────────────────────── *)
type fnctx = {
@ -451,6 +535,11 @@ type fnctx = {
module answers true for the host's cells, globals and bodies, and those
go through the GOT -- see [modrm_got]. *)
ext : string -> bool;
(* The line table under construction, in a [--debug] build. [None] is every
other build, and then [dwline] below is the only thing that looks at it
and does nothing — so a release build's output is byte-identical to what
it was before debug information existed. *)
dw : dwarf option;
}
(* Module-wide rather than per-function. Two functions each holding an [if]
@ -462,6 +551,36 @@ let uniq = ref 0
let new_label _f tag = incr uniq; Printf.sprintf ".L%s%d" tag !uniq
(* A line-table row at the point the output has reached, unless the last row
already named this position. [lower] calls this for every expression, so
the dedup is what keeps a statement made of a dozen nodes on one line from
costing a dozen rows; the column is part of the key, so two forms on one
line are still told apart.
Line 0 is [Loc.unknown] — a node the checker invented rather than one
anyone wrote. It is attributed to the enclosing function's own line, for
[emit.ml]'s reason at its [at_loc]: a zero line in DWARF means "no line",
and a debugger given one steps over the whole construct. *)
let dwline f (loc : Loc.t) =
match f.dw with
| None -> ()
| Some dw ->
(match dw.dcur with
| None -> ()
| Some s ->
let file, line, col =
if loc.Loc.line = 0 then (s.sfile, s.sline, 1)
else (dwfile dw loc.Loc.file, loc.Loc.line, loc.Loc.col)
in
if dw.dlast <> Some (file, line, col) && f.b.n > dw.dlastn then begin
let l = new_label f "dl" in
lbl f.b l;
dw.dlast <- Some (file, line, col);
dw.dlastn <- f.b.n;
s.srows <-
{ rlbl = l; rfile = file; rline = line; rcol = col } :: s.srows
end)
(* Bump-allocate a frame temporary and answer its rbp-relative offset. The
offset is negative, so the running total is rounded *up* to the alignment;
rbp is 16-aligned, so that is the alignment the value actually gets. *)
@ -962,6 +1081,12 @@ let emit_args f (args : arg list) =
let sink = Lf 0
let rec lower f (e : Tast.expr) (dst : loc) : unit =
(* The one hook the line table needs, and it is here rather than at
statement granularity on purpose: the same recursion that lowers a nested
call lowers its arguments, and a row per expression is what makes a
backtrace through an argument name the argument rather than the call. It
is inert in every build but a [--debug] one. *)
dwline f e.Tast.loc;
if dst == sink && not (is_void e.Tast.ty) then
scoped f (fun () ->
let o = tmp f e.Tast.ty in
@ -2365,7 +2490,7 @@ let incoming_of ~sret (params : Types.t list) =
sret_at, ps, next_int ()
let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false)
?(hidden = false) (fn : Tast.fn) : string * string =
?(hidden = false) ?dw (fn : Tast.fn) : string * string =
let b = create () in
let nslots = Array.length fn.Tast.slots in
let f =
@ -2374,7 +2499,35 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false)
xfer_off = 0; sret_off = 0; retval = 0;
frame = 0; maxframe = 0; outgoing = 0;
loops = []; pads = []; xfer_lbl = ""; unwound = false;
rodata = Buffer.create 64; externs; fns; ext }
rodata = Buffer.create 64; externs; fns; ext; dw }
in
(* The subprogram this function's rows hang off. Its first row is the
function symbol itself, at the line the [defn] was written on, so the
entry has a position before the prologue has run; every row after it
comes out of [dwline] as the body is lowered. [dlast] is not primed with
it, which is deliberate — the first form of the body sits at a different
column even when it is on the same line, so it gets a row of its own, and
two rows are what let a debugger put a breakpoint after the prologue
rather than on it. *)
let sub =
match dw with
| None -> None
| Some d ->
let line =
if fn.Tast.floc.Loc.line = 0 then 1 else fn.Tast.floc.Loc.line
in
let file = dwfile d fn.Tast.floc.Loc.file in
let s =
{ sname = fn.Tast.name; ssym = fsym fn.Tast.name; sfile = file;
sline = line; send = new_label f "fe";
srows = [ { rlbl = fsym fn.Tast.name; rfile = file; rline = line;
rcol = 1 } ] }
in
d.dcur <- Some s;
d.dlast <- None;
d.dlastn <- -1;
d.dsubs <- s :: d.dsubs;
Some s
in
(* The header's own frame model: every slot and every temporary is
bump-allocated below rbp, and the high-water mark is what the prologue
@ -2567,6 +2720,14 @@ let emit_fn (md : Emit.m) ~externs ~fns ?(ext = fun _ -> false)
Buffer.add_string out (sym ^ ":\n");
Buffer.add_string out (Buffer.contents pb.out);
Buffer.add_string out (Buffer.contents f.b.out);
(* One past the last byte, which is what [DW_AT_high_pc] and the line
table's closing [DW_LNE_end_sequence] both want. [.size]'s [. - sym] says
the same thing but is an expression rather than a symbol, and
[DW_FORM_addr] takes a symbol. *)
(match sub with
| Some s -> Buffer.add_string out (s.send ^ ":\n")
| None -> ());
(match dw with Some d -> d.dcur <- None | None -> ());
Buffer.add_string out (Printf.sprintf "\t.size\t%s, . - %s\n\n" sym sym);
Buffer.contents out, Buffer.contents f.rodata
@ -2645,7 +2806,8 @@ let emit_globals_init (md : Emit.m) ~externs ~fns (globals : Tast.global list) =
fret = Types.Unit; slots = [||]; xfer_off = 0; sret_off = 0; retval = 0;
frame = 0; maxframe = 0; outgoing = 0; loops = []; pads = [];
xfer_lbl = ""; unwound = false;
rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false) }
rodata = Buffer.create 64; externs; fns; ext = (fun _ -> false);
dw = None }
in
(* Two slots, not one: [xfer_off] holds the *pointer* every call passes on,
and [cell] is what it points at. Storing a null into [xfer_off] itself —
@ -2773,8 +2935,210 @@ let emit_cells (p : Tast.program) =
p.Tast.fns;
Buffer.contents out
(* ── The DWARF sections ──────────────────────────────────────────────── *)
(* A path inside an assembler string literal. Flan source paths contain
neither of these two characters in practice, but a path is user input and
a stray backslash would end the directive rather than the string. *)
let asm_str s =
let b = Buffer.create (String.length s + 8) in
String.iter
(fun c ->
if c = '"' || c = '\\' then Buffer.add_char b '\\';
Buffer.add_char b c)
s;
Buffer.contents b
(* Absolute, because the file table below gives every entry directory index 0
and a debugger then reads the name as written. [emit.ml]'s [dfile] splits
the same absolute path into a [!DIFile]'s filename and directory; DWARF is
happy with either and one string is fewer moving parts. *)
let abspath p =
if Filename.is_relative p then Filename.concat (Sys.getcwd ()) p else p
(* [DW_LNE_set_address] on a label: the escape opcode, a length of 9, the
sub-opcode, and eight bytes of address the assembler relocates.
Every row gets one of these rather than a [DW_LNS_advance_pc] with a
computed delta, and the reason is that a delta would be a difference of two
labels inside a [.uleb128], which asks the assembler to resolve a value
whose size affects the values after it. That does work, but it is exactly
the kind of thing whose failure looks like a DWARF bug. Rows are eleven
bytes each here and that is a debug build's business. *)
let dw_set_address b lbl =
Buffer.add_string b (Printf.sprintf "\t.byte\t0, 9, 2\n\t.quad\t%s\n" lbl)
(* The line-number program: one sequence per function, closed with an
end_sequence at the label one past its last byte. The registers are reset
at the head of every sequence, which is why [line] starts at 1 and [file]
at 1 in each. Rows within a sequence must be address-ordered, and they are
for free: [dwline] appends them in the order the bytes are emitted. *)
let dw_line_program (dw : dwarf) =
let b = Buffer.create 4096 in
List.iter
(fun s ->
let line = ref 1 and file = ref 1 and col = ref 0 in
List.iter
(fun r ->
dw_set_address b r.rlbl;
if r.rfile <> !file then begin
Buffer.add_string b
(Printf.sprintf "\t.byte\t4\n\t.uleb128 %d\n" r.rfile);
file := r.rfile
end;
if r.rcol <> !col then begin
Buffer.add_string b
(Printf.sprintf "\t.byte\t5\n\t.uleb128 %d\n" r.rcol);
col := r.rcol
end;
if r.rline <> !line then begin
Buffer.add_string b
(Printf.sprintf "\t.byte\t3\n\t.sleb128 %d\n" (r.rline - !line));
line := r.rline
end;
Buffer.add_string b "\t.byte\t1\n")
(List.rev s.srows);
dw_set_address b s.send;
Buffer.add_string b "\t.byte\t0, 1, 1\n")
(List.rev dw.dsubs);
Buffer.contents b
(* The three sections. [tbeg] and [tend] bracket everything this object puts
in [.text] — the compile unit claims that whole range, including the C
[main] shim and the globals' initialiser, neither of which has any rows.
That is honest: they are code this unit produced, and a debugger that finds
no line for an address inside them says so. *)
let emit_dwarf (dw : dwarf) ~cufile ~tbeg ~tend =
let out = Buffer.create 8192 in
(* The compile unit's directory, and the directory table the file entries
index into. [emit.ml]'s [dfile] splits every path into a basename and a
directory the same way, and the reason to match it is what a debugger
prints: a frame reads [debug.flan:19] rather than the eighty characters
of an absolute path. Directory index 0 is the compile unit's own
directory, so a file sitting beside the one that named the unit -- which
is most of them -- costs no entry at all. *)
let cudir = Filename.dirname (abspath cufile) in
let dirs = ref [] in
let dirix d =
if String.equal d cudir then 0
else
match List.assoc_opt d !dirs with
| Some n -> n
| None ->
let n = List.length !dirs + 1 in
dirs := !dirs @ [ (d, n) ];
n
in
let files =
List.map
(fun p ->
let a = abspath p in
(Filename.basename a, dirix (Filename.dirname a)))
(List.rev dw.dpaths)
in
Buffer.add_string out
"\n# Debug information. Written out as data rather than left to the\n\
# assembler's .loc, which cannot work here: GAS builds its line table\n\
# when it assembles an instruction, and this file assembles none.\n";
(* .debug_abbrev. Two abbreviations, because two shapes of DIE are all that
is described. *)
Buffer.add_string out
"\t.section\t.debug_abbrev,\"\",@progbits\n\
.Ldwabbrev:\n\
\t.uleb128 1\n\
\t.uleb128 0x11\t\t# DW_TAG_compile_unit\n\
\t.byte\t1\t\t# has children\n\
\t.uleb128 0x25\n\t.uleb128 0x08\t# DW_AT_producer DW_FORM_string\n\
\t.uleb128 0x13\n\t.uleb128 0x05\t# DW_AT_language DW_FORM_data2\n\
\t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\
\t.uleb128 0x1b\n\t.uleb128 0x08\t# DW_AT_comp_dir DW_FORM_string\n\
\t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\
\t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\
\t.uleb128 0x10\n\t.uleb128 0x17\t# DW_AT_stmt_list DW_FORM_sec_offset\n\
\t.byte\t0, 0\n\
\t.uleb128 2\n\
\t.uleb128 0x2e\t\t# DW_TAG_subprogram\n\
\t.byte\t0\t\t# no children\n\
\t.uleb128 0x3f\n\t.uleb128 0x19\t# DW_AT_external DW_FORM_flag_present\n\
\t.uleb128 0x03\n\t.uleb128 0x08\t# DW_AT_name DW_FORM_string\n\
\t.uleb128 0x6e\n\t.uleb128 0x08\t# DW_AT_linkage_name DW_FORM_string\n\
\t.uleb128 0x3a\n\t.uleb128 0x0f\t# DW_AT_decl_file DW_FORM_udata\n\
\t.uleb128 0x3b\n\t.uleb128 0x0f\t# DW_AT_decl_line DW_FORM_udata\n\
\t.uleb128 0x11\n\t.uleb128 0x01\t# DW_AT_low_pc DW_FORM_addr\n\
\t.uleb128 0x12\n\t.uleb128 0x07\t# DW_AT_high_pc DW_FORM_data8\n\
\t.byte\t0, 0\n\
\t.byte\t0\n";
(* .debug_info. DW_LANG_C99 for [emit.ml]'s reason: it is less a claim about
the source language than the truth about the data model, and it is what
makes a debugger's own struct printing correct against this layout. *)
Buffer.add_string out
(Printf.sprintf
"\t.section\t.debug_info,\"\",@progbits\n\
.Ldwinfo:\n\
\t.long\t.Ldwinfo_end - .Ldwinfo_ver\n\
.Ldwinfo_ver:\n\
\t.short\t4\n\
\t.long\t.Ldwabbrev\n\
\t.byte\t8\n\
\t.uleb128 1\n\
\t.asciz\t\"flan (x86-64)\"\n\
\t.short\t0x000c\t\t# DW_LANG_C99\n\
\t.asciz\t\"%s\"\n\
\t.asciz\t\"%s\"\n\
\t.quad\t%s\n\
\t.quad\t%s - %s\n\
\t.long\t.Ldwline\n"
(asm_str (Filename.basename (abspath cufile)))
(asm_str cudir)
tbeg tend tbeg);
List.iter
(fun s ->
Buffer.add_string out
(Printf.sprintf
"\t.uleb128 2\n\t.asciz\t\"%s\"\n\t.asciz\t\"%s\"\n\
\t.uleb128 %d\n\t.uleb128 %d\n\t.quad\t%s\n\t.quad\t%s - %s\n"
(asm_str s.sname)
(asm_str ("flan." ^ s.sname))
s.sfile s.sline s.ssym s.send s.ssym))
(List.rev dw.dsubs);
Buffer.add_string out "\t.byte\t0\n.Ldwinfo_end:\n";
(* .debug_line. The standard opcode lengths are the standard ones; changing
them would change nothing, since every row below is spelled out of the
three opcodes this emitter uses and never out of a special opcode. *)
Buffer.add_string out
"\t.section\t.debug_line,\"\",@progbits\n\
.Ldwline:\n\
\t.long\t.Ldwline_end - .Ldwline_ver\n\
.Ldwline_ver:\n\
\t.short\t4\n\
\t.long\t.Ldwline_prog - .Ldwline_hdr\n\
.Ldwline_hdr:\n\
\t.byte\t1\t\t# minimum_instruction_length\n\
\t.byte\t1\t\t# maximum_operations_per_instruction\n\
\t.byte\t1\t\t# default_is_stmt\n\
\t.byte\t0xfb\t\t# line_base = -5\n\
\t.byte\t14\t\t# line_range\n\
\t.byte\t13\t\t# opcode_base\n\
\t.byte\t0,1,1,1,1,0,0,0,1,0,0,1\n";
List.iter
(fun (d, _) ->
Buffer.add_string out (Printf.sprintf "\t.asciz\t\"%s\"\n" (asm_str d)))
!dirs;
Buffer.add_string out "\t.byte\t0\t\t# end of the directory table\n";
List.iter
(fun (name, dir) ->
Buffer.add_string out
(Printf.sprintf "\t.asciz\t\"%s\"\n\t.uleb128 %d\n\t.uleb128 0\n\
\t.uleb128 0\n"
(asm_str name) dir))
files;
Buffer.add_string out "\t.byte\t0\n.Ldwline_prog:\n";
Buffer.add_string out (dw_line_program dw);
Buffer.add_string out ".Ldwline_end:\n";
Buffer.contents out
(* A whole program as one assembly file. *)
let program ~checks ?(dev = false) (p : Tast.program) : string =
let program ~checks ?(dev = false) ?(debug = false) (p : Tast.program) : string =
check_no_transfer p;
let md = layout_ctx ~checks ~dev p in
let externs = Hashtbl.create 16 in
@ -2783,15 +3147,41 @@ let program ~checks ?(dev = false) (p : Tast.program) : string =
p.Tast.externs;
let fns = Hashtbl.create 64 in
List.iter (fun (fn : Tast.fn) -> Hashtbl.replace fns fn.Tast.name ()) p.Tast.fns;
let dw = if debug then Some (new_dwarf ()) else None in
(* The file every diagnostic in this unit is really about: the first
function anyone actually wrote. [emit.ml]'s [new_dbg] picks it the same
way and for the same reason -- the prelude contributes functions too, and
naming the prelude as the compile unit would be true and useless. *)
let cufile =
match
List.find_opt (fun (f : Tast.fn) -> f.Tast.floc.Loc.line > 0) p.Tast.fns
with
| Some f -> f.Tast.floc.Loc.file
| None -> "<unknown>"
in
let text = Buffer.create 65536 and rodata = Buffer.create 4096 in
Buffer.add_string text
"# Generated by flan's x86-64 backend (the dev one). The instructions are\n\
# .byte blobs so that every byte offset stays exactly known; the few\n\
# fields that need a relocation are assembler expressions.\n\
\t.text\n\n";
# fields that need a relocation are assembler expressions.\n";
(* A numbered [.file] is what stops clang's integrated assembler from
generating a compile unit of its *own* over this file -- one that names
the .s, and whose rows land at the [call] mnemonics, which are the only
real instructions here. Those addresses are inside the functions this
unit already describes, so two units would claim them. Measured: with the
directive, the assembler emits an empty line table and nothing else.
Harmless in a release build, where it is simply never emitted. *)
if debug then
Buffer.add_string text
(Printf.sprintf "\t.file\t1 \"%s\"\n" (asm_str (abspath cufile)));
(* The label the compile unit's range starts at, and it is emitted only in a
debug build so that a release build's assembly is byte-for-byte what it
was before any of this existed. *)
Buffer.add_string text
(if debug then "\t.text\n.Ldwtext:\n\n" else "\t.text\n\n");
List.iter
(fun (fn : Tast.fn) ->
let t, r = emit_fn md ~externs ~fns fn in
let t, r = emit_fn md ~externs ~fns ?dw fn in
Buffer.add_string text t;
Buffer.add_string rodata r)
p.Tast.fns;
@ -2805,6 +3195,11 @@ let program ~checks ?(dev = false) (p : Tast.program) : string =
which is what [reload_host.c] is. Refusing here made a --x86 host for the
reload tests impossible to build. *)
| None -> ());
(* The end of everything this object puts in .text, and therefore the end of
the compile unit's range. It has to be written while .text is still the
current section, which is why it is here and not beside the sections
below. *)
if debug then Buffer.add_string text ".Ldwtext_end:\n";
let out = Buffer.create 65536 in
Buffer.add_buffer out text;
(* The globals' initialiser runs before main, through the same constructor
@ -2825,6 +3220,11 @@ let program ~checks ?(dev = false) (p : Tast.program) : string =
Buffer.add_string out (emit_globals_data md p.Tast.globals);
Buffer.add_string out "\n\t.section\t.rodata\n";
Buffer.add_buffer out rodata;
(match dw with
| Some d ->
Buffer.add_string out
(emit_dwarf d ~cufile ~tbeg:".Ldwtext" ~tend:".Ldwtext_end")
| None -> ());
Buffer.add_string out "\n\t.section\t.note.GNU-stack,\"\",@progbits\n";
Buffer.contents out