From 391fc62cc817303ebb1ace2f892e104ada441615 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Mon, 14 Sep 2026 10:40:56 +0700 Subject: [PATCH] The handoff, with both backends measured through a real daemon --- docs/handoffs/HANDOFF-x86-devloop.md | 249 ++++++++++++++++++++++++--- 1 file changed, 226 insertions(+), 23 deletions(-) diff --git a/docs/handoffs/HANDOFF-x86-devloop.md b/docs/handoffs/HANDOFF-x86-devloop.md index 04ba70b..22739fe 100644 --- a/docs/handoffs/HANDOFF-x86-devloop.md +++ b/docs/handoffs/HANDOFF-x86-devloop.md @@ -1,36 +1,239 @@ # Handoff — the x86 backend, wired to the dev loop -Branch `dev-loop`, worktree `agent-aaf84e55296df121c`, from `f459352`. This is items 1, 2 and 3 of +Branch `dev-loop`, worktree `agent-aaf84e55296df121c`, from `f459352`. Items 1, 2 and 3 of `HANDOFF-x86-redef.md`'s "What remains": the new-name path, the transient thunk, and the wiring that lets `flan dev` choose the backend for a host and its modules together. -**Status: in progress.** This file is written before the work rather than after it, so that a session that ends -early leaves its plan behind rather than nothing. +**All three landed, and the numbers say the lane was worth it.** A `C-c C-c` round trip through a real daemon +is **62–66ms on LLVM and 27–30ms on `--x86`** on this machine; `C-x C-e` is **60ms and 24ms**. The build inside +those is **53ms and 17–18ms**. Details and caveats in §Measured. -## Why +There is one finding that is not good news and it is in §The merged daemon: `flan dev --x86` refuses the +*merged* daemon and needs `--two-process`, because the compiler's macro module is a third way the two backends +can meet in one process and `flan.abi.x86` does not guard it. -`llc` is 15–17ms of a ~19ms `C-c C-c` (`docs/BUILT.md`). The hand-written backend exists to replace that -number and has never been given the chance: `--x86` is read only by `flan build`'s argument list, and the -daemon builds both halves through LLVM. Until it is wired, the backend saves nothing. +## What was built -## The plan, in the order it will be committed +| file | what | +|---|---| +| `lib/x86.ml` — `loc`'s new `Lslot`, `fnctx.slot` | a name the host was never built with: the address lives in a module-local slot the installer fills by string. `Lgot`'s shape with `Sym` where it has `Got` | +| `lib/x86.ml` — `redefinition`'s installer | `flan_dev_cell` / `flan_dev_global` lookups, an init image per new global, the `consts` republish, and a real frame so those calls are made on an aligned stack | +| `lib/x86.ml` — `flan_reload_call`, `flan_reload_transient` | what an expression evaluation compiles to, and the claim the agent unloads on | +| `lib/x86.ml` — `string_const` | now bumps `Emit.m.nstr`, which is the third transient condition | +| `lib/session.ml` — `t.x86`, `change.x86`, `redefinition` | the backend choice, as a session setting, made in one place | +| `lib/dev.ml` — `build_module`, `module_ext`, `Dev.start ?x86` | the daemon picking builder and extension off the change, and the two refusals | +| `lib/dev.ml` — `rename_program_main_asm`, `merged_executable`'s fork | an `--x86` merged host, which builds and is currently refused (see below) | +| `bin/main.ml` | `flan dev --x86`, `flan reload --x86`, and a bare `Failure` printed as a sentence | +| `test/programs/reload-v6.flan` | a run-time-new global with a value of its own | +| `test/test_reload.ml` | the x86 section runs all four modules against the LLVM transcript; the refusal it asserted is gone | +| `test/test_dev.ml` | an `--x86` daemon driven through `C-c C-c`, `C-x C-e`, a literal, a new `defvar` and a new `defn`; plus the merged refusal | -1. **The new-name path** in `X86.redefinition`. A `slot` predicate beside `fnctx.ext`, a fourth `loc` case - for a module-local slot, `flan_dev_cell` / `flan_dev_global` calls in `flan_reload_install`, and an init - image for a new global. `test/test_reload.ml`'s v3/v4/v5 fixtures already exist and currently assert a - refusal; the transcript replaces it. -2. **The transient thunk**: `flan_reload_call` and the `@flan_reload_transient` marker, with an x86 analogue - of `emit.ml`'s `m.nstr` condition — a module that holds a string literal can never claim to be transient, - because the string outlives the call and the agent unloads the mapping. -3. **The wiring**: an `x86` field on `Session.t` and on `Session.change`, set once in `Dev.start` from - `flan dev --x86`, exactly parallel to `debug` — whose own comment already says why both ends have to come - from one flag. Never a fallback: if `X86.redefinition` refuses, the daemon reports the refusal. A silent - fallback to LLVM is precisely the crossed pair `flan.abi.x86` exists to refuse. +## 1. The new-name path -## Decisions, recorded as they are made +A function or a `defvar` the running process has no symbol for. ELF cannot grow one, so the address is asked +for by string at install time and parked in a slot this module defines — `flan.cellp.` for a function's +cell, `flan.gp.` for a global's storage, spelled as `Emit.cellptr` and `Emit.globalptr` spell them so the +two sides read against each other. -(to be filled in) +The reference side is **one new `loc` case and nothing else**. `Lslot` loads the slot and answers +`Reg (scratch, d)`, which is exactly what `Lgot` already did with `Got` where this has `Sym`. Every site that +reaches a cell already double-loads, so no call site, no place expression and no `sym_loc` caller had to learn +a third case. `fnctx.slot` is a *second* predicate rather than a widened `ext` because the two answer different +questions: `ext` says "the host's, reach it through the GOT", `slot` says "nobody's yet, reach it through a +slot I filled". It defaults to `fun _ -> None`, so the whole-program path emits byte-identical output and +`survey.sh` goes on being a structural check on all of this. -## Measurements +Three things that would each have cost a session: -(to be filled in: emit / assemble / link / dlopen, both backends) +- **`flan_reload_install` had to become a function with a frame.** It was a run of loads and stores plus `ret`, + which was fine while it called nothing. The moment it calls `flan_dev_cell`, rsp is 8-mod-16 at the call and + glibc's `movaps` inside `strcmp` faults — a failure that reads as a backend bug and is an ABI bug. Its shape + is now `emit_globals_init`'s, down to owning the null transfer cell no caller hands it. +- **A new global's declared value has to travel with it.** `flan_dev_global` copies it onto the allocation the + first time the name is interned and ignores it after, which is where "a reload must not reset the program's + state" lives. `emit.ml` folds that value into an LLVM constant and this file has no folder, so the image is a + module-local `.bss` buffer written by the initialiser *lowered as ordinary code* — which is the bargain + `emit_globals_data`'s comment already makes for a whole program, and it needs no second evaluator that could + disagree with the first about what a struct literal means. +- **Order inside the installer is load-bearing and is not checkable from outside.** Every lookup resolves + before any body is published; publishing first exposes a function whose slots are still null to every call + site in the host. `emit.ml` says the same and `test_reload.ml` checks it *there* by grepping the IR text. + There is no text to grep on this side, so the guarantee is the loop order and the comment above it. + +`reload-v6.flan` is new and is the reason the fixture set was not enough. v3's `extra` is declared zero, which +`calloc` also gives, so a run-time-new global whose image never arrived would still have passed. v6's `tuning` +is 42 and the host prints 88. + +Republishing a `defconst` came free once the rest was there — one store of the new constant into the host's +global, which is what `emit.ml` does — so that refusal is gone too. + +## 2. The transient thunk + +`flan_reload_call` is `emit_main` without the argv and the exit: no caller hands it a transfer channel, so it +owns a null cell on its own frame and passes that cell's address on. Sixteen bytes of frame rather than eight, +because rsp has to be 16-aligned at the call. + +The thunk is excluded from everything else the module does — no cell, no publish, no registry slot. There are +4096 slots and an expression evaluated in a loop would exhaust them, and a module with nothing pointing into it +is what lets the agent `dlclose` it at all. + +`@flan_reload_transient` is that claim, under `emit.ml`'s three conditions. The third is about **data** rather +than text and is the one that can be got wrong in the dangerous direction: a string literal lives in this +module's image, an expression may store one anywhere it likes, and a global left pointing into an unmapped +image is silent garbage rather than a fault. So the count is kept where the literals are made — `string_const` +bumps the same `Emit.m.nstr` field `emit.ml` counts on. Two deliberate decisions in that: + +- **A float constant is not counted.** It is a label in the same `.rodata` and it is *loaded*, never retained. + Counting rodata bytes would be the wrong test; `emit.ml` keeps frame descriptors in a separate `nfi` for + exactly this reason and says so. +- **The installer's own registry name strings are counted.** That is right rather than incidental: a module + that interned a name left something behind, and `flan_dev.c` `strdup`s it precisely because the module that + passed it may go away. + +## 3. The wiring + +**The choice is a session setting, not a per-command flag**, and it is spelled exactly as `debug` already is: +one field on `Session.t`, set once in `Dev.start`, carried on every `Session.change` the session emits. +`session.ml`'s own comment on `debug` gives the reason and it is the same one — the modules have to match the +process they are loaded into. `Session.redefinition` is the single place that picks a backend, so the six call +sites cannot disagree and the refusal has one home; `change.x86` rides beside `change.ir` so the text and the +builder can never come from two different answers to the same question. + +**There is no fallback and there must not be one.** If `X86.redefinition` refuses a form the daemon reports the +refusal. Quietly building an LLVM module instead is precisely the crossed pair `flan.abi.x86` exists to refuse +at `dlopen`. A refusal reaches the editor as a diagnostic like any other: `X86.Unsupported` is re-raised as a +`Loc.Error` at the form it is about, because every caller already handles that and none handled the other, and +a daemon that died on the first unsupported form would be worse than one that says so and stays up. + +`flan reload` got the same flag at the same time. A command that could build a module for a host the other +backend compiled is how the crossed pair was reachable from the CLI at all; `HANDOFF-x86-aggregates.md`'s +two-line reproduction no longer has a second half. + +**`--x86 --debug` is refused for the daemon**, and this is a decision rather than an oversight. +`flan build --x86 --debug` stays allowed — `X86.program` emits a hand-written DWARF 4 unit — but +`X86.redefinition` emits none, so a `--debug` session would build a host with a line table and then send it +modules without one. A breakpoint set on a line in the buffer would fire before the first `C-c C-c` and stop +firing after it. Accepting the flag and ignoring it would be worse. + +## The merged daemon, which is the finding + +**`flan dev --x86` refuses the merged daemon and needs `--two-process`.** + +A merged build is the program and the compiler in one process. The compiler expands macros by `dlopen`ing a +module `Build.macro_module` made — through `Emit.program`, always, and cached on disk under +`~/.cache/flan/objcache` by the *macro source*, not by the backend. A merged host is linked `-rdynamic` so a +redefinition module can reach its cells, and that exports every `flan.*` body it has. So the macro module's own +copy of a prelude function is **interposed by the host's**. + +With an LLVM host both halves are LLVM and nobody notices. With an `--x86` host the caller is LLVM and the body +it lands in is this backend's. Measured, before the refusal went in: + +``` +Thread 2 "program" received signal SIGSEGV +#0 0x0000000000403884 in flan[clamp] () <- the --x86 host's body +#1 0x00007ffff6f6b58c in flan.macro[clamp] () from ~/.cache/flan/objcache/flan-macros-....so +#2 flan_macro_call (dynload_stubs.c:68) +... +#19 camlFlan__Session.create_inner (lib/session.ml:117) +#20 camlFlan__Dev.merged_setup (lib/dev.ml:3013) +``` + +During the **first macro expansion**, before the program had started. `flan dev --x86` printed its build line +and exited 139. + +Three things worth stating about it: + +- **`flan.abi.x86` does not catch this and was never meant to.** It guards a redefinition module. + `HANDOFF-x86-abi-marker.md` reasoned explicitly that macro modules are unaffected because + `Build.macro_module` calls `Emit.program` without `~dev`, so a macro module neither defines nor requires a + marker — and that reasoning was correct for what it covered. This is a *third* path, and it exists only + because the merged daemon puts the compiler and the program in one address space. +- **`-Bsymbolic` is the wrong fix**, though it is one line. The macro module links its own `flan_rt.c`, and + binding symbolically would bind those calls to a copy of the runtime `flan_rt_init` never ran on — a second + bug hiding behind a green suite — and it would change the cached object for the LLVM path, which is the + default and must not move. `-Bsymbolic-functions` has the same defect. +- **The honest fix is hidden visibility on a macro module's Flan bodies.** That leaves the C runtime symbols + binding to the host's exactly as they do now and stops only the Flan ones from being interposed. It still + changes the cached object for both backends, so it wants a lane and a suite run of its own. + +`--two-process` has no such meeting: the compiler is a separate binary that LLVM built, the macro module is +loaded into *it* and never into the program, and the only thing crossing between them is a redefinition module +— which this session now builds with the same backend as the host. + +`start_merged` keeps its `--x86` plumbing (`rename_program_main_asm`, the `merged_executable` fork, the +`FLAN_DEV_X86` environment variable). It is unreachable today. It is the half that is right and it is what will +be wanted the day the macro module is fixed: the `--x86` merged host does build and link, and it is the *macro* +module and nothing about the host that stops it. + +## Measured + +One machine, `llc`/`clang` as installed here, warm caches. **This machine's `llc` is slower than the one +`docs/BUILT.md` was written on** — 42–48ms against that file's 15–17ms — so both rows below are measured here +rather than compared against that table. The shape of the answer is what matters and it does not depend on +which machine: `llc` is replaced by `as`, and `as` is five times cheaper. + +`flan reload`, five runs each, on one changed `defn`: + +| | LLVM | `--x86` | +|---|---|---| +| codegen | `llc` **42–48ms** | `as` **8.0–8.4ms** | +| link | `ld -shared` 9.7–11.4ms | `ld -shared` 8.9–9.3ms | + +**Read `timing.llc_ms` carefully**: `Build.shared_x86` puts the *assembler's* time in that field, because the +record is shared between the two. `flan reload` now prints `as` or `llc` according to what ran, so its output +does not mislead; anything reading the field directly still has to know. + +The round trip a user feels, through a real `flan dev --two-process` daemon on `programs/dev-repl.flan`, +measured at the socket — so this includes checking the form, emitting, building, delivering to the agent and +waiting for a frame boundary: + +| | LLVM | `--x86` | +|---|---|---| +| `C-c C-c` (`eval`, one `defn`) | 62, 66, 65ms | **27, 28, 30ms** | +| of which the build reports | 53.0, 52.6, 53.5ms | **17.9, 17.2, 18.2ms** | +| `C-x C-e` (`eval-expr`) | 61, 60, 60ms | **24, 24, 25ms** | + +**Better than twice as fast end to end, and three times on the build.** That is the claim the lane was written +to test and it holds. + +Two things the numbers say that the headline does not: + +- **`ld -shared` is now the bulk of the `--x86` build.** 9ms of an 18ms build, and it did not move between the + two backends. `docs/BUILT.md` records it at 3ms on the other machine; whatever the cause, the next + millisecond in this loop is in the linker and not in codegen. Nothing here tried to move it. +- **About 9ms of the round trip is neither codegen nor link.** 27ms at the socket against an 18ms build leaves + the frontend re-checking the program, the wire, the `dlopen` and the wait for a frame boundary. + `docs/BUILT.md` puts the re-check "under 10ms" and that is consistent. It was 14% of the LLVM round trip and + is 33% of this one, which is the ordinary consequence of removing the big term: the next thing worth + measuring has changed. + +## Verification + +| | before (`f459352`) | after | +|---|---|---| +| `dune test --root .` | exit 0, 232 checks, 0 failures | **exit 0, 232 checks, 0 failures**, three runs | +| `spike/x86/survey.sh` | 103 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86 | **103 / 0 / 0 / 0** | +| `spike/x86/cells.sh` | 4/4 ok | **4/4 ok** | +| `bash web/examples/check.sh` | green | **green** | + +The survey is the measurement that could have moved and did not, which is why it is run: `Lslot` and +`fnctx.slot` default to the whole-program answer and `string_const` now mutates `md.Emit.nstr` on that path +too, harmlessly, because nothing there reads it. The survey is what says so structurally rather than by +argument. + +Run `dune test --root .` **without a pipe** — piping to `tail` gives you `tail`'s exit status, and the +`dev-robust` fixture puts `ld` and `clang` failure text in the output either way. Run the survey **detached** +(`setsid timeout 2400 spike/x86/survey.sh > log 2>&1