Joseph Ferano
3eaa3e23bd
flan dev's merged build is the program and the compiler in one -rdynamic executable, so it exports every flan.* body it has, and ELF gives it precedence over anything dlopened afterwards. The compiler expands a macro by dlopening a module into that same process, and the module is built by Emit.program whatever backend the session uses -- so under --x86 the caller was LLVM's and the body it landed in was the dev backend's, which is a crossed pair. It died with SIGSEGV inside flan.[clamp] during the first expansion, before the program had run a line, and Dev.start refused the combination rather than do that. Build.macro_module now asks Emit.program for hidden visibility on the module's own Flan definitions. There is nothing left for the host to interpose, and the flan.macro.* thunks stay exported because dlsym is how the compiler reaches them -- nm -D on the built module lists those three and nothing else of Flan's. The -Wl,-Bsymbolic that had been binding everything locally since 65d14f4 goes with it: the module links its own flan_rt.c, and binding that locally aimed its calls at a runtime flan_rt_init never ran on, with a null flan_exit_hook, so a trap raised inside an expansion would have exited the process instead of parking it. Nothing about the host moved, which is what keeps redefinition modules reaching its cells, its globals and flan_dev_cell. hidden defaults to false, and the 540 IR files this compiler emits for the test corpus are byte-identical to the ones before it. test_dev.ml's assertion that the merged daemon refuses --x86 becomes the session it was standing in for: dev-macro.flan calls a prelude macro at the top level, so the daemon coming up at all is the old crash not happening, and one build then carries C-x C-e, a C-c C-c whose body calls a macro again, the park and the rerun.
261 lines
18 KiB
Markdown
261 lines
18 KiB
Markdown
# Handoff — the x86 backend, wired to the dev loop
|
||
|
||
Branch `dev-loop`, worktree `agent-aaf84e55296df121c`, from `f459352`. Items 1, 2 and 3 of
|
||
`HANDOFF-x86-redef.md`'s "What remains": the new-name path, the transient thunk, and the wiring that lets
|
||
`flan dev` choose the backend for a host and its modules together.
|
||
|
||
**All three landed, and the numbers say the lane was worth it.** A `C-c C-c` round trip through a real daemon
|
||
is **62–66ms on LLVM and 27–30ms on `--x86`** on this machine; `C-x C-e` is **60ms and 24ms**. The build inside
|
||
those is **53ms and 17–18ms**. Details and caveats in §Measured.
|
||
|
||
There is one finding that is not good news and it is in §The merged daemon: `flan dev --x86` refuses the
|
||
*merged* daemon and needs `--two-process`, because the compiler's macro module is a third way the two backends
|
||
can meet in one process and `flan.abi.x86` does not guard it.
|
||
|
||
> **Closed.** `docs/handoffs/HANDOFF-x86-macro-visibility.md` took the recommendation in §The merged daemon —
|
||
> hidden visibility on a macro module's Flan bodies — and `flan dev --x86` starts a merged daemon now. The
|
||
> refusal in `Dev.start` is gone, `test_dev.ml` drives a merged `--x86` session through C-x C-e, C-c C-c, a
|
||
> park and a `rerun`, and the analysis below stands as the account of what the failure *was*. One correction
|
||
> to it: by the time that lane ran, `-Wl,-Bsymbolic` had already landed for an unrelated reason (a package's
|
||
> `defmacro` resolving to the merged host's copy, commit 65d14f4) and was incidentally suppressing this crash
|
||
> too. It has been removed in favour of the narrow fix, for the reason §The merged daemon gives for not
|
||
> wanting it.
|
||
|
||
## What was built
|
||
|
||
| file | what |
|
||
|---|---|
|
||
| `lib/x86.ml` — `loc`'s new `Lslot`, `fnctx.slot` | a name the host was never built with: the address lives in a module-local slot the installer fills by string. `Lgot`'s shape with `Sym` where it has `Got` |
|
||
| `lib/x86.ml` — `redefinition`'s installer | `flan_dev_cell` / `flan_dev_global` lookups, an init image per new global, the `consts` republish, and a real frame so those calls are made on an aligned stack |
|
||
| `lib/x86.ml` — `flan_reload_call`, `flan_reload_transient` | what an expression evaluation compiles to, and the claim the agent unloads on |
|
||
| `lib/x86.ml` — `string_const` | now bumps `Emit.m.nstr`, which is the third transient condition |
|
||
| `lib/session.ml` — `t.x86`, `change.x86`, `redefinition` | the backend choice, as a session setting, made in one place |
|
||
| `lib/dev.ml` — `build_module`, `module_ext`, `Dev.start ?x86` | the daemon picking builder and extension off the change, and the two refusals |
|
||
| `lib/dev.ml` — `rename_program_main_asm`, `merged_executable`'s fork | an `--x86` merged host, which builds and is currently refused (see below) |
|
||
| `bin/main.ml` | `flan dev --x86`, `flan reload --x86`, and a bare `Failure` printed as a sentence |
|
||
| `test/programs/reload-v6.flan` | a run-time-new global with a value of its own |
|
||
| `test/test_reload.ml` | the x86 section runs all four modules against the LLVM transcript; the refusal it asserted is gone |
|
||
| `test/test_dev.ml` | an `--x86` daemon driven through `C-c C-c`, `C-x C-e`, a literal, a new `defvar` and a new `defn`; plus the merged refusal |
|
||
|
||
## 1. The new-name path
|
||
|
||
A function or a `defvar` the running process has no symbol for. ELF cannot grow one, so the address is asked
|
||
for by string at install time and parked in a slot this module defines — `flan.cellp.<n>` for a function's
|
||
cell, `flan.gp.<n>` for a global's storage, spelled as `Emit.cellptr` and `Emit.globalptr` spell them so the
|
||
two sides read against each other.
|
||
|
||
The reference side is **one new `loc` case and nothing else**. `Lslot` loads the slot and answers
|
||
`Reg (scratch, d)`, which is exactly what `Lgot` already did with `Got` where this has `Sym`. Every site that
|
||
reaches a cell already double-loads, so no call site, no place expression and no `sym_loc` caller had to learn
|
||
a third case. `fnctx.slot` is a *second* predicate rather than a widened `ext` because the two answer different
|
||
questions: `ext` says "the host's, reach it through the GOT", `slot` says "nobody's yet, reach it through a
|
||
slot I filled". It defaults to `fun _ -> None`, so the whole-program path emits byte-identical output and
|
||
`survey.sh` goes on being a structural check on all of this.
|
||
|
||
Three things that would each have cost a session:
|
||
|
||
- **`flan_reload_install` had to become a function with a frame.** It was a run of loads and stores plus `ret`,
|
||
which was fine while it called nothing. The moment it calls `flan_dev_cell`, rsp is 8-mod-16 at the call and
|
||
glibc's `movaps` inside `strcmp` faults — a failure that reads as a backend bug and is an ABI bug. Its shape
|
||
is now `emit_globals_init`'s, down to owning the null transfer cell no caller hands it.
|
||
- **A new global's declared value has to travel with it.** `flan_dev_global` copies it onto the allocation the
|
||
first time the name is interned and ignores it after, which is where "a reload must not reset the program's
|
||
state" lives. `emit.ml` folds that value into an LLVM constant and this file has no folder, so the image is a
|
||
module-local `.bss` buffer written by the initialiser *lowered as ordinary code* — which is the bargain
|
||
`emit_globals_data`'s comment already makes for a whole program, and it needs no second evaluator that could
|
||
disagree with the first about what a struct literal means.
|
||
- **Order inside the installer is load-bearing and is not checkable from outside.** Every lookup resolves
|
||
before any body is published; publishing first exposes a function whose slots are still null to every call
|
||
site in the host. `emit.ml` says the same and `test_reload.ml` checks it *there* by grepping the IR text.
|
||
There is no text to grep on this side, so the guarantee is the loop order and the comment above it.
|
||
|
||
`reload-v6.flan` is new and is the reason the fixture set was not enough. v3's `extra` is declared zero, which
|
||
`calloc` also gives, so a run-time-new global whose image never arrived would still have passed. v6's `tuning`
|
||
is 42 and the host prints 88.
|
||
|
||
Republishing a `defconst` came free once the rest was there — one store of the new constant into the host's
|
||
global, which is what `emit.ml` does — so that refusal is gone too.
|
||
|
||
## 2. The transient thunk
|
||
|
||
`flan_reload_call` is `emit_main` without the argv and the exit: no caller hands it a transfer channel, so it
|
||
owns a null cell on its own frame and passes that cell's address on. Sixteen bytes of frame rather than eight,
|
||
because rsp has to be 16-aligned at the call.
|
||
|
||
The thunk is excluded from everything else the module does — no cell, no publish, no registry slot. There are
|
||
4096 slots and an expression evaluated in a loop would exhaust them, and a module with nothing pointing into it
|
||
is what lets the agent `dlclose` it at all.
|
||
|
||
`@flan_reload_transient` is that claim, under `emit.ml`'s three conditions. The third is about **data** rather
|
||
than text and is the one that can be got wrong in the dangerous direction: a string literal lives in this
|
||
module's image, an expression may store one anywhere it likes, and a global left pointing into an unmapped
|
||
image is silent garbage rather than a fault. So the count is kept where the literals are made — `string_const`
|
||
bumps the same `Emit.m.nstr` field `emit.ml` counts on. Two deliberate decisions in that:
|
||
|
||
- **A float constant is not counted.** It is a label in the same `.rodata` and it is *loaded*, never retained.
|
||
Counting rodata bytes would be the wrong test; `emit.ml` keeps frame descriptors in a separate `nfi` for
|
||
exactly this reason and says so.
|
||
- **The installer's own registry name strings are counted.** That is right rather than incidental: a module
|
||
that interned a name left something behind, and `flan_dev.c` `strdup`s it precisely because the module that
|
||
passed it may go away.
|
||
|
||
## 3. The wiring
|
||
|
||
**The choice is a session setting, not a per-command flag**, and it is spelled exactly as `debug` already is:
|
||
one field on `Session.t`, set once in `Dev.start`, carried on every `Session.change` the session emits.
|
||
`session.ml`'s own comment on `debug` gives the reason and it is the same one — the modules have to match the
|
||
process they are loaded into. `Session.redefinition` is the single place that picks a backend, so the six call
|
||
sites cannot disagree and the refusal has one home; `change.x86` rides beside `change.ir` so the text and the
|
||
builder can never come from two different answers to the same question.
|
||
|
||
**There is no fallback and there must not be one.** If `X86.redefinition` refuses a form the daemon reports the
|
||
refusal. Quietly building an LLVM module instead is precisely the crossed pair `flan.abi.x86` exists to refuse
|
||
at `dlopen`. A refusal reaches the editor as a diagnostic like any other: `X86.Unsupported` is re-raised as a
|
||
`Loc.Error` at the form it is about, because every caller already handles that and none handled the other, and
|
||
a daemon that died on the first unsupported form would be worse than one that says so and stays up.
|
||
|
||
`flan reload` got the same flag at the same time. A command that could build a module for a host the other
|
||
backend compiled is how the crossed pair was reachable from the CLI at all; `HANDOFF-x86-aggregates.md`'s
|
||
two-line reproduction no longer has a second half.
|
||
|
||
**`--x86 --debug` is refused for the daemon**, and this is a decision rather than an oversight.
|
||
`flan build --x86 --debug` stays allowed — `X86.program` emits a hand-written DWARF 4 unit — but
|
||
`X86.redefinition` emits none, so a `--debug` session would build a host with a line table and then send it
|
||
modules without one. A breakpoint set on a line in the buffer would fire before the first `C-c C-c` and stop
|
||
firing after it. Accepting the flag and ignoring it would be worse.
|
||
|
||
## The merged daemon, which is the finding
|
||
|
||
**`flan dev --x86` refuses the merged daemon and needs `--two-process`.**
|
||
|
||
A merged build is the program and the compiler in one process. The compiler expands macros by `dlopen`ing a
|
||
module `Build.macro_module` made — through `Emit.program`, always, and cached on disk under
|
||
`~/.cache/flan/objcache` by the *macro source*, not by the backend. A merged host is linked `-rdynamic` so a
|
||
redefinition module can reach its cells, and that exports every `flan.*` body it has. So the macro module's own
|
||
copy of a prelude function is **interposed by the host's**.
|
||
|
||
With an LLVM host both halves are LLVM and nobody notices. With an `--x86` host the caller is LLVM and the body
|
||
it lands in is this backend's. Measured, before the refusal went in:
|
||
|
||
```
|
||
Thread 2 "program" received signal SIGSEGV
|
||
#0 0x0000000000403884 in flan[clamp] () <- the --x86 host's body
|
||
#1 0x00007ffff6f6b58c in flan.macro[clamp] () from ~/.cache/flan/objcache/flan-macros-....so
|
||
#2 flan_macro_call (dynload_stubs.c:68)
|
||
...
|
||
#19 camlFlan__Session.create_inner (lib/session.ml:117)
|
||
#20 camlFlan__Dev.merged_setup (lib/dev.ml:3013)
|
||
```
|
||
|
||
During the **first macro expansion**, before the program had started. `flan dev --x86` printed its build line
|
||
and exited 139.
|
||
|
||
Three things worth stating about it:
|
||
|
||
- **`flan.abi.x86` does not catch this and was never meant to.** It guards a redefinition module.
|
||
`HANDOFF-x86-abi-marker.md` reasoned explicitly that macro modules are unaffected because
|
||
`Build.macro_module` calls `Emit.program` without `~dev`, so a macro module neither defines nor requires a
|
||
marker — and that reasoning was correct for what it covered. This is a *third* path, and it exists only
|
||
because the merged daemon puts the compiler and the program in one address space.
|
||
- **`-Bsymbolic` is the wrong fix**, though it is one line. The macro module links its own `flan_rt.c`, and
|
||
binding symbolically would bind those calls to a copy of the runtime `flan_rt_init` never ran on — a second
|
||
bug hiding behind a green suite — and it would change the cached object for the LLVM path, which is the
|
||
default and must not move. `-Bsymbolic-functions` has the same defect.
|
||
- **The honest fix is hidden visibility on a macro module's Flan bodies.** That leaves the C runtime symbols
|
||
binding to the host's exactly as they do now and stops only the Flan ones from being interposed. It still
|
||
changes the cached object for both backends, so it wants a lane and a suite run of its own.
|
||
|
||
`--two-process` has no such meeting: the compiler is a separate binary that LLVM built, the macro module is
|
||
loaded into *it* and never into the program, and the only thing crossing between them is a redefinition module
|
||
— which this session now builds with the same backend as the host.
|
||
|
||
`start_merged` keeps its `--x86` plumbing (`rename_program_main_asm`, the `merged_executable` fork, the
|
||
`FLAN_DEV_X86` environment variable). It is unreachable today. It is the half that is right and it is what will
|
||
be wanted the day the macro module is fixed: the `--x86` merged host does build and link, and it is the *macro*
|
||
module and nothing about the host that stops it.
|
||
|
||
## Measured
|
||
|
||
One machine, `llc`/`clang` as installed here, warm caches. **This machine's `llc` is slower than the one
|
||
`docs/BUILT.md` was written on** — 42–48ms against that file's 15–17ms — so both rows below are measured here
|
||
rather than compared against that table. The shape of the answer is what matters and it does not depend on
|
||
which machine: `llc` is replaced by `as`, and `as` is five times cheaper.
|
||
|
||
`flan reload`, five runs each, on one changed `defn`:
|
||
|
||
| | LLVM | `--x86` |
|
||
|---|---|---|
|
||
| codegen | `llc` **42–48ms** | `as` **8.0–8.4ms** |
|
||
| link | `ld -shared` 9.7–11.4ms | `ld -shared` 8.9–9.3ms |
|
||
|
||
**Read `timing.llc_ms` carefully**: `Build.shared_x86` puts the *assembler's* time in that field, because the
|
||
record is shared between the two. `flan reload` now prints `as` or `llc` according to what ran, so its output
|
||
does not mislead; anything reading the field directly still has to know.
|
||
|
||
The round trip a user feels, through a real `flan dev --two-process` daemon on `programs/dev-repl.flan`,
|
||
measured at the socket — so this includes checking the form, emitting, building, delivering to the agent and
|
||
waiting for a frame boundary:
|
||
|
||
| | LLVM | `--x86` |
|
||
|---|---|---|
|
||
| `C-c C-c` (`eval`, one `defn`) | 62, 66, 65ms | **27, 28, 30ms** |
|
||
| of which the build reports | 53.0, 52.6, 53.5ms | **17.9, 17.2, 18.2ms** |
|
||
| `C-x C-e` (`eval-expr`) | 61, 60, 60ms | **24, 24, 25ms** |
|
||
|
||
**Better than twice as fast end to end, and three times on the build.** That is the claim the lane was written
|
||
to test and it holds.
|
||
|
||
Two things the numbers say that the headline does not:
|
||
|
||
- **`ld -shared` is now the bulk of the `--x86` build.** 9ms of an 18ms build, and it did not move between the
|
||
two backends. `docs/BUILT.md` records it at 3ms on the other machine; whatever the cause, the next
|
||
millisecond in this loop is in the linker and not in codegen. Nothing here tried to move it.
|
||
- **About 9ms of the round trip is neither codegen nor link.** 27ms at the socket against an 18ms build leaves
|
||
the frontend re-checking the program, the wire, the `dlopen` and the wait for a frame boundary.
|
||
`docs/BUILT.md` puts the re-check "under 10ms" and that is consistent. It was 14% of the LLVM round trip and
|
||
is 33% of this one, which is the ordinary consequence of removing the big term: the next thing worth
|
||
measuring has changed.
|
||
|
||
## Verification
|
||
|
||
| | before (`f459352`) | after |
|
||
|---|---|---|
|
||
| `dune test --root .` | exit 0, 232 checks, 0 failures | **exit 0, 232 checks, 0 failures**, three runs |
|
||
| `spike/x86/survey.sh` | 103 MATCH / 0 DIFFER / 0 REFUSED / 0 NOX86 | **103 / 0 / 0 / 0** |
|
||
| skip breakdown | 28 does-not-compile / 8 no-main / 2 runs-forever | 28 / **9** / 2 |
|
||
| `spike/x86/cells.sh` | 4/4 ok | **4/4 ok** |
|
||
| `bash web/examples/check.sh` | green | **green** |
|
||
|
||
The `no-main` count moving from 8 to 9 is `reload-v6.flan` existing: `survey.sh` globs
|
||
`test/programs/*.flan`, and a program with no `main` fails the LLVM link and is classified there before the x86
|
||
build is attempted. `reload-agg.flan` moved it from 6 to 8 for the same reason. The **MATCH** count is what
|
||
must not move, and it did not.
|
||
|
||
The survey is the measurement that could have moved and did not, which is why it is run: `Lslot` and
|
||
`fnctx.slot` default to the whole-program answer and `string_const` now mutates `md.Emit.nstr` on that path
|
||
too, harmlessly, because nothing there reads it. The survey is what says so structurally rather than by
|
||
argument.
|
||
|
||
One more trap, and it wasted a run here. `dune test` caches a test's *result*: the second and third runs of an
|
||
unchanged tree print nothing and exit 0 without having executed anything. If what you want is three real runs
|
||
— which is what the flakiness in this area asks for — it is `dune test --root . --force`.
|
||
|
||
Run `dune test --root .` **without a pipe** — piping to `tail` gives you `tail`'s exit status, and the
|
||
`dev-robust` fixture puts `ld` and `clang` failure text in the output either way. Run the survey **detached**
|
||
(`setsid timeout 2400 spike/x86/survey.sh > log 2>&1 </dev/null`) or a signal to the harness's process group
|
||
comes back as 143, which is not a result. And do not run the survey while `dune test` is running: both take the
|
||
`_build` lock, and the loser reports a lock error rather than a count.
|
||
|
||
## What remains
|
||
|
||
- ~~**The macro module, so the merged daemon can be `--x86`.**~~ Done — `HANDOFF-x86-macro-visibility.md`.
|
||
- **The linker.** 9ms, unchanged between backends, now the bulk of an `--x86` build.
|
||
- **DWARF from `X86.redefinition`**, which is what `flan dev --x86 --debug` is refused for.
|
||
- **`X86.program ~macros`**, if the macro module is ever to be built by this backend rather than merely made
|
||
immune to interposition. Not needed for the fix recommended above.
|
||
- **`test/test_emacs.ml` got no `--x86` case, deliberately.** It drives the Elisp client against a daemon, and
|
||
the client neither knows nor can say which backend a session uses — the flag is `flan dev`'s and the protocol
|
||
above it is identical. The real editor path for *this* lane is `test_dev.ml`, which drives the daemon's own
|
||
verbs and is where the case went. Worth revisiting only if `--x86` ever grows a spelling in the protocol.
|
||
- Items 2–7 of `HANDOFF-x86-rt.md` §6, unchanged. And still worth doing: **run `spike/x86/survey.sh` in CI**.
|