A dev backend, ORC, and which one actually answers the want

This commit is contained in:
Joseph Ferano 2026-09-12 17:17:50 +07:00
parent 76ef56343d
commit 50b3feaa86

View File

@ -345,3 +345,43 @@ a pure-Emacs one — which also means it should not be built as a pure-Emacs hac
**Settled: it sticks, like Clojure's.** The mark stays until the form is evaluated again plainly. That is the
behaviour the workflow expects — you mark it, run the game, hit it repeatedly, and clear it with an ordinary `C-c C-c`
rather than having to re-mark before every run.
## 10. A second backend for dev builds — study SBCL and drop LLVM there?
Raised at the end of the session, not answered. The proposal: write our own code generator for **debug builds and the
interactive workflow**, keep LLVM for release. The motivation is that evaluating arbitrary code — full functions, real
test code, not just constant arithmetic — costs a compile, a file and a `dlopen` every time.
**What SBCL actually does**, since it is the model: its own assembler and code generator, emitting machine code
straight into the heap. No files, no external tools, no dynamic loading. Code objects are ordinary heap objects and are
collected when unreachable. That is also why its `disassemble` can annotate richly and why it can show machine state —
it owns everything.
**The argument against, and it is the same one that killed the interpreter.** Two backends must agree about arithmetic,
struct layout, overflow, evaluation order and calling convention. When they disagree you get *works interactively,
breaks when shipped* — the worst bug class in a live-programming system, and arguably worse than the interpreter
version because the divergence is in code generation rather than in semantics you can read.
Note what plan.org's dev-vs-release table already accepts: indirection cells, a shadow stack, version words, code
never freed. Those are **additive instrumentation on one code generator**, not a second one. This proposal is a
different kind of divergence.
And the cost is enormous: an instruction selector and assembler per architecture — x86-64, arm64, wasm32 — which is
where SBCL's decades went.
**The cheaper route to the same goal, and the thing to weigh it against: ORC, LLVM's in-process JIT.** Measured today,
a 35ms redefinition is **21ms of `llc` plus `ld`** — external tools generating and linking machine code. An in-process
JIT replaces exactly that, with **the same IR and therefore the same semantics**, no second implementation, and code
that can be unloaded. It gets what the proposal wants without the divergence hazard.
plan.org already says ORC "remains addable later behind the same typed IR without touching the language", and the only
column text-IR-plus-clang loses in its comparison table is the JIT one.
**So the honest position:** the thing that would justify revisiting the LLVM decision is not slower eval in the
abstract — it is wanting *arbitrary interactive evaluation of real code* as a first-class workflow. That is now stated
as a want. If it stays a want, ORC is the answer and a hand-written backend is not.
**Where the shared objects live**, since it was asked: `Build.workdir ()` — a per-build temporary directory — with the
object cache separately at `Build.cachedir ()`, stable across builds. Nothing sweeps the workdir, and nothing is ever
`dlclose`d, so a long session accumulates both files and mappings. Sweeping at session end is cheap and unrelated to
any of the above.