From b49160b887ed783ffdb4616470bce2d5cb610ec2 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sat, 12 Sep 2026 17:28:13 +0700 Subject: [PATCH] One process or two, which subsumes the backend question and several settled ones --- DISCUSS.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/DISCUSS.md b/DISCUSS.md index 92d3454..31ab9ec 100644 --- a/DISCUSS.md +++ b/DISCUSS.md @@ -385,3 +385,48 @@ as a want. If it stays a want, ORC is the answer and a hand-written backend is n object cache separately at `Build.cachedir ()`, stable across builds. Nothing sweeps the workdir, and nothing is ever `dlclose`d, so a long session accumulates both files and mappings. Sweeping at session end is cheap and unrelated to any of the above. + +## 11. One process, not two — the biggest open question + +Raised at the very end of the session and **not resolved**. It subsumes items 9 and 10, and possibly several decisions +recorded elsewhere as settled. Do not treat any of those as closed until this is answered. + +**The claim:** the daemon and the running program should not be two processes. SBCL is one image; Clojure is one +image; in both, the interactive workflow is cheap *because nothing is transported*. The author's reading is that the +split here was forced by not having threads — and that is worth checking, because **threads are already in use**: the +agent runs a listener thread and a loader thread inside the program today. + +**The concrete shape**, if it were done: load the *program* into the daemon's process as a shared library rather than +launching it as a child. The reload primitive already `dlopen`s modules; the program itself would be one. + +**What would evaporate, and it is a lot.** No socket and no wire protocol. Cell updates become plain pointer stores. +Locals and globals become memory the compiler can read directly — no render thunks compiled per inspection, no 4K +result cap, no seqlock, no snapshot copying, no generation stamping. The macro expander stops needing the compiler's +own `dlopen`. `flan reload` stops being a fresh process paying startup every time. A large amount of machinery +recorded in `BUILT.md` exists **only** to move data between two address spaces. + +**What is given up, in order of sharpness:** + +1. **Crash isolation.** A segfault in the game currently cannot touch the compiler. `plan.org`'s jank note — corrected + earlier for citing the wrong mechanism — concludes "we are safe from the repro *because we compile out of process*". + In one image a bad pointer takes the session with it. Lisp users live with this and mostly accept it; it is a real + loss and should be chosen knowingly, not discovered. +2. **macOS requires a window on the main thread.** "Run raylib on a separate thread" is fine on Linux and not portable. + The inversion — compiler on the side thread, window on main — probably works, but it needs deciding rather than + assuming, and it changes who owns the main loop. +3. **The OCaml runtime and the game in one address space**: signals, `SIGSEGV` handling for the break loop, the OCaml + GC running while game code holds raw pointers into its own arenas. Each is probably fine; none is free. + +**What it does NOT fix, and this is where an earlier answer in this session was wrong.** A hand-written debug backend +was discussed as the way to make evaluation instant. The correction: a new backend removes `llc` and `ld` — about +21ms of a 35ms redefinition — but **not the transport**, because the transport exists due to the process split, not +the backend. Conversely, **merging the processes removes the transport but not the code generation**. They are +independent, and merging is the larger prize of the two. + +**Decisions to revisit if this is taken:** the watch design (push was chosen partly because polling costs a compile); +the 4K result cap and its seqlock; the snapshot machinery and its generation stamping; the render-thunk-per-inspection +design for locals and globals; the inspector's inability to retain a value; and whether ORC or a hand-written lowering +is needed at all. + +**Suggested next step:** answer the macOS main-thread question first. It is the only one of the three costs that could +make the whole idea non-portable, and it is answerable by reading raylib and GLFW rather than by building anything.