diff --git a/NEXT.md b/NEXT.md index 135deb3..812c683 100644 --- a/NEXT.md +++ b/NEXT.md @@ -4,121 +4,93 @@ **Branch `dev-loop`, 34 commits, working tree clean, `dune test` green.** -The dev loop works end to end: `flan dev program.flan`, then `C-c C-c`, -`C-x C-e` and `C-c C-r` in Emacs against the running process. Conditions are -two steps in of four. +The dev loop works end to end: `flan dev program.flan`, then `C-c C-c`, `C-x C-e` and `C-c C-r` in Emacs against the +running process. Conditions are two steps in of four. -`(error c)` is in — §2's diverging variant, same lookup, type `Never`. A -handler that returns normally has not answered it, so only a transfer gets -past; with nothing transferring the program stops and names the condition. -`flan_error` is where the break loop goes. +`(error c)` is in — §2's diverging variant, same lookup, type `Never`. A handler that returns normally has not answered +it, so only a transfer gets past; with nothing transferring the program stops and names the condition. `flan_error` is +where the break loop goes. -**The break loop is in.** An unhandled `error` now stops the program on the -frame that erred, lists the restarts on offer, and waits; a choice sent to the -agent's socket resumes it. See "The break loop" below. What is left of §2 is the -*editor* half — a `break` notification on the daemon protocol and a minibuffer -prompt — which is what makes it something you use rather than something you -drive with a socket. +**The break loop is in.** An unhandled `error` now stops the program on the frame that erred, lists the restarts on +offer, and waits; a choice sent to the agent's socket resumes it. See "The break loop" below. What is left of §2 is the +*editor* half — a `break` notification on the daemon protocol and a minibuffer prompt — which is what makes it something +you use rather than something you drive with a socket. -**The old next task, now done, was the dev-build break loop, `spec-conditions.md` §2** — where -an unhandled `error` stops and talks to the daemon instead of `rt_die()`, and -where **"a crash kills the program"** finally gets fixed. The transfer it needs -exists now: §6's channel is in every signature and `restart-case` catches on -it, so a break loop is a place to *stand* while the program is stopped, not a -new way to move. +**The old next task, now done, was the dev-build break loop, `spec-conditions.md` §2** — where an unhandled `error` +stops and talks to the daemon instead of `rt_die()`, and where **"a crash kills the program"** finally gets fixed. The +transfer it needs exists now: §6's channel is in every signature and `restart-case` catches on it, so a break loop is a +place to *stand* while the program is stopped, not a new way to move. -Then restarts offered in the Emacs minibuffer, which wants `compute-restarts` -plus two protocol ops. SBCL's restart struct carries `report-function` and -`interactive-function` for exactly that prompt and `spec-conditions.md` -mentions neither — worth adding before that step. `find-restart` and -`compute-restarts` are both named in §4 and neither exists yet; the runtime -already has the stack they would walk. +Then restarts offered in the Emacs minibuffer, which wants `compute-restarts` plus two protocol ops. SBCL's restart +struct carries `report-function` and `interactive-function` for exactly that prompt and `spec-conditions.md` mentions +neither — worth adding before that step. `find-restart` and `compute-restarts` are both named in §4 and neither exists +yet; the runtime already has the stack they would walk. -Still open from §3, and each refused by name today: **restarts with -parameters** (argument marshalling plus the runtime arity check), and -`handler-case`, which §"What this does not settle" leaves open as possibly a -macro over `handler-bind` plus a transfer. `error`, `find-restart` and -`compute-restarts` now refuse by name too — they used to fall through to a call -and come back as *unknown name*, which is the house rule's own class of bug. +Still open from §3, and each refused by name today: **restarts with parameters** (argument marshalling plus the runtime +arity check), and `handler-case`, which §"What this does not settle" leaves open as possibly a macro over `handler-bind` +plus a transfer. `error`, `find-restart` and `compute-restarts` now refuse by name too — they used to fall through to a +call and come back as *unknown name*, which is the house rule's own class of bug. -Read SBCL for what restarts should *mean* and ignore how it moves control: it -transfers with `block`/`return-from`, which §6 rules out. +Read SBCL for what restarts should *mean* and ignore how it moves control: it transfers with `block`/`return-from`, +which §6 rules out. ### Managed classes are planned. Do not start them. -plan.org grew a `class` facility beside `struct`: identity, runtime shape -metadata, an implementation-defined representation, generic-function dispatch, -and live schema change with an explicit migration at a frame boundary. Its own -last line is the rule — nothing until ordinary `struct`, `Handle` and reload -semantics are working. It is here so that a session reading plan.org cold does -not take it as the next task. Three things found while reviewing it, none of +plan.org grew a `class` facility beside `struct`: identity, runtime shape metadata, an implementation-defined +representation, generic-function dispatch, and live schema change with an explicit migration at a frame boundary. Its +own last line is the rule — nothing until ordinary `struct`, `Handle` and reload semantics are working. It is here so +that a session reading plan.org cold does not take it as the next task. Three things found while reviewing it, none of them in plan.org yet: -- **A generic function is a cell.** "A later module can add `(defmethod draw - ((e Enemy)) ...)` without editing the original" means every compiled call site - of `draw` has to find the new method — which is the problem the indirection - cells already solve. A generic function is a cell whose body is a dispatch - table and a reload extends the table. The expensive half of classes is - therefore already built and tested. -- **The pool is not one storage option among three.** `migrate-instances` has to - *enumerate* live instances. A pool behind generational `(Handle T)` gives that - by construction; a world arena and an owned region do not obviously. plan.org - presents the three as a free choice and they are not. -- **`Enemy@1` has to stay resolvable** for `migrate` to dispatch on it, so the - session retains every layout version's metadata for as long as any instance - holds it. Same rule as "nothing is ever `dlclose`d", and worth stating as one. +- **A generic function is a cell.** "A later module can add `(defmethod draw ((e Enemy)) ...)` without editing the +original" means every compiled call site of `draw` has to find the new method — which is the problem the indirection +cells already solve. A generic function is a cell whose body is a dispatch table and a reload extends the table. The +expensive half of classes is therefore already built and tested. +- **The pool is not one storage option among three.** `migrate-instances` has to *enumerate* live instances. A pool +behind generational `(Handle T)` gives that by construction; a world arena and an owned region do not obviously. +plan.org presents the three as a free choice and they are not. +- **`Enemy@1` has to stay resolvable** for `migrate` to dispatch on it, so the session retains every layout version's +metadata for as long as any instance holds it. Same rule as "nothing is ever `dlclose`d", and worth stating as one. ### Open: can a condition be a class? -Unanswered, and it wants answering before `handler-case`, because it decides -whether handler matching has one path or two. +Unanswered, and it wants answering before `handler-case`, because it decides whether handler matching has one path or +two. -It would buy the thing conditions most lack: a **hierarchy**. §1 says flatly -there is none, which is why nothing can say "any condition" — no catch-all -handler and nothing for a break loop to match on. Class inheritance gives it. +It would buy the thing conditions most lack: a **hierarchy**. §1 says flatly there is none, which is why nothing can say +"any condition" — no catch-all handler and nothing for a break loop to match on. Class inheritance gives it. Three costs, one serious: -- **Signalling would allocate.** A struct condition is a stack value and - `signal` takes its address; a class instance needs a pool slot at the signal - site. That is the failure path, sometimes the hot path, and sometimes the - thing that failed is allocation itself. plan.org also says no implicit - allocation anywhere in the core. -- **§5's lifetime inverts.** Today the condition dies with the signalling frame - and a handler that keeps it copies, which is free for a value struct. A class - instance survives the transfer — nicer, but now something owns and frees it. -- **Layout versions meet handler frames.** A struct condition cannot change - layout; it is refused. A class can, and then a frame pushed against - `MyError@1` is on the stack while the signaller builds `MyError@2`. +- **Signalling would allocate.** A struct condition is a stack value and `signal` takes its address; a class instance +needs a pool slot at the signal site. That is the failure path, sometimes the hot path, and sometimes the thing that +failed is allocation itself. plan.org also says no implicit allocation anywhere in the core. +- **§5's lifetime inverts.** Today the condition dies with the signalling frame and a handler that keeps it copies, +which is free for a value struct. A class instance survives the transfer — nicer, but now something owns and frees it. +- **Layout versions meet handler frames.** A struct condition cannot change layout; it is refused. A class can, and then +a frame pushed against `MyError@1` is on the stack while the signaller builds `MyError@2`. -The shape that probably wins is both: a struct condition stays exactly what it -is — no allocation, matched by name hash, dies with the frame — and a class -condition is allocated, survives, and matches by walking its class chain. -That is two matching paths, which is the same bill the struct/class split -already signs, so it is consistent rather than a new cost. Either way it is an -amendment to a **frozen** `spec-conditions.md`, not a gap in it. +The shape that probably wins is both: a struct condition stays exactly what it is — no allocation, matched by name hash, +dies with the frame — and a class condition is allocated, survives, and matches by walking its class chain. That is two +matching paths, which is the same bill the struct/class split already signs, so it is consistent rather than a new cost. +Either way it is an amendment to a **frozen** `spec-conditions.md`, not a gap in it. -**The dev loop is closed.** `C-c C-c` in Emacs recompiles the top-level form -at point and installs it in a running program, at that program's next frame -boundary. Verified against sand: an unsaved buffer edit to `game-draw`, and 240 +**The dev loop is closed.** `C-c C-c` in Emacs recompiles the top-level form at point and installs it in a running +program, at that program's next frame boundary. Verified against sand: an unsaved buffer edit to `game-draw`, and 240 consecutive frames drew it. -Steps 1, 2 and 3 are done — see *The reload primitive* below. A list -of top-level forms can be recompiled and installed into a running process; call -sites compiled before they existed follow them, and a `defn` or `defvar` the -process was never built with can be added and then redefined again. That is the -whole of `C-c C-c`, minus an editor: sand.flan takes a redefinition over a -socket and installs it between frames. +Steps 1, 2 and 3 are done — see *The reload primitive* below. A list of top-level forms can be recompiled and installed +into a running process; call sites compiled before they existed follow them, and a `defn` or `defvar` the process was +never built with can be added and then redefined again. That is the whole of `C-c C-c`, minus an editor: sand.flan takes +a redefinition over a socket and installs it between frames. -What is left is the *session* — something that holds the checker environment -between evaluations, tracks which names the running process was built with, and -speaks a protocol an editor can talk to. +What is left is the *session* — something that holds the checker environment between evaluations, tracks which names the +running process was built with, and speaks a protocol an editor can talk to. -Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its -simulation has a headless acceptance case that runs on the `dune test` path at -`-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and -runs; the interpreter was dropped — open decision #7, settled, see below). +Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its simulation has a headless acceptance case that +runs on the `dune test` path at `-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and runs; the +interpreter was dropped — open decision #7, settled, see below). ``` reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅ @@ -173,45 +145,35 @@ $ flan run sand.flan # a window, 120 fps, hold space ## What milestone 4 added -**`dotimes`** desugars in `check.ml` to a `Let` plus a `While` — no new IR node. -The bound is evaluated once into a hidden slot before the loop, so a body that -changes it cannot change the trip count, and the loop variable is not +**`dotimes`** desugars in `check.ml` to a `Let` plus a `While` — no new IR node. The bound is evaluated once into a +hidden slot before the loop, so a body that changes it cannot change the trip count, and the loop variable is not assignable, which makes the generated step its only writer. -**`defer`** is recognised in `check_fn` and nowhere else, because that is the -only place that knows a form is at the top level of a function body. Each one -is checked in place, then registered on the context; it emits nothing where it -stands. Function exit runs them innermost-first, and an explicit `return` runs -the ones registered *above* it — a defer written below a return has not -executed yet and must not fire. A trap runs none of them, which follows from -the bounds-check shape (`noreturn` then `unreachable`) rather than being a -separate decision. +**`defer`** is recognised in `check_fn` and nowhere else, because that is the only place that knows a form is at the top +level of a function body. Each one is checked in place, then registered on the context; it emits nothing where it +stands. Function exit runs them innermost-first, and an explicit `return` runs the ones registered *above* it — a defer +written below a return has not executed yet and must not fire. A trap runs none of them, which follows from the +bounds-check shape (`noreturn` then `unreachable`) rather than being a separate decision. -`defer` inside a `let`, a loop or a branch is **rejected**, not accepted with -function scope. It would run once at function exit rather than once per -iteration, and that is the silent-wrongness class the rule below is about. +`defer` inside a `let`, a loop or a branch is **rejected**, not accepted with function scope. It would run once at +function exit rather than once per iteration, and that is the silent-wrongness class the rule below is about. Block-scoped defer is real work and is not done. -**New builtins:** `zeroed` (takes its type from the place it is stored into), -`min`/`max` (each operand through a slot, so neither is evaluated twice), -`bit-and`/`bit-or`/`bit-xor`/`<<`/`>>` (integers only; `>>` is arithmetic on a -signed type and logical on an unsigned one), and `rand-f32`. +**New builtins:** `zeroed` (takes its type from the place it is stored into), `min`/`max` (each operand through a slot, +so neither is evaluated twice), `bit-and`/`bit-or`/`bit-xor`/`<<`/`>>` (integers only; `>>` is arithmetic on a signed +type and logical on an unsigned one), and `rand-f32`. -**`rand-f32` is in the prelude, in Flan** — PCG-XSH-RR 32 over a `u64` state. -It is not libc's, because a grid hash is only a regression test if the sequence -is byte-identical on native and wasm32 (plan.org, RNG is ours). `rand-seed` -sets the state. This is what the bitwise operators were added for. +**`rand-f32` is in the prelude, in Flan** — PCG-XSH-RR 32 over a `u64` state. It is not libc's, because a grid hash is +only a regression test if the sequence is byte-identical on native and wasm32 (plan.org, RNG is ours). `rand-seed` sets +the state. This is what the bitwise operators were added for. -**Enums and keywords.** `(defenum Name [member value ...])` gives a type that -is an `i32` at run time and its own type in the checker, so `:space` at a call -site resolves against the parameter's enum and a typo is an error there rather -than a wrong number later. A keyword means nothing where no enum is expected — -there is no keyword type to fall back on. +**Enums and keywords.** `(defenum Name [member value ...])` gives a type that is an `i32` at run time and its own type +in the checker, so `:space` at a call site resolves against the parameter's enum and a typo is an error there rather +than a wrong number later. A keyword means nothing where no enum is expected — there is no keyword type to fall back on. ## Why the FFI goes through a C shim -The decision that shapes the whole raylib package. What clang generates for -raylib's own prototypes on x86-64: +The decision that shapes the whole raylib package. What clang generates for raylib's own prototypes on x86-64: ``` Vector2 {float,float} → declare <2 x float> @GetMousePosition() @@ -219,27 +181,22 @@ Color {u8,u8,u8,u8} → declare void @ClearBackground(i32) Rectangle {4 × int} → declare { i64, i64 } @mkrect() ``` -None of those is the struct's own LLVM type. A small aggregate's calling -convention is not part of its layout — it is a per-target classification the -*caller* has to reproduce, and x86-64, arm64 and wasm32 classify differently. -Putting that in `emit.ml` is three classifiers to write and then keep correct -forever, and a mistake shows up as `(.y m)` returning garbage rather than as a -link error. +None of those is the struct's own LLVM type. A small aggregate's calling convention is not part of its layout — it is a +per-target classification the *caller* has to reproduce, and x86-64, arm64 and wasm32 classify differently. Putting that +in `emit.ml` is three classifiers to write and then keep correct forever, and a mistake shows up as `(.y m)` returning +garbage rather than as a link error. -So the boundary has one wrapper per binding, each one flattening the -aggregates: a struct returns through an out-pointer, a struct argument is -passed by pointer, a Flan string crosses as ptr+len and the shim NUL-terminates -a copy. clang classifies all of it, per target, for free. `check.ml` enforces -the rule — an aggregate in a `declare` signature is rejected with the reason — -so the boundary cannot quietly acquire one. This is plan.org's "one narrow host +So the boundary has one wrapper per binding, each one flattening the aggregates: a struct returns through an +out-pointer, a struct argument is passed by pointer, a Flan string crosses as ptr+len and the shim NUL-terminates a +copy. clang classifies all of it, per target, for free. `check.ml` enforces the rule — an aggregate in a `declare` +signature is rejected with the reason — so the boundary cannot quietly acquire one. This is plan.org's "one narrow host ABI, implemented twice", and `flan_rt.c` is the same pattern. ### The wrappers are generated now — `declare-c`, `lib/shim.ml` -The price above was a hand-written wrapper per raylib call, and the prediction -that they were mechanical enough to generate "if that ever becomes the -bottleneck" came true at 84 of them. `vendor/raylib/shim.c` is gone; the -directory holds `raylib.flan` and `link` and no C at all. +The price above was a hand-written wrapper per raylib call, and the prediction that they were mechanical enough to +generate "if that ever becomes the bottleneck" came true at 84 of them. `vendor/raylib/shim.c` is gone; the directory +holds `raylib.flan` and `link` and no C at all. One binding is now one line: @@ -247,297 +204,218 @@ One binding is now one line: (declare-c draw-texture [t Texture2D x i32 y i32 tint Color] "DrawTexture") ``` -`declare-c` names raylib's own function in raylib's own signature, and the -compiler emits, into a C file compiled like any other: the typedefs for the -structs involved, made from the Flan `defstruct`s; the `extern` prototype in -the function's true signature; the wrapper that flattens it; and the flattened -`declare` the Flan side calls, with an ordinary Flan `defn` above it when the -signature has a struct in it. `flan shim ` prints the whole file. +`declare-c` names raylib's own function in raylib's own signature, and the compiler emits, into a C file compiled like +any other: the typedefs for the structs involved, made from the Flan `defstruct`s; the `extern` prototype in the +function's true signature; the wrapper that flattens it; and the flattened `declare` the Flan side calls, with an +ordinary Flan `defn` above it when the signature has a struct in it. `flan shim ` prints the whole file. -**It is a second form and not a change to `declare`, for one reason worth -remembering:** `(declare start-raw [path string] i32 "flan_agent_start")` in -`vendor/agent` means the symbol takes ptr+len, and `(declare-c init-window [w -i32 h i32 title string] "InitWindow")` means it takes a NUL-terminated `char -*`. Same shape, opposite claims, so no structural rule can separate them. -`declare` is untouched, and `sqrtf` and the agent still work unedited. +**It is a second form and not a change to `declare`, for one reason worth remembering:** `(declare start-raw [path +string] i32 "flan_agent_start")` in `vendor/agent` means the symbol takes ptr+len, and `(declare-c init-window [w i32 h +i32 title string] "InitWindow")` means it takes a NUL-terminated `char *`. Same shape, opposite claims, so no structural +rule can separate them. `declare` is untouched, and `sqrtf` and the agent still work unedited. -**What the generator guarantees, and what it trusts.** Guaranteed: the C -typedef and the Flan struct come from the same `defstruct`, so they cannot -disagree — permute the `defstruct` and the typedef permutes with it, which is -exactly what makes the permutation runs below meaningful. And clang -type-checks the wrapper against the generated prototype. Trusted: that the -`defstruct` matches the library's real struct, and that the `declare-c` -signature is the function's real signature — no header is read, deliberately, -so nothing can check either. A `_Static_assert` on `sizeof`/`offsetof` was -considered and rejected as circular: both sides would come from the same field -list. Padding is not a separate hazard: for every field type the generator -admits — the machine integers, the two floats, `bool`, a pointer and a nested -struct — LLVM's layout is C's, and `emit.ml` writes no datalayout, so clang -applies the target's rules to both halves. Everything where they could diverge -is refused at the field. +**What the generator guarantees, and what it trusts.** Guaranteed: the C typedef and the Flan struct come from the same +`defstruct`, so they cannot disagree — permute the `defstruct` and the typedef permutes with it, which is exactly what +makes the permutation runs below meaningful. And clang type-checks the wrapper against the generated prototype. Trusted: +that the `defstruct` matches the library's real struct, and that the `declare-c` signature is the function's real +signature — no header is read, deliberately, so nothing can check either. A `_Static_assert` on `sizeof`/`offsetof` was +considered and rejected as circular: both sides would come from the same field list. Padding is not a separate hazard: +for every field type the generator admits — the machine integers, the two floats, `bool`, a pointer and a nested struct +— LLVM's layout is C's, and `emit.ml` writes no datalayout, so clang applies the target's rules to both halves. +Everything where they could diverge is refused at the field. -**One thing got sharper and should be said plainly:** the prototype is now -generated *from the declaration*, so a scalar's width carries ABI weight it did -not before. `f64` where raylib says `float` used to be narrowed by clang at the -hand-written call site; now it emits `double` and raylib reads garbage. All 84 -migrated prototypes were diffed against the deleted `shim.c`'s — which was the -ground truth for the true signatures — and agree. +**One thing got sharper and should be said plainly:** the prototype is now generated *from the declaration*, so a +scalar's width carries ABI weight it did not before. `f64` where raylib says `float` used to be narrowed by clang at the +hand-written call site; now it emits `double` and raylib reads garbage. All 84 migrated prototypes were diffed against +the deleted `shim.c`'s — which was the ground truth for the true signatures — and agree. -**Strings.** The hand-written wrappers sized the NUL-copy per call site: 256 -for a window title, `PATH_MAX` for a path, 512 for drawn text, truncating past -it. A generator has no call site to look at, so it must not be the thing -deciding a string is too long: 256 bytes on the stack, the heap past that, -freed after the call. The only truncation left is on malloc failure, where the -alternative is handing C a null pointer. +**Strings.** The hand-written wrappers sized the NUL-copy per call site: 256 for a window title, `PATH_MAX` for a path, +512 for drawn text, truncating past it. A generator has no call site to look at, so it must not be the thing deciding a +string is too long: 256 bytes on the stack, the heap past that, freed after the call. The only truncation left is on +malloc failure, where the alternative is handing C a null pointer. -**Two bindings keep a hand-written wrapper, and both wrappers are Flan, not C.** -`collision-point-poly?` takes a slice and `collision-lines` answers with an -`Option`; neither is raylib's own signature. A slice parameter in a `declare-c` -is refused by name, because a slice's length crosses as i64 and the type of the -C count parameter beside the pointer is not recoverable from `[T]` — so that -one declares `(Ptr Vector2)` with an explicit `count i32` and the Flan wrapper -passes `(addr (at points 0))` and `(len points)`. Every other refusal — an -Option, a union, a fixed array, a map, a returned string, a callback, an -unknown type, an unrepresentable struct field, two Flan names for one C symbol -— is by name with the reason, and the acceptance table asserts on the reasons. +**Two bindings keep a hand-written wrapper, and both wrappers are Flan, not C.** `collision-point-poly?` takes a slice +and `collision-lines` answers with an `Option`; neither is raylib's own signature. A slice parameter in a `declare-c` is +refused by name, because a slice's length crosses as i64 and the type of the C count parameter beside the pointer is not +recoverable from `[T]` — so that one declares `(Ptr Vector2)` with an explicit `count i32` and the Flan wrapper passes +`(addr (at points 0))` and `(len points)`. Every other refusal — an Option, a union, a fixed array, a map, a returned +string, a callback, an unknown type, an unrepresentable struct field, two Flan names for one C symbol — is by name with +the reason, and the acceptance table asserts on the reasons. -**Known edge, not fixed:** a REPL redefinition that introduces a *new* -`declare-c` cannot work. `Build.shared` is llc + `ld -shared` and compiles no -C, so the wrapper would not exist in the running process. Editing the body of a -function that calls an existing binding is unaffected. +**Known edge, not fixed:** a REPL redefinition that introduces a *new* `declare-c` cannot work. `Build.shared` is llc + +`ld -shared` and compiles no C, so the wrapper would not exist in the running process. Editing the body of a function +that calls an existing binding is unaffected. -`raylib.flan` carries the nice signature and the compiler writes the rest, so -the surface sand.flan sees is `(rl/get-mouse-position)` -returning a `Vector2`. Verified end to end, headless: `GetColor(0x11223344)` -comes back as `17 34 51 68`, four separate bytes — a `Color` is *not* the -little-endian reading of the packed integer, so an identity would have passed a -weaker test. That case is in the acceptance table, skipped if `libraylib` is -not installed. +`raylib.flan` carries the nice signature and the compiler writes the rest, so the surface sand.flan sees is +`(rl/get-mouse-position)` returning a `Vector2`. Verified end to end, headless: `GetColor(0x11223344)` comes back as `17 +34 51 68`, four separate bytes — a `Color` is *not* the little-endian reading of the packed integer, so an identity +would have passed a weaker test. That case is in the acceptance table, skipped if `libraylib` is not installed. -The bindings are 29 calls: window (`init-window`, `close-window`, -`window-should-close?`, `set-target-fps`, `set-trace-log-level`), keyboard -(`key-pressed?`/`down?`/`released?`), mouse (`mouse-button-pressed?`/`down?`/ -`released?`, `get-mouse-position`), `get-color`, drawing (`begin-drawing`, -`end-drawing`, `draw-fps`, `clear-background`, `draw-rectangle`), textures -(`load-texture`, `texture-valid?`, `unload-texture`, `draw-texture`, -`draw-texture-v`, `draw-texture-ex`, `draw-texture-rec`), and the shapes -texture and rectangle intersection (`set-shapes-texture`, -`get-shapes-texture`, `get-shapes-texture-rectangle`, `get-collision-rec`), -plus the `Key`, `MouseButton` and `TraceLogLevel` enums and the `Vector2`, -`Color`, `Texture2D` and `Rectangle` structs. Adding one was three lines — a -`declare`, an `extern` prototype and a one-line wrapper — and is now one -`declare-c`. +The bindings are 29 calls: window (`init-window`, `close-window`, `window-should-close?`, `set-target-fps`, +`set-trace-log-level`), keyboard (`key-pressed?`/`down?`/`released?`), mouse (`mouse-button-pressed?`/`down?`/ +`released?`, `get-mouse-position`), `get-color`, drawing (`begin-drawing`, `end-drawing`, `draw-fps`, +`clear-background`, `draw-rectangle`), textures (`load-texture`, `texture-valid?`, `unload-texture`, `draw-texture`, +`draw-texture-v`, `draw-texture-ex`, `draw-texture-rec`), and the shapes texture and rectangle intersection +(`set-shapes-texture`, `get-shapes-texture`, `get-shapes-texture-rectangle`, `get-collision-rec`), plus the `Key`, +`MouseButton` and `TraceLogLevel` enums and the `Vector2`, `Color`, `Texture2D` and `Rectangle` structs. Adding one was +three lines — a `declare`, an `extern` prototype and a one-line wrapper — and is now one `declare-c`. -The texture calls are the first ones with no headless test, because loading -one needs a GL context. What the acceptance case does instead is pin the two -new struct layouts using the only things raylib computes from those fields -without a GPU: `GetCollisionRec`, which pins `Rectangle` completely, and -`SetShapesTexture`'s default substitution, which pins `Texture2D`'s `id` and -`format` and nothing else. Handing a struct over and reading it back proves -nothing at all — storing and returning is symmetric, so a permuted layout -comes back permuted the same way and the case passes. `width`, `height` and -`mipmaps` are therefore checked only by looking at `sand.flan` running, which -draws the brush sprite four ways for that reason. +The texture calls are the first ones with no headless test, because loading one needs a GL context. What the acceptance +case does instead is pin the two new struct layouts using the only things raylib computes from those fields without a +GPU: `GetCollisionRec`, which pins `Rectangle` completely, and `SetShapesTexture`'s default substitution, which pins +`Texture2D`'s `id` and `format` and nothing else. Handing a struct over and reading it back proves nothing at all — +storing and returning is symmetric, so a permuted layout comes back permuted the same way and the case passes. `width`, +`height` and `mipmaps` are therefore checked only by looking at `sand.flan` running, which draws the brush sprite four +ways for that reason. -No raylib headers are needed: the generated C declares the prototypes it uses, -so the build depends on the shared library being linkable and not on -`raylib-devel`. -`vendor/raylib/link` carries `-l:libraylib.so.550` because Fedora ships the -runtime library without the `.so` symlink. +No raylib headers are needed: the generated C declares the prototypes it uses, so the build depends on the shared +library being linkable and not on `raylib-devel`. `vendor/raylib/link` carries `-l:libraylib.so.550` because Fedora +ships the runtime library without the `.so` symlink. ### What a headless FFI test can and cannot pin -Worth knowing before writing another one, because two plausible tests in a row -turned out to check nothing. +Worth knowing before writing another one, because two plausible tests in a row turned out to check nothing. -- **A struct round trip is worthless.** Hand raylib a struct, read it back, - compare: store-and-return is symmetric, so C writes and reads the same wrong - slots and the test passes for *any* field order. Found by permuting two - fields and getting identical output. -- **Axis-aligned geometry cannot pin `Vector2`.** Exchanging `x` and `y` is a - reflection, applied to the inputs on the way in and undone on the way out, so - the printed answer is unchanged. Every collision predicate, and every - distance, passes with the fields swapped — verified by swapping the - `defstruct`, which is what the typedef is now made from. Distances are worse: the reflection does not even reach them. -- **What does pin `Vector2` is the rotated camera**, because a 90-degree - rotation is not axis-aligned and therefore does not commute with the - reflection. That case is load-bearing and must not be deleted on the grounds - that the collision cases look like they cover it. -- **What pins `Rectangle` is arithmetic on its fields** — `GetCollisionRec` - computes four numbers from four different field pairs, and the point/rect - predicates turn the wrong way when width and height are exchanged. +- **A struct round trip is worthless.** Hand raylib a struct, read it back, compare: store-and-return is symmetric, so C +writes and reads the same wrong slots and the test passes for *any* field order. Found by permuting two fields and +getting identical output. +- **Axis-aligned geometry cannot pin `Vector2`.** Exchanging `x` and `y` is a reflection, applied to the inputs on the +way in and undone on the way out, so the printed answer is unchanged. Every collision predicate, and every distance, +passes with the fields swapped — verified by swapping the `defstruct`, which is what the typedef is now made from. +Distances are worse: the reflection does not even reach them. +- **What does pin `Vector2` is the rotated camera**, because a 90-degree rotation is not axis-aligned and therefore does +not commute with the reflection. That case is load-bearing and must not be deleted on the grounds that the collision +cases look like they cover it. +- **What pins `Rectangle` is arithmetic on its fields** — `GetCollisionRec` computes four numbers from four different +field pairs, and the point/rect predicates turn the wrong way when width and height are exchanged. -- **Scalars in, fields out is the strongest shape there is**, and the Image - family is where it was finally available. `GenImageColor(4, 2, colour)` is - handed two integers and answers with a struct reading 4, 2, 1, 7 — four - distinct values in four adjacent `i32` slots, with no input struct for a - permutation to cancel against. That pins `Image` completely, including that - `data` is present and first; `Texture2D` could never be pinned that way - because nothing without a GPU reads its width, height or mipmaps at all. -- **A non-square image is an axis discriminator.** `GetImageColor` indexes - `y*width + x`, so on a 4-wide, 2-tall image `(3,0)` exists and its transpose - does not: exchange the wrapper's `x` and `y` and the read goes out of bounds - and answers transparent black. `ImageFlipHorizontal` against - `ImageFlipVertical` says the same thing twice more. -- **A file is external ground truth**, so `ExportImage` then `LoadImage` is not - the symmetric round trip the rest of the package has to avoid — stb's encoder - and decoder agree with each other, not with Flan's field order. Verified - red by the `x`/`y` permutation above. +- **Scalars in, fields out is the strongest shape there is**, and the Image family is where it was finally available. +`GenImageColor(4, 2, colour)` is handed two integers and answers with a struct reading 4, 2, 1, 7 — four distinct values +in four adjacent `i32` slots, with no input struct for a permutation to cancel against. That pins `Image` completely, +including that `data` is present and first; `Texture2D` could never be pinned that way because nothing without a GPU +reads its width, height or mipmaps at all. +- **A non-square image is an axis discriminator.** `GetImageColor` indexes `y*width + x`, so on a 4-wide, 2-tall image +`(3,0)` exists and its transpose does not: exchange the wrapper's `x` and `y` and the read goes out of bounds and +answers transparent black. `ImageFlipHorizontal` against `ImageFlipVertical` says the same thing twice more. +- **A file is external ground truth**, so `ExportImage` then `LoadImage` is not the symmetric round trip the rest of the +package has to avoid — stb's encoder and decoder agree with each other, not with Flan's field order. Verified red by the +`x`/`y` permutation above. -The rule that falls out: make raylib **compute** something whose answer differs -per axis, then verify the test can fail by permuting the fields and watching it -go red. A case not verified that way is decoration. +The rule that falls out: make raylib **compute** something whose answer differs per axis, then verify the test can fail +by permuting the fields and watching it go red. A case not verified that way is decoration. -One correction to an assumption that has now cost two lanes a guess: -**`MeasureText` is not headless material.** It measures with the default font, -`LoadFontDefault` is not exported, and nothing but `InitWindow` loads it — so -with no window it answers 0 for every string. Measured against -`libraylib.so.550`, not reasoned about. `GetFrameTime`, `GetTime` and -`GetScreenWidth`/`Height` are all 0 headless for the same kind of reason. All -five are bound, and all five are exercised by running `sand.flan` and looking, -which is the whole of what can be claimed for them. +One correction to an assumption that has now cost two lanes a guess: **`MeasureText` is not headless material.** It +measures with the default font, `LoadFontDefault` is not exported, and nothing but `InitWindow` loads it — so with no +window it answers 0 for every string. Measured against `libraylib.so.550`, not reasoned about. `GetFrameTime`, `GetTime` +and `GetScreenWidth`/`Height` are all 0 headless for the same kind of reason. All five are bound, and all five are +exercised by running `sand.flan` and looking, which is the whole of what can be claimed for them. ## Packages -`lib/load.ml` resolves `(import rl "vendor:raylib")` before the checker runs. -The directory is the package; `vendor:` is a collection, resolved by walking up -from the importing file until a directory of that name is found; a path with no -collection is relative to the importing file. Importing is a **rename**: every -top-level name the package declares becomes `alias/name`, and every use of one -— in a type, in a body, in a struct literal, in an *array length* — is -rewritten to match. Local bindings shadow. Nothing downstream knows a package -existed; the checker sees one flat list of declarations whose names contain a -slash. +`lib/load.ml` resolves `(import rl "vendor:raylib")` before the checker runs. The directory is the package; `vendor:` is +a collection, resolved by walking up from the importing file until a directory of that name is found; a path with no +collection is relative to the importing file. Importing is a **rename**: every top-level name the package declares +becomes `alias/name`, and every use of one — in a type, in a body, in a struct literal, in an *array length* — is +rewritten to match. Local bindings shadow. Nothing downstream knows a package existed; the checker sees one flat list of +declarations whose names contain a slash. -A package may also carry the C it binds to: every `.c` file in the directory is -compiled into the build, and a file named `link` lists extra linker arguments. -Whether those reach the build at all is decided *after* checking — see below. +A package may also carry the C it binds to: every `.c` file in the directory is compiled into the build, and a file +named `link` lists extra linker arguments. Whether those reach the build at all is decided *after* checking — see below. -**A package may be a single `.flan` file** named outright, rather than a -directory. That is for the program that is also a library: `sand.flan` shares -the repository root with three other loose programs, so naming its directory -would import all four. A file carries no `.c` and no `link` file; those belong -to a directory. +**A package may be a single `.flan` file** named outright, rather than a directory. That is for the program that is also +a library: `sand.flan` shares the repository root with three other loose programs, so naming its directory would import +all four. A file carries no `.c` and no `link` file; those belong to a directory. -**A package may import a package.** The qualification flattens to the *inner* -alias — raylib imported by a package that is itself imported is still `rl/…`, -never `sand/rl/…` — because a directory reached along two routes has to arrive -under one set of names or the checker sees every declaration twice. A directory -is keyed by its real path and read once, which is also what ends a cycle: a -package that imports itself meets its own entry and contributes nothing the -second time, and the namespace being flat, mutually dependent packages simply -work. The same directory under two *different* aliases is refused. +**A package may import a package.** The qualification flattens to the *inner* alias — raylib imported by a package that +is itself imported is still `rl/…`, never `sand/rl/…` — because a directory reached along two routes has to arrive under +one set of names or the checker sees every declaration twice. A directory is keyed by its real path and read once, which +is also what ends a cycle: a package that imports itself meets its own entry and contributes nothing the second time, +and the namespace being flat, mutually dependent packages simply work. The same directory under two *different* aliases +is refused. -**Visibility is one rule: `main` is not exported.** A package carrying one -would collide with the importer's the moment anything imported it, so a program -could never be a package; and `main` is a reachability root, so an imported one -would keep everything it calls alive. Writing `sand/main` is refused at the -line that wrote it, with the reason — left to the checker it would be "unknown -name", which is true and useless. +**Visibility is one rule: `main` is not exported.** A package carrying one would collide with the importer's the moment +anything imported it, so a program could never be a package; and `main` is a reachability root, so an imported one would +keep everything it calls alive. Writing `sand/main` is refused at the line that wrote it, with the reason — left to the +checker it would be "unknown name", which is true and useless. -Still missing: a package-private marker for anything other than `main`, which -is why `rl/get-color-raw` is callable. +Still missing: a package-private marker for anything other than `main`, which is why `rl/get-color-raw` is callable. ## The link follows the program -`Load` used to hand a package's `.c` files and `link` arguments to the build -the moment it was imported, whatever the importing program did with them. So -anything naming `vendor:raylib` linked libraylib on every target, and on wasm32 -that link cannot succeed. +`Load` used to hand a package's `.c` files and `link` arguments to the build the moment it was imported, whatever the +importing program did with them. So anything naming `vendor:raylib` linked libraylib on every target, and on wasm32 that +link cannot succeed. -`lib/reach.ml` answers it from the checked program instead. Start at `main` and -at the globals that run before it, follow every call — including the `Handled` -frames, where a lifted handler clause is reached by address and by nothing else -— and keep what is reached. A package none of whose externs survive contributes -no C and no linker argument. +`lib/reach.ml` answers it from the checked program instead. Start at `main` and at the globals that run before it, +follow every call — including the `Handled` frames, where a lifted handler clause is reached by address and by nothing +else — and keep what is reached. A package none of whose externs survive contributes no C and no linker argument. -Dropping the flags alone would only move the failure: the bodies that called -into raylib would still be emitted and `wasm-ld` would fail on the symbols -rather than on the argument list. So the same walk prunes **functions and -externs** from the program. Only those. Globals, structs and unions stay, -because an unreferenced global is bytes in BSS and a dropped one is a silently -different program. +Dropping the flags alone would only move the failure: the bodies that called into raylib would still be emitted and +`wasm-ld` would fail on the symbols rather than on the argument list. So the same walk prunes **functions and externs** +from the program. Only those. Globals, structs and unions stay, because an unreferenced global is bytes in BSS and a +dropped one is a silently different program. -**Dev builds are not pruned.** What a REPL may redefine next is not a function -of what has been called so far. +**Dev builds are not pruned.** What a REPL may redefine next is not a function of what has been called so far. -The filtering happens at the call sites — `bin/main.ml`, the tests — because -`Build.executable` receives `csrcs` and `lflags` from its caller and never sees -the import list. `Reach.link` returns the pruned program and its C and linker +The filtering happens at the call sites — `bin/main.ml`, the tests — because `Build.executable` receives `csrcs` and +`lflags` from its caller and never sees the import list. `Reach.link` returns the pruned program and its C and linker arguments together, so a caller cannot take one without the other. ## sand.flan is one program -It was two files, and only ever for the reason above: the headless run is the -one CI does on native *and* wasm32, and a program that imported raylib linked -libraylib whatever its `main` did. So the simulation lived in `sand-sim/` and -both drivers imported it. +It was two files, and only ever for the reason above: the headless run is the one CI does on native *and* wasm32, and a +program that imported raylib linked libraylib whatever its `main` did. So the simulation lived in `sand-sim/` and both +drivers imported it. -Now `sand.flan` holds the simulation *and* the raylib front-end, and -`test/programs/sand-headless.flan` imports `sand.flan` itself — window, raylib -bindings, dev agent and all — and still builds for wasm32. Nothing it calls -reaches raylib; `sand.flan`'s `main` is not exported, so the only `main` is the -headless one; and the hash is unchanged on both targets at `-O2` and `-O0`, -which is the point. A refactor that moved that number would have moved the -simulation. +Now `sand.flan` holds the simulation *and* the raylib front-end, and `test/programs/sand-headless.flan` imports +`sand.flan` itself — window, raylib bindings, dev agent and all — and still builds for wasm32. Nothing it calls reaches +raylib; `sand.flan`'s `main` is not exported, so the only `main` is the headless one; and the hash is unchanged on both +targets at `-O2` and `-O0`, which is the point. A refactor that moved that number would have moved the simulation. -What still justifies *two entry points* is smaller and stands on its own: **the -headless test needs no window and no input on any target.** `sand.flan` could -not be that test even natively — with no mouse the grid stays empty and -`settle` and `move-grain` never run on real data. Measured through the probe: -168 grains painted around row 4–8, still 168 after 40 frames, lowest occupied -row 68. Grains fall, and none are lost. +What still justifies *two entry points* is smaller and stands on its own: **the headless test needs no window and no +input on any target.** `sand.flan` could not be that test even natively — with no mouse the grid stays empty and +`settle` and `move-grain` never run on real data. Measured through the probe: 168 grains painted around row 4–8, still +168 after 40 frames, lowest occupied row 68. Grains fall, and none are lost. Two claims that got run together in an earlier note, for the record: -- *raylib does not work on wasm* — false. It works through emscripten. What is - true is that it does not work on the **wasi** path, which is what the - headless table targets, and which has no GL and no browser. -- *a game loop cannot be expressed on wasm* — false. The browser cannot be - blocked, so a web build drives the loop with `emscripten_set_main_loop` - instead of a `while`. That is a different `main`, not a different program. +- *raylib does not work on wasm* — false. It works through emscripten. What is true is that it does not work on the +**wasi** path, which is what the headless table targets, and which has no GL and no browser. +- *a game loop cannot be expressed on wasm* — false. The browser cannot be blocked, so a web build drives the loop with +`emscripten_set_main_loop` instead of a `while`. That is a different `main`, not a different program. -**Three edits were made to sand.flan's own text** when it was ported, and they -are language decisions rather than fixes: +**Three edits were made to sand.flan's own text** when it was ported, and they are language decisions rather than fixes: -- `(defconst gravity 0.05)` → `(defconst gravity f32 0.05)`. An untyped float - constant is `f64`, `velocity` is `[f32]`, and there is no implicit widening. -- `(defvar current-color u32)` → `i32`. It is an index into `colors`, and - `(len colors)` is an `i32`. +- `(defconst gravity 0.05)` → `(defconst gravity f32 0.05)`. An untyped float constant is `f64`, `velocity` is `[f32]`, +and there is no implicit widening. +- `(defvar current-color u32)` → `i32`. It is an index into `colors`, and `(len colors)` is an `i32`. - `(defn main [])` is unchanged — the short form, as plan.org says. -Painting is on **hold left mouse button** rather than on space, since the mouse -bindings exist now. Space is still what cycles the colour, on release, which is -a leftover and probably wants to move to the right button or to a key press. +Painting is on **hold left mouse button** rather than on space, since the mouse bindings exist now. Space is still what +cycles the colour, on release, which is a leftover and probably wants to move to the right button or to a key press. ## Bounds checks — done at milestone 3 -`at` and `slice` emit `icmp` → `br` → cold block → `call` → `unreachable`; a -failure names the source location. Three check sites: `at` on `[n T]` (static -bound, folded by LLVM for a literal index — and a literal that is out of bounds -never reaches emit, `check.ml` rejects it), `at` on a slice or string (runtime -len), and `slice` (two comparisons — `lo <= hi` is not redundant, without it a -reversed range yields a huge unsigned length). All comparisons unsigned. -`Build.opts.checks` is on by default and **not** tied to `opts.opt`, which is -what lets the acceptance table run the same programs at `-O0` and `-O2` with -identical checks. The flag is `--no-bounds-checks`. +`at` and `slice` emit `icmp` → `br` → cold block → `call` → `unreachable`; a failure names the source location. Three +check sites: `at` on `[n T]` (static bound, folded by LLVM for a literal index — and a literal that is out of bounds +never reaches emit, `check.ml` rejects it), `at` on a slice or string (runtime len), and `slice` (two comparisons — `lo +<= hi` is not redundant, without it a reversed range yields a huge unsigned length). All comparisons unsigned. +`Build.opts.checks` is on by default and **not** tied to `opts.opt`, which is what lets the acceptance table run the +same programs at `-O0` and `-O2` with identical checks. The flag is `--no-bounds-checks`. -The write path is its own case: `(set (at arr n) …)` lowers through -`place`/`Pindex`, not through `At`, so a refactor that split them would break -the write check silently. The test covers both. +The write path is its own case: `(set (at arr n) …)` lowers through `place`/`Pindex`, not through `At`, so a refactor +that split them would break the write check silently. The test covers both. -Cost, measured: a 50M-iteration dependency chain over a 1024-element array runs -at 0.11–0.12s checked against 0.12–0.13s unchecked. Indistinguishable. +Cost, measured: a 50M-iteration dependency chain over a 1024-element array runs at 0.11–0.12s checked against 0.12–0.13s +unchecked. Indistinguishable. ## Why there is no interpreter -Open decision #7 is settled: **the compiled path is the only backend.** Both -arguments for a permanent interpreter had expired — the instrumentation step -debugger that wanted it is cut, and compiled redefinition measured at ~16ms, -perceptually instant for expression eval too. Milestone 3 did not need an -oracle either: the acceptance table is hand-written, so the table *is* the -oracle. Consequences already applied: milestone 2's "interpreted calls per -second" criterion is dropped, and the host ABI moved onto the critical path. +Open decision #7 is settled: **the compiled path is the only backend.** Both arguments for a permanent interpreter had +expired — the instrumentation step debugger that wanted it is cut, and compiled redefinition measured at ~16ms, +perceptually instant for expression eval too. Milestone 3 did not need an oracle either: the acceptance table is +hand-written, so the table *is* the oracle. Consequences already applied: milestone 2's "interpreted calls per second" +criterion is dropped, and the host ABI moved onto the critical path. ## The layout, which is the whole backend design @@ -554,75 +432,54 @@ a struct a literal struct, declaration order Unit and Never {} ``` -No object headers anywhere, so a Flan struct is exactly its C struct and -nothing marshals. Two consequences carry the semantics: +No object headers anywhere, so a Flan struct is exactly its C struct and nothing marshals. Two consequences carry the +semantics: -- **Every slot is an `alloca`.** Reading a local is a `load`, assigning is a - `store`, and a `store` of an aggregate *is* the copy `spec-memory.md` - requires. `addr` of a local is then just the alloca, and `mem2reg` removes - the ones nobody addressed. `test/programs/values.flan` pins this down. -- **A place is a pointer, a value is a load from it.** `(set (.pos c) …)` - through a `(Ptr Cursor)` becomes a `getelementptr` on the pointer, not on a - copy. This is the split that would have made a tree-walker silently wrong. +- **Every slot is an `alloca`.** Reading a local is a `load`, assigning is a `store`, and a `store` of an aggregate *is* +the copy `spec-memory.md` requires. `addr` of a local is then just the alloca, and `mem2reg` removes the ones nobody +addressed. `test/programs/values.flan` pins this down. +- **A place is a pointer, a value is a load from it.** `(set (.pos c) …)` through a `(Ptr Cursor)` becomes a +`getelementptr` on the pointer, not on a copy. This is the split that would have made a tree-walker silently wrong. -Non-local exit is lowered explicitly: `return`, `some` and a failed bounds -check are branches, never platform unwinding, so wasm32 needs no exception -proposal. +Non-local exit is lowered explicitly: `return`, `some` and a failed bounds check are branches, never platform unwinding, +so wasm32 needs no exception proposal. ## Sharp edges -Most of these are edges the language keeps and you should know about. Two — -the top-level namespace and the shift count, both found by review after -milestone 4 — were bugs that reached LLVM or ran wrong, and are **fixed**; each -says so. They stay written down because each one is now a rule the checker -enforces, and a later change could quietly drop it. +Most of these are edges the language keeps and you should know about. Two — the top-level namespace and the shift count, +both found by review after milestone 4 — were bugs that reached LLVM or ran wrong, and are **fixed**; each says so. They +stay written down because each one is now a rule the checker enforces, and a later change could quietly drop it. -- **An index converts from a narrower integer and never from a wider one.** - `(nth colors current-color)` with a `u32` index works — anything above 2³¹ - truncates to a negative `i32` and the unsigned bounds check rejects it. An - `i64` index is refused with the reason: 2³²+5 truncates to 5 and would read - the wrong element with no trap at all. -- **There is one top-level namespace, and `check.ml` now enforces it.** The - environment's tables are per-kind — structs, unions, aliases, enums, - functions, externs and globals each have their own — so only a function was - ever checked for a duplicate. `(defn item …)` beside `(defvar item …)` type - checked and then died in LLVM as `redefinition of function '@flan.item'`, a - message about an emitted symbol with no source location left, and two - colliding *type* declarations were not caught anywhere. One pass over - `Ast.declared_name` now runs before every other collection pass and rejects - the second declaration of a name whatever kind either one is. `declared_name` - lives in `ast.ml` because `Load` needs exactly the same set — the names an - import renames — and two copies of that list would drift. -- **A shift count is bounded, two different ways.** A shift by the operand's - own width or more is *poison* in LLVM, not a wrong number: `(defn main [] i32 - (<< 1 32))` compiled at -O2 to a bare `retq`, returning an undefined value. A - literal count out of range is now rejected in `check.ml` — that is the typo - case — and `emit.ml` masks a computed count to `width - 1`, which is what the - hardware does anyway and which LLVM folds away whenever the count is - constant. The prelude's rotate masks its own count; that is now redundant but - harmless. -- **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real - `u64` and not an error. The cost is that a negative *decimal* literal is - accepted as a `u64` too, because the reader records the value and not how it - was written. Narrower unsigned types keep the strict check, which is where a - typo like `300` for a `u8` actually shows up. -- **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted - from the folding pass's value, because a global's initialiser has to be a - compile-time constant and only that pass knows this one is. Its range check - is therefore its own call to `in_range`; there is a regression test. -- A `let` binding takes no type annotation, which is why `sand.flan` names its - FNV constants instead of writing them inline. -- `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean - f64*: with a single body form the parser cannot tell a return type from the - first expression. Only the parameter position and `(Option …)` are - unambiguous. +- **An index converts from a narrower integer and never from a wider one.** `(nth colors current-color)` with a `u32` +index works — anything above 2³¹ truncates to a negative `i32` and the unsigned bounds check rejects it. An `i64` index +is refused with the reason: 2³²+5 truncates to 5 and would read the wrong element with no trap at all. +- **There is one top-level namespace, and `check.ml` now enforces it.** The environment's tables are per-kind — structs, +unions, aliases, enums, functions, externs and globals each have their own — so only a function was ever checked for a +duplicate. `(defn item …)` beside `(defvar item …)` type checked and then died in LLVM as `redefinition of function +'@flan.item'`, a message about an emitted symbol with no source location left, and two colliding *type* declarations +were not caught anywhere. One pass over `Ast.declared_name` now runs before every other collection pass and rejects the +second declaration of a name whatever kind either one is. `declared_name` lives in `ast.ml` because `Load` needs exactly +the same set — the names an import renames — and two copies of that list would drift. +- **A shift count is bounded, two different ways.** A shift by the operand's own width or more is *poison* in LLVM, not +a wrong number: `(defn main [] i32 (<< 1 32))` compiled at -O2 to a bare `retq`, returning an undefined value. A literal +count out of range is now rejected in `check.ml` — that is the typo case — and `emit.ml` masks a computed count to +`width - 1`, which is what the hardware does anyway and which LLVM folds away whenever the count is constant. The +prelude's rotate masks its own count; that is now redundant but harmless. +- **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real `u64` and not an error. The cost is that +a negative *decimal* literal is accepted as a `u64` too, because the reader records the value and not how it was +written. Narrower unsigned types keep the strict check, which is where a typo like `300` for a `u8` actually shows up. +- **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted from the folding pass's value, because a +global's initialiser has to be a compile-time constant and only that pass knows this one is. Its range check is +therefore its own call to `in_range`; there is a regression test. +- A `let` binding takes no type annotation, which is why `sand.flan` names its FNV constants instead of writing them +inline. +- `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean f64*: with a single body form the parser +cannot tell a return type from the first expression. Only the parameter position and `(Option …)` are unambiguous. ## The reload primitive — dev loop steps 1 and 2, measured -`llc` → `ld -shared` → `dlopen` → call, with no protocol and no daemon. -`dune test` runs it: one function is recompiled into its own object and called -inside a process that is already running, twice, with a changed body the second -time. +`llc` → `ld -shared` → `dlopen` → call, with no protocol and no daemon. `dune test` runs it: one function is recompiled +into its own object and called inside a process that is already running, twice, with a changed body the second time. | Step | Cost | |---|---| @@ -631,70 +488,53 @@ time. | `ld -shared` | 3ms | | `dlopen` + `dlsym` | **0.04ms** | -**~19ms end to end**, and the load itself is free. plan.org's 16ms was measured -with clang somewhere else; this is the number from this codebase. For contrast, -`clang -shared` on the same IR is 50ms — the driver is again most of the cost, -which is why the dev path skips it. `llc` and `clang` are both 20.1.8 here; -check that before trusting the `.ll`, since the driver absorbs IR the bare -tools reject. +**~19ms end to end**, and the load itself is free. plan.org's 16ms was measured with clang somewhere else; this is the +number from this codebase. For contrast, `clang -shared` on the same IR is 50ms — the driver is again most of the cost, +which is why the dev path skips it. `llc` and `clang` are both 20.1.8 here; check that before trusting the `.ll`, since +the driver absorbs IR the bare tools reject. -`ld -shared` rather than `clang -shared` for a second reason: a shared object -is allowed undefined symbols, and that *is* the mechanism. What the new module -does **not** define is the whole design: +`ld -shared` rather than `clang -shared` for a second reason: a shared object is allowed undefined symbols, and that +*is* the mechanism. What the new module does **not** define is the whole design: -- **a global is `external`.** This settles the open question below in the only - direction that supports the demo: a redefinition can change a function's - body and can never re-initialise the program's data. Define the global and - the loaded object gets a second copy — sand's `grid` would reset on every - reload, and "edit the code, keep the sand" is the thesis. -- **every other function is a `declare`**, so a redefined `settle` calls the - host's `move-grain` rather than freezing a private copy of it. +- **a global is `external`.** This settles the open question below in the only direction that supports the demo: a +redefinition can change a function's body and can never re-initialise the program's data. Define the global and the +loaded object gets a second copy — sand's `grid` would reset on every reload, and "edit the code, keep the sand" is the +thesis. +- **every other function is a `declare`**, so a redefined `settle` calls the host's `move-grain` rather than freezing a +private copy of it. - **no `main`.** This module is loaded, not started. -Its string constants still come along; omitting them is an undefined `@.str.N` -at link time, and it is easy to miss because a one-function module usually has -none. `Emit.signature` is now the single place a function's LLVM signature is -spelled, because a `define` here and a `declare` there drift the moment one of -them grows a case for `Unit` or for a slice parameter. +Its string constants still come along; omitting them is an undefined `@.str.N` at link time, and it is easy to miss +because a one-function module usually has none. `Emit.signature` is now the single place a function's LLVM signature is +spelled, because a `define` here and a `declare` there drift the moment one of them grows a case for `Unit` or for a +slice parameter. -**`flan_dev.c` is compiled into every build, not only a dev one.** Nothing in a -release build calls into it — the compiler only emits a registry lookup for a -name the host was not built with, which cannot arise without cells — but the -agent package's C refers to it, and a package's C sources are collected -whatever `main` does. Leaving it out of release builds made `flan build -sand.flan` fail at the link with an undefined `flan_dev_result_get`, which -reads as a compiler bug rather than as a missing flag. The table is BSS, so the -cost is address space and not binary size; `-rdynamic` and the cells are still -what `--dev` means. `test_agent.ml` links the agent program both ways for this -reason. +**`flan_dev.c` is compiled into every build, not only a dev one.** Nothing in a release build calls into it — the +compiler only emits a registry lookup for a name the host was not built with, which cannot arise without cells — but the +agent package's C refers to it, and a package's C sources are collected whatever `main` does. Leaving it out of release +builds made `flan build sand.flan` fail at the link with an undefined `flan_dev_result_get`, which reads as a compiler +bug rather than as a missing flag. The table is BSS, so the cost is address space and not binary size; `-rdynamic` and +the cells are still what `--dev` means. `test_agent.ml` links the agent program both ways for this reason. -**`-rdynamic` is load-bearing.** A normal executable exports nothing: `nm -D -calc-me | grep 'flan\.'` is empty, so a loaded module's `declare`s would have -nothing to bind to. The test passes it through `lflags`, which keeps it a -property of the dev build rather than of every build. `dlsym` on `"flan.bump"` -works — a dot is legal in an ELF symbol. +**`-rdynamic` is load-bearing.** A normal executable exports nothing: `nm -D calc-me | grep 'flan\.'` is empty, so a +loaded module's `declare`s would have nothing to bind to. The test passes it through `lflags`, which keeps it a property +of the dev build rather than of every build. `dlsym` on `"flan.bump"` works — a dot is legal in an ELF symbol. -Two things about the test are deliberate and are what make it prove anything: -both loads happen in **one process**, since two runs would pass while saying -nothing about an in-process swap; and the versions are **two paths**, since -`dlopen` caches by path and re-opening one would hand back the handle it -already had, so the check would lie. And `helper` is `(* x 2)` in one fixture -and `(* x 3)` in the other: the second body is dead text, since the module -declares `helper` rather than defining it, so the expected 1024 coming back -instead of 1036 is what proves the call landed on the host's copy. With the two -bodies identical nothing at run time would notice a module that grew its own. +Two things about the test are deliberate and are what make it prove anything: both loads happen in **one process**, +since two runs would pass while saying nothing about an in-process swap; and the versions are **two paths**, since +`dlopen` caches by path and re-opening one would hand back the handle it already had, so the check would lie. And +`helper` is `(* x 2)` in one fixture and `(* x 3)` in the other: the second body is dead text, since the module declares +`helper` rather than defining it, so the expected 1024 coming back instead of 1036 is what proves the call landed on the +host's copy. With the two bodies identical nothing at run time would notice a module that grew its own. -String constants are emitted `private unnamed_addr`, so the module's own -`@.str.N` cannot be interposed by the host's — worth knowing, because with -external linkage a redefined function would silently print the *old* text and -nothing would fail at link time. The fixtures each print a literal so that path -is actually exercised. +String constants are emitted `private unnamed_addr`, so the module's own `@.str.N` cannot be interposed by the host's — +worth knowing, because with external linkage a redefined function would silently print the *old* text and nothing would +fail at link time. The fixtures each print a literal so that path is actually exercised. ### Cells — how a call site follows a redefinition -Loading a new body is not installing it. A call bound at link time cannot be -made to notice one, so **a dev build routes every Flan-to-Flan call through a -cell**: a mutable global holding the address of the function that is current. +Loading a new body is not installing it. A call bound at link time cannot be made to notice one, so **a dev build routes +every Flan-to-Flan call through a cell**: a mutable global holding the address of the function that is current. ``` @"flan.cell.bump" = global ptr @"flan.bump" ; the host defines it @@ -702,101 +542,78 @@ cell**: a mutable global holding the address of the function that is current. %r = call i64 %p() ``` -Redefinition is then one store. A redefinition module declares the cells -`external`, exactly like the globals, and exposes `flan_reload_install()` that -stores its own body into its own cell — cost **below a microsecond**, which is -what makes a frame-boundary swap a non-event. +Redefinition is then one store. A redefinition module declares the cells `external`, exactly like the globals, and +exposes `flan_reload_install()` that stores its own body into its own cell — cost **below a microsecond**, which is what +makes a frame-boundary swap a non-event. -The cell load is emitted *after* the arguments, so a redefinition landing -between two calls cannot land in the middle of one. +The cell load is emitted *after* the arguments, so a redefinition landing between two calls cannot land in the middle of +one. Four things about this that are not free choices: -- **`flan_reload_install` is a named function and not an ELF constructor.** A - constructor runs during `dlopen`, on whatever thread called it, mid-frame. - The agent has to choose when the store happens. Loading and installing are - separate on purpose. -- **A redefinition's own body is `hidden`.** Default visibility in a shared - object is interposable, and that applies to *taking the address* too: plain - `@"flan.bump"` inside the module resolves to the host's copy, so the - installer would publish the very function it was replacing and the reload - would appear to do nothing. There is a test on the linkage, because the - failure is silent. -- **This also fixes the self-call edge**, which the previous version of this - section listed as a sharp edge: a redefined function calling itself goes - through the cell like any other call, so it reaches the new body. v2 of the - fixture recurses on purpose, and would print the old body's text if it did - not. -- **`-rdynamic` is what exports the cells**, so it and cells are one flag: - `Build.opts.dev`, `flan build --dev`. This is the first time `opts` means - something semantic rather than an optimisation level. +- **`flan_reload_install` is a named function and not an ELF constructor.** A constructor runs during `dlopen`, on +whatever thread called it, mid-frame. The agent has to choose when the store happens. Loading and installing are +separate on purpose. +- **A redefinition's own body is `hidden`.** Default visibility in a shared object is interposable, and that applies to +*taking the address* too: plain `@"flan.bump"` inside the module resolves to the host's copy, so the installer would +publish the very function it was replacing and the reload would appear to do nothing. There is a test on the linkage, +because the failure is silent. +- **This also fixes the self-call edge**, which the previous version of this section listed as a sharp edge: a redefined +function calling itself goes through the cell like any other call, so it reaches the new body. v2 of the fixture +recurses on purpose, and would print the old body's text if it did not. +- **`-rdynamic` is what exports the cells**, so it and cells are one flag: `Build.opts.dev`, `flan build --dev`. This is +the first time `opts` means something semantic rather than an optimisation level. -LLVM cannot fold the indirection away — the cell is an external mutable global -— and a `--dev` build of calc-me keeps 46 indirect calls at `-O2`. The -acceptance table now runs `values`, `machine` and `sand-headless` as dev builds -as well; the sand hash is the case that matters, since it is the one result -that would notice a call reaching the wrong function. +LLVM cannot fold the indirection away — the cell is an external mutable global — and a `--dev` build of calc-me keeps 46 +indirect calls at `-O2`. The acceptance table now runs `values`, `machine` and `sand-headless` as dev builds as well; +the sand hash is the case that matters, since it is the one result that would notice a call reaching the wrong function. ### Names that did not exist when the process started -Editing a `defvar` or a `defn` is a symbol the host exports. *Adding* one is -not: there is no symbol to bind to and ELF cannot grow one. Those go through -`runtime/flan_dev.c`, which is two lookups and nothing else: +Editing a `defvar` or a `defn` is a symbol the host exports. *Adding* one is not: there is no symbol to bind to and ELF +cannot grow one. Those go through `runtime/flan_dev.c`, which is two lookups and nothing else: ``` void **flan_dev_cell(const char *name); /* a new function's cell */ void *flan_dev_global(const char *name, uint64_t); /* a new global's storage */ ``` -Both are idempotent, so the second module to mention a name gets what the first -one got — which is the entire point. A new global's **declared initial value -travels with it**, as a constant the runtime copies on the allocation and -ignores on every call after: `calloc` alone is only right for ZII, and the -"ignores afterwards" half is where "a reload must not reset the program's -state" lives. Putting it in the allocation path rather than in a branch at the -call site means the rule cannot be got wrong at one of them. The compiler picks per name: a name the -host has is a symbol (one load at a call site), a name it lacks is a registry -lookup cached at install time in a module-local slot (two loads). So the common -case pays nothing for the general one. +Both are idempotent, so the second module to mention a name gets what the first one got — which is the entire point. A +new global's **declared initial value travels with it**, as a constant the runtime copies on the allocation and ignores +on every call after: `calloc` alone is only right for ZII, and the "ignores afterwards" half is where "a reload must not +reset the program's state" lives. Putting it in the allocation path rather than in a branch at the call site means the +rule cannot be got wrong at one of them. The compiler picks per name: a name the host has is a symbol (one load at a +call site), a name it lacks is a registry lookup cached at install time in a module-local slot (two loads). So the +common case pays nothing for the general one. -**The unit is a list of top-level forms**, not one function — `Emit.redefinition -~fns`. `C-c C-c` passes one name, `C-c C-k` passes a file's worth, one code -path either way. It has to be: v3 of the fixture adds `extra` and uses it from -a redefined `bump`, and splitting that into two loads would leave a module -referring to storage that does not exist yet. +**The unit is a list of top-level forms**, not one function — `Emit.redefinition ~fns`. `C-c C-c` passes one name, `C-c +C-k` passes a file's worth, one code path either way. It has to be: v3 of the fixture adds `extra` and uses it from a +redefined `bump`, and splitting that into two loads would leave a module referring to storage that does not exist yet. Four rules, each of which is a silent failure if broken: -- **Every lookup resolves before any body is published.** Publish first and a - caller reaches a function whose slots are still null. Not race-testable, so - it is asserted on the emitted `flan_reload_install`. -- **`flan_dev_global` refuses a size change.** The running process has already - laid that memory out; handing back the old allocation for a differently - shaped type means the new body reads fields at the wrong offsets and nothing - says so. This is the layout-drift rule's first enforcement point. Retyping a - var needs a restart. -- **Nothing is ever `dlclose`d.** A cell holds an address inside a module's - text; unloading it leaves every call site pointing at unmapped memory. That - is a constraint on the agent too. -- **The registry never moves.** A module holds a cell's address for as long as - it is loaded, so the table is fixed capacity with a loud failure rather than - growable. +- **Every lookup resolves before any body is published.** Publish first and a caller reaches a function whose slots are +still null. Not race-testable, so it is asserted on the emitted `flan_reload_install`. +- **`flan_dev_global` refuses a size change.** The running process has already laid that memory out; handing back the +old allocation for a differently shaped type means the new body reads fields at the wrong offsets and nothing says so. +This is the layout-drift rule's first enforcement point. Retyping a var needs a restart. +- **Nothing is ever `dlclose`d.** A cell holds an address inside a module's text; unloading it leaves every call site +pointing at unmapped memory. That is a constraint on the agent too. +- **The registry never moves.** A module holds a cell's address for as long as it is loaded, so the table is fixed +capacity with a loud failure rather than growable. -The test that separates this from a plausible wrong version is **v4**, which -redefines `added` — a name v3 introduced at run time. v3's `bump` is already -installed and is not rebuilt, so it picks v4 up only if its call goes through a -*cell* both modules found by the same name. Had v3 cached the function's -address instead, every other assertion would still pass and the transcript -would read 246 instead of 432. +The test that separates this from a plausible wrong version is **v4**, which redefines `added` — a name v3 introduced at +run time. v3's `bump` is already installed and is not rebuilt, so it picks v4 up only if its call goes through a *cell* +both modules found by the same name. Had v3 cached the function's address instead, every other assertion would still +pass and the transcript would read 246 instead of 432. -Sizes are spelled LLVM's way — `ptrtoint (ptr getelementptr (T, ptr null, i32 -1) to i64)` — rather than by a layout calculator in OCaml that would have to -agree with LLVM's on every target. +Sizes are spelled LLVM's way — `ptrtoint (ptr getelementptr (T, ptr null, i32 1) to i64)` — rather than by a layout +calculator in OCaml that would have to agree with LLVM's on every target. ### The agent — dev loop step 3 -`vendor/agent/` is a package like any other: `agent.flan` declares three calls, -`flan_agent.c` implements them, `link` asks for `-lpthread`. +`vendor/agent/` is a package like any other: `agent.flan` declares three calls, `flan_agent.c` implements them, `link` +asks for `-lpthread`. ``` (agent/start path) listen on a unix socket; once, at startup @@ -804,163 +621,126 @@ agree with LLVM's on every target. (agent/wait ms) the same, but waits for something first ``` -The split between them is the design. `dlopen` relocates a module and takes the -loader lock — milliseconds, unbounded — so it happens on the listener thread. -`flan_reload_install` is one store per function and must not land while a -redefined function is on the stack, so it happens on the game thread, at the -top of the frame, when the program asks. The two are connected by a -single-producer/single-consumer ring and two atomics; the game thread never -blocks on the loader. +The split between them is the design. `dlopen` relocates a module and takes the loader lock — milliseconds, unbounded — +so it happens on the listener thread. `flan_reload_install` is one store per function and must not land while a +redefined function is on the stack, so it happens on the game thread, at the top of the frame, when the program asks. +The two are connected by a single-producer/single-consumer ring and two atomics; the game thread never blocks on the +loader. -`wait` exists for tests. A test that races the frame rate fails on a loaded -machine, so `test/programs/agent.flan` waits for the reload instead of sleeping -past it. It takes **two** reloads, which is the daemon's actual loop: the first -introduces a global the process was never built with, the second only reads it, -and the second can only answer 1007 if it found the storage the first one -allocated rather than a fresh zeroed copy. One reload would not have shown -that. +`wait` exists for tests. A test that races the frame rate fails on a loaded machine, so `test/programs/agent.flan` waits +for the reload instead of sleeping past it. It takes **two** reloads, which is the daemon's actual loop: the first +introduces a global the process was never built with, the second only reads it, and the second can only answer 1007 if +it found the storage the first one allocated rather than a fresh zeroed copy. One reload would not have shown that. Two details found by running it: -- **stdout is line buffered**, set in `flan_rt_init`. The C default when - stdout is a file or a pipe is a 4K block, so a program running with a REPL - attached shows nothing until it exits — and a test driving one cannot see - its progress at all, which is how this was found. -- **The reply goes out before the module is queued.** The other way round, the - game thread can install and the program can exit between the two, and the - answer reaches the sender as a connection reset rather than as `ok`. -- **`ok` means queued, not installed.** The sender does not get to know when - the swap happened; only the program knows when it is between frames. +- **stdout is line buffered**, set in `flan_rt_init`. The C default when stdout is a file or a pipe is a 4K block, so a +program running with a REPL attached shows nothing until it exits — and a test driving one cannot see its progress at +all, which is how this was found. +- **The reply goes out before the module is queued.** The other way round, the game thread can install and the program +can exit between the two, and the answer reaches the sender as a connection reset rather than as `ok`. +- **`ok` means queued, not installed.** The sender does not get to know when the swap happened; only the program knows +when it is between frames. -**sand.flan calls `agent/poll` at the top of its loop**, which is what step 3 -was for. Verified: with sand running under Xvfb, `flan reload sand-probe.flan -game-draw` and one line on the socket, and 455 consecutive frames drew from a -body that did not exist when the process started. Building without `--dev` is -fine — there are no cells, so a module is refused on the listener thread and -the loop never notices. +**sand.flan calls `agent/poll` at the top of its loop**, which is what step 3 was for. Verified: with sand running under +Xvfb, `flan reload sand-probe.flan game-draw` and one line on the socket, and 455 consecutive frames drew from a body +that did not exist when the process started. Building without `--dev` is fine — there are no cells, so a module is +refused on the listener thread and the loop never notices. -`flan reload ... [-o out.so] [--new name,...]` builds one -module the way the daemon will. `--new` is the names the host was *not* built -with; it is the one thing the command cannot work out for itself, and it is -exactly what the session will track automatically. +`flan reload ... [-o out.so] [--new name,...]` builds one module the way the daemon will. `--new` is the +names the host was *not* built with; it is the one thing the command cannot work out for itself, and it is exactly what +the session will track automatically. ### The session -`lib/session.ml` is the program as a live thing: the declarations the running -process was built from, plus every change accepted since. +`lib/session.ml` is the program as a live thing: the declarations the running process was built from, plus every change +accepted since. -**Transactionality came for free and needed no machinery.** `Check.program` -builds a fresh environment from a declaration list on every call, so a form -that fails to check mutates nothing — the accumulated list is simply not -replaced. Re-checking the whole program each evaluation costs the entire -frontend, under 10ms, less than the `llc` that follows. There is a test for the -case that actually matters: a typo, then a good form, in the same session. +**Transactionality came for free and needed no machinery.** `Check.program` builds a fresh environment from a +declaration list on every call, so a form that fails to check mutates nothing — the accumulated list is simply not +replaced. Re-checking the whole program each evaluation costs the entire frontend, under 10ms, less than the `llc` that +follows. There is a test for the case that actually matters: a typo, then a good form, in the same session. Two things the session knows that no single evaluation could: -- **Which names the running process was built with.** It comes from the - *checked* program, not from any accumulated AST, because `Check.program` - prepends the prelude and no AST contains it. Derive it from declarations and - `print-line` reads as new, gets a registry cell nobody publishes, and the - first call jumps to null with no diagnostic. -- **What that process's memory looks like.** Three changes are refused with a - reason rather than loaded: +- **Which names the running process was built with.** It comes from the *checked* program, not from any accumulated AST, +because `Check.program` prepends the prelude and no AST contains it. Derive it from declarations and `print-line` reads +as new, gets a registry cell nobody publishes, and the first call jumps to null with no diagnostic. +- **What that process's memory looks like.** Three changes are refused with a reason rather than loaded: - | Change | What it would have broken | - |---|---| - | a function's signature | a cell is a bare `ptr`; every call site compiled before the change still passes the old arguments through it — **and this is now a stopgap**, see below | - | a global's type | the storage exists and has a shape — reuse reads at the wrong offsets, replacement discards the state the reload exists to preserve | - | a struct's fields | the values the process is holding have the old layout | - | a `defconst`'s value, **when the checker consumed it** | it is in the *shape* of the program — `(defconst rows (/ h c))` decides `grid`'s type before anything else resolves — so no store can reach it | - | a `defenum` member | `:space` is erased to an `i32` literal in the caller, so it is folded there too | +| Change | What it would have broken | +|---|---| +| a function's signature | a cell is a bare `ptr`; every call site compiled before the change still passes the old arguments through it — **and this is now a stopgap**, see below | +| a global's type | the storage exists and has a shape — reuse reads at the wrong offsets, replacement discards the state the reload exists to preserve | +| a struct's fields | the values the process is holding have the old layout | +| a `defconst`'s value, **when the checker consumed it** | it is in the *shape* of the program — `(defconst rows (/ h c))` decides `grid`'s type before anything else resolves — so no store can reach it | +| a `defenum` member | `:space` is erased to an `i32` literal in the caller, so it is folded there too | - **The signature row is the one the plan has moved past.** plan.org now says a - signature-changing redefinition should make a new internal function version - with its own trampoline: newly compiled code resolves the name to it, while - existing callers and stored `Fn` values keep the old version and stay safe, - and the session warns at every tracked caller site still targeting the old - signature — recompiling one either retargets it or gives an ordinary type - error. Open decision #6 records it the same way. None of the three parts - exists: there are no function versions, no trampolines (a cell holds a body - address today), and no record of which source locations called what. So the - refusal stays, because the alternative to refusing is not the new design, it - is a silent argument mismatch. It is a stopgap and the message should not be - read as the final answer. +**The signature row is the one the plan has moved past.** plan.org now says a signature-changing redefinition should +make a new internal function version with its own trampoline: newly compiled code resolves the name to it, while +existing callers and stored `Fn` values keep the old version and stay safe, and the session warns at every tracked +caller site still targeting the old signature — recompiling one either retargets it or gives an ordinary type error. +Open decision #6 records it the same way. None of the three parts exists: there are no function versions, no trampolines +(a cell holds a body address today), and no record of which source locations called what. So the refusal stays, because +the alternative to refusing is not the new design, it is a silent argument mismatch. It is a stopgap and the message +should not be read as the final answer. - A `defvar`'s *initial value* is deliberately **not** in that table. Its - storage holds live state the program moved past long ago, and refusing to - change the initialiser would be refusing "edit the code, keep the sand". Same - `Tast.global` record as a `defconst`, opposite answers, told apart by - `gconst`. The enum comparison runs over declarations rather than the checked - program, because `Tast.program` carries no enums at all — they are erased to - `i32` in the checker, which is the same fact that makes them unreloadable. +A `defvar`'s *initial value* is deliberately **not** in that table. Its storage holds live state the program moved past +long ago, and refusing to change the initialiser would be refusing "edit the code, keep the sand". Same `Tast.global` +record as a `defconst`, opposite answers, told apart by `gconst`. The enum comparison runs over declarations rather than +the checked program, because `Tast.program` carries no enums at all — they are erased to `i32` in the checker, which is +the same fact that makes them unreloadable. - Note what the checker catches on its own: change `helper`'s parameter type - and the *caller* fails to type check first, loudly. The session's rules only - get a turn on a change the checker accepts — one to a name nothing else in - the program uses, which is exactly where the silent version lives. The - fixtures carry an unused `defvar` and a C-called `defn` for that reason. +Note what the checker catches on its own: change `helper`'s parameter type and the *caller* fails to type check first, +loudly. The session's rules only get a turn on a change the checker accepts — one to a name nothing else in the program +uses, which is exactly where the silent version lives. The fixtures carry an unused `defvar` and a C-called `defn` for +that reason. -A `defconst` the checker never consumed is a different matter and **can** be -changed: it is only ever bytes in memory. A dev build emits every `defconst` as -a mutable `global` rather than a `constant` — so LLVM cannot fold a read of it -and a module can store into it — and a changed one is published at the frame -boundary exactly as a new function body is. That is how sand's `colors` gets -tuned live while `rows` stays refused. Release builds emit `constant` and get -all the folding back; `Tast.global.gfolded` is what tells the two apart, because -nothing downstream of the checker could. +A `defconst` the checker never consumed is a different matter and **can** be changed: it is only ever bytes in memory. A +dev build emits every `defconst` as a mutable `global` rather than a `constant` — so LLVM cannot fold a read of it and a +module can store into it — and a changed one is published at the frame boundary exactly as a new function body is. That +is how sand's `colors` gets tuned live while `rows` stays refused. Release builds emit `constant` and get all the +folding back; `Tast.global.gfolded` is what tells the two apart, because nothing downstream of the checker could. -**A form typed into a file that is imported as a package is qualified the way -the import qualified it.** `poll` in `vendor/agent/agent.flan` becomes -`agent/poll`, and its call to `poll-raw` becomes `agent/poll-raw` — through `Load`'s own -`qualify_decl`, so the rule cannot drift from the one used at import time. -Without this the form spliced as a brand-new unrelated name: the evaluation -answered `ok`, and the running program went on calling the `sim/settle` it -already had. Since sand's simulation lives in a package, the one thing worth -tuning live was the one thing that silently did nothing. +**A form typed into a file that is imported as a package is qualified the way the import qualified it.** `poll` in +`vendor/agent/agent.flan` becomes `agent/poll`, and its call to `poll-raw` becomes `agent/poll-raw` — through `Load`'s +own `qualify_decl`, so the rule cannot drift from the one used at import time. Without this the form spliced as a +brand-new unrelated name: the evaluation answered `ok`, and the running program went on calling the `sim/settle` it +already had. Since sand's simulation lives in a package, the one thing worth tuning live was the one thing that silently +did nothing. -It is derived from the file's path and **not sent by the editor**, which is -where this departs from CIDER's `ns` key: a Clojure namespace is declared in -the file, but a Flan alias is chosen by whatever imported the directory and is -written nowhere the editor can see. One directory imported under two aliases is -refused with the reason rather than resolved to either. +It is derived from the file's path and **not sent by the editor**, which is where this departs from CIDER's `ns` key: a +Clojure namespace is declared in the file, but a Flan alias is chosen by whatever imported the directory and is written +nowhere the editor can see. One directory imported under two aliases is refused with the reason rather than resolved to +either. -The accumulated list is the **post-`Load`** one, so an evaluated `(import …)` -is spliced as its expansion. Otherwise re-evaluating a file that imports -something appends a second import, `Load` expands it again, and the -duplicate-name pass rejects it. `C-c C-k` on sand.flan's own text is the test. +The accumulated list is the **post-`Load`** one, so an evaluated `(import …)` is spliced as its expansion. Otherwise +re-evaluating a file that imports something appends a second import, `Load` expands it again, and the duplicate-name +pass rejects it. `C-c C-k` on sand.flan's own text is the test. -`flan reload ` is that path from the command line: a -session over the program the process was built from, and a file of the forms -that changed. Verified against a running sand under Xvfb — a one-form -`game-draw` and 910 consecutive frames drew it. +`flan reload ` is that path from the command line: a session over the program the process was +built from, and a file of the forms that changed. Verified against a running sand under Xvfb — a one-form `game-draw` +and 910 consecutive frames drew it. -Two limits of that command specifically, neither of them true of sessions: -it builds a fresh session from source on every invocation, so if the program -file has been edited since the process launched, its idea of which names the -host has and what its memory looks like describes a binary that is not running. -And `Session.eval`'s `origin` defaults to ``, so an error in forms sent -without one reports positions in a file that does not exist — the daemon has to -pass the real buffer path, which is the same key CIDER's `eval` carries. +Two limits of that command specifically, neither of them true of sessions: it builds a fresh session from source on +every invocation, so if the program file has been edited since the process launched, its idea of which names the host +has and what its memory looks like describes a binary that is not running. And `Session.eval`'s `origin` defaults to +``, so an error in forms sent without one reports positions in a file that does not exist — the daemon has to pass +the real buffer path, which is the same key CIDER's `eval` carries. ### The daemon — `flan dev` -`flan dev ` holds one `Session`, builds the program, launches it, -and listens on `.flan-dev.sock` beside the source. What it adds over `flan -reload` is that the session *persists* — a `defvar` added by one evaluation is -part of what the next one is checked against — and that it **owns the build**, -which is what makes its layout rules describe the process that is actually -running rather than a guess about it. +`flan dev ` holds one `Session`, builds the program, launches it, and listens on `.flan-dev.sock` beside +the source. What it adds over `flan reload` is that the session *persists* — a `defvar` added by one evaluation is part +of what the next one is checked against — and that it **owns the build**, which is what makes its layout rules describe +the process that is actually running rather than a guess about it. -**The protocol is s-expressions, not bencode.** nREPL was the plan and the -argument for it evaporated once the client became ours too: there is no CIDER -to be compatible with, `eval` is string-in/string-out with no slot for *which -form, from which file*, and Emacs already has `read` and `prin1`. So it is one -sexp per message — no parsing code on the editor side, and on this side the -parser is the language's own reader, where `:op` is already a keyword and a -payload of Flan source is already a string literal. Framing is a decimal byte -count and a newline, because the payload contains newlines. An nREPL front end -can sit on the same `Session` later; it should not have gated the editor. +**The protocol is s-expressions, not bencode.** nREPL was the plan and the argument for it evaporated once the client +became ours too: there is no CIDER to be compatible with, `eval` is string-in/string-out with no slot for *which form, +from which file*, and Emacs already has `read` and `prin1`. So it is one sexp per message — no parsing code on the +editor side, and on this side the parser is the language's own reader, where `:op` is already a keyword and a payload of +Flan source is already a string literal. Framing is a decimal byte count and a newline, because the payload contains +newlines. An nREPL front end can sit on the same `Session` later; it should not have gated the editor. ``` (:op "describe") → (:status "ok" :fns (…) :globals (…) :alive t) @@ -970,59 +750,46 @@ can sit on the same `Session` later; it should not have gated the editor. (:op "close") ``` -`defs` is its own op rather than more fields on `describe`, because `describe` -is what an editor *polls* — it is how the program's output is drained — and -signatures on that would be paid for every time anyone glanced at the output -buffer. It is asked once on connect and again after each accepted install. -Four strings an editor reads with `read` and nothing else: eldoc, completion -and find-definition want the same three facts about a name. `loc` is empty -where there is none to give, because only `Tast.fn` carries one — an editor -must refuse rather than go looking for the definition itself, which in a -program of several files finds the wrong one. Parameter *names* are not in the -Tast, so a signature is `step [i64 f32] i64`: types only. +`defs` is its own op rather than more fields on `describe`, because `describe` is what an editor *polls* — it is how the +program's output is drained — and signatures on that would be paid for every time anyone glanced at the output buffer. +It is asked once on connect and again after each accepted install. Four strings an editor reads with `read` and nothing +else: eldoc, completion and find-definition want the same three facts about a name. `loc` is empty where there is none +to give, because only `Tast.fn` carries one — an editor must refuse rather than go looking for the definition itself, +which in a program of several files finds the wrong one. Parameter *names* are not in the Tast, so a signature is `step +[i64 f32] i64`: types only. -The daemon makes its own source path absolute before building, because every -location it reports derives from it. `flan dev src/game.flan` run from a -project root otherwise answered `src/game.flan:12:7`, which an editor can only -resolve by guessing which directory it was relative to. +The daemon makes its own source path absolute before building, because every location it reports derives from it. `flan +dev src/game.flan` run from a project root otherwise answered `src/game.flan:12:7`, which an editor can only resolve by +guessing which directory it was relative to. -An evaluation that declares nothing to install — a declaration the program -already has, with no body and no new storage — is accepted and answered with -`:note "nothing to install"` rather than by shipping an empty module. Building -one anyway reports success for a change that cannot have taken effect, and -costs the program a reload it did not need. +An evaluation that declares nothing to install — a declaration the program already has, with no body and no new storage +— is accepted and answered with `:note "nothing to install"` rather than by shipping an empty module. Building one +anyway reports success for a change that cannot have taken effect, and costs the program a reload it did not need. -`:file` is not decoration: `Session.eval`'s origin defaults to ``, so -without it every error an editor shows points into a file that does not exist. +`:file` is not decoration: `Session.eval`'s origin defaults to ``, so without it every error an editor shows +points into a file that does not exist. -Two things the daemon must not paper over, both of which would look like a -successful evaluation: +Two things the daemon must not paper over, both of which would look like a successful evaluation: -- **The agent socket is chosen by the daemon**, not by the program. A program's - source has to name some path — sand.flan says `/tmp/flan-sand.sock` — and the - daemon overrides it through `FLAN_AGENT_SOCKET` before spawning. Guessing - instead fails silently: the module compiles, is built, and nobody receives - it. -- **Delivery is checked.** `agent/start` returning 0 means a socket was bound, - not that anyone connected. A failed connect or a reply that is not `ok` - becomes an error the editor sees. +- **The agent socket is chosen by the daemon**, not by the program. A program's source has to name some path — sand.flan +says `/tmp/flan-sand.sock` — and the daemon overrides it through `FLAN_AGENT_SOCKET` before spawning. Guessing instead +fails silently: the module compiles, is built, and nobody receives it. +- **Delivery is checked.** `agent/start` returning 0 means a socket was bound, not that anyone connected. A failed +connect or a reply that is not `ok` becomes an error the editor sees. -It waits for the program to bind before accepting an evaluation — one arriving -first would fail for a reason that reads like a compiler bug — and it accepts -with a timeout so that a program which has exited takes the daemon with it -rather than leaving an editor waiting on a socket nobody is serving. +It waits for the program to bind before accepting an evaluation — one arriving first would fail for a reason that reads +like a compiler bug — and it accepts with a timeout so that a program which has exited takes the daemon with it rather +than leaving an editor waiting on a socket nobody is serving. ### The Emacs client -`emacs/flan-mode.el` derives from `prog-mode` with `lisp-mode`'s syntax table, -which is most of the work: Flan is s-expressions, so sexp motion, paren -matching, `beginning-of-defun` and indentation are already right. What it adds -is Flan's own bracket syntax (`[` and `{` are brackets, not symbol characters — -every binding list and every type is written with them), the characters a Flan -name may contain (`-`, `?`, `/`, `.`), and its keywords. +`emacs/flan-mode.el` derives from `prog-mode` with `lisp-mode`'s syntax table, which is most of the work: Flan is +s-expressions, so sexp motion, paren matching, `beginning-of-defun` and indentation are already right. What it adds is +Flan's own bracket syntax (`[` and `{` are brackets, not symbol characters — every binding list and every type is +written with them), the characters a Flan name may contain (`-`, `?`, `/`, `.`), and its keywords. -`emacs/flan-dev.el` is the client. There is no parser in it, which is the point -of the protocol choice: `prin1` writes a request and `read` reads a reply. +`emacs/flan-dev.el` is the client. There is no parser in it, which is the point of the protocol choice: `prin1` writes a +request and `read` reads a reply. | | | |---|---| @@ -1036,94 +803,81 @@ of the protocol choice: `prin1` writes a request and `read` reads a reply. | `C-c C-d` | what the running program currently defines | | `M-.` / `M-,` | where a name is written, through an `xref` backend | -eldoc, `completion-at-point` and `M-.` all read one cached `defs` reply rather -than asking per keystroke: eldoc fires on an idle timer and completion inside -redisplay, and neither may block on a socket or signal. The cache is refreshed -at the two moments the answer can have changed — on connect, and after an -evaluation the daemon accepted — so a `defn` just installed completes at once. +eldoc, `completion-at-point` and `M-.` all read one cached `defs` reply rather than asking per keystroke: eldoc fires on +an idle timer and completion inside redisplay, and neither may block on a socket or signal. The cache is refreshed at +the two moments the answer can have changed — on connect, and after an evaluation the daemon accepted — so a `defn` just +installed completes at once. -The modeline says whether there is a program on the other end, in three states. -`lost` is a daemon that has gone away, which is ordinary rather than an error — -`flan dev` ends when its program does — so the next request reconnects on the -socket it was on. Strictly *before* a send, never after one: a connection that -died mid-request may have died after the daemon ran what it was given, and -resending would install it twice or evaluate a side-effecting expression twice. +The modeline says whether there is a program on the other end, in three states. `lost` is a daemon that has gone away, +which is ordinary rather than an error — `flan dev` ends when its program does — so the next request reconnects on the +socket it was on. Strictly *before* a send, never after one: a connection that died mid-request may have died after the +daemon ran what it was given, and resending would install it twice or evaluate a side-effecting expression twice. -`C-c C-k` sends one module rather than a form at a time on purpose: a `defvar` -and the function that uses it have to arrive in the same load, or the first -refers to storage that does not exist yet. +`C-c C-k` sends one module rather than a form at a time on purpose: a `defvar` and the function that uses it have to +arrive in the same load, or the first refers to storage that does not exist yet. -**Framing is in bytes and Emacs counts characters.** Every length goes through -`string-bytes` and the process is binary, or a single non-ASCII character in a -buffer puts the reply stream out of step by exactly as many bytes as the -payload has of them — a bug that would look like a corrupt protocol and appear -only for some users. `test/test_emacs.ml` drives the real client against a real -daemon for this reason: it is not the same claim as the daemon answering -correctly, and a mistake in the framing, in `beginning-of-defun` over Flan's -syntax table, or in the reply reader passes `test_dev.ml` and fails here. +**Framing is in bytes and Emacs counts characters.** Every length goes through `string-bytes` and the process is binary, +or a single non-ASCII character in a buffer puts the reply stream out of step by exactly as many bytes as the payload +has of them — a bug that would look like a corrupt protocol and appear only for some users. `test/test_emacs.ml` drives +the real client against a real daemon for this reason: it is not the same claim as the daemon answering correctly, and a +mistake in the framing, in `beginning-of-defun` over Flan's syntax table, or in the reply reader passes `test_dev.ml` +and fails here. -An error comes back with a location and the client draws an overlay there, with -the message beside the code, cleared the next time that buffer's evaluation is -accepted. Two things had to be right first. **The column in a `:loc` is a byte -offset**, because the reader walks the source a byte at a time — the same rule -as the framing, in a different place, and `forward-char` with it put the marker -as many columns right as the line had non-ASCII characters before it. And the -daemon numbers lines from the start of what it was *sent*, so `C-c C-c` on a -defn halfway down a buffer answered line 1 and every overlay would have sat on -the file's first line; the client pads the form with leading newlines, which -the reader skips, so the reply's line numbers are the buffer's own. +An error comes back with a location and the client draws an overlay there, with the message beside the code, cleared the +next time that buffer's evaluation is accepted. Two things had to be right first. **The column in a `:loc` is a byte +offset**, because the reader walks the source a byte at a time — the same rule as the framing, in a different place, and +`forward-char` with it put the marker as many columns right as the line had non-ASCII characters before it. And the +daemon numbers lines from the start of what it was *sent*, so `C-c C-c` on a defn halfway down a buffer answered line 1 +and every overlay would have sat on the file's first line; the client pads the form with leading newlines, which the +reader skips, so the reply's line numbers are the buffer's own. -An accepted evaluation says which names landed and what the build cost, and -flashes the region that was sent. Silent success is indistinguishable from -silent failure, and `beginning-of-defun` may well have found a different form -from the one point looked like it was in. +An accepted evaluation says which names landed and what the build cost, and flashes the region that was sent. Silent +success is indistinguishable from silent failure, and `beginning-of-defun` may well have found a different form from the +one point looked like it was in. -**The program's stdout is a pipe into the daemon**, and whatever it printed -since the last reply rides along with the next one into `*flan-output*`. Having -it arrive *with* a reply rather than by a separate request is the point: the -output an evaluation itself caused is the output anyone wants to see. Draining -that pipe is a liveness requirement and not a nicety — a pipe nobody reads -fills at 64K and the next write blocks the program forever — so it is read from -the accept loop's `select`, not only when an editor asks, and the buffer is -capped so a program printing every frame cannot grow the daemon without -limit. +**TODO — live disassembly.** Add an editor command that asks the live Flan session for a named function's generated code +and shows a disassembly (with the source location and reload generation it came from). `flan emit --dev` can show LLVM +IR for a whole source file today, but neither the daemon nor Emacs can inspect the native code currently installed in an +indirection cell. There should be options to both show the LLVM IR but also the target platform assembly, which in my +case is amd64, if that's possible. Take a look at how SBCL does it for reference. + +**TODO — transient error overlays.** The diagnostic ghost text should vanish as soon as the user does anything else in +that buffer — edit, move, evaluate, or invoke another command — rather than surviving until a later evaluation is +accepted. It is feedback about the action that just failed, not a durable annotation on the source. + +**The program's stdout is a pipe into the daemon**, and whatever it printed since the last reply rides along with the +next one into `*flan-output*`. Having it arrive *with* a reply rather than by a separate request is the point: the +output an evaluation itself caused is the output anyone wants to see. Draining that pipe is a liveness requirement and +not a nicety — a pipe nobody reads fills at 64K and the next write blocks the program forever — so it is read from the +accept loop's `select`, not only when an editor asks, and the buffer is capped so a program printing every frame cannot +grow the daemon without limit. ### `C-x C-e` — evaluating an expression -A different primitive from redefining a name, and the difference is the whole -design. There is no name to install a body into, so the expression is wrapped -in a function with nowhere to be called from; the module says *run this once* -by exporting `flan_reload_call`, and the agent calls it after the install — on -the game thread, at a frame boundary, so an expression reading the program's -state sees a point the program agrees is consistent. +A different primitive from redefining a name, and the difference is the whole design. There is no name to install a body +into, so the expression is wrapped in a function with nowhere to be called from; the module says *run this once* by +exporting `flan_reload_call`, and the agent calls it after the install — on the game thread, at a frame boundary, so an +expression reading the program's state sees a point the program agrees is consistent. -**Nothing is marshalled back, because nothing could be.** A Flan value carries -no header, so no code at run time can say what it is. The compiler knows the -type and renders it *there*, in the thunk, into `flan_dev_result`. That is the -layout decision's bill, and it is why the printer set is small rather than -universal. +**Nothing is marshalled back, because nothing could be.** A Flan value carries no header, so no code at run time can say +what it is. The compiler knows the type and renders it *there*, in the thunk, into `flan_dev_result`. That is the layout +decision's bill, and it is why the printer set is small rather than universal. -It does not go through stdout. Stdout belongs to the program, it is in the hot -path for anything that prints, and a dev-only feature must not put a branch in -it — so `flan_rt.c` is untouched and the value is read back over the agent's -socket. The read is safe without a handshake because `flan_dev_result` bumps a -generation counter last; the daemon waits for it to move rather than assuming -the program has reached a frame boundary. +It does not go through stdout. Stdout belongs to the program, it is in the hot path for anything that prints, and a +dev-only feature must not put a branch in it — so `flan_rt.c` is untouched and the value is read back over the agent's +socket. The read is safe without a handshake because `flan_dev_result` bumps a generation counter last; the daemon waits +for it to move rather than assuming the program has reached a frame boundary. -**This renderer is most of `println`**, which is worth knowing before anyone -schedules it. plan.org describes a compiler-provided, type-directed intrinsic -that selects or emits a structural printer per concrete instantiation, prints -structs, fixed arrays and options structurally, prints a `Ptr` as its address -rather than following it, and bounds depth and length. That is a description of -what `Session` already does for `C-x C-e` — same walk, same refusals, same -three bounds — aimed at `flan_dev_emit` and the wire instead of at stdout. What -`println` needs on top is a stdout sink, a builtin that takes its printer from -the argument's type, and the `any`/`Error` dynamic cases, which have no -compile-time type to walk. Not the walk itself. +**This renderer is most of `println`**, which is worth knowing before anyone schedules it. plan.org describes a +compiler-provided, type-directed intrinsic that selects or emits a structural printer per concrete instantiation, prints +structs, fixed arrays and options structurally, prints a `Ptr` as its address rather than following it, and bounds depth +and length. That is a description of what `Session` already does for `C-x C-e` — same walk, same refusals, same three +bounds — aimed at `flan_dev_emit` and the wire instead of at stdout. What `println` needs on top is a stdout sink, a +builtin that takes its printer from the argument's type, and the `any`/`Error` dynamic cases, which have no compile-time +type to walk. Not the walk itself. -The renderer is a **compile-time walk over the type**, emitting a piece at a -time through `flan_dev_emit`. Piecewise because a struct is its fields with -punctuation between them, and concatenating that in generated IR would need an +The renderer is a **compile-time walk over the type**, emitting a piece at a time through `flan_dev_emit`. Piecewise +because a struct is its fields with punctuation between them, and concatenating that in generated IR would need an allocator the language does not have. ``` @@ -1138,155 +892,120 @@ sim/grid [ [ 0 0 0 0 0 0 0 0 ...] [ 0 ... ] ...] Details that are decisions rather than formatting: -- **`u64` renders in C**, with `%llu`. The language's own `i64->bytes` is - signed, so it used to refuse rather than come back as `-1` — but refusing a - whole struct because one field is a `u64` is much worse, so the runtime got - an entry point instead. -- **Strings are quoted and escaped**, also in C. Unescaped content does not - round-trip and reads as a framing bug rather than as the value it is. -- **An enum renders as `:name`**, recovered from the checker's table as a chain - of comparisons, because members are erased to `i32` before the backend sees - them. A value outside the declared members falls through to its number, which - is exactly what you would want to see. -- **A pointer is never followed** — ``. It is the only thing that could - make the walk cycle, and dereferencing one a REPL was handed is not a safe - thing to do on someone's behalf. -- **Three separate bounds**, easy to conflate. `depth` (4) and `span` (8) bound - the *walk*, so `[100 [100 u32]]` does not become ten thousand render sites in - one module. The *output* is bounded once in the runtime — `emit` truncates at - 4K and `end` appends `...` — because a slice renders through a loop the - compiler cannot bound, and one place enforcing it means no renderer carries a - budget. -- A slice is the one case needing a runtime loop, and the slice goes into a - slot first so the expression it came from is not evaluated once per element. +- **`u64` renders in C**, with `%llu`. The language's own `i64->bytes` is signed, so it used to refuse rather than come +back as `-1` — but refusing a whole struct because one field is a `u64` is much worse, so the runtime got an entry point +instead. +- **Strings are quoted and escaped**, also in C. Unescaped content does not round-trip and reads as a framing bug rather +than as the value it is. +- **An enum renders as `:name`**, recovered from the checker's table as a chain of comparisons, because members are +erased to `i32` before the backend sees them. A value outside the declared members falls through to its number, which is +exactly what you would want to see. +- **A pointer is never followed** — ``. It is the only thing that could make the walk cycle, and dereferencing one +a REPL was handed is not a safe thing to do on someone's behalf. +- **Three separate bounds**, easy to conflate. `depth` (4) and `span` (8) bound the *walk*, so `[100 [100 u32]]` does +not become ten thousand render sites in one module. The *output* is bounded once in the runtime — `emit` truncates at 4K +and `end` appends `...` — because a slice renders through a loop the compiler cannot bound, and one place enforcing it +means no renderer carries a budget. +- A slice is the one case needing a runtime loop, and the slice goes into a slot first so the expression it came from is +not evaluated once per element. What still refuses by name: `Map`, `Fn`, a type variable. -A caveat inherited from the language, not introduced here: `3.0` renders as -`3`, indistinguishable from the integer. `flan run calc-me.flan "1.5 * 2.0"` -has always said `3`. +A caveat inherited from the language, not introduced here: `3.0` renders as `3`, indistinguishable from the integer. +`flan run calc-me.flan "1.5 * 2.0"` has always said `3`. -An evaluation is **not** a declaration: the thunk is built against the program -and never spliced into it, so `describe` does not fill up with `eval/N` for -every expression ever typed. +An evaluation is **not** a declaration: the thunk is built against the program and never spliced into it, so `describe` +does not fill up with `eval/N` for every expression ever typed. -**The module is unloaded afterwards**, which is the one case where that is -safe. The thunk is called directly by `flan_reload_call` rather than through a -cell, and it takes no registry slot — so once it has returned, nothing points -into its text and the value it produced has been copied out. It declares that -with `@flan_reload_transient` and the agent `dlclose`s it. Measured: sixteen -expression evaluations retain **zero** mappings, where each *redefinition* -retains three, permanently and correctly — a module that publishes a body -exists precisely to leave a pointer behind, and can never claim this. +**The module is unloaded afterwards**, which is the one case where that is safe. The thunk is called directly by +`flan_reload_call` rather than through a cell, and it takes no registry slot — so once it has returned, nothing points +into its text and the value it produced has been copied out. It declares that with `@flan_reload_transient` and the +agent `dlclose`s it. Measured: sixteen expression evaluations retain **zero** mappings, where each *redefinition* +retains three, permanently and correctly — a module that publishes a body exists precisely to leave a pointer behind, +and can never claim this. -Skipping the registry matters for more than tidiness: the table holds 4096 -names and an expression evaluated in a loop would exhaust it. +Skipping the registry matters for more than tidiness: the table holds 4096 names and an expression evaluated in a loop +would exhaust it. -The test that matters is the same expression twice: the fixture increments -`ticks` every frame, so two evaluations must disagree. A value computed in the -compiler, or read out of a copy of the program's state, would not. +The test that matters is the same expression twice: the fixture increments `ticks` every frame, so two evaluations must +disagree. A value computed in the compiler, or read out of a copy of the program's state, would not. ### The REPL buffer -`flan-repl.el` is a `comint-mode` buffer whose every line goes through the same -`eval-expr` request `C-x C-e` uses. No new protocol and no compiler support. -Deriving from `comint` rather than hand-rolling a prompt is the same call as -deriving `flan-mode` from `lisp-mode`: history, the input ring and kill/yank -already exist. There is no subprocess behind it — the "process" is a stub -comint needs in order to have a prompt at all. +`flan-repl.el` is a `comint-mode` buffer whose every line goes through the same `eval-expr` request `C-x C-e` uses. No +new protocol and no compiler support. Deriving from `comint` rather than hand-rolling a prompt is the same call as +deriving `flan-mode` from `lisp-mode`: history, the input ring and kill/yank already exist. There is no subprocess +behind it — the "process" is a stub comint needs in order to have a prompt at all. Three things about it that are decisions: -- **It is program-scoped.** A name typed at the prompt resolves against the - running program's top-level namespace, so in sand you write `sim/settle` and - not `settle`. A buffer visiting a package's own file gets the alias applied - for it because the file says which package it belongs to; a prompt has no - file and nothing to derive one from. -- **RET on a half-typed form opens a line instead of sending it.** Balance is - checked with the Flan syntax table, so a paren inside a string does not - count. -- **A value and the program's output are different things and arrive by - different routes.** The value is the result of the request and appears at the - prompt; anything the program printed while evaluating it rides along on the - same reply and goes to `*flan-output*`. Showing them in one place would be - convenient and wrong, so there is a test for the separation. +- **It is program-scoped.** A name typed at the prompt resolves against the running program's top-level namespace, so in +sand you write `sim/settle` and not `settle`. A buffer visiting a package's own file gets the alias applied for it +because the file says which package it belongs to; a prompt has no file and nothing to derive one from. +- **RET on a half-typed form opens a line instead of sending it.** Balance is checked with the Flan syntax table, so a +paren inside a string does not count. +- **A value and the program's output are different things and arrive by different routes.** The value is the result of +the request and appears at the prompt; anything the program printed while evaluating it rides along on the same reply +and goes to `*flan-output*`. Showing them in one place would be convenient and wrong, so there is a test for the +separation. -That test is what caught a real bug: the renderer's `Unit` case emitted `()` -without evaluating the expression, so `(print-line "x")` — the most ordinary -thing anyone types at a prompt — answered `()` while nothing happened. A Unit -expression is almost always a call made for its effect, and is now evaluated -and *then* reported. +That test is what caught a real bug: the renderer's `Unit` case emitted `()` without evaluating the expression, so +`(print-line "x")` — the most ordinary thing anyone types at a prompt — answered `()` while nothing happened. A Unit +expression is almost always a call made for its effect, and is now evaluated and *then* reported. ### Conditions — step 1: `handler-bind` and `signal` -`spec-conditions.md` §1 and §2, and nothing else yet. They are worth having on -their own because **neither alters control flow**: `signal` returns `Unit` -whatever it finds, a handler that returns normally leaves the signalling -function to carry on, and with nothing matching it is a no-op. So none of the -transfer machinery §6 describes exists yet, and no signature changed. +`spec-conditions.md` §1 and §2, and nothing else yet. They are worth having on their own because **neither alters +control flow**: `signal` returns `Unit` whatever it finds, a handler that returns normally leaves the signalling +function to carry on, and with nothing matching it is a no-op. So none of the transfer machinery §6 describes exists +yet, and no signature changed. ``` (handler-bind [(AssetMissing [c] (set seen (+ seen (i64 (.id c)))))] (load-all)) ``` -The runtime is a linked list: establishing a handler is two stores and a push -onto a frame allocated on the establishing function's own stack, and `signal` -with an empty stack is a null check — which is what §2 asks for. Popping is by -frame rather than by count, so restoring what this one displaced is right even -if something below it left the stack out of step. +The runtime is a linked list: establishing a handler is two stores and a push onto a frame allocated on the establishing +function's own stack, and `signal` with an empty stack is a null check — which is what §2 asks for. Popping is by frame +rather than by count, so restoring what this one displaced is right even if something below it left the stack out of +step. Three decisions worth keeping: -- **A condition's type is a hash of its name**, not an index. An index would - shift the moment a struct were added, and every handler a running program had - already pushed would then match the wrong type. FNV-1a over the name. -- **The condition crosses as a pointer**, because a handler runs while the - signalling frame is still alive and there is nothing to copy. What the clause - *binds* is the condition itself, though — the pointer is a hidden parameter - and the name is a slot loaded from it, so a handler passing `c` to something - expecting the struct is not handed an address instead. -- **A clause is lifted into a function of its own.** A handler runs from - wherever the signal was, so it cannot be a branch in the function that wrote - it. -- **A pushed handler frame holds the clause's body address, not a cell.** This - is a deliberate divergence from plan.org's rule that a top-level function - value is a stable trampoline over the cell and never the address of a - particular body. A handler frame is not a `Fn` value — nothing in the - language can name it — and it is live only for the duration of the - `handler-bind` body, so a reload landing while it is on the stack finds the - clause it pushed still valid, which is exactly the "old code is never - unloaded" guarantee. The consequence to know: a handler already on the stack - does *not* observe a redefinition of its own clause; the next entry to the - `handler-bind` pushes the new one. When `Fn` values arrive, this is the one - place that stores a body address on purpose and must not be swept up with - them. +- **A condition's type is a hash of its name**, not an index. An index would shift the moment a struct were added, and +every handler a running program had already pushed would then match the wrong type. FNV-1a over the name. +- **The condition crosses as a pointer**, because a handler runs while the signalling frame is still alive and there is +nothing to copy. What the clause *binds* is the condition itself, though — the pointer is a hidden parameter and the +name is a slot loaded from it, so a handler passing `c` to something expecting the struct is not handed an address +instead. +- **A clause is lifted into a function of its own.** A handler runs from wherever the signal was, so it cannot be a +branch in the function that wrote it. +- **A pushed handler frame holds the clause's body address, not a cell.** This is a deliberate divergence from +plan.org's rule that a top-level function value is a stable trampoline over the cell and never the address of a +particular body. A handler frame is not a `Fn` value — nothing in the language can name it — and it is live only for the +duration of the `handler-bind` body, so a reload landing while it is on the stack finds the clause it pushed still +valid, which is exactly the "old code is never unloaded" guarantee. The consequence to know: a handler already on the +stack does *not* observe a redefinition of its own clause; the next entry to the `handler-bind` pushes the new one. When +`Fn` values arrive, this is the one place that stores a body address on purpose and must not be swept up with them. Which gives the two refusals, both by the house rule rather than by accident: -- **A handler cannot see the establishing function's locals.** That is a - closure with an explicit environment, so a reference to one is refused *for - that reason* rather than reported as an unknown name. Globals and the - condition are in scope, which is what the accumulation case needs. +- **A handler cannot see the establishing function's locals.** That is a closure with an explicit environment, so a +reference to one is refused *for that reason* rather than reported as an unknown name. Globals and the condition are in +scope, which is what the accumulation case needs. - What it needs is narrower than it looks, and worth getting right before - anyone schedules it: a handler frame does not outlive the function that - established it, so this is spec-memory.md's **case 2** — a non-escaping `fn` - capturing by value into a stack environment — and *not* the escaping closure - that plan.org's open decision #5 defers until a concrete use case. Case 2 is - settled, and #5 says in as many words that without it "conditions are not - worth building". So the biggest usability limit in conditions is not behind - the thing that was just deferred. -- **`return` inside a `handler-bind` body is refused.** The frames are popped - on the way out and an early exit would leave them on the stack pointing into - a function that has gone. Same shape as `defer` inside a block. +What it needs is narrower than it looks, and worth getting right before anyone schedules it: a handler frame does not +outlive the function that established it, so this is spec-memory.md's **case 2** — a non-escaping `fn` capturing by +value into a stack environment — and *not* the escaping closure that plan.org's open decision #5 defers until a concrete +use case. Case 2 is settled, and #5 says in as many words that without it "conditions are not worth building". So the +biggest usability limit in conditions is not behind the thing that was just deferred. +- **`return` inside a `handler-bind` body is refused.** The frames are popped on the way out and an early exit would +leave them on the stack pointing into a function that has gone. Same shape as `defer` inside a block. ### The break loop — conditions step 3 -Where **"a crash kills the program"** stops being true. An unhandled `error` -runs a hook instead of `rt_die()`, on the frame that erred with nothing -unwound, so the condition and every restart between there and the top are still -live. +Where **"a crash kills the program"** stops being true. An unhandled `error` runs a hook instead of `rt_die()`, on the +frame that erred with nothing unwound, so the condition and every restart between there and the top are still live. ``` flan: unhandled Missing — stopped, not dead. @@ -1296,71 +1015,54 @@ flan: unhandled Missing — stopped, not dead. Four decisions, each of which is the reason something is where it is: -- **It is a hook, not a call.** The loop lives in `vendor/agent/`, which is an - optional package; `flan_rt.c` is the release runtime and must not depend on - something a program may never import. A program with no agent leaves the hook - null and dies exactly as it did before. -- **The hook resumes by writing a restart into the transfer channel** — the - same channel an `invoke-restart` writes, reaching the same guard. Choosing a - restart from the break loop and choosing one from a handler are therefore the - same act, lowered once. Nothing about §6 needed changing to support it. -- **The break loop *is* the poll loop**, run from the error instead of from the - frame boundary. That is not a convenience: an expression evaluated while - stopped is a module the listener queues and the game thread runs, so a loop - that did not drain that queue would hang `C-x C-e` exactly when it is most - wanted. -- **Installing while stopped is allowed**, which contradicts a rule stated - above and should. "A redefined function must not be swapped while it is on - the stack" is about *mid-frame consistency* — half a frame of old code and - half of new — and there is no frame in progress here. The old body on the - stack keeps running; a `retry` restart calls through the cell and reaches the - new one. That is the fix-it-and-retry loop, and refusing the install would - remove the point of stopping. +- **It is a hook, not a call.** The loop lives in `vendor/agent/`, which is an optional package; `flan_rt.c` is the +release runtime and must not depend on something a program may never import. A program with no agent leaves the hook +null and dies exactly as it did before. +- **The hook resumes by writing a restart into the transfer channel** — the same channel an `invoke-restart` writes, +reaching the same guard. Choosing a restart from the break loop and choosing one from a handler are therefore the same +act, lowered once. Nothing about §6 needed changing to support it. +- **The break loop *is* the poll loop**, run from the error instead of from the frame boundary. That is not a +convenience: an expression evaluated while stopped is a module the listener queues and the game thread runs, so a loop +that did not drain that queue would hang `C-x C-e` exactly when it is most wanted. +- **Installing while stopped is allowed**, which contradicts a rule stated above and should. "A redefined function must +not be swapped while it is on the stack" is about *mid-frame consistency* — half a frame of old code and half of new — +and there is no frame in progress here. The old body on the stack keeps running; a `retry` restart calls through the +cell and reaches the new one. That is the fix-it-and-retry loop, and refusing the install would remove the point of +stopping. -**A restart frame carries its name now**, beside the hash. Matching never needs -it — that is what the hash is for — but a break loop has to *show* someone -their choices and nothing at run time can turn a hash back into a name. It is -also `compute-restarts`' data, whenever that arrives. +**A restart frame carries its name now**, beside the hash. Matching never needs it — that is what the hash is for — but +a break loop has to *show* someone their choices and nothing at run time can turn a hash back into a name. It is also +`compute-restarts`' data, whenever that arrives. -**A choice is validated on the listener thread**, against a stack the stopped -game thread is holding still, and refused there. Accepting it and discovering -on the game thread that no frame offers it would answer `ok` for something that +**A choice is validated on the listener thread**, against a stack the stopped game thread is holding still, and refused +there. Accepting it and discovering on the game thread that no frame offers it would answer `ok` for something that cannot happen. -The socket verbs are `restarts`, `restart ` and `abort`, and all three -are refused with the reason when the program is not stopped — there is no -restart stack to walk from a running one. `test/programs/break.flan` errors -twice and the test takes a *different* restart each time, so a loop that always -resumed the same way could not pass. +The socket verbs are `restarts`, `restart ` and `abort`, and all three are refused with the reason when the +program is not stopped — there is no restart stack to walk from a running one. `test/programs/break.flan` errors twice +and the test takes a *different* restart each time, so a loop that always resumed the same way could not pass. ### The break loop in the editor — conditions step 3, the other half -The loop above is reachable from a raw socket. This is the half that makes it -reachable from Emacs, and the whole of it follows from one fact: **a program -stops at a moment nobody asked about.** Every other op in the protocol answers -a question an editor chose to ask. +The loop above is reachable from a raw socket. This is the half that makes it reachable from Emacs, and the whole of it +follows from one fact: **a program stops at a moment nobody asked about.** Every other op in the protocol answers a +question an editor chose to ask. So the state is learned **twice, deliberately**: -- **It rides on every reply**, beside the program's output and for the same - reason. `:stopped t :condition "Missing"`, or `:stopped nil`. The likeliest - instant for a program to stop is the one just after an evaluation — a body - that now errors — and that is a reply the client is already reading. Finding - out a second later from a poll would mean finding out *after* the echo area - had said the evaluation was fine. -- **And a timer asks anyway**, once a second, with `describe` — the cheap op, - which is also how the output pipe is drained. A program that stops in a frame - of its own game loop produces no reply at all, and folding state into replies - that never come says nothing. The timer never *reconnects*: `flan-dev--request` - reopens a socket a restarted daemon left behind, which is right for something - a person did and wrong for a background poll, because it would quietly erase - the `lost` state that exists to be seen. It also skips while another request - is in flight — `accept-process-output` runs timers, so a poll firing inside a - read would eat the reply that read was waiting for. +- **It rides on every reply**, beside the program's output and for the same reason. `:stopped t :condition "Missing"`, +or `:stopped nil`. The likeliest instant for a program to stop is the one just after an evaluation — a body that now +errors — and that is a reply the client is already reading. Finding out a second later from a poll would mean finding +out *after* the echo area had said the evaluation was fine. +- **And a timer asks anyway**, once a second, with `describe` — the cheap op, which is also how the output pipe is +drained. A program that stops in a frame of its own game loop produces no reply at all, and folding state into replies +that never come says nothing. The timer never *reconnects*: `flan-dev--request` reopens a socket a restarted daemon left +behind, which is right for something a person did and wrong for a background poll, because it would quietly erase the +`lost` state that exists to be seen. It also skips while another request is in flight — `accept-process-output` runs +timers, so a poll firing inside a read would eat the reply that read was waiting for. -Three ops, and the *annotation* owns `:stopped`, not the ops — one place in the -daemon decides whether the program is stopped, so the poll and the prompt -cannot disagree. +Three ops, and the *annotation* owns `:stopped`, not the ops — one place in the daemon decides whether the program is +stopped, so the poll and the prompt cannot disagree. ``` (:op "break") → (:status "ok" :restarts ("retry" …) :stopped t :condition "Missing") @@ -1368,52 +1070,41 @@ cannot disagree. (:op "abort") → (:status "ok" :note "the program is exiting; …") ``` -`break` carries only the restart names, because those cost a second round trip -to the program and are wanted only by someone about to choose one. +`break` carries only the restart names, because those cost a second round trip to the program and are wanted only by +someone about to choose one. -**`ok` from `restart` means accepted, not resumed.** The choice is validated on -the program's listener thread against the stopped stack, then taken when that -thread next comes round its loop. A client that read it as "running again" -would poll once, find it still stopped, and re-open the prompt it had just -answered — so the client clears its own flag and lets the next poll settle it. +**`ok` from `restart` means accepted, not resumed.** The choice is validated on the program's listener thread against +the stopped stack, then taken when that thread next comes round its loop. A client that read it as "running again" would +poll once, find it still stopped, and re-open the prompt it had just answered — so the client clears its own flag and +lets the next poll settle it. -The modeline is a fourth state, `flan:stopped(Missing)`, before `live`: a -stopped program looks exactly like a running one from anywhere else in Emacs. -The prompt is a `completing-read` over the names with `require-match`, which is -exactly right for a closed set the program computed, and `abort` is the last -entry on that same list rather than a second key — it is the thing you pick -when none of the restarts is the answer. +The modeline is a fourth state, `flan:stopped(Missing)`, before `live`: a stopped program looks exactly like a running +one from anywhere else in Emacs. The prompt is a `completing-read` over the names with `require-match`, which is exactly +right for a closed set the program computed, and `abort` is the last entry on that same list rather than a second key — +it is the thing you pick when none of the restarts is the answer. -**`flan_agent_poll` had to become re-entrant**, and that is the one thing here -that was a bug rather than an addition. A `C-x C-e` thunk may itself error; the -break loop that catches it polls again from inside that very call. The old loop -cached `head` and `tail` and wrote `tail` back at the end, so the outer call -rewound the index over everything the nested one had consumed — and re-ran the -thunk that had just stopped the program, which is an unbounded recursion of -breaks rather than a stumble. It now claims each job by advancing `tail` before -running it, and re-reads both indices each time round. Still single-consumer: -only the game thread writes `tail`, nesting included. `test_dev.ml` evaluates an +**`flan_agent_poll` had to become re-entrant**, and that is the one thing here that was a bug rather than an addition. A +`C-x C-e` thunk may itself error; the break loop that catches it polls again from inside that very call. The old loop +cached `head` and `tail` and wrote `tail` back at the end, so the outer call rewound the index over everything the +nested one had consumed — and re-ran the thunk that had just stopped the program, which is an unbounded recursion of +breaks rather than a stumble. It now claims each job by advancing `tail` before running it, and re-reads both indices +each time round. Still single-consumer: only the game thread writes `tail`, nesting included. `test_dev.ml` evaluates an expression that errors and resumes it, which fails against the old shape. -The agent grew one verb, `status`, answered in **both** states — `running` or -`stopped `. Everything else the break loop offers is refused while -running, and rightly; but the question an editor asks *without already knowing* -had to have an answer either way, or there would be nothing to poll. The -condition is its class name and nothing more: the hook is handed a name and an -opaque pointer, and nothing at run time can render a value whose type it does -not know. +The agent grew one verb, `status`, answered in **both** states — `running` or `stopped `. Everything else the +break loop offers is refused while running, and rightly; but the question an editor asks *without already knowing* had +to have an answer either way, or there would be nothing to poll. The condition is its class name and nothing more: the +hook is handed a name and an opaque pointer, and nothing at run time can render a value whose type it does not know. -`test/programs/dev-break.flan` stops on its first frame, so the daemon meets a -program that is *already* stopped — the state an editor has to cope with and the -hardest one to arrange later. The Emacs test breaks a program the other way -round, by installing a `step` that errors into a loop that calls it, fixes it -while stopped, and then resumes: `C-x C-e` answering while the program sits in -the break loop is checked there against the real client, not only in OCaml. +`test/programs/dev-break.flan` stops on its first frame, so the daemon meets a program that is *already* stopped — the +state an editor has to cope with and the hardest one to arrange later. The Emacs test breaks a program the other way +round, by installing a `step` that errors into a loop that calls it, fixes it while stopped, and then resumes: `C-x C-e` +answering while the program sits in the break loop is checked there against the real client, not only in OCaml. ### Conditions — step 2: `restart-case` and `invoke-restart` -`spec-conditions.md` §3 to §6: the transfer. A handler runs where the signal -was, decides, and control resumes at a `restart-case` further out. +`spec-conditions.md` §3 to §6: the transfer. A handler runs where the signal was, decides, and control resumes at a +`restart-case` further out. ``` (defn fetch [n i32] i32 @@ -1425,139 +1116,112 @@ was, decides, and control resumes at a `restart-case` further out. (fetch 2)) ; -1 ``` -**The channel is an out-parameter**, as §6 now says: one `ptr` appended to -every Flan signature, written by an `invoke-restart` and checked after every -call. The return type stays what the source says, so the disassembly is the -release one plus a guard, and one pointer threads down the whole chain — a -callee writes the target into its caller's slot and each frame only has to -check and return early, which reuses the existing `return` path and with it -§5's defers. `Emit.signature` was already the one place a signature is spelled, -which is what made this a three-line change rather than a hunt. +**The channel is an out-parameter**, as §6 now says: one `ptr` appended to every Flan signature, written by an +`invoke-restart` and checked after every call. The return type stays what the source says, so the disassembly is the +release one plus a guard, and one pointer threads down the whole chain — a callee writes the target into its caller's +slot and each frame only has to check and return early, which reuses the existing `return` path and with it §5's defers. +`Emit.signature` was already the one place a signature is spelled, which is what made this a three-line change rather +than a hunt. -**Every function is transfer-transparent**, release included — and that is the -ABI, not a stopgap. A cell holds a bare pointer, so the honest answer to "what -can this call?" is "anything"; the same bargain as the indirect call. §6 and -plan.org both now say the later optimisation may stop a function *checking* the -channel, or pass the pointer straight through, but may not drop the parameter — -a signature that depended on an analysis could not be reloaded into. Uniform +**Every function is transfer-transparent**, release included — and that is the ABI, not a stopgap. A cell holds a bare +pointer, so the honest answer to "what can this call?" is "anything"; the same bargain as the indirect call. §6 and +plan.org both now say the later optimisation may stop a function *checking* the channel, or pass the pointer straight +through, but may not drop the parameter — a signature that depended on an analysis could not be reloaded into. Uniform also means redefinition acquires no new refusal class. -**The transfer target is the restart frame's own address, not a clause id.** -This is a correction to what the previous note settled. A static id has to be -unique against every module a running program may *later* load, and a hash is -only probably unique — two `restart-case`s colliding means the inner one -silently catches a transfer aimed at the outer. The frame is an `alloca` in the -function that offers it, so its address is exact, and it also says *which* -clause, which is how clause ids disappeared entirely. §6 says "transferring to -frame N" and this is closer to it than the number was. Re-entering a -`restart-case` then works with nothing extra: each activation allocates its -own frames, and §4's "innermost offering the name" is just the order of the -walk. +**The transfer target is the restart frame's own address, not a clause id.** This is a correction to what the previous +note settled. A static id has to be unique against every module a running program may *later* load, and a hash is only +probably unique — two `restart-case`s colliding means the inner one silently catches a transfer aimed at the outer. The +frame is an `alloca` in the function that offers it, so its address is exact, and it also says *which* clause, which is +how clause ids disappeared entirely. §6 says "transferring to frame N" and this is closer to it than the number was. +Re-entering a `restart-case` then works with nothing extra: each activation allocates its own frames, and §4's +"innermost offering the name" is just the order of the walk. -**Cleanup happens in landing blocks, one per region.** A guard branches to the -innermost open one, which pops whatever frames it established and either -catches the transfer or forwards it outward: +**Cleanup happens in landing blocks, one per region.** A guard branches to the innermost open one, which pops whatever +frames it established and either catches the transfer or forwards it outward: -- a `restart-case`'s pops its restart frames, compares the target against its - own, and either runs that clause or puts the target back and goes on out; -- a `handler-bind`'s pops its handler frames and goes on out — which is the - path a transfer out of a handled body takes, and without it the handler stack - would be left pointing into a frame that has gone; -- the function's own runs its defers (§5) and returns early. `errdefer` does not - run and never could: `try`/`Result` is still refused by name. +- a `restart-case`'s pops its restart frames, compares the target against its own, and either runs that clause or puts +the target back and goes on out; +- a `handler-bind`'s pops its handler frames and goes on out — which is the path a transfer out of a handled body takes, +and without it the handler stack would be left pointing into a frame that has gone; +- the function's own runs its defers (§5) and returns early. `errdefer` does not run and never could: `try`/`Result` is +still refused by name. -A single function-wide unwind block would have been wrong for the first two: -a call inside a `restart-case` body would jump straight past the very form that -was supposed to catch it. +A single function-wide unwind block would have been wrong for the first two: a call inside a `restart-case` body would +jump straight past the very form that was supposed to catch it. -**The channel is cleared before any cleanup runs and put back after.** A defer -makes ordinary calls and each one is guarded; with the channel still set the -first of them would branch straight back into the landing block it came from. +**The channel is cleared before any cleanup runs and put back after.** A defer makes ordinary calls and each one is +guarded; with the channel still set the first of them would branch straight back into the landing block it came from. Same reason the clause body starts with it null. -`flan_signal` takes the channel and passes it to each handler, and stops -walking once one has written to it. That makes the one C frame every handler is -reached through transparent to a transfer — it has to be, or §6's "a transfer -cannot cross a foreign frame" would make `restart-case` useless. It is also the -only such frame: `extern` is Flan-to-C only and there are no function values -yet, so nothing can call *back* into Flan across one. +`flan_signal` takes the channel and passes it to each handler, and stops walking once one has written to it. That makes +the one C frame every handler is reached through transparent to a transfer — it has to be, or §6's "a transfer cannot +cross a foreign frame" would make `restart-case` useless. It is also the only such frame: `extern` is Flan-to-C only and +there are no function values yet, so nothing can call *back* into Flan across one. Scope, each piece refused by name with its reason and a test on the reason: -- **restarts take no parameters.** That covers §1's own `load-texture` example - and skips argument marshalling and §3's runtime arity check. -- `return` inside a `restart-case` body, exactly as inside `handler-bind`: a - bare `ret` skips the pops. -- one `restart-case` offering a name twice — §4 finds the first frame offering - it, and two in one frame makes that a choice nothing in the source shows. -- `invoke-restart` inside a `defer`. A defer *is* the cleanup a transfer runs - on its way out, so a transfer starting there leaves the function's defers - half run with two targets and no way to choose. The lexical case is the - checker's; a defer that reaches one through a call is trapped at run time by - `flan_transfer_fail`, because nothing static could see it. -- no restart of that name is active: a runtime error at the invoke site, named - and located, rather than an unwind past everything. There is nowhere to - resume, so there is nothing else to do. +- **restarts take no parameters.** That covers §1's own `load-texture` example and skips argument marshalling and §3's +runtime arity check. +- **TODO — CL-style interactive recovery.** A stopped program should be able to offer a typed restart such as +`(use-value [value T] value)` or `(use-function [replacement (Fn ...)] ...)`, and the editor should show its signature +and ask for the replacement before invoking it. That is the missing "this variable is None; what should I use instead?" +path: named, hard-coded branches are useful for `retry` and `skip`, but not a substitute for an interactive value or +alternate implementation. It needs restart argument marshalling and validation in the runtime, plus an editor protocol +for entering and checking the supplied value/expression. +- `return` inside a `restart-case` body, exactly as inside `handler-bind`: a bare `ret` skips the pops. +- one `restart-case` offering a name twice — §4 finds the first frame offering it, and two in one frame makes that a +choice nothing in the source shows. +- `invoke-restart` inside a `defer`. A defer *is* the cleanup a transfer runs on its way out, so a transfer starting +there leaves the function's defers half run with two targets and no way to choose. The lexical case is the checker's; a +defer that reaches one through a call is trapped at run time by `flan_transfer_fail`, because nothing static could see +it. +- no restart of that name is active: a runtime error at the invoke site, named and located, rather than an unwind past +everything. There is nowhere to resume, so there is nothing else to do. -**`(error c)`, §2.** The same walk as `signal`, and the difference is entirely -what happens when the walk ends: `signal` returns `Unit` and the signalling -function carries on, `error` has type `Never` and stops. So only a transfer -gets past it, which is why `emit` puts a guard after the call and then -`unreachable` — and why `flan_error` cannot be marked `noreturn`, since it does -return, on exactly one path. Being `Never` is also what lets it stand as a -`restart-case` body's fall-through, which is the shape §1's `load-texture` -example needs. `test/programs/error.flan` is the unhandled case: it cannot be -an `outputs` row, because it does not exit 0. +**`(error c)`, §2.** The same walk as `signal`, and the difference is entirely what happens when the walk ends: `signal` +returns `Unit` and the signalling function carries on, `error` has type `Never` and stops. So only a transfer gets past +it, which is why `emit` puts a guard after the call and then `unreachable` — and why `flan_error` cannot be marked +`noreturn`, since it does return, on exactly one path. Being `Never` is also what lets it stand as a `restart-case` +body's fall-through, which is the shape §1's `load-texture` example needs. `test/programs/error.flan` is the unhandled +case: it cannot be an `outputs` row, because it does not exit 0. -`flan_transfer_fail` covers the ordinary return path as well as the unwind one: -a defer that reaches an `invoke-restart` through a call traps either way, and -the message names the rule rather than the path, since the rule is the same. +`flan_transfer_fail` covers the ordinary return path as well as the unwind one: a defer that reaches an `invoke-restart` +through a call traps either way, and the message names the rule rather than the path, since the rule is the same. -**A lifted handler clause is named after the function it came out of** — -`handler/step/0/Missing` — and is emitted by a redefinition module alongside -the body it belongs to, hidden, for the same interposition reason the body is. -This was a hole step 1 left: the name used to be numbered by position in the -whole program's lifted list, so it was neither stable nor attributable, and a -redefinition of a function containing a `handler-bind` failed in `llc` with an -undefined value. A clause is reached by address from its parent's body and from -nowhere else, so it takes no cell and no registry slot. +**A lifted handler clause is named after the function it came out of** — `handler/step/0/Missing` — and is emitted by a +redefinition module alongside the body it belongs to, hidden, for the same interposition reason the body is. This was a +hole step 1 left: the name used to be numbered by position in the whole program's lifted list, so it was neither stable +nor attributable, and a redefinition of a function containing a `handler-bind` failed in `llc` with an undefined value. +A clause is reached by address from its parent's body and from nowhere else, so it takes no cell and no registry slot. -`test/test_dev.ml` drives that path: a third evaluation redefines `step` to a -`restart-case` whose frame is an `alloca` in the newly loaded module, whose -guarded call goes through the host's cell, and whose transfer starts in a -handler and crosses `probe`, which the host was compiled with. Those three do -not meet anywhere else. +`test/test_dev.ml` drives that path: a third evaluation redefines `step` to a `restart-case` whose frame is an `alloca` +in the newly loaded module, whose guarded call goes through the host's cell, and whose transfer starts in a handler and +crosses `probe`, which the host was compiled with. Those three do not meet anywhere else. Two things found by writing it: -- **`{ ctx with in_handler = true }` was a latent bug.** `ctx.slots` and - `ctx.slot_tys` are mutable, so a copy allocates the body's slots into a - record the function never sees again and the indices collide. It was harmless - only because no `handler-bind` body in the tests had a `let` in it. The flags - are set on `ctx` and restored now. -- **`test/reload_host.c` had to learn the parameter.** It calls `flan.outer` - through an `__asm__` label, which does not fail at link time when the - prototype is a parameter short — it reads a garbage pointer as the channel - and dies somewhere else entirely. +- **`{ ctx with in_handler = true }` was a latent bug.** `ctx.slots` and `ctx.slot_tys` are mutable, so a copy allocates +the body's slots into a record the function never sees again and the indices collide. It was harmless only because no +`handler-bind` body in the tests had a `let` in it. The flags are set on `ctx` and restored now. +- **`test/reload_host.c` had to learn the parameter.** It calls `flan.outer` through an `__asm__` label, which does not +fail at link time when the prototype is a parameter short — it reads a garbage pointer as the channel and dies somewhere +else entirely. -`test/programs/restarts.flan` runs in the acceptance table at `-O2`, at `-O0` -and as a dev build. `-O0` is not redundant here: the guard after every call is -control flow the optimiser would otherwise launder, and the dev build is where -each of those calls goes through a cell. +`test/programs/restarts.flan` runs in the acceptance table at `-O2`, at `-O0` and as a dev build. `-O0` is not redundant +here: the guard after every call is control flow the optimiser would otherwise launder, and the dev build is where each +of those calls goes through a cell. ### What is left - **Editor comforts**: completion, eldoc, jump-to-definition, error overlays. -**Session identity is the daemon that owns the build.** A session's struct -layouts and global types have to describe the memory of the process it is -talking to, which is only guaranteed if it is the session that compiled the -running binary. Attaching to a process someone else built is not a thing to -support by default. +**Session identity is the daemon that owns the build.** A session's struct layouts and global types have to describe the +memory of the process it is talking to, which is only guaranteed if it is the session that compiled the running binary. +Attaching to a process someone else built is not a thing to support by default. ## Where build time goes -`flan build calc-me.flan` was ~160ms, and ~95% of it was clang. **The object -cache is in**, and it is now ~110ms: +`flan build calc-me.flan` was ~160ms, and ~95% of it was clang. **The object cache is in**, and it is now ~110ms: | Step | Cost | |---|---| @@ -1566,226 +1230,168 @@ cache is in**, and it is now ~110ms: | `clang` on `flan_rt.c` | 40ms — **now cached, paid once** | | link | 20ms | -Every C translation unit a build needs — the host shim and each package's shim -— goes through `Build.compile_c`, which compiles to a `.o` under -`$TMPDIR/flan-objcache` and reuses it. The key is a digest of the source text, -the compiler (its path, size and mtime, so an upgrade invalidates without -paying a `clang --version` subprocess per build), `opts.opt` and `opts.target`. -The opt level has to be in there: the acceptance table builds the same programs -at `-O0` and `-O2`, and an `-O2` object must not serve an `-O0` build. The -object is written to a temporary name and `rename`d into place, so two -concurrent builds cannot see a half-written one. +Every C translation unit a build needs — the host shim and each package's shim — goes through `Build.compile_c`, which +compiles to a `.o` under `$TMPDIR/flan-objcache` and reuses it. The key is a digest of the source text, the compiler +(its path, size and mtime, so an upgrade invalidates without paying a `clang --version` subprocess per build), +`opts.opt` and `opts.target`. The opt level has to be in there: the acceptance table builds the same programs at `-O0` +and `-O2`, and an `-O2` object must not serve an `-O0` build. The object is written to a temporary name and `rename`d +into place, so two concurrent builds cannot see a half-written one. -Measured: calc-me 160ms → 110ms; sand ~720ms → ~700ms, since sand's time is -mostly linking libraylib and its C was never the cost. The cache is -keyed by content, so it never needs invalidating by hand — `rm -rf` on the -directory is only ever a disk-space decision. +Measured: calc-me 160ms → 110ms; sand ~720ms → ~700ms, since sand's time is mostly linking libraylib and its C was never +the cost. The cache is keyed by content, so it never needs invalidating by hand — `rm -rf` on the directory is only ever +a disk-space decision. -The other cheap win is still open: skip the clang driver for the `.ll` (`llc` + -link directly), worth another ~40ms. It is a subset of the dev path's -machinery. Check `llc`'s major version against clang's before relying on it — -the emitted IR text is currently absorbed by the driver behind -`-Wno-override-module`, and a version mismatch surfaces as IR parse errors. +The other cheap win is still open: skip the clang driver for the `.ll` (`llc` + link directly), worth another ~40ms. It +is a subset of the dev path's machinery. Check `llc`'s major version against clang's before relying on it — the emitted +IR text is currently absorbed by the driver behind `-Wno-override-module`, and a version mismatch surfaces as IR parse +errors. -**There is still no REPL.** Nothing does redefinition, `dlopen`, or nREPL. -`build` is the only way to run code. +**There is still no REPL.** Nothing does redefinition, `dlopen`, or nREPL. `build` is the only way to run code. ## Next — the REPL is the priority -Decided in conversation: wasm32 can wait (it is believed to be a solved problem -once the builtins archive is in place), and **the dev loop is the thesis of the -project**, so it comes first. Staged so each step is runnable on its own — -the failure mode is building a daemon and a protocol before knowing the reload -primitive works. +Decided in conversation: wasm32 can wait (it is believed to be a solved problem once the builtins archive is in place), +and **the dev loop is the thesis of the project**, so it comes first. Staged so each step is runnable on its own — the +failure mode is building a daemon and a protocol before knowing the reload primitive works. -1. ~~**The reload primitive, measured.**~~ **Done** — `Emit.redefinition`, - `Build.shared`, `test/reload_host.c`, ~19ms. See the section above. -2. ~~**Indirection cells.**~~ **Done** — `Build.opts.dev` / `flan build --dev`, - `flan_reload_install`, `runtime/flan_dev.c` for names introduced at run - time, and a fixture where an untouched call site follows the swap and a - run-time-added function is itself redefined. See the section above. -3. ~~**The agent, in C.**~~ **Done** — `vendor/agent/`, a listener thread that - loads and a game thread that installs, and sand.flan polling at the top of - its frame. See the section above. -4. ~~**The daemon**~~ and ~~**5. the Emacs client**~~ — **both done**, and the - protocol is s-expressions rather than nREPL's bencode; see the two sections - above for why that changed. An nREPL front end can sit on the same - `Session` if something else ever needs to talk to it. +1. ~~**The reload primitive, measured.**~~ **Done** — `Emit.redefinition`, `Build.shared`, `test/reload_host.c`, ~19ms. +See the section above. +2. ~~**Indirection cells.**~~ **Done** — `Build.opts.dev` / `flan build --dev`, `flan_reload_install`, +`runtime/flan_dev.c` for names introduced at run time, and a fixture where an untouched call site follows the swap and a +run-time-added function is itself redefined. See the section above. +3. ~~**The agent, in C.**~~ **Done** — `vendor/agent/`, a listener thread that loads and a game thread that installs, +and sand.flan polling at the top of its frame. See the section above. +4. ~~**The daemon**~~ and ~~**5. the Emacs client**~~ — **both done**, and the protocol is s-expressions rather than +nREPL's bencode; see the two sections above for why that changed. An nREPL front end can sit on the same `Session` if +something else ever needs to talk to it. -**One decision left to settle before step 2**, because both change codegen and are -painful to retrofit: +**One decision left to settle before step 2**, because both change codegen and are painful to retrofit: -- ~~**Do cells cover globals, or only functions?**~~ **Settled by step 1: - functions only.** A redefinition module declares every global `external`, so - globals live in the host and survive a reload — which is what "edit the code, - keep the sand" needs. The consequence to watch is the other half: adding a - `defvar` to a file cannot take effect on reload, and changing one's type is a - silent mismatch against storage the host already laid out. Nothing detects - that yet. -- **What is a redefinition unit — one function, or a file?** A file is much - easier to make correct and is what `load-file` wants anyway; one function is - what `C-c C-c` wants and is where the 16ms number comes from. +- ~~**Do cells cover globals, or only functions?**~~ **Settled by step 1: functions only.** A redefinition module +declares every global `external`, so globals live in the host and survive a reload — which is what "edit the code, keep +the sand" needs. The consequence to watch is the other half: adding a `defvar` to a file cannot take effect on reload, +and changing one's type is a silent mismatch against storage the host already laid out. Nothing detects that yet. +- **What is a redefinition unit — one function, or a file?** A file is much easier to make correct and is what +`load-file` wants anyway; one function is what `C-c C-c` wants and is where the 16ms number comes from. Deferred until after the dev loop: -6. ~~**wasm32.**~~ **Done, with one glued joint.** `flan build - --target=wasm32-wasi` produces a module, and `test/programs/sand-headless.flan` - prints `2256461126764447066` under it — the same hash as native, byte for - byte, at `-O2` and at `-O0`. That is the milestone: the RNG is ours rather - than libc's precisely so that number can be compared across targets, and it - compares equal. `values.flan` and `machine.flan` run there too, which is - where a 32-bit pointer would have shown. The acceptance table runs all four, - and skips them by *probing* — it builds the smallest program and runs it — - rather than by looking for a binary on PATH. +6. ~~**wasm32.**~~ **Done, with one glued joint.** `flan build --target=wasm32-wasi` produces a module, and +`test/programs/sand-headless.flan` prints `2256461126764447066` under it — the same hash as native, byte for byte, at +`-O2` and at `-O0`. That is the milestone: the RNG is ours rather than libc's precisely so that number can be compared +across targets, and it compares equal. `values.flan` and `machine.flan` run there too, which is where a 32-bit pointer +would have shown. The acceptance table runs all four, and skips them by *probing* — it builds the smallest program and +runs it — rather than by looking for a binary on PATH. - Three things this cost that were not in the old note: +Three things this cost that were not in the old note: - - **The entry point is not `main`.** wasi-libc's start code calls - `__main_argc_argv`; clang renames C's argc/argv `main` to that, and the - `.ll` `Emit` writes says `@main` literally. The link succeeds and the - program traps on a signature-mismatched weak stub. `Build.wasm_main_source` - is a two-line C shim that bridges it, and the `__asm__("main")` label in it - is load-bearing: spelling the callee `main` makes clang rename *that* too - and the shim becomes an infinite self-call. - - **The target has to reach the C compiles, not just the link.** `flan_rt.c` - includes ``; without `--sysroot` it never gets that far. - `target_flags` is computed once and passed to both, and the whole flag - list — not just the triple — is in the object cache key, so repointing a - sysroot cannot serve a stale `.o`. - - **Fedora's sysroot is one level deeper** than wasi-sdk's: - `include/wasm32-wasi/stdio.h`, not `include/stdio.h`. Both shapes count. +- **The entry point is not `main`.** wasi-libc's start code calls `__main_argc_argv`; clang renames C's argc/argv `main` +to that, and the `.ll` `Emit` writes says `@main` literally. The link succeeds and the program traps on a +signature-mismatched weak stub. `Build.wasm_main_source` is a two-line C shim that bridges it, and the `__asm__("main")` +label in it is load-bearing: spelling the callee `main` makes clang rename *that* too and the shim becomes an infinite +self-call. +- **The target has to reach the C compiles, not just the link.** `flan_rt.c` includes ``; without `--sysroot` +it never gets that far. `target_flags` is computed once and passed to both, and the whole flag list — not just the +triple — is in the object cache key, so repointing a sysroot cannot serve a stale `.o`. +- **Fedora's sysroot is one level deeper** than wasi-sdk's: `include/wasm32-wasi/stdio.h`, not `include/stdio.h`. Both +shapes count. - **The glued joint, and the one thing this contradicts in the old note.** The - old note said the builtins archive has to come from a wasi-sdk release. It - does not have to: emscripten builds the same compiler-rt for wasm32 and - calls it `libcompiler_rt.a`, and dropping that in as - `libclang_rt.builtins.a` links and runs. It is a different triple - (`wasm32-unknown-emscripten`) built by a different clang (22 against - Fedora's 20), so it is *substituting*, and wasi-sdk is still the proper - article. `build.ml` looks for `FLAN_WASM_BUILTINS`, then - `/opt/wasi-sdk/...`, then emscripten's beside `emcc` on PATH, and refuses by - name listing every path it tried when none is there. clang's resource - directory is root-owned, so the archive is not dropped into it — a shadow - resource directory is built under the object cache, named by a digest of - clang's own resource dir plus the archive's path, size and mtime, with the - real `include` symlinked in. +**The glued joint, and the one thing this contradicts in the old note.** The old note said the builtins archive has to +come from a wasi-sdk release. It does not have to: emscripten builds the same compiler-rt for wasm32 and calls it +`libcompiler_rt.a`, and dropping that in as `libclang_rt.builtins.a` links and runs. It is a different triple +(`wasm32-unknown-emscripten`) built by a different clang (22 against Fedora's 20), so it is *substituting*, and wasi-sdk +is still the proper article. `build.ml` looks for `FLAN_WASM_BUILTINS`, then `/opt/wasi-sdk/...`, then emscripten's +beside `emcc` on PATH, and refuses by name listing every path it tried when none is there. clang's resource directory is +root-owned, so the archive is not dropped into it — a shadow resource directory is built under the object cache, named +by a digest of clang's own resource dir plus the archive's path, size and mtime, with the real `include` symlinked in. - **The runtime is Node.** No `wasmtime` and no `wasmer` on this machine; - `test/wasm-run.mjs` is twenty lines of `node:wasi` and the table prefers - `wasmtime` or `wasmer` if either appears. `--no-warnings`, because - `node:wasi` writes an `ExperimentalWarning` to stderr on every run and the - harness compares combined output. +**The runtime is Node.** No `wasmtime` and no `wasmer` on this machine; `test/wasm-run.mjs` is twenty lines of +`node:wasi` and the table prefers `wasmtime` or `wasmer` if either appears. `--no-warnings`, because `node:wasi` writes +an `ExperimentalWarning` to stderr on every run and the harness compares combined output. - **Refused by name, not half-supported:** `--dev` with a wasm target (the - reload path is `dlopen`), `Build.shared` with one (same reason), and `flan - run --target=` (a `.wasm` is not something this host execs — build it and - point a runtime at it). +**Refused by name, not half-supported:** `--dev` with a wasm target (the reload path is `dlopen`), `Build.shared` with +one (same reason), and `flan run --target=` (a `.wasm` is not something this host execs — build it and point a runtime +at it). - Still open: raylib on wasm, which plan.org wants through emscripten and its - own sysroot. wasi-sdk is right for the headless table; it is not necessarily - right for the eventual game build. +Still open: raylib on wasm, which plan.org wants through emscripten and its own sysroot. wasi-sdk is right for the +headless table; it is not necessarily right for the eventual game build. -7. **Loose ends from milestone 4**, none of them blocking: block-scoped - `defer`; package visibility, so `rl/get-color-raw` is not callable; a - package importing a package; imported unions. +7. **Loose ends from milestone 4**, none of them blocking: block-scoped `defer`; package visibility, so +`rl/get-color-raw` is not callable; a package importing a package; imported unions. ## Macros — the reader and the declaration are in, the expander is not The front half landed. What exists: -- **The reader** reads `` `x ``, `~x` and `~@x` as `(quasiquote x)`, - `(unquote x)` and `(unquote-splicing x)`, exactly as `'x` reads as - `(quote x)`. It stays dumb: it does not count nesting levels, does not know - whether an unquote is inside a quasiquote, and attaches no meaning to the - three names. Clojure's spelling, not Common Lisp's, because a comma is - whitespace in `is_delimiter` and every binding vector in the corpus relies - on that. Backtick and tilde are delimiters now, so `a~b` is two things. -- **`parse.ml` refuses all four by name.** `quasiquote` and `gensym` say - expansion is not wired up; `unquote` and `unquote-splicing` say they mean - nothing outside a quasiquote, which is a mistake rather than a missing - feature. `(defmacro name [params] body ...)` at the top level is checked for - shape and *then* refused — a malformed defmacro and an unimplemented one get - different reasons, so the shape rule is enforced before the feature exists. +- **The reader** reads `` `x ``, `~x` and `~@x` as `(quasiquote x)`, `(unquote x)` and `(unquote-splicing x)`, exactly +as `'x` reads as `(quote x)`. It stays dumb: it does not count nesting levels, does not know whether an unquote is +inside a quasiquote, and attaches no meaning to the three names. Clojure's spelling, not Common Lisp's, because a comma +is whitespace in `is_delimiter` and every binding vector in the corpus relies on that. Backtick and tilde are delimiters +now, so `a~b` is two things. +- **`parse.ml` refuses all four by name.** `quasiquote` and `gensym` say expansion is not wired up; `unquote` and +`unquote-splicing` say they mean nothing outside a quasiquote, which is a mistake rather than a missing feature. +`(defmacro name [params] body ...)` at the top level is checked for shape and *then* refused — a malformed defmacro and +an unimplemented one get different reasons, so the shape rule is enforced before the feature exists. -Nothing is stored. There is deliberately no macro table and no `Ast.Defmacro`, -because a table nothing reads is a place for a design to rot, and the storage -shape is the expander author's first decision, not a decision to inherit. +Nothing is stored. There is deliberately no macro table and no `Ast.Defmacro`, because a table nothing reads is a place +for a design to rot, and the storage shape is the expander author's first decision, not a decision to inherit. ### How the expander should work -**There is no interpreter** (see "Why there is no interpreter") and there is not -going to be one, so running a macro at compile time means *compiling it and -loading it into the compiler*. That machinery already exists and is measured: -`Emit.redefinition` → `Build.shared` → `dlopen` is ~19ms end to end, with the -load itself at 0.04ms (see "The reload primitive"). A macro is that pipeline -pointed at the compiler's own process instead of the program's. +**There is no interpreter** (see "Why there is no interpreter") and there is not going to be one, so running a macro at +compile time means *compiling it and loading it into the compiler*. That machinery already exists and is measured: +`Emit.redefinition` → `Build.shared` → `dlopen` is ~19ms end to end, with the load itself at 0.04ms (see "The reload +primitive"). A macro is that pipeline pointed at the compiler's own process instead of the program's. The shape it wants: -1. **A macro is a function `[Form] -> Form`.** Its parameters are forms and its - result is a form, which means `Form.t` has to exist on the Flan side — a - `defunion` mirroring `lib/form.ml`, in the prelude, plus constructors and - accessors. That is the real work, and it is bigger than the expander itself: - the compiler and the compiled macro have to agree on the *layout* of a - `Form`, not merely its shape, so whatever the checker does for unions has to - be exact here. Until unions are values this cannot start — `check.ml` puts - union values and `match` on a union at **milestone 6**, so that is milestone - 6 work landing before milestone 5's. -2. **Expansion runs over `Form`, before `Parse`.** Not a pass over `Ast`: - there is no `Ast.Defmacro` and `Parse` refuses `defmacro` outright, so an - `Ast`-level pass would have nothing to work with. That refusal is not a dead - end, it is the ordering — the expander runs first and `Parse` never sees a - macro call at all. It is also the Clojure ordering, and the reason a macro - expanding to a special form is ordinary rather than a special case. -3. **Order matters and files do not have one.** Top-level names in a package - are order-independent everywhere else (`declared_types`, the constant - fixpoint in `check.ml`). Macros cannot be: a macro must be compiled and - loaded before a call to it is expanded. Either collect every `defmacro` in a - pre-pass and compile them as one module, or require definition-before-use for - macros specifically and say so in the error. The pre-pass is better and - matches how the rest of the frontend already behaves. -4. **A macro's own body may call macros**, so the pre-pass is a fixpoint, not a - single sweep, and a cycle has to be detected and named rather than looping. -5. **`gensym` is a runtime function of the compiler**, called by the loaded - macro while it runs. It needs a counter that lives in the compiler process - and a name that cannot collide with a reader-produced symbol — the usual - trick is a character no symbol may contain, and this reader now has two new - ones it could reserve. Hygiene is settled (plan.org, open decision 2): - deliberately non-hygienic, Common Lisp/Clojure style, explicit `gensym`, no - `macrolet` until a concrete use case appears. -6. **Quasiquote itself is a macro-shaped desugaring**, not a compiler feature: - `` `(a ~b) `` becomes list-construction over quoted pieces, with - `~@` splicing. Written once, in the expander, over `Form`. +1. **A macro is a function `[Form] -> Form`.** Its parameters are forms and its result is a form, which means `Form.t` +has to exist on the Flan side — a `defunion` mirroring `lib/form.ml`, in the prelude, plus constructors and accessors. +That is the real work, and it is bigger than the expander itself: the compiler and the compiled macro have to agree on +the *layout* of a `Form`, not merely its shape, so whatever the checker does for unions has to be exact here. Until +unions are values this cannot start — `check.ml` puts union values and `match` on a union at **milestone 6**, so that is +milestone 6 work landing before milestone 5's. +2. **Expansion runs over `Form`, before `Parse`.** Not a pass over `Ast`: there is no `Ast.Defmacro` and `Parse` refuses +`defmacro` outright, so an `Ast`-level pass would have nothing to work with. That refusal is not a dead end, it is the +ordering — the expander runs first and `Parse` never sees a macro call at all. It is also the Clojure ordering, and the +reason a macro expanding to a special form is ordinary rather than a special case. +3. **Order matters and files do not have one.** Top-level names in a package are order-independent everywhere else +(`declared_types`, the constant fixpoint in `check.ml`). Macros cannot be: a macro must be compiled and loaded before a +call to it is expanded. Either collect every `defmacro` in a pre-pass and compile them as one module, or require +definition-before-use for macros specifically and say so in the error. The pre-pass is better and matches how the rest +of the frontend already behaves. +4. **A macro's own body may call macros**, so the pre-pass is a fixpoint, not a single sweep, and a cycle has to be +detected and named rather than looping. +5. **`gensym` is a runtime function of the compiler**, called by the loaded macro while it runs. It needs a counter that +lives in the compiler process and a name that cannot collide with a reader-produced symbol — the usual trick is a +character no symbol may contain, and this reader now has two new ones it could reserve. Hygiene is settled (plan.org, +open decision 2): deliberately non-hygienic, Common Lisp/Clojure style, explicit `gensym`, no `macrolet` until a +concrete use case appears. +6. **Quasiquote itself is a macro-shaped desugaring**, not a compiler feature: `` `(a ~b) `` becomes list-construction +over quoted pieces, with `~@` splicing. Written once, in the expander, over `Form`. -The four files this touches — `build.ml`, `check.ml`, `emit.ml`, `load.ml` — -were owned by other lanes when the front half landed, which is the only reason -the expander is not here too. +The four files this touches — `build.ml`, `check.ml`, `emit.ml`, `load.ml` — were owned by other lanes when the front +half landed, which is the only reason the expander is not here too. ### What would tell you it works -`when`, `unless`, `until`, `cond` and `dotimes` are special forms in `parse.ml` -today, and plan.org milestone 5 says they are special forms *only until macros -land*. Moving one of them out of the compiler and into the prelude as a -`defmacro`, with the existing tests unchanged and still green, is the exit -criterion — it proves expansion, quasiquote, `gensym` and the ordering pre-pass -at once, against a test suite written before any of them existed. +`when`, `unless`, `until`, `cond` and `dotimes` are special forms in `parse.ml` today, and plan.org milestone 5 says +they are special forms *only until macros land*. Moving one of them out of the compiler and into the prelude as a +`defmacro`, with the existing tests unchanged and still green, is the exit criterion — it proves expansion, quasiquote, +`gensym` and the ordering pre-pass at once, against a test suite written before any of them existed. ## Watch for -The rule that caught the two misparse bugs applies unchanged: **anything that -binds a name, alters control flow, or is not yet implemented must be recognised -explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`, -`Result`/`try`, union values, closures, quoted symbols, generics and function -values *by name*, each with the milestone it belongs to; `load.ml` rejects the -package shapes it does not handle; and the FFI boundary rejects an aggregate. -The tests assert on the reason, not just on the failure. +The rule that caught the two misparse bugs applies unchanged: **anything that binds a name, alters control flow, or is +not yet implemented must be recognised explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`, +`Result`/`try`, union values, closures, quoted symbols, generics and function values *by name*, each with the milestone +it belongs to; `load.ml` rejects the package shapes it does not handle; and the FFI boundary rejects an aggregate. The +tests assert on the reason, not just on the failure. ## Untracked on purpose -`calc-me` and `sand`, the executables `flan build` drops beside their sources, -are now in `.gitignore` — anchored (`/calc-me`, `/sand`) so the patterns cannot -match anything nested. +`calc-me` and `sand`, the executables `flan build` drops beside their sources, are now in `.gitignore` — anchored +(`/calc-me`, `/sand`) so the patterns cannot match anything nested. -`old-ocaml/` — the pre-rewrite menhir/ocamllex frontend, kept as reference and -excluded from the build by the root `dune` file. Its contents are also in git -history at `2c232dd`. +`old-ocaml/` — the pre-rewrite menhir/ocamllex frontend, kept as reference and excluded from the build by the root +`dune` file. Its contents are also in git history at `2c232dd`. diff --git a/emacs/flan-dev.el b/emacs/flan-dev.el index 3983c63..333bac2 100644 --- a/emacs/flan-dev.el +++ b/emacs/flan-dev.el @@ -427,9 +427,9 @@ for it — this is only what Emacs is responsible for killing.") SOCKET defaults to `flan-dev-socket-name' beside FILE, which is where the daemon puts it when it is not told otherwise. -Refuses while a daemon this Emacs started is still alive, by name: killing it -would take its program — and everything that program has in memory — with it, -which is the one thing a dev loop exists to avoid doing by accident." +When called interactively while a daemon this Emacs started is alive, asks +before stopping it and switching programs. A noninteractive call still +refuses: callers cannot silently discard a running program's state." (interactive ;; The program last started, where there was one: a restart after a quit is ;; the common case, and it is rarely the buffer point happens to be in — @@ -439,8 +439,23 @@ which is the one thing a dev loop exists to avoid doing by accident." (string-suffix-p ".flan" buffer-file-name) (file-name-nondirectory buffer-file-name))))) (when (process-live-p flan-dev--daemon) - (user-error "flan dev: already running on %s; M-x flan-dev-quit first" - (abbreviate-file-name (or flan-dev--socket "a socket")))) + ;; `interactive' has already read FILE. Refusing only here used to make + ;; that selection look as though it had been ignored: the old daemon kept + ;; running, even though the minibuffer had just accepted a different + ;; program. Keep the state-preserving default for Lisp callers, but let a + ;; person explicitly choose to replace their own session. + (if (called-interactively-p 'interactive) + (if (y-or-n-p + (format "Stop %s and start %s? " + (abbreviate-file-name + (or flan-dev--file "the running program")) + (abbreviate-file-name (expand-file-name file)))) + (flan-dev-quit) + (user-error "flan dev: keeping %s" + (abbreviate-file-name + (or flan-dev--file "the running program")))) + (user-error "flan dev: already running on %s; M-x flan-dev-quit first" + (abbreviate-file-name (or flan-dev--socket "a socket"))))) (let* ((file (expand-file-name file)) (socket (or socket (expand-file-name flan-dev-socket-name diff --git a/emacs/flan-mode.el b/emacs/flan-mode.el index c981a3b..f515972 100644 --- a/emacs/flan-mode.el +++ b/emacs/flan-mode.el @@ -12,21 +12,22 @@ ;; otherwise be made buffer-local before its own defvar had run. (require 'imenu) -;; The keymap binds them; loading the client is what defines them, and a user -;; may well edit Flan without ever connecting to a running program. -(declare-function flan-eval-defun "flan-dev") -(declare-function flan-eval-buffer "flan-dev") -(declare-function flan-eval-last-sexp "flan-dev") -(declare-function flan-connect "flan-dev") -(declare-function flan-disconnect "flan-dev") -(declare-function flan-describe "flan-dev") -(declare-function flan-show-output "flan-dev") -(declare-function flan-repl "flan-repl") -(declare-function flan-break "flan-dev") -(declare-function flan-doc "flan-dev") -(declare-function flan-dev "flan-dev") -(declare-function flan-dev-quit "flan-dev") -(declare-function flan-dev-restart-program "flan-dev") +;; The keymap binds them, but do not load the client merely to edit a file. +;; These must be real autoloads, not just `declare-function`s: otherwise a +;; user who has loaded only flan-mode cannot invoke M-x flan-dev at all. +(autoload 'flan-eval-defun "flan-dev" nil t) +(autoload 'flan-eval-buffer "flan-dev" nil t) +(autoload 'flan-eval-last-sexp "flan-dev" nil t) +(autoload 'flan-connect "flan-dev" nil t) +(autoload 'flan-disconnect "flan-dev" nil t) +(autoload 'flan-describe "flan-dev" nil t) +(autoload 'flan-show-output "flan-dev" nil t) +(autoload 'flan-repl "flan-repl" nil t) +(autoload 'flan-break "flan-dev" nil t) +(autoload 'flan-doc "flan-dev" nil t) +(autoload 'flan-dev "flan-dev" nil t) +(autoload 'flan-dev-quit "flan-dev" nil t) +(autoload 'flan-dev-restart-program "flan-dev" nil t) (defgroup flan nil "Editing and evaluating Flan." diff --git a/sand.flan b/sand.flan index 90f5d42..1e11d78 100644 --- a/sand.flan +++ b/sand.flan @@ -36,8 +36,8 @@ ;;;; No raylib between here and the next banner, which is what the headless ;;;; driver imports this file for. The hash is over exactly this. -(defconst screen-width 1400) -(defconst screen-height 1000) +(defconst screen-width 900) +(defconst screen-height 600) (defconst cell-size 5) ;; f32: velocity is [f32], and there is no implicit widening. (defconst gravity f32 0.05) @@ -46,7 +46,7 @@ (defconst brush-size 10) ;; Packed 0xRRGGBBAA. A cell of 0 means empty, so no Option and no tag word. -(defconst colors [4 u32] [0xE6B800FF 0x3B6E8CFF 0xA83232FF 0xCC6B1FFF]) +(defconst colors [4 u32] [0xFFF00FFF 0x3B6E8CFF 0xA83232FF 0xCC6B1FFF]) ;; Flat, unboxed, statically sized. No headers, so these are exactly ;; rows*cols*4 bytes each — the same memory the Odin port has. Fixed arrays are @@ -283,6 +283,7 @@ (when (rl/mouse-button-released? :left) (next-color)) (step)) + (defn draw-grid [] (dotimes [row rows] (dotimes [col cols] @@ -438,10 +439,16 @@ ;; Bare (defn main []) — argv and the i32 status are both optional. ;; Nothing in this loop allocates, so context/temp is never even touched. (until (rl/window-should-close?) - ;; The frame boundary, and the only place a redefinition becomes visible: - ;; nothing that could be redefined is on the stack here. - (agent/poll) - (game-update) + ;; The frame boundary, and the only place a redefinition becomes visible. + ;; An error while installing/evaluating a dev form, or during the update, + ;; leaves one CL-style escape hatch: choose `continue' in the editor to + ;; abandon this frame and return to the next one with the game still live. + ;; Keep drawing outside it. Skipping between BeginDrawing and EndDrawing + ;; would leave raylib's frame unbalanced. + (restart-case + (do (agent/poll) + (game-update)) + (continue [] (do))) (rl/begin-drawing) (game-draw) (rl/end-drawing)))