flan/NEXT.md
Joseph Ferano bb90f6e65e The reload primitive, and the cells that make it mean something
Two things, and either alone is useless, so they are one commit.

Emit.redefinition compiles one function into its own module against a host
that is already running. What it does *not* define is the design: a global is
external, so state survives a reload and sand's grid is not reset by editing
the code; every other function is a declare, so a redefined settle calls the
host's move-grain rather than a frozen copy; there is no main. Build.shared
puts that text through llc + ld -shared. ld, not clang, because a shared object
is allowed undefined symbols and that is the whole mechanism - and because the
driver is 50ms of a 20ms job. Measured here: llc 16ms, ld 3ms, dlopen 0.04ms.

Loading a body is not installing it, though. A call bound at link time cannot
notice a new one, so a dev build routes every Flan-to-Flan call through a cell
- a mutable global holding the address of the function that is current - and a
module publishes itself with one store. The cell load is emitted after the
arguments, so a redefinition between two calls cannot land inside one.

Three details that are not free choices. flan_reload_install is a named
function rather than an ELF constructor, because the agent has to choose when
the store happens and a constructor would do it during dlopen, mid-frame, on
whatever thread called it. A redefinition's own body is hidden, because default
visibility in a shared object is interposable and that applies to taking the
address too: plain @"flan.bump" inside the module resolves to the host's copy,
so the installer would publish the function it was replacing and the reload
would silently do nothing. And -rdynamic is what exports the cells at all, so
it and cells are one flag: Build.opts.dev, flan build --dev, the first time
opts means something semantic rather than an optimisation level.

The test is one process, because two runs would prove nothing about a swap,
and two .so paths, because dlopen caches by path and would hand back the first
handle. Every call in it goes through outer, compiled once into the host and
never rebuilt, so a changed answer can only mean its call site followed. v2
recurses through its own cell, which is the interposition case; it would print
the old body's text if it did not. helper differs between the fixtures purely
as a tripwire for a module that grew its own copy.

LLVM cannot fold the indirection - the cell is an external mutable global - and
a --dev calc-me keeps 46 indirect calls at -O2. values, machine and
sand-headless now run as dev builds in the acceptance table too; the sand hash
is the one result that would notice a call reaching the wrong function.
2026-09-10 21:27:11 +07:00

537 lines
29 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Where this is
**Dev loop steps 1 and 2 are done** — see *The reload primitive* below.
A function can be recompiled and installed into a running process, and call
sites compiled before it existed follow it. That is `C-c C-c` on a `defn`,
without an editor attached to it yet.
Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its
simulation has a headless acceptance case that runs on the `dune test` path at
`-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and
runs; the interpreter was dropped — open decision #7, settled, see below).
```
reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅
```
| File | What it does |
|---|---|
| `lib/loc.ml` | source locations + `Loc.Error`, the frontend's one exception |
| `lib/form.ml` | reader output: `Sym Kw Int Float Str Byte List Vec Map` |
| `lib/reader.ml` | hand-written S-expression reader, no menhir/ocamllex |
| `lib/ast.ml` | AST: `texpr`, `expr`, `place`, `pattern`, `decl` |
| `lib/parse.ml` | forms → AST; special forms, desugaring, declarations |
| `lib/load.ml` | **imports: a package directory → qualified declarations** |
| `lib/types.ml` | resolved types; structural equality, `Never` fits anywhere |
| `lib/tast.ml` | the typed IR the backend consumes |
| `lib/check.ml` | AST → typed IR; two passes, bidirectional |
| `lib/prelude.ml` | printers + `rand-f32`, written in Flan |
| `lib/emit.ml` | typed IR → LLVM IR text |
| `lib/build.ml` | `.ll` + the shim + the packages' C → clang → executable |
| `runtime/flan_rt.c` | the host ABI: argv, stdout, exit, 4 conversions |
| `vendor/raylib/` | **the raylib package: `raylib.flan`, `shim.c`, `link`** |
| `sand-sim/` | **the falling-sand simulation, with no raylib in it** |
| `bin/main.ml` | `flan read \| parse \| check \| emit \| build \| run` |
| `test/test_flan.ml` | reader, parser and checker |
| `test/test_acceptance.ml` | expression/result pairs + whole programs + the traps |
| `test/test_reload.ml` | **the reload primitive: recompile one function, load it, call it** |
| `test/reload_host.c` | the C host that loads and installs two rebuilds, in one process |
```
$ flan run calc-me.flan "1 + 2 * (3 - 0.5) / 2"
3.5
$ flan run test/programs/sand-headless.flan
2256461126764447066
$ flan run sand.flan # a window, 120 fps, hold space
```
## What milestone 4 added
**`dotimes`** desugars in `check.ml` to a `Let` plus a `While` — no new IR node.
The bound is evaluated once into a hidden slot before the loop, so a body that
changes it cannot change the trip count, and the loop variable is not
assignable, which makes the generated step its only writer.
**`defer`** is recognised in `check_fn` and nowhere else, because that is the
only place that knows a form is at the top level of a function body. Each one
is checked in place, then registered on the context; it emits nothing where it
stands. Function exit runs them innermost-first, and an explicit `return` runs
the ones registered *above* it — a defer written below a return has not
executed yet and must not fire. A trap runs none of them, which follows from
the bounds-check shape (`noreturn` then `unreachable`) rather than being a
separate decision.
`defer` inside a `let`, a loop or a branch is **rejected**, not accepted with
function scope. It would run once at function exit rather than once per
iteration, and that is the silent-wrongness class the rule below is about.
Block-scoped defer is real work and is not done.
**New builtins:** `zeroed` (takes its type from the place it is stored into),
`min`/`max` (each operand through a slot, so neither is evaluated twice),
`bit-and`/`bit-or`/`bit-xor`/`<<`/`>>` (integers only; `>>` is arithmetic on a
signed type and logical on an unsigned one), and `rand-f32`.
**`rand-f32` is in the prelude, in Flan** — PCG-XSH-RR 32 over a `u64` state.
It is not libc's, because a grid hash is only a regression test if the sequence
is byte-identical on native and wasm32 (plan.org, RNG is ours). `rand-seed`
sets the state. This is what the bitwise operators were added for.
**Enums and keywords.** `(defenum Name [member value ...])` gives a type that
is an `i32` at run time and its own type in the checker, so `:space` at a call
site resolves against the parameter's enum and a typo is an error there rather
than a wrong number later. A keyword means nothing where no enum is expected —
there is no keyword type to fall back on.
## Why the FFI goes through a C shim
The decision that shapes the whole raylib package. What clang generates for
raylib's own prototypes on x86-64:
```
Vector2 {float,float} → declare <2 x float> @GetMousePosition()
Color {u8,u8,u8,u8} → declare void @ClearBackground(i32)
Rectangle {4 × int} → declare { i64, i64 } @mkrect()
```
None of those is the struct's own LLVM type. A small aggregate's calling
convention is not part of its layout — it is a per-target classification the
*caller* has to reproduce, and x86-64, arm64 and wasm32 classify differently.
Putting that in `emit.ml` is three classifiers to write and then keep correct
forever, and a mistake shows up as `(.y m)` returning garbage rather than as a
link error.
So `vendor/raylib/shim.c` has one wrapper per binding, each one flattening the
aggregates: a struct returns through an out-pointer, a struct argument is
passed by pointer, a Flan string crosses as ptr+len and the shim NUL-terminates
a copy. clang classifies all of it, per target, for free. `check.ml` enforces
the rule — an aggregate in a `declare` signature is rejected with the reason —
so the boundary cannot quietly acquire one. This is plan.org's "one narrow host
ABI, implemented twice", and `flan_rt.c` is the same pattern.
The price is a hand-written wrapper per raylib call. They are one-liners and
mechanical enough to generate if that ever becomes the bottleneck.
`raylib.flan` declares each `-raw` entry point and wraps it in an ordinary Flan
function just below, so the surface sand.flan sees is `(rl/get-mouse-position)`
returning a `Vector2`. Verified end to end, headless: `GetColor(0x11223344)`
comes back as `17 34 51 68`, four separate bytes — a `Color` is *not* the
little-endian reading of the packed integer, so an identity would have passed a
weaker test. That case is in the acceptance table, skipped if `libraylib` is
not installed.
The bindings are 18 calls: window (`init-window`, `close-window`,
`window-should-close?`, `set-target-fps`, `set-trace-log-level`), keyboard
(`key-pressed?`/`down?`/`released?`), mouse (`mouse-button-pressed?`/`down?`/
`released?`, `get-mouse-position`), `get-color`, and drawing (`begin-drawing`,
`end-drawing`, `draw-fps`, `clear-background`, `draw-rectangle`), plus the
`Key`, `MouseButton` and `TraceLogLevel` enums. Adding one is three lines: a
`declare`, an `extern` prototype, and a one-line wrapper.
No raylib headers are needed: `shim.c` declares the prototypes it uses, so the
build depends on the shared library being linkable and not on `raylib-devel`.
`vendor/raylib/link` carries `-l:libraylib.so.550` because Fedora ships the
runtime library without the `.so` symlink.
## Packages
`lib/load.ml` resolves `(import rl "vendor:raylib")` before the checker runs.
The directory is the package; `vendor:` is a collection, resolved by walking up
from the importing file until a directory of that name is found; a path with no
collection is relative to the importing file. Importing is a **rename**: every
top-level name the package declares becomes `alias/name`, and every use of one
— in a type, in a body, in a struct literal, in an *array length* — is
rewritten to match. Local bindings shadow. Nothing downstream knows a package
existed; the checker sees one flat list of declarations whose names contain a
slash.
A package may also carry the C it binds to: every `.c` file in the directory is
compiled into the build, and a file named `link` lists extra linker arguments.
This is not a module system yet. No visibility (hence `rl/get-color-raw` being
callable), no cycle detection, and a package cannot import another one.
## sand.flan is two programs
plan.org wants sand tested twice — interactive at 120 fps, and headless over N
frames with the grid hashed, the version CI runs on native *and* wasm32. Those
cannot be one binary: `Load` collects a package's C sources and linker
arguments unconditionally, so anything importing the raylib package links
libraylib on every target regardless of what its `main` does, and on wasm32
that link cannot succeed.
So the simulation moved to `sand-sim/`, which imports nothing. `sand.flan`
imports it as `sim/` and adds the window, the mouse and the drawing;
`test/programs/sand-headless.flan` imports it and adds a seed, four
deterministic clouds, 40 frames and an FNV-1a hash. One copy of the physics.
The headless case is what actually *verifies* milestone 4 — running the
interactive build only proves it enters its loop, because with no mouse input
the grid stays empty and `paint-at`, `settle` and `move-grain` never execute on
real data. Measured through the probe: 168 grains painted around row 48, still
168 after 40 frames, lowest occupied row 68. Grains fall, and none are lost.
**Three edits were made to sand.flan's own text**, and they are language
decisions rather than fixes:
- `(defconst gravity 0.05)``(defconst gravity f32 0.05)`. An untyped float
constant is `f64`, `velocity` is `[f32]`, and there is no implicit widening.
- `(defvar current-color u32)``i32`. It is an index into `colors`, and
`(len colors)` is an `i32`.
- The file was split as above, so its body now says `sim/rows` and so on.
`(defn main [])` is unchanged — the short form, as plan.org says.
Painting is on **hold left mouse button** rather than on space, since the mouse
bindings exist now. Space is still what cycles the colour, on release, which is
a leftover and probably wants to move to the right button or to a key press.
## Bounds checks — done at milestone 3
`at` and `slice` emit `icmp``br` → cold block → `call``unreachable`; a
failure names the source location. Three check sites: `at` on `[n T]` (static
bound, folded by LLVM for a literal index — and a literal that is out of bounds
never reaches emit, `check.ml` rejects it), `at` on a slice or string (runtime
len), and `slice` (two comparisons — `lo <= hi` is not redundant, without it a
reversed range yields a huge unsigned length). All comparisons unsigned.
`Build.opts.checks` is on by default and **not** tied to `opts.opt`, which is
what lets the acceptance table run the same programs at `-O0` and `-O2` with
identical checks. The flag is `--no-bounds-checks`.
The write path is its own case: `(set (at arr n) …)` lowers through
`place`/`Pindex`, not through `At`, so a refactor that split them would break
the write check silently. The test covers both.
Cost, measured: a 50M-iteration dependency chain over a 1024-element array runs
at 0.110.12s checked against 0.120.13s unchecked. Indistinguishable.
## Why there is no interpreter
Open decision #7 is settled: **the compiled path is the only backend.** Both
arguments for a permanent interpreter had expired — the instrumentation step
debugger that wanted it is cut, and compiled redefinition measured at ~16ms,
perceptually instant for expression eval too. Milestone 3 did not need an
oracle either: the acceptance table is hand-written, so the table *is* the
oracle. Consequences already applied: milestone 2's "interpreted calls per
second" criterion is dropped, and the host ABI moved onto the critical path.
## The layout, which is the whole backend design
```
i8..i64 / u8..u64 i8..i64 signedness lives in the ops
f32 f64 float double
bool i1
an enum i32
[T] and string { ptr, i64 } ptr+len, non-owning
[n T] [n x T] inline, a value
(Ptr T) ptr opaque pointers
(Option T) { i8, T } tag 0 None, 1 Some
a struct a literal struct, declaration order
Unit and Never {}
```
No object headers anywhere, so a Flan struct is exactly its C struct and
nothing marshals. Two consequences carry the semantics:
- **Every slot is an `alloca`.** Reading a local is a `load`, assigning is a
`store`, and a `store` of an aggregate *is* the copy `spec-memory.md`
requires. `addr` of a local is then just the alloca, and `mem2reg` removes
the ones nobody addressed. `test/programs/values.flan` pins this down.
- **A place is a pointer, a value is a load from it.** `(set (.pos c) …)`
through a `(Ptr Cursor)` becomes a `getelementptr` on the pointer, not on a
copy. This is the split that would have made a tree-walker silently wrong.
Non-local exit is lowered explicitly: `return`, `some` and a failed bounds
check are branches, never platform unwinding, so wasm32 needs no exception
proposal.
## Sharp edges
Most of these are edges the language keeps and you should know about. Two —
the top-level namespace and the shift count, both found by review after
milestone 4 — were bugs that reached LLVM or ran wrong, and are **fixed**; each
says so. They stay written down because each one is now a rule the checker
enforces, and a later change could quietly drop it.
- **An index converts from a narrower integer and never from a wider one.**
`(nth colors current-color)` with a `u32` index works — anything above 2³¹
truncates to a negative `i32` and the unsigned bounds check rejects it. An
`i64` index is refused with the reason: 2³²+5 truncates to 5 and would read
the wrong element with no trap at all.
- **There is one top-level namespace, and `check.ml` now enforces it.** The
environment's tables are per-kind — structs, unions, aliases, enums,
functions, externs and globals each have their own — so only a function was
ever checked for a duplicate. `(defn item …)` beside `(defvar item …)` type
checked and then died in LLVM as `redefinition of function '@flan.item'`, a
message about an emitted symbol with no source location left, and two
colliding *type* declarations were not caught anywhere. One pass over
`Ast.declared_name` now runs before every other collection pass and rejects
the second declaration of a name whatever kind either one is. `declared_name`
lives in `ast.ml` because `Load` needs exactly the same set — the names an
import renames — and two copies of that list would drift.
- **A shift count is bounded, two different ways.** A shift by the operand's
own width or more is *poison* in LLVM, not a wrong number: `(defn main [] i32
(<< 1 32))` compiled at -O2 to a bare `retq`, returning an undefined value. A
literal count out of range is now rejected in `check.ml` — that is the typo
case — and `emit.ml` masks a computed count to `width - 1`, which is what the
hardware does anyway and which LLVM folds away whenever the count is
constant. The prelude's rotate masks its own count; that is now redundant but
harmless.
- **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real
`u64` and not an error. The cost is that a negative *decimal* literal is
accepted as a `u64` too, because the reader records the value and not how it
was written. Narrower unsigned types keep the strict check, which is where a
typo like `300` for a `u8` actually shows up.
- **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted
from the folding pass's value, because a global's initialiser has to be a
compile-time constant and only that pass knows this one is. Its range check
is therefore its own call to `in_range`; there is a regression test.
- A `let` binding takes no type annotation, which is why `sand-sim` names its
FNV constants instead of writing them inline.
- `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean
f64*: with a single body form the parser cannot tell a return type from the
first expression. Only the parameter position and `(Option …)` are
unambiguous.
## The reload primitive — dev loop steps 1 and 2, measured
`llc``ld -shared``dlopen` → call, with no protocol and no daemon.
`dune test` runs it: one function is recompiled into its own object and called
inside a process that is already running, twice, with a changed body the second
time.
| Step | Cost |
|---|---|
| `Emit.redefinition` | below the timer (<0.1ms) |
| `llc -O2 -filetype=obj` | 1517ms |
| `ld -shared` | 3ms |
| `dlopen` + `dlsym` | **0.04ms** |
**~19ms end to end**, and the load itself is free. plan.org's 16ms was measured
with clang somewhere else; this is the number from this codebase. For contrast,
`clang -shared` on the same IR is 50ms the driver is again most of the cost,
which is why the dev path skips it. `llc` and `clang` are both 20.1.8 here;
check that before trusting the `.ll`, since the driver absorbs IR the bare
tools reject.
`ld -shared` rather than `clang -shared` for a second reason: a shared object
is allowed undefined symbols, and that *is* the mechanism. What the new module
does **not** define is the whole design:
- **a global is `external`.** This settles the open question below in the only
direction that supports the demo: a redefinition can change a function's
body and can never re-initialise the program's data. Define the global and
the loaded object gets a second copy sand's `grid` would reset on every
reload, and "edit the code, keep the sand" is the thesis.
- **every other function is a `declare`**, so a redefined `settle` calls the
host's `move-grain` rather than freezing a private copy of it.
- **no `main`.** This module is loaded, not started.
Its string constants still come along; omitting them is an undefined `@.str.N`
at link time, and it is easy to miss because a one-function module usually has
none. `Emit.signature` is now the single place a function's LLVM signature is
spelled, because a `define` here and a `declare` there drift the moment one of
them grows a case for `Unit` or for a slice parameter.
**`-rdynamic` is load-bearing.** A normal executable exports nothing: `nm -D
calc-me | grep 'flan\.'` is empty, so a loaded module's `declare`s would have
nothing to bind to. The test passes it through `lflags`, which keeps it a
property of the dev build rather than of every build. `dlsym` on `"flan.bump"`
works a dot is legal in an ELF symbol.
Two things about the test are deliberate and are what make it prove anything:
both loads happen in **one process**, since two runs would pass while saying
nothing about an in-process swap; and the versions are **two paths**, since
`dlopen` caches by path and re-opening one would hand back the handle it
already had, so the check would lie. And `helper` is `(* x 2)` in one fixture
and `(* x 3)` in the other: the second body is dead text, since the module
declares `helper` rather than defining it, so the expected 1024 coming back
instead of 1036 is what proves the call landed on the host's copy. With the two
bodies identical nothing at run time would notice a module that grew its own.
String constants are emitted `private unnamed_addr`, so the module's own
`@.str.N` cannot be interposed by the host's worth knowing, because with
external linkage a redefined function would silently print the *old* text and
nothing would fail at link time. The fixtures each print a literal so that path
is actually exercised.
### Cells — how a call site follows a redefinition
Loading a new body is not installing it. A call bound at link time cannot be
made to notice one, so **a dev build routes every Flan-to-Flan call through a
cell**: a mutable global holding the address of the function that is current.
```
@"flan.cell.bump" = global ptr @"flan.bump" ; the host defines it
%p = load ptr, ptr @"flan.cell.bump" ; every call site
%r = call i64 %p()
```
Redefinition is then one store. A redefinition module declares the cells
`external`, exactly like the globals, and exposes `flan_reload_install()` that
stores its own body into its own cell cost **below a microsecond**, which is
what makes a frame-boundary swap a non-event.
The cell load is emitted *after* the arguments, so a redefinition landing
between two calls cannot land in the middle of one.
Four things about this that are not free choices:
- **`flan_reload_install` is a named function and not an ELF constructor.** A
constructor runs during `dlopen`, on whatever thread called it, mid-frame.
The agent has to choose when the store happens. Loading and installing are
separate on purpose.
- **A redefinition's own body is `hidden`.** Default visibility in a shared
object is interposable, and that applies to *taking the address* too: plain
`@"flan.bump"` inside the module resolves to the host's copy, so the
installer would publish the very function it was replacing and the reload
would appear to do nothing. There is a test on the linkage, because the
failure is silent.
- **This also fixes the self-call edge**, which the previous version of this
section listed as a sharp edge: a redefined function calling itself goes
through the cell like any other call, so it reaches the new body. v2 of the
fixture recurses on purpose, and would print the old body's text if it did
not.
- **`-rdynamic` is what exports the cells**, so it and cells are one flag:
`Build.opts.dev`, `flan build --dev`. This is the first time `opts` means
something semantic rather than an optimisation level.
LLVM cannot fold the indirection away the cell is an external mutable global
and a `--dev` build of calc-me keeps 46 indirect calls at `-O2`. The
acceptance table now runs `values`, `machine` and `sand-headless` as dev builds
as well; the sand hash is the case that matters, since it is the one result
that would notice a call reaching the wrong function.
### Still missing for `C-c C-c`
- **A new `defvar` has nowhere to live.** Editing one works a redefinition
module declares it `external`, so the storage stays the host's. *Adding* one
needs storage the host never laid out, which needs a runtime registry
(`flan_dev_cell(name)` handing out stable cell addresses, allocating on
first use) and globals reached through cells too. That is the next commit.
- **The agent**, so the install happens at a frame boundary in a real process
rather than in a C test harness. Step 3.
- **A session that holds the checker environment.** `Check.program` builds a
`new_env ()`, prepends the prelude, mutates it through `collect` and throws
it away. A REPL keeps it and has to check each new form into a scratch copy
and commit only on success, or one typo leaves a half-declared name behind
and every later eval sees it.
- **Layout drift has to be rejected.** Editing a `defstruct` or retyping a
`defvar` changes the shape of memory the running process already laid out.
The house rule below says compare against the declaration the session was
built with and refuse with a reason, rather than load a module that reads a
field at the wrong offset. Nothing does this yet.
## Where build time goes
`flan build calc-me.flan` was ~160ms, and ~95% of it was clang. **The object
cache is in**, and it is now ~110ms:
| Step | Cost |
|---|---|
| frontend: read parse load check emit | <10ms, below the timer |
| `clang` on the `.ll` | 60ms `llc` does the same codegen in **20ms** |
| `clang` on `flan_rt.c` | 40ms **now cached, paid once** |
| link | 20ms |
Every C translation unit a build needs the host shim and each package's shim
goes through `Build.compile_c`, which compiles to a `.o` under
`$TMPDIR/flan-objcache` and reuses it. The key is a digest of the source text,
the compiler (its path, size and mtime, so an upgrade invalidates without
paying a `clang --version` subprocess per build), `opts.opt` and `opts.target`.
The opt level has to be in there: the acceptance table builds the same programs
at `-O0` and `-O2`, and an `-O2` object must not serve an `-O0` build. The
object is written to a temporary name and `rename`d into place, so two
concurrent builds cannot see a half-written one.
Measured: calc-me 160ms 110ms; sand ~720ms ~700ms, since sand's time is
mostly linking libraylib and its `shim.c` was never the cost. The cache is
keyed by content, so it never needs invalidating by hand `rm -rf` on the
directory is only ever a disk-space decision.
The other cheap win is still open: skip the clang driver for the `.ll` (`llc` +
link directly), worth another ~40ms. It is a subset of the dev path's
machinery. Check `llc`'s major version against clang's before relying on it
the emitted IR text is currently absorbed by the driver behind
`-Wno-override-module`, and a version mismatch surfaces as IR parse errors.
**There is still no REPL.** Nothing does redefinition, `dlopen`, or nREPL.
`build` is the only way to run code.
## Next — the REPL is the priority
Decided in conversation: wasm32 can wait (it is believed to be a solved problem
once the builtins archive is in place), and **the dev loop is the thesis of the
project**, so it comes first. Staged so each step is runnable on its own
the failure mode is building a daemon and a protocol before knowing the reload
primitive works.
1. ~~**The reload primitive, measured.**~~ **Done** `Emit.redefinition`,
`Build.shared`, `test/reload_host.c`, ~19ms. See the section above.
2. ~~**Indirection cells.**~~ **Done** `Build.opts.dev` / `flan build --dev`,
`flan_reload_install`, and a fixture whose untouched call site follows the
swap. See the section above. Still to do here: a *new* `defvar`, which needs
a runtime cell registry.
3. **The agent, in C.** A socket listener in the game process, `dlopen` off the
game thread with `RTLD_NOW`, and the staged cell publish at a frame
boundary. It lives next to `flan_rt.c` no OCaml runtime in the game
binary. sand.flan is the test: redefine `settle` while grains are falling
and see the behaviour change with no stutter and no dropped frame.
4. **The daemon and nREPL** (bencode over a socket; `eval`, `load-file`,
`describe`, `interrupt`), then **5. the Emacs client** a focused ~35k
line client, not a CIDER fork. Deliberately last and deliberately separate:
the protocol is mechanical once 13 exist, and the editor client is where
the taste is.
**One decision left to settle before step 2**, because both change codegen and are
painful to retrofit:
- ~~**Do cells cover globals, or only functions?**~~ **Settled by step 1:
functions only.** A redefinition module declares every global `external`, so
globals live in the host and survive a reload which is what "edit the code,
keep the sand" needs. The consequence to watch is the other half: adding a
`defvar` to a file cannot take effect on reload, and changing one's type is a
silent mismatch against storage the host already laid out. Nothing detects
that yet.
- **What is a redefinition unit one function, or a file?** A file is much
easier to make correct and is what `load-file` wants anyway; one function is
what `C-c C-c` wants and is where the 16ms number comes from.
Deferred until after the dev loop:
6. **wasm32.** The user installed `wasi-libc-devel` and `wasi-libc-static`; the
sysroot is `/usr/wasm32-wasi` and `wasm-ld` is present. `clang
--target=wasm32-wasi --sysroot=/usr/wasm32-wasi` gets past the headers and
then **fails to link**: it wants
`lib/clang/20/lib/wasm32-unknown-wasi/libclang_rt.builtins.a`, which no
Fedora package provides (`dnf provides '*libclang_rt.builtins*wasm*'` finds
nothing). It has to come from a wasi-sdk release, dropped into clang's
resource directory. After that: teach `build.ml` `--sysroot`, and run the
acceptance table `sand-headless.flan` included, which is exactly why it
does not import raylib on both targets in CI.
Note plan.org has the *web* build linking raylib via emscripten, which
brings its own sysroot: wasi-sdk is right for the headless table, not
necessarily for the eventual game build.
7. **Loose ends from milestone 4**, none of them blocking: block-scoped
`defer`; package visibility, so `rl/get-color-raw` is not callable; a
package importing a package; imported unions.
## Watch for
The rule that caught the two misparse bugs applies unchanged: **anything that
binds a name, alters control flow, or is not yet implemented must be recognised
explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`,
`Result`/`try`, union values, closures, quoted symbols, generics and function
values *by name*, each with the milestone it belongs to; `load.ml` rejects the
package shapes it does not handle; and the FFI boundary rejects an aggregate.
The tests assert on the reason, not just on the failure.
## Untracked on purpose
`calc-me` and `sand`, the executables `flan build` drops beside their sources,
are now in `.gitignore` anchored (`/calc-me`, `/sand`) so the patterns cannot
also match `sand-sim/` or anything nested.
`old-ocaml/` the pre-rewrite menhir/ocamllex frontend, kept as reference and
excluded from the build by the root `dune` file. Its contents are also in git
history at `2c232dd`.