# Where this is Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its simulation has a headless acceptance case that runs on the `dune test` path at `-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and runs; the interpreter was dropped — open decision #7, settled, see below). ``` reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅ ``` | File | What it does | |---|---| | `lib/loc.ml` | source locations + `Loc.Error`, the frontend's one exception | | `lib/form.ml` | reader output: `Sym Kw Int Float Str Byte List Vec Map` | | `lib/reader.ml` | hand-written S-expression reader, no menhir/ocamllex | | `lib/ast.ml` | AST: `texpr`, `expr`, `place`, `pattern`, `decl` | | `lib/parse.ml` | forms → AST; special forms, desugaring, declarations | | `lib/load.ml` | **imports: a package directory → qualified declarations** | | `lib/types.ml` | resolved types; structural equality, `Never` fits anywhere | | `lib/tast.ml` | the typed IR the backend consumes | | `lib/check.ml` | AST → typed IR; two passes, bidirectional | | `lib/prelude.ml` | printers + `rand-f32`, written in Flan | | `lib/emit.ml` | typed IR → LLVM IR text | | `lib/build.ml` | `.ll` + the shim + the packages' C → clang → executable | | `runtime/flan_rt.c` | the host ABI: argv, stdout, exit, 4 conversions | | `vendor/raylib/` | **the raylib package: `raylib.flan`, `shim.c`, `link`** | | `sand-sim/` | **the falling-sand simulation, with no raylib in it** | | `bin/main.ml` | `flan read \| parse \| check \| emit \| build \| run` | | `test/test_flan.ml` | reader, parser and checker | | `test/test_acceptance.ml` | expression/result pairs + whole programs + the traps | ``` $ flan run calc-me.flan "1 + 2 * (3 - 0.5) / 2" 3.5 $ flan run test/programs/sand-headless.flan 2256461126764447066 $ flan run sand.flan # a window, 120 fps, hold space ``` ## What milestone 4 added **`dotimes`** desugars in `check.ml` to a `Let` plus a `While` — no new IR node. The bound is evaluated once into a hidden slot before the loop, so a body that changes it cannot change the trip count, and the loop variable is not assignable, which makes the generated step its only writer. **`defer`** is recognised in `check_fn` and nowhere else, because that is the only place that knows a form is at the top level of a function body. Each one is checked in place, then registered on the context; it emits nothing where it stands. Function exit runs them innermost-first, and an explicit `return` runs the ones registered *above* it — a defer written below a return has not executed yet and must not fire. A trap runs none of them, which follows from the bounds-check shape (`noreturn` then `unreachable`) rather than being a separate decision. `defer` inside a `let`, a loop or a branch is **rejected**, not accepted with function scope. It would run once at function exit rather than once per iteration, and that is the silent-wrongness class the rule below is about. Block-scoped defer is real work and is not done. **New builtins:** `zeroed` (takes its type from the place it is stored into), `min`/`max` (each operand through a slot, so neither is evaluated twice), `bit-and`/`bit-or`/`bit-xor`/`<<`/`>>` (integers only; `>>` is arithmetic on a signed type and logical on an unsigned one), and `rand-f32`. **`rand-f32` is in the prelude, in Flan** — PCG-XSH-RR 32 over a `u64` state. It is not libc's, because a grid hash is only a regression test if the sequence is byte-identical on native and wasm32 (plan.org, RNG is ours). `rand-seed` sets the state. This is what the bitwise operators were added for. **Enums and keywords.** `(defenum Name [member value ...])` gives a type that is an `i32` at run time and its own type in the checker, so `:space` at a call site resolves against the parameter's enum and a typo is an error there rather than a wrong number later. A keyword means nothing where no enum is expected — there is no keyword type to fall back on. ## Why the FFI goes through a C shim The decision that shapes the whole raylib package. What clang generates for raylib's own prototypes on x86-64: ``` Vector2 {float,float} → declare <2 x float> @GetMousePosition() Color {u8,u8,u8,u8} → declare void @ClearBackground(i32) Rectangle {4 × int} → declare { i64, i64 } @mkrect() ``` None of those is the struct's own LLVM type. A small aggregate's calling convention is not part of its layout — it is a per-target classification the *caller* has to reproduce, and x86-64, arm64 and wasm32 classify differently. Putting that in `emit.ml` is three classifiers to write and then keep correct forever, and a mistake shows up as `(.y m)` returning garbage rather than as a link error. So `vendor/raylib/shim.c` has one wrapper per binding, each one flattening the aggregates: a struct returns through an out-pointer, a struct argument is passed by pointer, a Flan string crosses as ptr+len and the shim NUL-terminates a copy. clang classifies all of it, per target, for free. `check.ml` enforces the rule — an aggregate in a `declare` signature is rejected with the reason — so the boundary cannot quietly acquire one. This is plan.org's "one narrow host ABI, implemented twice", and `flan_rt.c` is the same pattern. The price is a hand-written wrapper per raylib call. They are one-liners and mechanical enough to generate if that ever becomes the bottleneck. `raylib.flan` declares each `-raw` entry point and wraps it in an ordinary Flan function just below, so the surface sand.flan sees is `(rl/get-mouse-position)` returning a `Vector2`. Verified end to end, headless: `GetColor(0x11223344)` comes back as `17 34 51 68`, four separate bytes — a `Color` is *not* the little-endian reading of the packed integer, so an identity would have passed a weaker test. That case is in the acceptance table, skipped if `libraylib` is not installed. No raylib headers are needed: `shim.c` declares the prototypes it uses, so the build depends on the shared library being linkable and not on `raylib-devel`. `vendor/raylib/link` carries `-l:libraylib.so.550` because Fedora ships the runtime library without the `.so` symlink. ## Packages `lib/load.ml` resolves `(import rl "vendor:raylib")` before the checker runs. The directory is the package; `vendor:` is a collection, resolved by walking up from the importing file until a directory of that name is found; a path with no collection is relative to the importing file. Importing is a **rename**: every top-level name the package declares becomes `alias/name`, and every use of one — in a type, in a body, in a struct literal, in an *array length* — is rewritten to match. Local bindings shadow. Nothing downstream knows a package existed; the checker sees one flat list of declarations whose names contain a slash. A package may also carry the C it binds to: every `.c` file in the directory is compiled into the build, and a file named `link` lists extra linker arguments. This is not a module system yet. No visibility (hence `rl/get-color-raw` being callable), no cycle detection, and a package cannot import another one. ## sand.flan is two programs plan.org wants sand tested twice — interactive at 120 fps, and headless over N frames with the grid hashed, the version CI runs on native *and* wasm32. Those cannot be one binary: `Load` collects a package's C sources and linker arguments unconditionally, so anything importing the raylib package links libraylib on every target regardless of what its `main` does, and on wasm32 that link cannot succeed. So the simulation moved to `sand-sim/`, which imports nothing. `sand.flan` imports it as `sim/` and adds the window, the mouse and the drawing; `test/programs/sand-headless.flan` imports it and adds a seed, four deterministic clouds, 40 frames and an FNV-1a hash. One copy of the physics. The headless case is what actually *verifies* milestone 4 — running the interactive build only proves it enters its loop, because with no mouse input the grid stays empty and `paint-at`, `settle` and `move-grain` never execute on real data. Measured through the probe: 168 grains painted around row 4–8, still 168 after 40 frames, lowest occupied row 68. Grains fall, and none are lost. **Three edits were made to sand.flan's own text**, and they are language decisions rather than fixes: - `(defconst gravity 0.05)` → `(defconst gravity f32 0.05)`. An untyped float constant is `f64`, `velocity` is `[f32]`, and there is no implicit widening. - `(defvar current-color u32)` → `i32`. It is an index into `colors`, and `(len colors)` is an `i32`. - The file was split as above, so its body now says `sim/rows` and so on. `(defn main [])` is unchanged — the short form, as plan.org says. ## Bounds checks — done at milestone 3 `at` and `slice` emit `icmp` → `br` → cold block → `call` → `unreachable`; a failure names the source location. Three check sites: `at` on `[n T]` (static bound, folded by LLVM for a literal index — and a literal that is out of bounds never reaches emit, `check.ml` rejects it), `at` on a slice or string (runtime len), and `slice` (two comparisons — `lo <= hi` is not redundant, without it a reversed range yields a huge unsigned length). All comparisons unsigned. `Build.opts.checks` is on by default and **not** tied to `opts.opt`, which is what lets the acceptance table run the same programs at `-O0` and `-O2` with identical checks. The flag is `--no-bounds-checks`. The write path is its own case: `(set (at arr n) …)` lowers through `place`/`Pindex`, not through `At`, so a refactor that split them would break the write check silently. The test covers both. Cost, measured: a 50M-iteration dependency chain over a 1024-element array runs at 0.11–0.12s checked against 0.12–0.13s unchecked. Indistinguishable. ## Why there is no interpreter Open decision #7 is settled: **the compiled path is the only backend.** Both arguments for a permanent interpreter had expired — the instrumentation step debugger that wanted it is cut, and compiled redefinition measured at ~16ms, perceptually instant for expression eval too. Milestone 3 did not need an oracle either: the acceptance table is hand-written, so the table *is* the oracle. Consequences already applied: milestone 2's "interpreted calls per second" criterion is dropped, and the host ABI moved onto the critical path. ## The layout, which is the whole backend design ``` i8..i64 / u8..u64 i8..i64 signedness lives in the ops f32 f64 float double bool i1 an enum i32 [T] and string { ptr, i64 } ptr+len, non-owning [n T] [n x T] inline, a value (Ptr T) ptr opaque pointers (Option T) { i8, T } tag 0 None, 1 Some a struct a literal struct, declaration order Unit and Never {} ``` No object headers anywhere, so a Flan struct is exactly its C struct and nothing marshals. Two consequences carry the semantics: - **Every slot is an `alloca`.** Reading a local is a `load`, assigning is a `store`, and a `store` of an aggregate *is* the copy `spec-memory.md` requires. `addr` of a local is then just the alloca, and `mem2reg` removes the ones nobody addressed. `test/programs/values.flan` pins this down. - **A place is a pointer, a value is a load from it.** `(set (.pos c) …)` through a `(Ptr Cursor)` becomes a `getelementptr` on the pointer, not on a copy. This is the split that would have made a tree-walker silently wrong. Non-local exit is lowered explicitly: `return`, `some` and a failed bounds check are branches, never platform unwinding, so wasm32 needs no exception proposal. ## Sharp edges found and left visible - **An index converts from a narrower integer and never from a wider one.** `(nth colors current-color)` with a `u32` index works — anything above 2³¹ truncates to a negative `i32` and the unsigned bounds check rejects it. An `i64` index is refused with the reason: 2³²+5 truncates to 5 and would read the wrong element with no trap at all. - **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real `u64` and not an error. The cost is that a negative *decimal* literal is accepted as a `u64` too, because the reader records the value and not how it was written. Narrower unsigned types keep the strict check, which is where a typo like `300` for a `u8` actually shows up. - **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted from the folding pass's value, because a global's initialiser has to be a compile-time constant and only that pass knows this one is. Its range check is therefore its own call to `in_range`; there is a regression test. - A `let` binding takes no type annotation, which is why `sand-sim` names its FNV constants instead of writing them inline. - `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean f64*: with a single body form the parser cannot tell a return type from the first expression. Only the parameter position and `(Option …)` are unambiguous. ## Where build time goes `flan build calc-me.flan` is ~140ms, and ~95% of it is clang: | Step | Cost | |---|---| | frontend: read → parse → load → check → emit | <10ms, below the timer | | `clang` on the `.ll` | 60ms — `llc` does the same codegen in **20ms** | | `clang` on `flan_rt.c` | 40ms — recompiled every build, never changes | | link | 20ms | sand.flan additionally recompiles `shim.c` every build. Two cheap wins take the base to ~60ms: cache `flan_rt.o` (and the packages' `.o`), and skip the clang driver for the `.ll` (`llc` + link directly). Both are a subset of the dev path's machinery, so doing them now is not wasted work. **There is still no REPL.** Nothing does redefinition, `dlopen`, or nREPL. `build` is the only way to run code. ## Next 1. **wasm32.** The user installed `wasi-libc-devel` and `wasi-libc-static`; the sysroot is `/usr/wasm32-wasi` and `wasm-ld` is present. `clang --target=wasm32-wasi --sysroot=/usr/wasm32-wasi` gets past the headers and then **fails to link**: it wants `lib/clang/20/lib/wasm32-unknown-wasi/libclang_rt.builtins.a`, which no Fedora package provides (`dnf provides '*libclang_rt.builtins*wasm*'` finds nothing). It has to come from a wasi-sdk release, dropped into clang's resource directory. After that: teach `build.ml` `--sysroot`, and run the acceptance table — `sand-headless.flan` included, which is exactly why it does not import raylib — on both targets in CI. Note plan.org has the *web* build linking raylib via emscripten, which brings its own sysroot: wasi-sdk is right for the headless table, not necessarily for the eventual game build. 2. **The dev path / REPL.** `llc` + `ld -shared` + `dlopen` ≈ 16ms, a compiler daemon plus an in-game reload agent (plan.org, Dev architecture). There is now a frame loop for it to not stutter, which was the reason to do it after milestone 4. 3. **Loose ends from milestone 4**, none of them blocking: block-scoped `defer`; package visibility, so `rl/get-color-raw` is not callable; a package importing a package; imported unions. ## Watch for The rule that caught the two misparse bugs applies unchanged: **anything that binds a name, alters control flow, or is not yet implemented must be recognised explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`, `Result`/`try`, union values, closures, quoted symbols, generics and function values *by name*, each with the milestone it belongs to; `load.ml` rejects the package shapes it does not handle; and the FFI boundary rejects an aggregate. The tests assert on the reason, not just on the failure. ## Untracked on purpose `old-ocaml/` — the pre-rewrite menhir/ocamllex frontend, kept as reference and excluded from the build by the root `dune` file. Its contents are also in git history at `2c232dd`.