flan/plan.org

Flan — Design Plan

• Specs
• What Flan is
  • References
• Non-goals
• Memory — no GC
  • Why no persistent collections
  • The consequence that drives everything
• Data model
  • Managed classes — planned, deliberately separate from structs
• Types
  • Consequences
• Semantics kept
  • Conditions and restarts
  • Error handling, layered
  • Multimethods
  • Macros
• Host language
• Milestone-2 primitives
• Modules
• Compilation
  • Measured redefinition latency
  • Redefinition must not stutter the running game
    • Dev architecture: daemon plus agent
  • Why LLVM IR as text
  • The interpreter, and why there is not one
    • Settled: the compiled path is the only backend
• Targets
  • One narrow host ABI, implemented twice
• Dev vs release builds
• Hot reload
  • What redefinition cannot do
• Tooling
  • Emacs client
  • Debugging: two layers, two protocols
    • Reload and DAP
    • Cut: the instrumentation-based step debugger
  • Pause on exception
• Build sequence
• Runtime budget
• Open decisions
• Unverified claims in this plan

Specs

Two documents are normative and are settled ahead of implementation. Anything in this plan that contradicts them is out of date.

• spec-memory.md — ownership, the four container types, copies and moves, assignable places, generics without type classes, function values.
• spec-conditions.md — the six hard cases of conditions/restarts: what signal returns, no-handler behaviour, restart signatures, name shadowing, cleanup during a transfer, and crossing compiler-generated and foreign frames.

What Flan is

A minimal Lisp for game development. Clojure's brackets and a small slice of its API, C's memory and value model. No mandatory GC.

Not a Common Lisp, not a Clojure. In one line: Odin with a Lisp frontend and a live REPL.

References

• Odin — closest existing language. LLVM backend, manual memory, no GC, ships amd64 + arm64 + wasm32 (js_wasm32, wasi_wasm32, freestanding_wasm32). Proves this exact pipeline. Take directly:

  • context.allocator / context.temp_allocator (already this plan's design; free_all(context.temp_allocator) per frame is the frame arena)
  • $T compile-time parametric polymorphism — monomorphisation without a heavy type system
  • #soa struct-of-arrays syntax
  • fixed arrays with component-wise ops and swizzles ([4]f32, v.xyzw), built-in matrix type
  • defer, tagged unions, distinct types, bit sets
  • vendor:raylib

  Flan diverges by adding what Odin deliberately lacks: s-expressions, macros, conditions/restarts, hot reload, interactive development.

• SBCL — indirection cells for redefinition, conditions/restarts, the break loop.
• Janet — reference only; its immutable/mutable split is rejected (see Data model).
• Carp — statically typed Lisp with inference, no GC (ownership-based), compiles to C, aimed at games. Closest precedent for the language shape, as Odin is for the implementation shape.
• jank — cautionary; see Hot reload.

Non-goals

• Numeric tower (no bignums, rationals, complex)
• Full CLOS/MOP (metaclasses, method combination, slot-missing, arbitrary change-class), format, pathnames, streams, sequences-over-anything, CL reader
• Clojure's lazy seqs, JVM interop, core.async
• Persistent collections, and immutable collection types generally (see Data model)
• Live-image development at SBCL's level
• Consoles

Memory — no GC

Allocators all the way down; malloc hidden behind them.

Tier	Strategy	Cost
Frame	arena, bulk reset each frame	free
Entities	pool + generational handles	free
Subsystem	region, freed wholesale	free
Dev/REPL	leaks by design, reset on reload	dev only

• Allocator is part of the calling convention, so a refcounted allocator can be added later without a language change.
• Generational handles instead of pointers for cross-references: a stale reference is detectable, not undefined behaviour.
• Symbols and code live in a permanent arena that only grows.

Why no persistent collections

Structure sharing destroys clear ownership, which is the only thing that forces a collector. Replaced by value structs that copy on assignment — see Data model.

The consequence that drives everything

Plain struct means no object headers. Every struct is exactly its C layout, arrays are C arrays, so there is no marshalling layer and no wrapper allocation. The future managed class facility below is deliberately a separate kind of value: it has identity, metadata, and an implementation-defined representation.

FFI is still a boundary — ownership, who allocates, string and slice representation, struct padding, callbacks into Flan, error handling, and the platform ABI all remain real work. What C layout buys is that the data crosses for free. See spec-conditions.md §6 for the one hard rule: a restart transfer cannot cross a foreign frame.

Data model

No immutable collection types. Value semantics plus const, as in C/Zig/Odin — not Janet's tuple/struct vs array/table split, which was an answer to a GC'd world.

• Four container types, distinct in both type and ownership — see spec-memory.md, which is normative:

  Notation	Layout	Assignment	Owns
  [n T]	inline, n items	copies	no
  [T]	ptr+len	copies the view	no
  (Vec T)	ptr+len+cap	moves	yes
  (Map K V)	open addressing	moves	yes

  Vec and Map are monomorphic on element type and record their allocator. Not a Lua-style array/hash hybrid — that is what makes Lua's layout and performance unpredictable.

• Map keys initially use compiler-provided structural equality and hashing for integers, enums, strings, fixed arrays and value structs; pointers, slices and owning containers are excluded. A map is homogeneous, and empty construction names its types: (let [enemies (map-new string Enemy)] ...). get returns (Option V); put is the ()-returning upsert. See spec-memory.md for the deferred move-aware operations.
• Operations: get, put, remove, push, pop, at, len, update. Copying is explicit: (clone m), and owning containers move rather than copy on assignment. No ! convention — nothing is immutable, so it would carry no information. No assoc; it only existed as the copy-returning form.
• const qualifier on references and slices: compile-time contract that a callee will not mutate. Zero runtime cost.
• Value structs copy on assignment — but only value structs. Ownership is structural: a struct is a value type iff every field is, so one Vec field makes it move-only. This is what keeps "copies on assignment" from meaning a shallow copy that aliases owned storage. Deep copies are always explicit: (clone x). Value structs are the snapshot / undo / replay story; they need no separate type.
• Literals live in read-only memory.
• A global's initialiser may be computed. A value the linker can write goes into the image and costs nothing to start; anything else is stored at startup, by a function main calls before a line of the program's own code — Odin's __$startup_runtime shape rather than a constructor, so the runtime is up and the order is the compiler's to choose. The computed ones are sorted by what they read, so (defvar b i64 (+ a 1)) works above a; a cycle between two of them is a compile error naming both. A transfer out of an initialiser — signal, restart-case — is refused: nothing has established a handler that early. A reload never re-runs an initialiser, which is what keeps the live state a reload exists to preserve.
• Struct literals name fields: (Cursor {.src src .pos 0}). Omitted fields are zeroed, as in Odin — the same rule as a declaration with no initialiser, so (Cursor {.src src}) is complete and means pos is 0.
• Zero is initialisation (ZII), with an opt-out. No initialiser means all-bytes-zero. (defvar buf [65536 u8] uninit) skips it, exactly as Odin's --- does, for a large buffer that is about to be overwritten. uninit is greppable and rare by design; reading an uninit value before writing it is undefined, and dev builds poison the memory so the bug is loud.
• Slices as ptr+len, non-owning; raw pointers (Ptr T) with explicit deref; visible casts. resolve yields (Ptr a) where deref yields a value — that is how a matched struct is mutated in place rather than as a copy.
• Flat and SoA arrays.
• No implicit allocation anywhere in the core.

Managed classes — planned, deliberately separate from structs

struct remains Flan's default: fixed-layout values suitable for stack storage, arrays, SIMD/SoA, FFI, and hot paths. It cannot acquire fields while live values exist. A future class is for long-lived gameplay/editor objects that need identity, extensibility, and live schema changes:

(defclass Enemy
  (x f32) (y f32) (health i32))

(defmethod update ((e Enemy) dt) ...)

Classes require a managed allocation strategy and runtime class/shape metadata, but not necessarily a tracing GC. The initial likely choices are a world or session arena, pool allocation behind generational (Handle T) values, or an explicitly owned region. A small tracing GC confined to class instances remains an option if cyclic graphs prove burdensome; it never changes struct layout or the C ABI.

Generic functions attach operations independently of class definitions. This addresses the expression problem for dynamic game objects: a later module can add either (defmethod draw ((e Enemy)) ...) or a new Capsule class plus its draw method without editing the original class or a central closed match. Start with exact-class, single-argument dispatch. Multi-argument dispatch is the later extension for interactions such as (collide Player Enemy). Method specificity/ambiguity rules are required before inheritance or multiple dispatch is enabled.

Class redefinition has a stable class identity and numbered layouts. Changing slots leaves existing instances at their old layout until an explicit migration at a reload/frame boundary. Added fields use declared defaults; removed fields are discarded; renamed or type-changed fields require user migration code. This is the small, eager and debuggable analogue of CLOS make-instances-obsolete / update-instance-for-redefined-class, rather than surprising lazy mutation on field access:

(redefine-class Enemy
  (x f32) (y f32) (health i32) (shield i32))
(defmethod migrate ((old Enemy@1) (new Enemy@2))
  (set (.shield new) (.health old)))     ; a default would not have been right
(migrate-instances Enemy)

The precise class syntax, inheritance, storage strategy, and migration API are not frozen. Do not add classes until ordinary struct, Handle, and reload semantics are working.

Immutability also serves the optimiser: a value known never to be mutated can be copied into registers and stack-allocated freely. Mutability is what forces heap identity.

Types

Statically typed. Not forced — tagging and boxing give dynamic typing without a collector, as Forth-lineage and refcounted dynamic languages show. It is chosen, and the reason is that dynamic typing would require paying a tag word on every value, which is exactly the header cost dropping the GC was meant to avoid. Under static typing the tag is paid only where it is asked for: in any, in Error, and on a managed class instance. An ordinary struct never carries one.

• Types are mandatory; inference makes them feel optional. Annotate function signatures, infer locals — Odin/Zig/Rust ergonomics.
• Signatures are annotated as inline name/type pairs, as in defstruct and a restart-case clause: (defn area [s Shape] f32 ...). A let is not one of them — a local is inferred from its initialiser and takes no annotation at all. No separate declare form — declare is kept only where there is no body (forward declarations, FFI).
• Annotations at function boundaries are unavoidable, because compile-time overloading is incompatible with full inference. Locals are inferred.
• The return type is always written, and a function that returns nothing writes () — a real zero-sized type with one value, not C's void. Generic code over it works, so there is no Action~/~Func split. The slot was optional once and the parser guessed between a return type and a body form from a table of type names; the guess was silently wrong twice, so the slot is mandatory.
• Every type notation reads as exactly one data item: [f32], [4 f32], (Vec f32), (Map string i32), (Ptr World), (Fn [f32] bool), (Option $t), (Handle $t). The map spelling was {string i32} once and is not any more: braces in type position are refused by name. The where clause below is a brace form at the head of a body, so keeping the brace type would have put (defn f [...] {string i32} {:where ...} body) in the language — two braces in a row meaning different things.
• i8..i64, u8..u64, f32, f64 as real machine types; wrapping arithmetic.
• Vector width 128-bit. Fixed arrays with component-wise ops and swizzles.
• Parametric polymorphism by monomorphisation (Odin's model, no type classes, no HKTs). A type variable is written $t wherever a type goes — parameter, return type, or nested as [$t] or (Vec $t) — and bare t wherever a type's name is an argument in expression position: (vec-new t), (map-new t i32), (pool-new t), and the cast (t x). This is what makes map-in-place~/~filter~/~reduce and the monomorphic containers work; it collapsed the prelude's per-type families into one function each. A generic body is checked abstractly, with nothing substituted, so =, <, + and hash over an unconstrained variable are rejected at the definition rather than at whichever call site first instantiates it. This is not Odin's rule — Odin checks a polymorphic body only per instantiation — and the two decisions are independent even though this line once bundled them. What makes the rejection liveable is a where clause over compile-time type predicates, which is Odin's (core/slice/slice.odin:289, where intrinsics.type_is_ordered(T)). It is written as a Clojure-style map at the head of the body — {:where (ordered? $t)}, or a vector for more than one, {:where [(copyable? $t) (copyable? $u)]} — on the precedent of Clojure's {:pre ... :post ...}, and because a bare {} in expression position is already refused so nothing else it could be. sort declares ordered? of its variable, the abstract pass then allows < in the body, and each instantiation checks the concrete type satisfies the predicate and refuses the call site if it does not. There are five predicates — ordered?, equal?, hashable?, numeric?, copyable? — against Odin's forty-one, and they entail one another in one direction, so one clause usually does: numeric? gives ordered?, ordered? gives equal?, and any of the four gives copyable?. copyable? has no Odin counterpart, because Odin has no move semantics and a $T there never has to answer the question. A type variable is move-only by default and copyable? is the opt-out: whether a variable is move-only is not decidable abstractly — i32 at one instantiation, (Vec i32) at the next — so the checker takes the stricter rule, which can only refuse a program that would have been fine and never admit one that double-frees. The prior art is Rust's T: Copy, differing in that the compiler answers the question rather than a user implementing a trait. hashable? is what lets a variable key a map: without it the type (Map $t i32) is refused where it is written, and with it the refusal moves to the call site that names an unhashable key. This is not a type class and the difference is worth keeping straight: a type class carries implementations selected per instance and extensible by anyone, and needs dictionaries and coherence rules. A predicate carries nothing — it gates a builtin the compiler already has. The ceiling is that no one can supply a user-defined <; every operation the prelude and the containers need is a primitive, so it does not bind. Compile-time interfaces, if they are ever wanted, come after the base checker is stable. println is a compiler-provided exception and is on an explicit allow-list of forms the abstract pass defers to instantiation: whether a printer exists for a variable is only decidable once it is substituted. The Map operations over a variable key are the other member, for the same reason — the hash and the equality are concrete symbols chosen from the concrete key type — and that one is paid for by hashable? being written in the signature, so the refusal it moves still lands against a requirement somebody wrote down. Keep the list short: every member moves a refusal from the definition to a call site, which is what the abstract rule exists to avoid.
• Function values split three ways (spec-memory.md): (Fn [T1 T2] R) is a plain pointer with no environment — the only kind that crosses FFI or sits in a reload cell; a non-escaping fn captures enclosing locals by value into a stack environment, which is what reduce callbacks and handler-bind handlers use; an escaping closure needs a heap environment and is still an open decision.
• No monads, no HKTs, no type classes. Effects are direct; error handling is conditions plus Option and or-else. Monadic sequencing, if ever wanted, is a macro.
• any is an explicit opt-in tagged union, for heterogeneous containers and debug printing. Error is a second, specialised erased carrier for fallible results; neither adds a header to ordinary values.
• Tagged unions, distinct types, bit sets.

Consequences

• For ~struct~s, multimethods are compile-time overload resolution, not runtime dispatch. Genuine runtime dispatch is on an explicit enum or tagged union. Planned managed classes instead use generic-function dispatch as described in "Managed classes"; this is an intentionally separate facility.
• Conditions still work: a condition is a struct, the signal channel is a tagged union, handler-case type matching resolves at compile time.
• The REPL works — the compiler knows each site's type and emits the right printer, as in the OCaml and Haskell REPLs.
• Macros are unaffected; they run on syntax before typing.

Semantics kept

Conditions and restarts

The headline feature. Four operators: handler-bind, handler-case, restart-case, invoke-restart. No condition class hierarchy — struct types plus predicate matching. Operational semantics: spec-conditions.md, which is normative. The ~500-line estimate below was for the operators alone and does not include the explicit transfer lowering, which is compiler work.

signal does not mean "fail". It means: here is something notable, here is the data, and here are the ways I know how to continue. Restarts are a menu; a handler installed by an outer caller reads the data and picks one — or picks nothing and returns, in which case the signaller simply carries on.

Handler does	Behaviour you get
returns normally	accumulate and continue
invokes a restart	recover / retry / substitute
unwinds (handler-case)	try/catch

Why this replaces most error types. A condition signalled deep in a call stack never appears in intermediate signatures. Nothing to thread, no From conversions, no anyhow equivalent. Compare Rust, where every layer must name every error type it passes through.

Accumulation. A handler that records and invokes a continue-style restart gives error collection with no applicative or monad. This is how CL compilers report every error in one pass.

Costs nothing when unused. The handler stack is a linked list of stack-allocated frames: handler-bind is a couple of stores, signal with no handler is a null check. No allocation, safe inside a frame loop.

Condition objects live on the signalling frame's stack, since nothing unwinds before the handler runs. Because conditions are value structs, accumulating one into an outliving array copies it. A pointer-to-condition would dangle.

Under static typing. Restarts are dynamically scoped and named, so (invoke-restart 'skip-form) cannot be fully checked at compile time. Accept a runtime error initially; a statically tracked restart set (as Zig tracks error sets) is a nice-to-have, not a blocker. signal has type (), invoke-restart and error have type Never, and every restart clause shares one type with the restart-case body — so a restart-case in value position needs a fall-through that produces the type or diverges.

Unwinding is only the transfer — invoking an outer restart. Lowered explicitly, see Compilation.

Error handling, layered

1. Option for expected absence: lookup miss, empty collection, end of stream.
2. Conditions for exceptional failure where a caller may have a recovery policy.
3. Result where failure should be visible in the signature (parsers, fallible pure functions). The ordinary form is (Result T Error): an erased built-in error carrier, not an external anyhow-style library. A precise (Result T E) remains available where it materially helps.

Error is a small value carrying a type id, pointer to an allocator-owned typed payload, source location, context and a rendered summary. It preserves structured fields for debugger/editor inspection while the summary is stable to print and send over the wire. This makes (Vec Error) the standard heterogeneous diagnostic collection. Wrapping an error uses the current implicit allocator; metadata is out-of-band, so ordinary values keep their header-free C layout.

Rule of thumb: if you can name the one correct recovery at the point of failure, return a Result. If the answer is "depends who is calling", signal a condition.

Mechanisms — two unwrap operators, because they are two different things (Zig's split):

• try — unwrap Ok, else early-return Err
• some — unwrap Some, else early-return None
• (ok-or opt err) / (ok res) — conversions, always explicit. try does not accept an Option in a Result-returning function. Converting a precise error to Error is explicit too; there is no trait-based conversion search.
• some-> (short-circuiting thread), or-else, if-let
• errdefer — cleanup on the failure path only, pairs with arenas

Async composes by nesting, no new mechanism: (try (await (http-get url))) — await unwraps the task, try unwraps the Result inside it.

try and some are macros expanding to early returns, and early return is the explicit non-local-exit lowering, so all of this compiles identically on native and wasm32.

Restarts go at the resync point, once — the loop over top-level forms in a parser, not inside every function below it. Intermediate frames stay silent about restarts for the same reason they stay silent about conditions.

No monads, no HKTs. Chaining that would want do-notation is a macro.

Async coupling: with a state-machine transform the handler stack must live in the task state, not thread-local, or a handler established before an await is out of scope after resumption. Cheap if designed in, painful later.

Multimethods

Compile-time overload resolution on argument types. No precedence lists, no method combination, no MOP, no runtime dispatch table — see Types.

Macros

• &env equivalent: macros seeing the lexical environment (names of locals in scope). No longer urgent — its "decide now" status came from the instrumentation-based step debugger, which is cut (see Tooling). Milestone 5.
• Macros are initially deliberately non-hygienic, in the Common Lisp/Clojure style. A macro uses explicit gensym for introduced bindings; there is no syntax-object hygiene system. Lexically local macros (macrolet-style) wait for a concrete use case.

Host language

OCaml. The compiler only; the runtime and stdlib are Flan with a few C primitives, and are never bootstrapped away.

The LLVM question does not bear on this: the release backend emits LLVM IR as text and shells out to clang, so no language needs LLVM bindings, and C++ or Rust buy nothing here. What the choice actually turns on is that milestones 2–5 are a reader, a typed IR, a checker and the code generators behind it — variants and exhaustive pattern matching, which is the one domain where OCaml is not a preference but a clear win. There is also a menhir lexer/parser already started in old-ocaml/.

The honest alternative is Rust, and it wins on exactly one axis: if the compiler is a language you will not enjoy maintaining in three months, that outweighs being 30% shorter. Nothing technical breaks either way.

Self-hosting is not a goal and must not drive this. It appears nowhere in the build sequence. For a game language it buys dogfooding at the price of a second compiler to maintain forever. Choose as if the host language is permanent.

Milestone-2 primitives

The runtime provides these; everything else is written in Flan. Keeping the list short is the whole strategy — it is what makes the LLVM backend and the wasm32 target cheap, because a primitive is the only thing implemented twice.

Primitive	Notes
argv	[string], borrowed, never freed
write-stdout	takes [u8]; the ONE output primitive
exit	i32 status
len at slice	on fixed arrays and slices
bytes	string → [u8], a view, no copy
bytes->f64 bytes->i64	and the inverses, for printing
addr	address of a place
arithmetic, comparison, casts	per machine type

Printing is not a primitive: write-stdout is the one output primitive and println and print are written over it. Both are compiler-provided — the checker walks the concrete type at the call site and emits the printer for it (lib/render.ml, shared with the REPL's C-x C-e). This is intentionally not user-defined overload resolution — ordinary values remain untagged, and there is nothing to dispatch on at run time — which is why it did not have to wait for generics as this line once said it would. Generic instantiations are concrete types by the time the checker sees them, so they need nothing further; any and Error are the remaining case, and they carry the metadata their dynamic printers need.

Entry point. (defn main [args [string]] i32). Both the parameter and the return type are optional: omitting args means the program ignores argv, a return type of () means an exit status of 0. sand.flan uses the short form, calc-me the long one.

RNG is ours, not libc's. rand-f32 is a seeded PRNG implemented in Flan (xoshiro or PCG), because a grid hash is only a regression test if the sequence is byte-identical on native and wasm32. Decided here rather than at milestone 4, since a headless deterministic sand run is the cross-target test.

Modules

There are modules — Odin calls them packages, and so does Flan. What is removed is Clojure's ns form: no path that must mirror the directory, no :require~/:refer~/~:as~/~:import~ vocabulary, no per-file namespace object.

• The directory is the package. Every file in a directory shares one top-level scope. Files in a package do not import each other, and top-level names are order-independent, so mutually recursive functions need no forward declaration.
• The package declaration is optional, which is the one place Flan diverges from Odin — Odin requires package foo as the first line of every file and requires it to agree across the directory. Flan infers the package name from the directory name, and (package parser) is written only when the name must differ from the directory (a directory named flan-parser, a scratch directory with a name that is not an identifier). When present it must agree across the directory, as in Odin.
• A loose file in ~/scratch/ is a package of one. No project file, no manifest, no declaration. Open it, connect the REPL, start working. The ceremony budget for "new file, running REPL" is zero — this is the requirement the whole scheme is designed around, and it is why the declaration is optional rather than mandatory.
• Cross-package: (import rl "vendor:raylib"), and everything from it is qualified rl/foo. One form, one meaning, no unqualified-import mode.
• Collections in the path (vendor:, core:) are Odin's, and are just root-directory aliases.

Compilation

One evaluator and three paths. The split is not dev-vs-release; it is does this code have a frame budget. There is no interpreter: open decision #7 is settled the other way from how this section was first written, and docs/BUILT.md's "Why there is no interpreter" carries the reasoning. Compiling is the only way a form is ever run, so there is no second evaluator that could disagree with the first about what a program means.

expression eval:    flan → typed IR → .ll → llc → ld -shared → dlopen → call
                                                                       ~19ms  (MEASURED)
dev redefinition:   the same path, ending in a cell store rather than a call
release build:      flan → typed IR → .ll → clang --target={native,wasm32}

Hard requirement: eval is immediate. Not "fast enough for a build" — immediate, because the whole point of the live loop is that you see the result. 19ms is around one frame at 60fps and under the 50ms threshold where a response stops feeling instantaneous. The rule that buys it: *never invoke the ~clang driver on the dev path.*

Expression eval — C-x C-e, calling a function, inspecting a var, running a test — is compiled like everything else, into its own shared object, which is then loaded and called. What made an interpreter look necessary was the assumption that this had to be sub-millisecond; the measurement below is that the compiled route is already inside the threshold, and the one thing an interpreter would have bought is an oracle the hand-written acceptance table supplies instead.

Dev redefinition — C-c C-c on a function inside a running game — has an 8ms frame budget to respect. It recompiles the one function, links it, and does the atomic indirection-cell store. This is what the Hot reload section has always described, and it is the same machinery expression eval uses, one step further on.

Release is whole-program AOT with direct calls and no cells.

A second code generator, not a second evaluator. lib/x86.ml emits x86-64 machine code directly and is selected with --x86; it exists because llc is most of the 19ms above. It is a different route from the same typed IR to the same observable behaviour, not a different semantics, and what holds it to that is spike/x86/survey.sh: every program in the corpus is built both ways and byte-compared on stdout, stderr and exit status. At the time of writing that is 103 MATCH, 0 DIFFER, 0 refused by name. It handles conditions, bounds checks, indirection cells, redefinition modules and DWARF line tables; what it does not have, and must not grow, is an aggregate classifier — an --x86 host therefore takes --x86 modules and an LLVM host takes LLVM ones, and lib/build.ml refuses the crossed pair by name.

Measured redefinition latency

Single function, x86-64, clang 20.1.8, 20 iterations each:

Step	Per call	Dev path?
clang -shared (driver: compile + link)	52.0ms	no
clang -c (driver: compile only)	22.5ms	no
llc -filetype=obj	13.9ms	yes
ld -shared from the .o	2.4ms	yes

llc + ld + dlopen ≈ 16ms. The clang driver is the cost, not codegen — it forks a second process and re-does argument and target resolution. Codegen itself barely scales with function size: 721 lines of IR took 16.7ms against 13.9ms for 8 lines, because 10ms is ~llc startup loading libLLVM. A realistic redefined function lands in the same 15–17ms.

This is why an in-process ORC JIT is not needed. It would take 16ms to ~3ms; the difference is below perception, and the price is a version-pinned libLLVM and C++ linkage from the host language, forever.

Redefinition must not stutter the running game

The 16ms is not paid by the game thread. llc and ld are already separate processes running on other cores. What the game process does is smaller:

Step	Cost	Game thread?
llc, ld	16.3ms	no, separate processes
dlopen the new .so	~0.1–1ms	must not be
atomic store into the cell	ns	yes, and free

Two design choices are load-bearing, and neither is automatic:

1. dlopen happens on the reload thread. It mmaps, relocates and takes the loader lock; off-thread it blocks nobody, because the game thread is not doing dynamic linking. Load with RTLD_NOW so lazy PLT resolution cannot ambush the game thread on a later first call.
2. Publish at a frame boundary, in a batch. This matters more than the threading. Storing each cell the moment it is ready lets the game observe a half-applied redefinition — two functions that changed together applied one frame apart, or a function swapped mid-frame with half the entities already updated by the old code. Instead the reload thread stages the complete set of new pointers and sets a flag; the game loop tests the flag once at the top of the frame and does N stores. One relaxed atomic load per frame when nothing changed.

Residual cost: the first call into new code page-faults and misses i-cache. Tens of microseconds, not visible.

Dev architecture: daemon plus agent

• Compiler daemon, a separate process: the OCaml frontend, the nREPL server, and the llc~/~ld invocations. Editors talk to this.
• Reload agent, linked into the game binary: a socket listener, dlopen, and the frame-boundary cell publisher. A few hundred lines, and no OCaml runtime in the game.

This is why the dev runtime is multithreaded — it needs the reload thread. That is settled, and is independent of whether the language exposes threads, which is still open decision #4.

This is also what settled the interpreter question: if the agent can dlopen and call anything in under 20ms, then even "eval this expression against live game state" is a compiled .so, and no interpreter is needed inside the game process. That is the route C-x C-e actually takes.

Why LLVM IR as text

	text .ll → clang	libLLVM bindings	emit C
Build dependency	a clang on PATH	matching libLLVM, version-pinned, C++ linkage	any C compiler
Breaks on LLVM upgrade	no	routinely	no
Debuggable	.ll is readable	print-from-memory	readable, but lies about origin
In-process JIT	no	yes (ORC)	no
Control of layout / ABI / tail calls	full	full	poor

The only column text loses is the JIT one, and the measurement above shows the loss is ~13ms — below perception. ORC remains addable later behind the same typed IR without touching the language, but nothing currently argues for it.

The interpreter, and why there is not one

An interpreter could never have run sand, and that was the first half of the argument. 200 × 280 = 56,000 cells, scanned by game-update and again by game-draw — ~112,000 interpreted cell-visits per frame against an 8.3ms budget at 120fps. At an optimistic 100ns per visit (environment allocation, argument binding, two index computations, a compare) that is 11ms before settle, paint, or a single raylib call. Expect 20–30fps. Milestone 4's interactive acceptance test was always going to run on the compiled dev build.

Settled: the compiled path is the only backend

This was open decision #7 and it is closed. Compiled redefinition measured at ~19ms is perceptually instant for expression eval too, so the one thing an interpreter was still wanted for went away; the instrumentation-based step debugger that wanted it is cut (see Tooling); and milestone 3 did not need it as an oracle either, because the acceptance table is hand-written and the table is the oracle. What is bought by dropping it is the standing obligation: two evaluators must agree on observable behaviour forever, and every divergence is a bug that reproduces in only one of them. docs/BUILT.md's "Why there is no interpreter" records the decision; lib/expand.ml states it at the top of the file, because macros are where the absence stopped being free — a macro has to run at compile time and there is nothing to interpret it with, so the compiler compiles it into a shared object and loads it with dlopen into its own process.

Consequences applied elsewhere in this document: milestone 2's "interpreted calls per second" exit criterion is dropped, and the host ABI moved onto the critical path in its place.

The paths that remain share the frontend and the typed IR and must agree on observable behaviour. That agreement is what the acceptance programs test, and for the two code generators it is tested byte for byte.

• Non-local exit lowered explicitly (result propagation + branch targets), not via platform unwinding. Same on both targets, no dependency on the WASM exception-handling proposal. Every Flan function carries the transfer channel; uniformity keeps indirect calls and hot reload ABI-safe. A later optimisation may narrow that cost, but it cannot change the ABI.
• Stdlib written in Flan, not the host language. A few hundred primitives per backend, everything else on top. This is what keeps a second backend cheap.

Targets

• Desktop: AOT to native, x86-64 and arm64. Primary development target.
• WASM: AOT build artifact only, not interactive-first — no REPL, no hot reload, no debugger there. That is not the same as untested: every runtime or ABI feature ships with automated wasm32 tests in CI from the first one, because a divergence found at ship time is a rewrite.
• Host binary links raylib natively; web links raylib via emscripten.

One narrow host ABI, implemented twice

The portability risk is the host interface, not the language. Divergence points:

• Filesystem — pack assets, one abstraction, never touch paths
• Threads — decide now; retrofitting is worse than the reverse
• Blocking — browser main thread cannot block
• Audio/input/window — constrain to the raylib subset identical on both

Dev vs release builds

Deliberately different.

	Dev	Release
Backend	LLVM, or --x86	LLVM/clang
Calls	indirection cells	direct
Code	never freed	static
Frames	shadow stack	none
Structs	version word	none
Reload	yes	no

Build and run the release config regularly, not just at ship time.

Hot reload

Every cross-function call goes through an indirection cell; body redefinition is one atomic pointer store. A top-level function value is a stable trampoline over the same cell, never a pointer to a specific body, so callers holding it observe a body redefinition too. Old code is never unloaded, so a thread mid-execution finishes safely in the old version.

A signature-changing redefinition creates a new internal function version and trampoline. Newly compiled callers use it; existing callers and stored function values safely retain the old version. The session immediately warns at each tracked old caller site, and recompiling that caller either updates it or gives a normal type error. This preserves live running code without hiding stale calls.

This is the fix for jank issue #947 (segfault redefining a running loop's function plus its callee — their JIT relinks and unloads under a running thread).

What redefinition cannot do

Patch a mid-execution frame and continue at the same PC — its register allocation belongs to the old compilation. No implementation does this. "Resume" means re-entering from an established restart point.

Tooling

Server speaks nREPL (bencode over socket) — the transport and the core ops (eval, load-file, describe, interrupt) are genuinely reusable, and that is what the ~500–1000 lines buys. It does not buy CIDER/Conjure/Calva compatibility: their useful operations assume Clojure-shaped vars, namespaces, nses-of-symbols and middleware. Treat "speaks nREPL" as milestone 7a and "an editor client that is pleasant" as a separate milestone 7b. In the dev runtime:

• eval string in package; compile form/file with source locations
• completion, arglist, describe, find-definition
• macroexpand, one step or to the fixpoint — the compiler builds the macro into a shared object and dlopens it to run the expansion, which is the same route a file's own macros take
• backtrace + restarts; interrupt

Emacs client

Focused client, 3–5k lines. Do *not* fork CIDER (~30k lines elisp, deeply Clojure-coupled) — reference it. ~clojure-mode-derived major mode, overlay rendering, hydra for stepping bindings (transient is the maintained alternative).

Debugging: two layers, two protocols

Do not extend nREPL with registers or memory. The protocol is the easy part — adding an op is trivial; implementing it means becoming a debugger (ptrace, trap handling, frame unwinding), duplicating an enormous existing project.

Capability	DAP (lldb-dap + dape)	nREPL
Registers, memory, native stack	yes	never
Breakpoints, watchpoints, stepping	yes	—
Eval Flan expressions with real semantics	no	yes
Redefine a function and continue	no	yes
Invoke a restart	no	yes
Pause before unwinding	no	yes

Neither covers the other's column, so this is not a choice between them.

DAP is nearly free. No debug adapter is written: emit DWARF from the backend and point lldb-dap at the binary; dape speaks to that. Both code generators do — the hand-written one writes its compile unit, subprograms and line table out as bytes, since .loc cannot work against a file whose instructions are .byte blobs, and what it does not describe is locals and types. lldb and gdb both ship DAP interfaces already.

This is where no object headers pays off a second time. Flan structs are C structs — real machine types, no tag words, no boxing — so DWARF describes them with no impedance mismatch and lldb shows a Cursor correctly with no plugin and no formatters.

Reload and DAP

Mostly automatic. lldb watches the loader rendezvous structure, so a dlopen'd generation is noticed and its DWARF read; and lldb keeps breakpoints as unresolved specs, re-resolving them against newly loaded modules, so a breakpoint on sand.flan:85 binds to the new function by itself.

The gotcha is our own never-unload rule. After N redefinitions there are N copies of a function, each with DWARF claiming sand.flan:85, so one breakpoint resolves to N locations and stops in stale generations. Needs glue, and it is small: give each generation a distinct source identity in its DWARF (sand.flan#7), and have the reload agent disable breakpoint locations in superseded modules on each reload. This is the one piece that cannot just be wired up and left alone.

Not available at any price: if execution is paused inside the old function, redefinition does not move it — register allocation belongs to the old compilation (see "What redefinition cannot do"). The current frame finishes in the old code; the next call enters the new one. Also, dape reads source from disk, so editing further after a generation was compiled drifts the highlighted line from what is executing.

Cut: the instrumentation-based step debugger

Previously planned as CIDER's model — macroexpansion wrapping subforms with breakpoints. Dropped. It is the most expensive piece of milestone 8 (instrumentation, an overlay protocol, &env, a bespoke Emacs client) and DAP+DWARF covers most of its use. What DAP cannot cover — pausing before the stack unwinds with restarts available — comes from handler-bind over nREPL.

Consequences: &env loses its urgency (its "decide now" status came from the step debugger requiring it — now an ordinary milestone-5 decision), and the shadow stack shrinks to serving only nREPL backtraces and restart enumeration.

Forcing the live-programming workflow into DAP would repeat the CIDER mistake in the other direction: DAP's model is stop/step/inspect, with no vocabulary for "invoke use-placeholder and resume". That stays on nREPL.

Pause on exception

Better than CIDER's, because handler-bind has not unwound the stack. Dev-mode global handler messages the editor and blocks with the full live stack and all restarts available. Fix the function, resume via restart.

Build sequence

Deliberately ordered so each step is runnable and the next one cannot start until the previous checker is stable. The failure mode this exists to prevent is building the whole live environment at once.

1. Freeze the model. spec-memory.md and spec-conditions.md — done before any code. Fixed arrays, non-owning slices, move-only Vec~/~Map, allocators, Ptr, explicit clone; the six restart cases. Done.

2. Run calc-me.flan. Reader, typed IR, checker, and a backend that can carry the program end to end. The exit criterion was once a measured interpreter throughput number; with no interpreter that criterion is gone and the narrow host ABI took its place on the critical path (see Compilation). Packages, structs, (Ptr T) + addr, byte slices, at~/~len, while, set on the fixed place list, cond, match, Option

   • some, i32~/~u8~/~f64, recursion, argv, stdout. No allocator, no Vec,

   no generics, no user macros, no FFI, no window. Headless, so the acceptance test is a table of expression/result pairs.

3. Emit LLVM IR and pass the same calc-me test AOT, on native and wasm32 in CI. Both targets, one test table, one narrow host ABI (argv, stdout, exit). This is where the second target gets proven — while there is almost nothing to port. The hand-written x86-64 code generator is not on this path: it arrived later, as a second route to the same behaviour rather than a milestone of its own, and is held to the LLVM backend's output byte for byte (see Compilation).
4. Run sand.flan. Fixed 2-D arrays, dotimes, defer, and typed FFI to raylib including keyword→enum coercion. Acceptance test twice: headless (N frames, hash the grid — runnable in CI on both targets) and interactive at 120 fps.
5. Generics and macro expansion, once the base checker is stable. defmacro, &env, hygiene. Until here, when~/~unless~/~until~/~cond~/~dotimes are special forms in the compiler.
6. Allocators, Vec~/~Map, Result~/~try~/~errdefer, then conditions and restarts against spec-conditions.md, with dedicated tests per numbered case.
7. Hot reload — indirection cells in dev builds, with signature generations and stale-caller warnings, plus the remaining compatibility limits written down and enforced: struct layout changes, live callbacks held by C, captured environments.
8. Debugger, nREPL, async — last, and 8 splits into transport (8a) and editor client (8b).

Milestones 1–4 are the project. Everything from 5 on is optional in the sense that a language that stops there is still usable; nothing before 5 is.

Ordering note: sand cannot be first even though it is the better demo, because it needs raylib FFI, keyword→enum coercion and a window before a single line of it runs. calc-me needs argv and stdout.

Runtime budget

Scoped to milestones 1–3 only. The earlier version of this table put the whole plan — conditions, explicit transfer lowering, macros, hot reload, debugger support — at 15–25k, which is not credible: each of those is architecture work that touches the frontend, the IR and the backend at once.

Piece	Lines	Milestone
Allocators	2–3k	2
Core data (fixed/slice/Vec/Map)	2–3k	2
Reader + frontend + checker	5–8k	2
LLVM IR lowering	3–5k	2
raylib FFI + host ABI ×2	1–2k	3
Subtotal, a language that runs sand	13–21k

Beyond that, estimated but not budgeted, because these are the parts that are architecture rather than volume:

Piece	Note
Macroexpander + &env	touches the reader and the checker
Generics / monomorphisation	touches the whole checker
Conditions + explicit transfer lowering	frontend and IR and backend
Hot reload cells + compatibility rules	changes the calling convention
Debugger instrumentation + nREPL	needs &env and source locations

Janet is 36k including a bytecode VM, and Janet has no static types, no monomorphisation, no restarts and no reload.

Open decisions

None of these block milestone 2. The milestone each one must be answered by is marked.

1. Host language: OCaml or Rust — settled: OCaml, and the compiler has been written in it since. See Host language for what the choice turned on.
2. Macro hygiene is settled for milestone 5: explicit gensym, deliberately non-hygienic expansion, no local macros until a concrete use case appears.
3. Borrow checking and escaping frame-arena values. Deferred; revisit after Vec/Map are useful in real programs. The first implementation deliberately follows Zig/Odin: slices and pointers have explicit lifetime and reallocation contracts, with dev generation checks. A later lightweight provenance pass may catch obvious escaping/stale-borrow mistakes, but must not impose Rust-style lifetime annotations, automatic promotion, or an ECS-shaped object model.
4. Threads in the language: yes or no, decided before the host ABI. (The dev runtime is multithreaded regardless — it needs a reload thread. Settled; see Compilation.) Milestone 3, before the host ABI.
5. Escaping closures. Deferred until a concrete use case. Settled in spec-memory.md: (Fn ...) is a bare pointer with no environment (callbacks, reload cells, FFI), and a non-escaping fn captures by value into a stack environment — which is what makes handler-bind handlers able to see enclosing locals, without which conditions are not worth building. Still open: a closure that is stored, returned, or pushed into a container. Which allocator owns its environment, and what happens when the frame arena resets? Do not design this speculatively; revisit it alongside the deferred provenance work only when it is needed.
6. Hot-reload compatibility rules. Milestone 7. A changed function signature creates a new version: new callers resolve it, old callers keep the old one, and the session warns at tracked stale caller sites. Still open: a changed struct layout with live values is rejected; a managed class layout has an explicit frame-boundary migration path, as described above. Still open: a function pointer already handed to C, a captured environment, and redefining a defvar. Each needs an answer of the form "rejected", "accepted with a migration", or "accepted and the old code keeps running".
7. Does the interpreter survive milestone 3, or is the compiled path the only backend? Settled: the compiled path is the only one, and there is no interpreter. See Compilation, and docs/BUILT.md's "Why there is no interpreter".
8. (Option a) settled: an ordinary stdlib union with Some~/~None; the compiler niche-optimises (Option (Ptr T)) to a nullable pointer. The CL-vs-Clojure truthiness question is moot under static typing.

Settled since the first draft (see the specs):

• Module system → the directory is the package, Odin's model. No ns form. See Modules.
• Zero values → ZII by default, with an opt-out. A declaration with no initialiser is all-bytes-zero; (zeroed) re-zeroes something later; uninit opts out for large buffers about to be overwritten. It is a memset, not a memcpy; zeroed globals live in BSS and cost nothing.
• Package declaration → optional, inferred from the directory name. See Modules.
• Keywords at typed call sites → yes. :space resolves at compile time against the parameter's enum type, rl/key-space names the same value, and a typo is a compile error checked against the enum's members. No runtime cost. Needs the FFI enum declared, so it lands with milestone 4.
• Dev redefinition latency → 16ms, measured: ~llc + ld -shared + dlopen, never the clang driver, dlopen off the game thread, cells published in a batch at a frame boundary. See Compilation.
• Dev backend → compiled, and only compiled. The interpreter that milestone 2 was going to be written against was never needed and does not exist: expression eval is a compiled shared object like everything else, and a macro is the case that made the absence load-bearing rather than merely tidy. See Compilation.
• set on places → a fixed list of assignable forms, not setf.
• Loop story → imperative while~/~until~/~dotimes with break~/~continue and return; loop~/~recur only if it later earns its place. It did: both are built. break and continue take a label for the loop that is not the innermost, and loop is an expression whose value is its body's, with recur rebinding every name at once and jumping rather than calling. sand.flan is ported.
• Generic parameters → inferred at call sites, no explicit instantiation. No type classes. An operator over a variable with no where clause asserting it is rejected at the definition; a where predicate is what admits it.

Unverified claims in this plan

• Scope is the biggest risk, not any single feature. A language, inference, monomorphisation, macros, an explicit-memory runtime, conditions/restarts, hot reload, a debugger, nREPL tooling, native and WASM — each is reasonable, the set is not one project. The build sequence above exists because of this; if something has to give, it gives from milestone 4 upward.
• Line-count estimates are extrapolations.
• LLVM → wasm32 with manual memory is validated by Odin shipping it; not yet validated for s-expression macros + conditions/restarts on top.