A statically typed Lisp for game development. Clojure's brackets, C's memory and value model, no garbage collector.
Flan compiles s-expressions to LLVM IR and then to a native binary. There are no object headers, so a Flan struct is exactly its C struct. There is no collector, so nothing runs between frames that you did not write. And a running program can be edited: a function recompiled in Emacs is installed into the live process at its next frame boundary, in about twenty milliseconds.
This page describes the compiler as it is, not as it is planned. Where
something is designed but not built, it says so and gives the message the compiler
prints for it. Every Flan program on this page is a file in web/examples/
with its output recorded beside it; sh web/examples/check.sh runs them all
and compares, and quotes.sh re-derives the blocks that are transcripts
rather than programs.
What Flan is
A minimal Lisp for games. In one line: Odin with a Lisp frontend and a live REPL.
Types are mandatory and inference makes them feel optional; memory is manual; the
frontend is OCaml, and the default backend writes LLVM IR as text and hands it to
clang. There is a second one, off by default, that emits x86-64 by hand —
see targets and builds.
What it is not:
- Not a Common Lisp and not a Clojure. No numeric tower, no CLOS,
no
format, no lazy seqs, no persistent collections, no JVM. - No immutable collection types. Structure sharing destroys clear ownership, and clear ownership is the only thing that removes the need for a collector. Value structs that copy on assignment replace them.
- No dynamic typing. A tag word on every value is exactly the header cost that dropping the GC was meant to avoid.
- No consoles, and no live-image development at SBCL's level.
Getting started
You need OCaml with dune, and a clang on PATH. Build the
compiler, then run something:
$ dune build
$ ./_build/default/bin/main.exe run calc-me.flan "1 + 2 * (3 - 0.5) / 2"
3.5
Call that binary flan. Its subcommands:
$ flan
usage: flan (read|parse|check|emit|shim) <file.flan>...
flan emit <file.flan> [--x86] [--dev] [--debug] [--no-bounds-checks]
flan import-c <header.h> [package.flan...] [clang flags...]
flan generate-c <package-dir>
flan build <file.flan> [-o out] [--no-bounds-checks] [--dev] [--debug] [--sanitize] [--x86] [--target=wasm32-wasi|web]
flan run <file.flan> [args...]
flan reload <program.flan> <forms.flan> [-o out.so] [--x86]
flan dev <program.flan> [-s socket] [--x86]
read, parse, check, emit and
shim each stop the pipeline one stage further along and print what it
produced. run builds to a temporary file and execs it.
flan emit --x86 prints the hand-written backend's assembly where plain
emit prints LLVM IR, and it is annotated: each run of bytes is
headed by the Flan form that produced it and its source position, and each function by a
map of its frame saying which displacement is which parameter, which is a named local,
and below which offset everything is a reusable temporary. Reading a listing of this
backend without that map is reading -0x20(%rbp) with no key. The comments
cost nothing in the object — the assembler discards them — but they are
emitted only by emit, never by a build, so the assembly a build hands to
clang is the same text it always was. spike/x86/dump.sh puts all
four lowerings of one function side by side, which is what C-c C-l shows in
the editor.
The smallest program:
(defn main [] ()
(println "hello from flan"))
The entry point is (defn main [args [string]] i32). The parameter is
optional — omitting args means the program ignores argv — and the return
type is not: () is unit, and a main that returns it exits 0.
Values and memory
There is no garbage collector and no hidden allocation. Every local is a stack slot; reading one is a load, assigning to one is a store, and a store of an aggregate is the copy. A struct is its C layout with nothing added.
Four rules carry most of the model:
- Zero is initialisation. A declaration with no initialiser is
all-bytes-zero, so a global array is BSS and costs nothing to start. A struct
literal that omits a field zeroes it.
(zeroed)re-zeroes something later — a memset, not an allocation. - Fixed arrays are values.
[n T]is inline storage and copies on assignment and on pass-by-value. - Slices are views.
[T]is ptr+len and owns nothing. Copying a slice copies the view, never the elements. - Pointers are visible.
(addr x)takes the address of any assignable place and gives(Ptr T). It does not extend anything's lifetime, and keeping one past its frame is your contract to honour — there is no borrow checker.
A value struct is shared mutably by passing its address down the call chain. No heap is involved.
Places — the forms set accepts — are a fixed list, not an extensible
setf:
(defstruct Enemy [hp i32 name string])
(defvar spawned i32)
(defconst room-size 4)
(defvar room [room-size i32])
;; `set` takes a fixed list of forms, not an extensible setf.
(defn main [] ()
(let [e (Enemy {.hp 10 .name "slime"})
p (addr e)]
(set spawned (+ spawned 1)) ; a local or a defvar
(set (.hp e) 7) ; a struct field
(set (.hp p) 8) ; through a (Ptr Enemy) — derefs one level
(set (at room 2) 5) ; a fixed array or slice element
(set (deref p) (Enemy {.hp 3 .name "wisp"})) ; a whole-object store
(print (.hp e)) (println "")
(println (.name e))
(print (at room 2)) (println "")
(print spawned) (println "")))
3
wisp
5
1
.field and at dereference exactly one pointer level, so
(set (.hp p) 8) above is legal when p is a
(Ptr Enemy). The whole-object store through p overwrote
e itself, so hp reads 3 and not 8.
Bounds are checked
(defconst xs [3 i32] [1 2 3])
;; (at xs 7) with a literal index does not reach the backend at all: check.ml
;; rejects it. This one goes through a local, so it is the runtime check that
;; catches it — the same message, and the program stops where it happened.
(defn main [] ()
(let [i 7]
(println "before")
(print (at xs i))
(println "unreachable")))
$ flan run bounds.flan
before
bounds.flan:9:19: index 7 is out of bounds for length 3
$ echo $?
134
at and slice emit a comparison and a branch to a cold
block that names the source location and stops. Checks are on by default and are not
tied to the optimisation level. --no-bounds-checks turns them off.
Measured cost on a
50-million-iteration dependency chain over a 1024-element array: 0.11–0.12s checked
against 0.12–0.13s unchecked.
So is arithmetic that has no answer
Three integer operations have no right result, and each of them used to be a bare
SIGFPE or an undefined value: a divide or remainder by zero, the one division
that overflows (INT64_MIN / -1, whose true quotient is one past the top of
the type), and a float-to-integer cast whose value does not fit. All three now signal
ArithError, the way a bad index signals BoundsError.
;; Three integer operations have no right answer. Each used to be a bare SIGFPE
;; or an undefined value; each signals ArithError now. The divisor goes through
;; a global so that constant folding cannot answer it before the backend does.
(defvar zero i32 0)
(defn main [] ()
(println "before")
(println (/ 10 zero))
(println "unreachable"))
$ flan run arith.flan
before
arith.flan:8:12: divide by zero: (/ 10 0)
$ echo $?
134
$ flan run cast.flan
cast.flan:7:17: this value does not fit the integer type it is cast to, which
holds [-2147483648 2147483647]
;; A float-to-integer cast whose value does not fit. The condition it violated
;; is reported as the range the destination type can hold, which is the same
;; shape BoundsError uses for a slice: the violated condition, written out.
(defvar big f64 1e30)
(defn main [] ()
(println (i32 big)))
A Lisp that stops naming the file and the line beats one that dies with
SIGFPE, and a program that genuinely does not care installs a handler once at
startup and never thinks about it again. Float division is deliberately left alone: IEEE
already answers it, with an infinity or a NaN.
No restart is established at the failing operation, which is the same
decision BoundsError made and for the same reason. A restart frame is
allocated by the restart-case that offers it, on that frame's own stack, so
nothing below the program can push one on its behalf; a use-value at a
division would mean an alloca and a push-and-pop emitted at every division in
every checked build, and what it would buy is a silently different answer. What answers a
division by zero is the restart the program already had — a frame loop's
continue — which is on the stack and reachable from a handler or from the
break loop without anything being pushed at the failure.
Types
Types are annotated at function boundaries and inferred everywhere else. Every type notation reads as exactly one data item.
| Notation | Meaning | Layout |
|---|---|---|
i8 … i64, u8 … u64 | machine integers, wrapping arithmetic | the obvious one |
f32, f64 | floats | float, double |
bool | i1 | |
string | a byte slice with no NUL | ptr + len |
[T] | slice, non-owning | ptr + len |
[n T] | fixed array, a value | n inline items |
(Vec T) | growable, owning — moves on assignment | ptr + len + cap + its allocator |
(Map K V) | open addressing, owning — moves. The only map spelling: braces in type position are not a type | data + len + log2cap + its allocator |
(Pool T) | generational slab storage, owning — moves | items + slots + its allocator |
(Handle T) | a reference into a pool that reports a dead referent | index and generation packed into an i64 |
(Ptr T) | raw pointer | a pointer |
(Option T) | Some / None | tag byte + T |
(Fn [T ...] R) | a function value | a pointer |
Allocator | an opaque builtin: a proc, its data and a capability set | a pointer to that |
$t | a type variable — see generics | whatever it is instantiated at |
| a struct | value type | fields in declaration order |
| a tagged data type | defdata, matched by case | tag + the widest payload |
| an untagged union | defunion, C's: read any member, no tag | the widest member, at the strictest alignment |
| an enum | its own type in the checker | i32 |
() | one value, zero size | empty |
Never | fits anywhere; nothing has it | empty |
There is no implicit widening. Both operands of a binary operator have one type, and every conversion is written as a cast:
(defn main [] i32
(let [n 40 ; i32, inferred
big (i64 n) ; every widening is written
x 1.5] ; f64
(print (+ big 2)) (println "")
(print (* x 2.5)) (println "")
(print (bit-xor (<< 1 8) 255)) (println "")
0))
42
3.75
511
An untyped integer literal is i32 and an untyped float literal is
f64, so (defconst gravity f32 0.05) names the type when
something narrower is wanted. One caveat: a whole-numbered float prints without its
fraction, so 3.0 comes out as 3.
Arithmetic wraps. Shifts are bounded two ways: a literal count at or past the
operand's width is a compile error, and a computed one is masked to the width.
>> is arithmetic on a signed type and logical on an unsigned
one.
An index converts from a narrower integer and never from a wider one. A
u32 index is fine — anything above 231 truncates to a negative
i32 and the unsigned bounds check rejects it. An i64 index is
refused:
an index is an i32, and i64 is wider — write (i32 …), because a value that does
not fit truncates to one that does and would read the wrong element without
tripping the bounds check
Structs and enums
A defstruct is a list of inline name/type pairs. A struct literal names
its fields, and omitted fields are zeroed.
(defstruct Cursor
[src [u8] ; a non-owning slice
pos i32]) ; no initialiser means zeroed
(defn peek [c (Ptr Cursor)] u8
(if (< (.pos c) (len (.src c)))
(at (.src c) (.pos c))
0))
(defn advance [c (Ptr Cursor)] ()
(set (.pos c) (+ (.pos c) 1))) ; field access derefs one level
(defn main [] ()
(let [c (Cursor {.src (bytes "hi")})] ; pos omitted, so pos is 0
(print (peek (addr c))) (println "")
(advance (addr c))
(print (peek (addr c))) (println "")))
104
105
Those are the bytes h and i. There is no character type — a
byte is a u8 — but there is a byte literal, so \h is 104 and
\space is 32, and the prelude's digit? reads as
(and (>= b \0) (<= b \9)). To see a byte as a letter rather than as a
number, print a slice of them: print writes a [u8] as
its bytes.
An enum is an i32 at run time and its own type in the checker. A
keyword at a call site resolves against the parameter's enum type at compile time, so a
typo is an error there rather than a wrong number later.
(defenum Key
[space 32 escape 256 left 263 right 262])
(defn key-name [k Key] string
(cond
(= k :space) "space"
(= k :escape) "escape"
:else "an arrow"))
(defn main [] ()
;; :space resolves against the parameter's enum at compile time.
;; A typo is an error here, not a wrong number later.
(println (key-name :space))
(println (key-name :left)))
space
an arrow
A keyword means nothing where no enum is expected. There is no keyword type to fall back on, and there is no way to name a member other than as a keyword in a position that expects that enum.
Struct and enum names share one top-level namespace with functions, globals and type aliases. A second declaration of a name is rejected whatever kind either one is.
Functions
(defn name [param Type ...] ReturnType body ...). The parameters are
inline name/type pairs, as in let and defstruct. The return
type is always written, and a function that returns nothing writes (),
which is unit. There is no separate declare form for a function with a
body — declare is kept only where there is none.
The slot used to be optional, and the parser decided return-type-versus-body by
looking the name up in a table of the file's types. It was sound only because one
top-level namespace means a name cannot be both a type and a value, and it was
silently wrong twice — once reading (Rune {.code 65}) at the head of a
body as the function's return type. Writing the type removes the guess, and a
mistyped one now says did you mean f64 rather than unknown name.
Top-level names are order-independent within a package, so mutually recursive functions need no forward declaration. Globals come in two kinds:
(defconst cell-size 5) ; a compile-time constant
(defconst gravity f32 0.05) ; with its type named
(defvar current-color i32) ; zeroed storage
(defconst rows 3)
(defconst cols 4)
(defvar grid [rows [cols u32]]) ; BSS, rows*cols*4 bytes
(defn main [] ()
(print cell-size) (println "")
(print gravity) (println "")
(print current-color) (println "")
(print (at grid 2 3)) (println ""))
5
0.05
0
0
A defconst the checker consumed — an array length, for instance — is
part of the shape of the program. One it did not is only ever bytes in memory. The
two reload differently; see the dev loop.
let binds name/value pairs and takes no type annotation, so a constant
whose type matters is named at the top level rather than written inline.
Control flow
if, when, unless, cond,
do, and, or, not,
while, until, dotimes, loop/recur,
break, continue, return,
match. and and or short-circuit.
:else is cond's catch-all.
(defconst nums [5 i32] [1 3 8 9 10])
(defn classify [n i32] string
(cond
(< n 0) "negative"
(= n 0) "zero"
:else "positive"))
(defn countdown [n i32] ()
(let [i n]
(while (> i 0)
(print i)
(print " ")
(set i (- i 1)))
(println "")))
(defn first-even [s [i32]] (Option i32)
(dotimes [i (len s)]
(when (= 0 (% (at s i) 2))
(return (Some (at s i)))))
None)
(defn main [] ()
(println (classify -3))
(countdown 4)
(unless false
(println "unless runs when the test is false"))
(match (first-even (slice nums 0 (len nums)))
(Some n) (do (print n) (println ""))
None (println "none")))
negative
4 3 2 1
unless runs when the test is false
8
dotimes evaluates its bound once into a hidden slot before the loop, so
a body that changes it cannot change the trip count, and the loop variable is not
assignable.
break and continue leave or restart the innermost loop, and
take a label when that is not the one meant. loop and
recur are the functional shape beside them: loop is an
expression, its value is the value of its body, and recur rebinds
every name at once and jumps rather than calls — ten million iterations do not grow the
stack.
(defn gcd [a i32 b i32] i32
(loop [x a y b] ; recur rebinds every name at once
(if (= y 0) x (recur y (% x y)))))
(defn main [] ()
(println (gcd 84 36))
;; loop is an expression: its value is the value of the body.
(println (loop [i 0 acc 0]
(if (= i 5) acc (recur (+ i 1) (+ acc i)))))
;; continue in a dotimes advances the counter on the skipped iteration too.
(let [sum 0]
(dotimes [k 5]
(when (= k 2) (continue))
(set sum (+ sum k)))
(println sum))
;; A label says which loop. Unlabelled, break leaves the innermost.
(dotimes :outer [a 3]
(dotimes [b 3]
(when (= b 2) (break :outer))
(println b))))
12
10
8
0
1
return is still the way out of a function from inside a loop,
as first-even does above.
Option, match and some
(Option T) is how absence is spelled: a lookup miss, an empty
collection, the end of a stream. match works on an Option and
on a defdata, and on nothing else. some unwraps
Some and early-returns None from the enclosing function.
(defconst nums [4 i32] [4 8 15 16])
;; `some` unwraps Some and early-returns None from *this* function.
(defn doubled-first [s [i32]] (Option i32)
(Some (* 2 (some (index-of s 15)))))
(defn main [] ()
(match (doubled-first (slice nums 0 4))
(Some i) (do (print i) (println "")) ; 4
None (println "not found"))
(match (index-of (slice nums 0 4) 99)
(Some i) (do (print i) (println ""))
None (println "not found")))
4
not found
defer
A defer runs at function exit, innermost first. An explicit
return runs the ones registered above it — a defer written below a return
has not executed yet and must not fire.
(defn work [n i32] i32
(defer (println "second"))
(defer (println "first")) ; innermost-first at exit
(when (< n 0)
(return 0)) ; runs both defers above it
(println "body")
n)
(defn main [] ()
(print (work 3))
(println ""))
body
first
second
3
defer is function-scoped: it is copied into every exit path of the
function, so it always registers and always runs at function exit. It is therefore
rejected inside a loop or a branch, where "always registers" would be a
lie. A let is fine, and that is not an exception — a let is not
a frame here, its bindings are function slots like any other and nothing is released at
scope exit, so a defer written in one has exactly the function's extent.
That is the shape the permission exists for: acquire, defer the release beside it, then
use it.
defer is not allowed inside a branch — a defer is copied into every exit path of
the function, so it always registers and always runs at function exit. Write it at
the top level of the function body, or in a let that is (a let has the function's
extent, because nothing is released at scope exit)
Arrays and slices
at indexes a fixed array or a slice, and takes any number of
indices, so (at grid r c) indexes a two-dimensional fixed array directly.
It is a place: (set (at grid r c) v) and (addr (at grid r c)) both work.
len works on a fixed array, a slice or a string.
(slice s lo hi) takes a half-open range and never copies.
(defconst rows 3)
(defconst cols 4)
;; A fixed array is a value: inline storage, copies on assignment.
(defconst palette [4 u32] [0xE6B800FF 0x3B6E8CFF 0xA83232FF 0xCC6B1FFF])
;; No initialiser means all-bytes-zero, so this is BSS and costs nothing.
(defvar grid [rows [cols i32]])
(defn main [] ()
(set (at grid 1 2) 7)
(print (at grid 1 2)) (println "") ; 7
(print (len palette)) (println "") ; 4
;; A slice is ptr+len and non-owning: it views the array, it does not copy it.
(let [row (slice (at grid 1) 0 cols)]
(set (at row 0) 5)
(print (at grid 1 0)) (println "") ; 5 — the same storage
(print (sum-i32 row)) (println "")) ; 12
;; (zeroed) is a memset, not an allocation.
(set grid (zeroed))
(print (at grid 1 2)) (println "")) ; 0
7
4
5
12
0
A reversed range — lo greater than hi — traps, rather than
yielding a huge unsigned length.
Generics
Parametric polymorphism by monomorphisation: one body is written, and every call site gets a copy compiled at the types it passed. There are no type classes, no dictionaries and nothing decided at run time.
A type variable is written $t where a type goes — a
parameter's type, the return type, or nested inside a type constructor such as
[$t] or (Vec $t). The sigil is not a binding-site-only
spelling: every type position writes it. Bare t is the same
variable in expression position, where a builtin takes the name of a
type as an argument — (vec-new t), (map-new t i32),
(pool-new t), and the cast (t x).
(defn ident [x $t] $t x) ; needs nothing declared
(defn twice [x $t] $t ; + - * / % need numeric?
{:where (numeric? $t)}
(+ x x))
(defn clamp-to [x $t lo $t hi $t] $t ; < <= > >= min max need ordered?
{:where (ordered? $t)}
(min (max x lo) hi))
(defn first-or [s [$t] d $t] $t ; the variable inside a slice type
{:where (copyable? $t)}
(if (= (len s) 0) d (at s 0)))
(defn one-of [x $t] (Vec $t) ; bare t is the type-name argument
{:where (copyable? $t)}
(let [v (vec-new t)]
(push v x)
v))
(defn main [] ()
(println (ident 3))
(println (ident "text"))
(println (twice 1.5))
(println (clamp-to 12 0 10))
(let [ns [5 3 9 1]
one (one-of 4.5)]
(println (first-or (slice ns 0 4) -1))
(println (at (as-slice one) 0))
(free one)))
3
text
3
10
5
4.5
The body is checked once, abstractly
A generic body is checked with nothing substituted, so an operator the variable is not declared to support is refused at the definition rather than at whichever call site first reaches a type that happens to work. That is deliberately not Odin's rule, which checks a polymorphic body per instantiation:
+ over the type variable t is refused: a type variable supports only what it is
declared to support, and nothing here says t is numeric?. Write {:where (numeric?
$t)} at the head of the body, or take the operation as a parameter — a
(Fn [t t] ...) — and call it here
What makes that liveable is a where clause, written as a Clojure-style
map at the head of the body — {:where (ordered? $t)}, or a vector when
there is more than one: {:where [(copyable? $t) (copyable? $u)]}. There
are five predicates, and each gates builtins the compiler already has:
| Predicate | What it admits |
|---|---|
numeric? | + - * / %, and a cast (t x) |
ordered? | < <= > >= min max |
equal? | = and != |
hashable? | the variable as a Map key — (map-new t V), get, put, has-key? |
copyable? | reading the value more than once; Pool and Vec element positions |
They entail each other in one direction, so one clause usually does:
numeric? gives ordered?, ordered? gives
equal?, and any of the four gives copyable?. A
sort! that compares its elements and reads them twice declares
ordered? and nothing else.
A type variable is move-only by default, and
copyable? is the opt-out. Types.is_move_only of a variable is
not decidable abstractly — the same variable is i32 at one instantiation
and (Vec i32) at the next — so the checker assumes the stricter rule,
which can only refuse a program that would have been fine and never admit one that
double-frees. It is Rust's T: Copy, with the difference that the compiler
answers the question rather than a user implementing a trait. So
(defn twice [x $t] $t (+ x x)) does not merely want
numeric?; reading x a second time is a use after move:
x was moved at twice.flan:1:26 and cannot be used again — t is move-only, so
binding, passing or returning one transfers ownership and the source binding is
dead afterwards (spec-memory.md). That rule is what makes a double free
unrepresentable; (clone x) if you wanted a second one
Each instantiation then checks the concrete type against what the signature declared, and refuses the call site when it does not answer:
this call instantiates twice at $t = bool, and bool does not answer numeric? —
which twice requires, being written {:where (numeric? $t)}. The requirement is the
signature's, so the refusal is here, at the call that asked for the type: pass one
the predicate admits
Two forms are deferred to the instantiation rather than settled abstractly, because
their legality is only decidable after substituting: println over a
variable, which selects the structural printer per copy, and the Map
operations over a variable key, whose hash and equality are concrete symbols chosen from
the concrete key type. The Map half is what hashable? buys —
without the clause, the type (Map $t i32) is refused where it is
written, and with it the refusal moves to the call site that names an
unhashable key.
This is not a type class and the difference is worth keeping straight. A type class
carries implementations, selected per instance and extensible by anyone, and
needs dictionaries and coherence rules. A predicate carries nothing — it gates a builtin
that already exists. The ceiling is that nobody can supply a < of their
own; every operation the prelude and the containers need is a primitive, so it does not
bind. test/programs/generics.flan exercises the whole of it.
Printing
println prints a value and a newline; print is the same
walk without the newline. There is one of each and they take any type, but neither is a
function and neither is overloading: the compiler walks the argument's type where the
call is written and emits the printing for it. Nothing is decided at run time — a Flan
value carries no header, so nothing at run time could say what it is — and there is no
user-supplied printer to choose between.
(defenum Key [space 32 left 263])
(defstruct Enemy [hp i32 name string key Key])
(defn look-up [k Key] (Option i32)
(if (= k :space) (Some 32) None))
(defn main [] ()
(println 42) ; an i32, uncast
(println 1.5)
(println (Enemy {.hp 3 .name "wisp" .key :left}))
(println (look-up :space))
(println (look-up :left))
(print "no newline: ") (println true))
42
1.5
(Enemy {.hp 3 .name "wisp" .key :left})
(some 32)
none
no newline: true
The walk covers every integer and float type, bool, (),
string, [u8], enums, Ptr, Option,
structs, unions, fixed arrays and slices. An owning container has no printer for its
contents and comes back as a marker instead — <vec>,
<pool>, <allocator> — while a Handle
shows its index and generation, and a Map has no printer at all.
An enum member comes back as its name: the value is
an i32 by the time the backend sees it, so the name is recovered here from
the checker's table, and a value outside the declared members falls through to the
number, which is what you would want to see. A Ptr prints as
<ptr> and is never followed — it is the one thing that could make
the walk cycle, and dereferencing a pointer on someone else's behalf is not safe.
A string prints raw at the top level and quoted-and-escaped inside a structure.
Those are not in conflict: (println "hello") has to print
hello or it is useless, and the name field above has to be
quoted or it could not be told from the punctuation around it.
The walk is bounded in depth and in span, so a deeply nested value or a
[100 [100 u32]] grid prints ... rather than a screenful — and,
since the walk is unrolled at compile time, rather than putting ten thousand printing
sites in the module.
That the walk takes the argument's own type matters more here than it would in a
language that widens implicitly. Nothing widens implicitly in this one, so a printer
that named a type would need a cast written at every call — and a u64
above 263 put through a signed one comes out negative. print
takes the value as it is and prints the number it holds.
The prelude
The prelude is written in Flan, all but its declare lines, and prepended to every
program, so nothing in it needs importing. It holds no printing of its own:
print and println are the compiler's, and
write-stdout — the one output primitive — is what they are written
over.
| Group | Names |
|---|---|
| slice algorithms, over one type variable | swap!, reverse!, sort!, sort-by!, index-of, min-of, max-of, map!, reduce, filter |
| the per-type layer that stays | sum-i32, sum-f32 — the element and the accumulator are different types, which one variable cannot say |
| bytes | bytes=?, bytes<?, bytes-ci=?, starts-with?, ends-with?, index-of-bytes, trim, digit?, space?, sort-bytes! |
| parsing | parse-i64, parse-f64 |
| text | split-on-byte, split-next!, split, lower-ascii, upper-ascii, to-lower, to-upper |
| building bytes | append!, append-i64!, append-f64!, concat, join, repeat-bytes, replace-bytes, slices-new, format-f64 |
| UTF-8 | decode-rune, rune-at, rune-count, rune-size, rune-start?, valid-utf8?, encode-rune! |
| numbers | sign-f32, lerp, clamp, floor-f32, ceil-f32, round-f32, abs-i32, abs-i64, the constants pi-f32, pi-f64, tau-f32, tau-f64, and libm through a declare at both widths: sqrt, abs, floor, ceil, round, fmod, sin, cos, tan, asin, acos, atan, atan2, log, log2, log10, exp, pow, hypot, cbrt — each spelled -f32 or -f64 |
| time | monotonic-ns, monotonic-seconds, unix-ns, unix-seconds, sleep-ns, sleep-seconds, and ns-per-second and its two smaller siblings |
| files | file-exists? and file-size, which answer a value; slurp, barf, delete-file, rename-file and make-directory, which signal FileError under retry and use-value |
| the operating system | getenv, which answers an (Option [u8]) viewing the process environment |
| random | rand-seed, rand-u32, rand-f32, rand-i32-range, rand-f32-range |
| forms, for macros | form-nil, form-cons, form-append, form-rest, form-items, form-pair, form-sym?, form-is-sym?, gensym, and unless and into, which are macros written here rather than special forms |
| the rest | pause, which signals the Pause condition the break loop stops on, and embed-find |
One family, not one per type. The slice algorithms used to be
sort-i32! beside sort-f32! beside sort-bytes!, and
generics collapsed them: sort! is written once and instantiated at whatever
element type the call passes. sum-i32 and sum-f32 are what did
not collapse, and they are the honest exception — each widens its element into a
different accumulator, which one variable cannot express.
The RNG is ours, not libc's — PCG-XSH-RR 32, written in Flan —
because a grid hash is only a regression test if the sequence is byte-identical on
native and on wasm32. The parsers are ours too:
strtoll answers 0 for "", 0 for
"abc" and 12 for "12x", which are three wrong answers a caller
cannot tell from a real 12.
The maths in the file is not Flan, and it is libm's:
(declare sqrt-f32 [x f32] f32 "sqrtf") and the same line for thirty-odd
more. Every other number here is reachable from the four operations and a cast; a square
root is not, and the usual trick of seeding Newton's method from the exponent bits needs
a bit-cast between f32 and u32 that the language does not have.
A declare is also the cheapest thing in the language to add — a line, a
symbol already on the link, and nothing in either backend — which is why the surface is
now the whole family at both widths rather than the five it started as.
One split is worth knowing before calling any of them. IEEE-754
specifies sqrt, fabs, floor, ceil,
round and fmod as exact or correctly rounded, so those give the
same bit pattern under glibc, musl and wasi-libc. It requires nothing of the
rest — sin, cos, tan, the inverses, the
logarithms, exp, pow, hypot, cbrt —
and the three libms do differ in the last bit, so the byte-identical-hash property the
RNG exists for does not survive a value routed through any of them. Every link carries
-lm.
The clock is two clocks and they are not interchangeable.
monotonic-ns measures: it never goes backwards, nothing adjusts it, and its
zero is the first time the program reads it, so it is meaningless alone and correct as a
difference. unix-ns dates: nanoseconds since 1970, which is what goes in a
save file or a log line, and which can jump in either direction when the system clock is
set. Odin draws the same line — Tick against Time in
core/time — and the nanosecond integer is the primitive on both sides, with
the -seconds faces derived from it. The f64 of seconds is the
shape raylib's get-time already answers with, so the two mix; it stays
integer-exact in nanoseconds for a hundred days of process life, which is why the
monotonic origin is the first read and not boot.
The file surface is split by whether a handler could do anything.
file-exists? and file-size answer a bool and an
(Option i64): absence is the reply, not a fault, and a condition would make
the ordinary case pay for a handler search. slurp, barf,
delete-file, rename-file and make-directory signal
FileError instead, under the two restarts Common Lisp establishes for a
file error — retry, because the handler may have just made the directory,
and use-value with another path. Nothing here returns an error code, which
is the same rule allocation follows. Streaming, stdin and directory listings are not
here; a whole file at a time is the surface.
The primitives underneath are few — a primitive is the only thing implemented
twice per backend: argv,
write-stdout, exit, len, at,
slice, bytes, bytes->f64,
bytes->i64, f64->bytes, i64->bytes,
addr, and arithmetic, comparison and casts.
Packages
The directory is the package. Every file in a directory shares one top-level scope; files within a package do not import each other, and their order does not matter. The package declaration is optional and the name is inferred from the directory, so a loose file in a scratch directory is a package of one with no manifest and no ceremony.
;; geom/vec.flan — no package declaration: the name comes from the directory.
(defstruct V2 [x f32 y f32])
(defn add [a V2 b V2] V2
(V2 {.x (+ (.x a) (.x b)) .y (+ (.y a) (.y b))}))
;; geom/len.flan — a second file in the same directory shares one top-level
;; scope: it does not import vec.flan, and the order of the two does not matter.
(defn length [v V2] f32
(sqrt-f32 (+ (* (.x v) (.x v)) (* (.y v) (.y v)))))
;; pkg.flan — the directory is the package, and everything it declares
;; arrives qualified by the alias this import chose.
(import g "geom")
(defn main [] ()
(let [v (g/add (g/V2 {.x 3.0 .y 0.0})
(g/V2 {.x 0.0 .y 4.0}))]
(print (g/length v))
(println "")))
5
There is one form and one meaning: (import alias "path"), and everything
from the package is qualified alias/name. There is no unqualified-import
mode, no :refer, no ns form and no per-file namespace object.
A path with no collection prefix is relative to the importing file;
vendor: and core: are collections, resolved by walking up from
the importing file until a directory of that name is found.
Importing is a rename. Every top-level name the package declares becomes
alias/name, and every use of one — in a type, in a body, in a struct
literal, in an array length — is rewritten to match. Nothing downstream knows a package
existed.
Three more rules:
- A package may be a single
.flanfile named outright, for the program that is also a library.sand.flanshares a directory with three other loose programs, so naming its directory would import all four. - A package may import a package, and the qualification flattens to
the inner alias: raylib imported by a package that is itself imported is still
rl/…. A directory is keyed by its real path and read once, so a cycle ends there. The same directory under two different aliases is refused. mainis not exported. A package carrying one would collide with the importer's, andmainis a reachability root, so an imported one would keep everything it calls alive. Writingsand/mainis refused at the line that wrote it.
Visibility is that one rule and no more: there is no package-private marker for
anything other than main yet.
A package may carry the C it binds to. Every .c file in the directory is
compiled into the build, and a file named link lists extra linker
arguments. Whether those reach the build is decided after checking, from the program
rather than from the import list: the compiler starts at main, follows every
call, and a package none of whose externs survive contributes no C and no linker
argument, so a file that imports raylib still builds for wasm32.
Conditions and restarts
A condition is a struct. There is no class hierarchy; matching is by type. The signalling end says here is something notable, here is the data, and an outer caller decides what to do about it — or decides nothing, in which case the signaller carries on.
(signal c) ; (). Handler returns -> carry on. No handler -> no-op.
(error c) ; Never. Only a transfer gets past; else the program stops.
(handler-bind [(Type [c] body ...) ...] body ...) ; match by type, no hierarchy
(restart-case BODY ; BODY and every clause have the same type = the form's
(name [p T ...] CLAUSE) ...)
(invoke-restart 'name arg ...) ; Never. Innermost frame offering the name wins.
signal has type (), always. A handler that returns
normally leaves the signaller to carry on — the accumulation case:
(defstruct AssetMissing [id i32])
(defvar seen i64)
(defn load-all [] ()
(signal (AssetMissing {.id 1})) ; () — the caller carries on
(signal (AssetMissing {.id 2})))
(defn main [] ()
(load-all) ; no handler: a no-op
(print seen) (println "") ; 0
;; A handler that returns normally accumulates and lets the signaller run on.
(handler-bind [(AssetMissing [c] (set seen (+ seen (i64 (.id c)))))]
(load-all))
(print seen) (println "")) ; 3
0
3
A handler that invokes a restart instead transfers control outward to the
restart-case that offers the name, and the clause's value becomes that
form's value. Every defer between the invoke and the target runs, innermost
first, before the clause body starts.
(defstruct AssetMissing [id i32])
(defvar cleanups i64)
(defn load [n i32] i32
(signal (AssetMissing {.id n}))
100)
(defn middle [n i32] i32
(defer (set cleanups (+ cleanups 1))) ; runs on the transfer too
(+ (load n) 1))
(defn fetch [n i32] i32
(restart-case (middle n) ; its value if nothing transfers
(use-placeholder [] -1)
(retry [] 7)))
(defn main [] ()
(print (fetch 1)) (println "") ; 101 — nothing handled it
(handler-bind [(AssetMissing [c] (invoke-restart 'use-placeholder))]
(print (fetch 2)) (println "")) ; -1
(print cleanups) (println "")) ; 2 — the defer ran both times
101
-1
2
A clause may take parameters, which is how the answer comes from outside the frame that offers the restart:
(defn supplied [n i32] i32
(restart-case (middle n)
(use-value [v i32] (* v 2)) ; the handler supplies v
(retry [] 7)))
(defn main [] ()
(handler-bind [(AssetMissing [c] (invoke-restart 'use-value 21))]
(println (supplied 7)))) ; 42
42
Restart lookup walks the dynamic restart stack from innermost outward and takes the
first frame offering the name, so an inner restart-case shadows an outer
one for the duration of its body. An inner parser's skip-form is found
before an outer one's.
How a transfer is lowered
A transfer is not platform unwinding. Every Flan signature carries a transfer channel
— one pointer appended as an out-parameter — which invoke-restart writes
and every call site checks. A callee writes the target into its caller's slot; each
frame checks, runs its defers and returns early. The disassembly is the release one plus
a guard after each call.
Three consequences:
- wasm32 works with no exception proposal, and native and wasm builds of the same program agree.
- Every function carries the channel, release builds included. A hot-reload cell holds a bare pointer, so the honest answer to "what can this call?" is "anything". A later optimisation may stop a function checking the channel; it may not drop the parameter.
- A transfer cannot cross a foreign frame. A handler installed across an FFI boundary must return normally.
Gotchas
- A handler closes over nothing. A clause is lifted into a function of its own, because it runs from wherever the signal was. Accumulate into a global, or put the value on the condition. A reference to an enclosing local is refused for that reason rather than reported as an unknown name.
- A restart is not a transaction. Control resumes at the
restart-caseand runs forward from there, so aretryrepeats every side effect between it and the target. Nothing rolls back — a global the frame already set stays set, and is set again. Common Lisp has exactly this property and offers no help either.So the author chooses where the retry boundary is. A
restart-caseat the top of a frame re-runs everything including mutations already applied; one placed after the mutations re-runs only what follows. Put the restart before anything mutates, make the retried section idempotent, or snapshot what will be re-applied.test/programs/frame-rollback.flanis the worked example of the snapshot, and the ordering in it is the part worth reading: restore in the restart clause and not in adefer, because a defer runs on the ordinary return path too and that version silently rolls back the frames that succeeded.This matters more here than in most Lisps because the intended use is a game loop, where the plan is to skip a frame and carry on rather than die. Now that a bad index signals
BoundsErrorand a bad division signalsArithErrorinstead of ending the process, abandoning a frame and retrying it is a real thing to do — and that is exactly the case a non-idempotent mutation spoils. - An unknown restart name is a hard stop — a located runtime error.
There is no
find-restartto test with yet. - No supertype, so nothing can say "any condition".
signalcannot hand a value back. Deliberate: the alternative forces every signal site to declare a default value and a result type.returnis refused inside ahandler-bindorrestart-casebody, and so isinvoke-restartinside adefer. In each case a bare exit would leave frames on the stack pointing into a function that has gone.- A restart's parameters are checked at run time, count then
spelling, because a restart is found by name on a dynamic stack and neither end can see
the other. Lookup is by the name alone and the signature is checked after it, so an
inner
(use-value [s string] …)shadows an outer(use-value [v i32] …)and(invoke-restart 'use-value 21)stops the program even though the outer clause would have taken it. The clause's parameters are slots of the function that wrote it, and the invoker fills a buffer that function owns — by the time a clause runs, the invoking frame has gone.
The break loop
An error nothing handles does not kill the program. It stops on the
frame that erred, with nothing unwound, so the condition and every restart between there
and the top are still live:
flan: unhandled Missing — stopped, not dead.
0. restart: retry
1. restart: use-placeholder
From there you fix the function, install it, and take a restart. The numbers are
how one is taken: a restart is chosen by position, because an inner one may shadow an
outer one of the same name and a name alone could not tell you which you were getting.
Control never left the erring frame, so retry calls through the
indirection cell and reaches the new body. Installing while stopped is allowed; there is no frame in progress.
The break loop lives in vendor/agent, which is an optional package. A
program that does not import it leaves the hook null and stops the old way — the message
and an exit status of 134:
(defstruct Missing [id i32])
(defn load [n i32] i32
(restart-case
(do (error (Missing {.id n})) ; Never — only a transfer gets past
0)
(use-placeholder [] -1)
(retry [] 7)))
(defn main [] ()
(print (load 1))
(println ""))
$ flan run boom.flan
unhandled Missing
$ echo $?
134
The FFI
There are two declaration forms, and they are two forms rather than one because no structural rule could tell them apart.
declare names a C symbol in a signature Flan can already spell. Nothing
is generated; a Flan string crosses as ptr+len, exactly as it is stored.
(declare cosh-f64 [x f64] f64 "cosh")
(defn main [] ()
(print (cosh-f64 0.0)) (println ""))
1
That is 1.0, printed by the same rule as before. It is cosh and not
cos because the prelude already declares
cos-f64, and a second declaration of a name is refused with both sites
named — which is the other half of what this example shows. The prelude is where the
common libm calls live; a declare is how you reach one it does not
name.
declare-c names the C library's own function in the C library's own
signature, and the compiler writes the wrapper. This is what raylib's package is made
of — one line per binding:
(declare-c unload-texture [texture Texture2D] "UnloadTexture")
(declare-c draw-texture
[texture Texture2D x i32 y i32 tint Color]
"DrawTexture")
An aggregate's calling convention is not part of its layout. On
x86-64, clang gives raylib's own prototypes
<2 x float> for a returned Vector2, i32 for
a Color argument, and { i64, i64 } for a returned
Rectangle — none of which is the struct's own LLVM type, and arm64 and
wasm32 classify differently again. Reproducing that in the backend would be three
classifiers to write and keep correct forever, and a mistake would show up as a field
full of garbage rather than as a link error.
So the boundary has one wrapper per binding, each flattening the aggregates: a struct
returns through an out-pointer, a struct argument is passed by pointer, and clang
classifies all of it, per target, for free. flan shim <file> prints
the whole generated file, of which this is the end — the rest is the typedefs
and a comment saying not to edit it:
(defstruct Vector2 [x f32 y f32])
(declare-c get-mouse-position [] Vector2 "GetMousePosition")
typedef struct flan_ty_Vector2_1bebc5ae_s flan_ty_Vector2_1bebc5ae;
struct flan_ty_Vector2_1bebc5ae_s { /* Vector2 */
float x;
float y;
};
/* get-mouse-position */
extern flan_ty_Vector2_1bebc5ae GetMousePosition(void);
void flan_shim_get_mouse_position_5ad0e205(flan_ty_Vector2_1bebc5ae *out) {
*out = GetMousePosition();
}
A build needs the shared library to be linkable and not the
-devel package to be installed: every declaration a package uses is written
down in the package itself, so nothing has to go looking for a system header.
Guaranteed: the C typedef
and the Flan struct come from the same defstruct, so they cannot disagree,
and clang type-checks the wrapper against the generated prototype.
Trusted: that the defstruct matches the library's real
struct, and that the declare-c signature is the function's real signature.
A scalar's width now carries ABI weight — f64 where the library says
float emits double, and the library reads garbage. That
trusted half is what the generator below checks.
Generated bindings, committed
Writing a binding per function by hand does not scale past the ones a program
happens to call, so most of raylib's package is not written by hand. A package
directory may carry a headers file naming the library's own C header;
flan generate-c <package-dir> reads it with
clang -Xclang -ast-dump=json, turns every function it can represent into
the same declare-c line a person would have written, and writes them to
generated.flan in the package — which is committed.
$ flan generate-c vendor/raylib
wrote vendor/raylib/generated.flan: 268 declarations, 117 refused, of 581 functions
in vendor/raylib/raylib-5.5.h.
Every defstruct, every hand-written declare-c and every mapped
constant agrees with it.
Committing the output is what makes the declarations a dependency of the repository rather than of the machine: every build gets all of them, they are greppable, and they show up in a diff when the library moves. The argument is not caching — the clang dump is already cached on disk and in memory.
The header is committed too, at
vendor/raylib/raylib-5.5.h, and headers names it by path with
no environment variable in front of it. That line used to be
?${FLAN_RAYLIB_H} — optional, on the argument that requiring a header would
cost everyone the no--devel property in order to give the check to whoever
had one. Committing the header dissolved that argument, because nobody needs
raylib-devel to have a file that ships with the repository. So the check now
runs on every build, and delete the header and the build says so by
name rather than going quiet. What being optional actually cost was found the hard way:
a gitignored web directory meant several working trees were checking against nothing and
were not told, and a check that silently does not run is worse than no check.
Regeneration is the stronger check. An ordinary build re-reads every
declaration against the header, but only the hand-written ones can disagree — the
generated half came out of that header and agrees with it by construction. The one
function that writes the file therefore compares first and refuses to
write when the package and the header disagree: every defstruct
against the header's record, and every hand-written declare-c against the
header's signature. Pointed at a raylib 5.1-dev header while the package is written for
5.5, it reports ten real differences and writes nothing — which is exactly the silent
version skew a generated file would otherwise bake in and make look reviewed.
This is also why the hand-written bindings are kept rather than replaced by generated ones. Everything the generator emits agrees with the header by construction, so diffing generated output against the header it came from proves nothing; the hand-written lines were transcribed by a person, so they are the only declarations a header can actually contradict. All ten of those differences came from them.
A committed generated file cannot be hand-corrected — the next regeneration destroys
the edit without telling anybody — so the corrections live in a bindings
file beside headers, which is read while the declarations are made.
Two directives:
# raylib's own malloc/realloc/free, which would be a second untracked heap
# behind three innocuous Flan names.
exclude Mem*
# The kebab rule gives is-window-ready. Lisp spells a predicate with a ?.
name IsWindowReady window-ready?
An excluded function still says it was excluded rather than going quiet, and a name
override changes only the Flan face — the C symbol is kept verbatim in the declaration,
so it is still what is called and still what the check compares. Renaming is also the
way out of a collision: Spin2D and spin2d both kebab to
spin-2d, so neither takes the name, because which one won would otherwise
depend on the order the header happens to declare them in.
Anything neither directive can express is a hand-written declare-c in the
package's own source, which wins over the generated file and is left alone by the
generator. That is the escape hatch for a signature the importer gets wrong and for a
Flan face the header cannot describe — raylib keeps two, each a raw binding wrapped by a
Flan function of the same name, one taking a slice and one answering with an
Option.
Everything the boundary cannot represent is refused by name with the reason, rather
than half-supported: an Option, a union, a fixed array, a map, a returned
string, a callback, a slice parameter, an unknown type, an unrepresentable struct field,
and two Flan names for one C symbol. An aggregate in a plain declare is
refused too, so the narrow boundary cannot quietly acquire one.
One known edge: a REPL redefinition that introduces a new
declare-c cannot work, because the reload path compiles no C and the wrapper
would not exist in the running process. Editing the body of a function that calls an
existing binding is unaffected.
The dev loop
Edit the code, keep the sand.
$ flan dev sand.flan
That builds the program, launches it, holds a session beside it, and listens on
.flan-dev.sock next to the source. From Emacs, C-c C-c on a
function recompiles it and installs it into the running process at that process's next
frame boundary. The window does not blink and the grid does not reset.
How it works
Four pieces, each runnable on its own.
The reload primitive. llc → ld -shared →
dlopen → call. Measured in this codebase:
| Step | Cost |
|---|---|
| emitting the redefinition's IR | below the timer (<0.1ms) |
llc -O2 -filetype=obj | 15–17ms |
ld -shared | 3ms |
dlopen + dlsym | 0.04ms |
About 19ms end to end. The clang driver on the same IR is 50ms, so the
dev path never invokes it. The driver forks a second process and re-does argument and
target resolution; codegen is not the cost.
Indirection cells. Loading a new body is not installing it. A call
bound at link time cannot be made to notice one, so a --dev build routes
every Flan-to-Flan call through a cell — a mutable global holding the address of the
function that is current.
Here is the defer example from above — its
main calls work — through flan emit --dev:
@"flan.cell.work" = global ptr @"flan.work"
define {} @"flan.main"(ptr %xfer) {
entry:
;; the frame push, elided — see conditions, above
%t8 = alloca %slice
%t2 = load ptr, ptr @"flan.cell.work"
%t3 = call i32 %t2(i32 3, ptr %xfer)
The call site loads the cell rather than naming @"flan.work"
directly. The signature carries ptr %xfer — the transfer channel from
conditions, on every Flan function, release builds
included. println is not in there: it is not a Flan function, so there is
no call to route and no cell for it.
Redefinition is then one store, below a microsecond. Three rules follow. A
redefinition module declares every global external, so globals live in the host
and survive a reload. Every other function is a declare, so a redefined
settle calls the host's move-grain rather than freezing a
private copy of it. And nothing is ever dlclosed: a cell holds an address
inside a module's text, so unloading it would leave call sites pointing at unmapped
memory. Old code is never unloaded, so a thread mid-execution finishes safely in the old
version.
The agent. vendor/agent is a package like any other: a
listener thread, a single-producer ring, and three calls. This is the whole of
agent.flan — no aggregate crosses the boundary, so a plain
declare does it and there is no shim.
(declare start-raw [path string] i32 "flan_agent_start")
(declare poll-raw [] i32 "flan_agent_poll")
(declare wait-raw [ms i32] i32 "flan_agent_wait")
(defn start [path string] i32 (start-raw path))
(defn poll [] i32 (poll-raw))
(defn wait [ms i32] i32 (wait-raw ms))
(agent/start path) listens on a unix socket, once, at startup.
(agent/poll) installs whatever has arrived and returns how many.
(agent/wait ms) is the same but waits for something first. A headless test
uses it so that a reload is deterministic rather than a race against the frame rate.
Loading and installing are separate. dlopen relocates
a module and takes the loader lock — milliseconds, unbounded — so it happens on the
listener thread. Installing is one store per function and must not land while a redefined
function is on the stack, so it happens on the game thread, at the top of the frame, when
the program asks. A game loop calls agent/poll at the top of its frame and
ignores the result.
The session and the daemon. flan dev holds the
declarations the running process was built from plus every change accepted since, and it
owns the build, so its rules describe the process that is actually running. Re-checking
the whole program on every evaluation costs under 10ms, less than the llc
that follows. An evaluation is transactional: a form that fails to check mutates
nothing.
The protocol is one s-expression per message, length-framed by a decimal byte count.
It is not nREPL: eval there is string-in/string-out with no slot for
which form, from which file, and once the editor client is ours too there is no
CIDER to be compatible with.
What a running process cannot be told
Some changes are refused with a reason rather than loaded, because the alternative is a silent mismatch against memory the process has already laid out:
| Change | What it would have broken |
|---|---|
| a function's signature | a cell is a bare pointer; every call site compiled before the change still passes the old arguments through it |
| a global's type | the storage exists and has a shape — reuse reads at the wrong offsets, replacement discards the state the reload exists to preserve |
| a struct's fields | the values the process is holding have the old layout |
a defconst the checker consumed | it is in the shape of the program — (defconst rows (/ h c)) decides grid's type before anything else resolves |
a defenum member | :space is erased to an i32 literal in the caller, so it is folded there too |
A defvar's initial value is deliberately not on that list:
its storage holds state the program moved past long ago. And a defconst
the checker never consumed can be changed, so a colour table can be tuned live while an
array length stays refused. A dev build emits those as mutable globals, so LLVM cannot
fold a read of one.
The signature row is a stopgap. The design is versioned functions with their own trampolines, so that new callers resolve the new version while existing ones keep the old. None of the three parts exists yet, and the alternative to refusing is a silent argument mismatch.
Evaluating an expression
C-x C-e is a different primitive from redefining a name. There is no name to install a body into, so the expression is wrapped in a thunk with nowhere to be called from. The module says run this once, and the agent calls it after the install, on the game thread, at a frame boundary. An expression reading the program's state therefore sees a consistent one.
Nothing is marshalled back, because nothing could be: a Flan value carries no header, so no code at run time can say what it is. The compiler knows the type and renders it there, in the thunk. What comes back looks like this:
big 18446744073709551615
col :blue
(.pos b) (V {.x 1.5 .y 0})
b (Blob {.id 7 .name "sandy \"quoted\"" .pos (V {.x 1.5 .y 0}) .tags [ 0 42 0]})
(slice (.tags b) 0 3) [ 0 42 0]
(rl/get-color 0x11223344) (rl/Color {.r 17 .g 34 .b 51 .a 68})
sim/grid [ [ 0 0 0 0 0 0 0 0 ...] [ 0 ... ] ...]
A pointer is never followed; it renders as <ptr>. Following one
would make the walk cycle, and dereferencing a pointer a REPL was handed is not safe.
The walk is bounded at depth 4 and 8 elements, and the output truncates at 4K. An owning
container renders as a marker rather than its contents — <vec>,
<pool>, <allocator>, <handle 3:1>
— and a function value renders as its signature, because the inspector reaches every local
of a stopped frame and a frame holding one has to render rather than refuse. A
Map is what still refuses by name.
The thunk's module is unloaded afterwards. Nothing points into its text once it has returned. Sixteen expression evaluations retain zero mappings, where each redefinition retains three, permanently and correctly.
Emacs
emacs/flan-mode.el derives from prog-mode with
lisp-mode's syntax table, so sexp motion, paren matching and indentation are
already right. It adds Flan's brackets — [ and { are brackets,
not symbol characters, since every binding list and every type is written with them — and
the characters a Flan name may contain. emacs/flan-dev.el is the client;
there is no parser in it.
Setting up
Put emacs/ on your load path and require the mode. Nothing else is needed:
the client, the REPL, the inspector and the conditions buffer all load on first use.
(add-to-list 'load-path "~/Development/flan/emacs")
(require 'flan-mode)
(require 'flan-dape) ; optional — lldb, and the only thing that binds C-c C-g
flan-dape.el is separate on purpose, so flan-mode works
without dape installed. You also need flan on your PATH, or
flan-dev-command pointed at it.
Starting a program
M-x flan-dev runs flan dev on a file, waits for it to come up
and connects. C-c C-z attaches to one that is already running, looking for
.flan-dev.sock upward from the buffer, so from anywhere in the project it
finds the one program you have going. C-c C-q disconnects and leaves it
running; M-x flan-dev-quit stops it, but only one this Emacs started — a
daemon you launched in a terminal is not Emacs' to kill, and it says so rather than doing
something surprising.
| Key | Does |
|---|---|
| C-c C-c | the top-level form at point, recompiled and installed |
| C-u C-c C-c | …and mark it, so the program stops at the form point is inside (C-u C-u: on entry) |
| C-c C-k | the whole buffer, as one module |
| C-x C-e | the expression before point, evaluated in the running program |
| C-u C-x C-e | …and stop at it instead of printing its value |
| C-c C-z / C-c C-q | connect (finds .flan-dev.sock upward) / disconnect |
| C-c C-o | the running program's own output, in *flan-output* |
| C-c C-r | a prompt on the running program (*flan-repl*) |
| C-c C-b | a stopped program: the condition, the restarts, the stack |
| C-c C-M-b | the same restarts, as a one-key prompt |
| C-c C-i | inspect a value, navigating into its fields |
| C-c C-m | what the macro call at point expands to, one step; C-u first for all the way |
| C-c C-a | disassemble a function; C-u first for its LLVM IR |
| C-c C-l | every lowering of a function at once: IR, -O0, -O2, and the hand-written backend |
| C-c C-g | debug under lldb, through dape — bound only once flan-dape.el is loaded, so flan-mode works without dape installed |
| C-c C-d | what the running program currently defines |
| C-c C-v | help on the name at point |
| C-c C-x | rebuild, relaunch and reconnect |
| M-. / M-, | where a name is written, through an xref backend |
Some commands have no key. M-x flan-dev starts a program and
M-x flan-dev-quit stops it. M-x flan-watch opens the watch
buffer, which shows values while the program runs rather than while it is
stopped, and M-x flan-watch-ghost-mode shows the same values inline at the
call that wrote each one. M-x flan-inspect-address roots an inspection at a
raw address rather than at an expression. M-x flan-allocations and
M-x flan-leaks read the allocation registry: every block it recorded grouped
by type, and the same walk with the dead left out.
C-c C-k sends one module rather than a form at a time on purpose: a
defvar and the function that uses it have to arrive in the same load, or the
first refers to storage that does not exist yet.
C-x C-e compiles the expression before point and runs it inside the running program — the actual process, with its actual state, not a copy. Both it and C-c C-c work on buffer text rather than the saved file, and a change installs at the next frame boundary.
The buffers, and their own keys
Three buffers have keymaps of their own. The conditions buffer (C-c C-b) shows the condition, then the restarts, then the stack — in that order, because the decision in front of you is which restart to take and the stack is only the explanation for it.
| Buffer | Keys |
|---|---|
| conditions | RET take the restart at point · 0–9 take one by number · TAB/n, S-TAB/p move · f fold a frame · i inspect a local · a abort · g re-read · q close |
| inspector | RET into the field at point · l back out · g re-read · TAB/n, S-TAB/p move · q close |
| repl | comint, plus C-c C-o, C-c C-d and C-c C-q |
A restart is taken by position, which is why the list is numbered: a name
resolves to the innermost frame offering it, so an outer retry shadowed by an
inner one is real, is on the list, and cannot be reached by name. One that genuinely cannot
be taken is drawn and refused with the reason rather than quietly omitted.
The inspector is unlike most: the view is never stale, because every step re-reads the
program as it is now. The cost is that the root expression runs again on every
step — going into a field sends (.pos b) where the last one sent
b, which is harmless, but inspecting (spawn-enemy) spawns one per
keystroke. That is why there is no auto-refresh and why g is a key you press.
The repl is program-scoped rather than buffer-scoped, so in sand you write
sim/settle and not settle. *flan-output* is separate
from it, because the program's stdout belongs to the program.
When a change is refused
Two different things wear the same refusal today, and only one of them is the design.
A changed signature is a placeholder refusal. The intended behaviour,
and what plan.org specifies, is that a signature change makes a new
version of the function: new callers resolve it, existing callers and any stored
Fn value stay safely on the old one, and the session warns at each
tracked stale caller site so you know what to re-evaluate. Nothing should have to restart.
That needs function versions, trampolines and caller tracking, none of which are built yet
— so until they are, the session refuses rather than letting a cell hand old arguments
to a new body. The refusal is a limitation with a date on it, not a rule.
A changed struct layout is the genuinely hard case, and is rejected while live values of that struct exist: storage already allocated has the old shape, and a new body would read its fields at the wrong offsets with nothing to say so. plan.org keeps this one as a rejection, and gives managed classes an explicit migration at a frame boundary as the eventual way through.
C-c C-x rebuilds, relaunches and reconnects, and is the way out while the above is true. It costs the program's state, which is why it is a key you press rather than something C-c C-c falls back to.
Around the edges
eldoc, completion and M-. all read one cached reply rather than asking per
keystroke, refreshed at the two moments the answer can have changed: on connect, and after
an evaluation the daemon accepted. The modeline says whether there is a program on the
other end, and says stopped when there is one sitting in the break loop — a
stopped program looks exactly like a running one from anywhere else in Emacs.
An error comes back with a location and the client draws an overlay there, cleared the
next time that buffer's evaluation is accepted. Under C-c C-g lldb needs no
plugin to read a Flan value, since a struct is its C struct; locals show under their real
names, and a shadowed one appears as v~2 while plain v still
answers with the outer binding.
| Setting | Default | What it is |
|---|---|---|
flan-dev-command | "flan" | the compiler binary |
flan-dev-socket-name | ".flan-dev.sock" | what C-c C-z searches for |
flan-dev-echo-result | t | print C-x C-e's value in the echo area |
flan-dev-names-shown | 4 | how many names to list before counting them |
flan-dev-output-buffer | "*flan-output*" | where the program's output goes |
flan-dev-poll-interval | 1.0 | seconds between checks for whether it stopped |
flan-dev-start-timeout | 60 | seconds to wait for a program to come up |
Targets and builds
Native x86-64 is the development target. --target=wasm32-wasi produces a
module, and the headless sand acceptance program prints the same 64-bit hash under it as
it does natively:
$ flan run test/programs/sand-headless.flan
15595743031174623232
$ flan build test/programs/sand-headless.flan --target=wasm32-wasi -o sand.wasm
$ node --no-warnings test/wasm-run.mjs sand.wasm
15595743031174623232
The RNG is written in Flan rather than called from libc for that number: a grid hash
is only a regression test if the sequence is byte-identical on both targets. It holds at
-O2 and at -O0. The wasm side needs a
wasm32 builtins archive — from wasi-sdk, or emscripten's substituting for
it — and the compiler names every path it looked in when it cannot find one.
Dev and release builds are deliberately different. --dev means
indirection cells and -rdynamic, which exports those cells for a loaded
module to bind to. Release builds call directly, emit constants as constants, and get all
the folding back. Dev builds are not pruned by reachability: what a REPL may redefine
next is not a function of what has been called so far.
There is a second backend, and it is off by default.
flan build --x86 lowers the checked program to x86-64 by hand —
lib/x86.ml, writing an assembly file directly — instead of going through
LLVM. It is the dev backend: LLVM stays the default and stays the release path,
and the two never meet in one process, which is what lets the hand-written one pick its
own internal calling convention (every aggregate by pointer, no eightbyte rule, no
classifier) and match SysV only at the C boundary, where the shim has already flattened
every struct.
It refuses by name anything it does not lower, so a build that succeeds is one it really compiled rather than one it half-compiled. Conditions were the visible gap once and are not any more: the transfer channel, the guard after every call, bounds and arithmetic failures, indirection cells, redefinition modules and DWARF line tables all landed, and what is left refused is narrow — an aggregate crossing the C boundary is the one worth naming, because closing it would mean the eightbyte classifier this backend is built on not having.
What holds it honest is that every program in the corpus is built both ways and the
two are compared byte for byte on stdout, stderr and exit status — not on a disassembly,
which has read perfectly beside a wrong answer more than once. spike/x86/survey.sh
is the script, and it currently reports 103 MATCH, 0 DIFFER, 0 refused by
name, with 38 programs skipped because they do not compile on either side, have
no main, or run forever. dune build @x86 runs it as part of the
build, so a refusal cannot sit unnoticed.
It serves the editor now, which is what it was written for.
flan dev --x86 and flan reload --x86 compile a redefinition
through the hand-written backend, and the round trip is about half what LLVM's is —
llc was nearly all of it, and an assembler is not:
| LLVM | --x86 | |
|---|---|---|
| code generation | llc 42–48ms | as 8.0–8.4ms |
| link | ld 9.7–11.4ms | ld 8.9–9.3ms |
C-c C-c, at the socket | 62–66ms | 27–30ms |
C-x C-e | 60–61ms | 24–25ms |
Both columns were measured on one machine, whose llc is slower than the
one the 15–17ms above came from; the comparison is the point and not the absolute
numbers. What the headline hides is where the time went: ld did not move and
is now half the x86 build, and about 9ms of the round trip is the frontend checking the
program again — a seventh of the LLVM loop but a third of this one. The next millisecond
is in the linker rather than the code generator.
It requires --two-process, and refuses the merged daemon by
name. A merged host is linked -rdynamic, so it exports every Flan
body it contains, and those interpose the prelude of the LLVM-built macro module the
compiler loads into itself to expand macros. An LLVM caller lands in an x86 body
and the process dies during the first expansion, before the program has started. The
marker symbol that refuses a crossed redefinition module does not catch this, and was
never meant to: a macro module deliberately carries none. Hidden visibility on a macro
module's Flan bodies is the fix and is not built.
--debug is a third flag beside --dev and the optimisation
level. --dev asks whether you can redefine the program while it runs;
--debug asks whether you can stop it and read it. It emits DWARF, sets
-O0, and is refused by name for wasm32. lldb needs no plugin to read a
Flan struct: the struct is its C struct. Both backends emit it, though not the same
amount: the hand-written one writes a compile unit, a subprogram per function and a line
table out as bytes, because .loc cannot work against a file whose
instructions are .byte blobs, so --x86 --debug gives a
backtrace naming Flan files, functions and lines while print x says the name
is not in the current context.
Some things are refused by name rather than half-supported, and both cross-target refusals say why:
$ flan run hello.flan --target=wasm32-wasi
flan run: --target is refused — a cross-built module is not something this host
can exec. Use flan build --target=... and a wasm runtime.
$ flan build hello.flan --dev --target=wasm32-wasi
wasm32: --dev is native only — the reload path is dlopen, which wasm32 has no
equivalent of
Build time for calc-me.flan is about 110ms, of which the frontend — read,
parse, load, check, emit — is under 10ms. Every C translation unit goes through an object
cache keyed by a digest of the source text, the compiler, the optimisation level and the
target flags, so it never needs invalidating by hand.
Not implemented yet
The house rule is that anything which binds a name, alters control flow, or is not yet implemented must be recognised explicitly and rejected. Each of these refuses by name, with the milestone it belongs to, and the tests assert on the reason.
| You write | The compiler says |
|---|---|
(Result T E) | (Result T E) is not implemented yet — milestone 6 (see plan.org) |
(try …) | try (Result) is not implemented yet — milestone 6 (see plan.org) |
'sym as a value | a quoted symbol (restart names) is not implemented yet — milestone 6 (see plan.org) |
| a bare lowercase type name | generic code over the type variable a is not implemented yet — milestone 5 (see plan.org) |
errdefer | errdefer is not implemented yet (see the build sequence in plan.org) |
await | await is not implemented yet (see the build sequence in plan.org) |
handler-case | handler-case is not implemented yet (see the build sequence in plan.org) |
find-restart, compute-restarts | … is not implemented yet (see the build sequence in plan.org) |
a union as a declare parameter | a parameter of g is U, which cannot cross to C directly — pass (Ptr U) and let the shim read it |
| a user-written allocator | a user-written allocator is not implemented yet, and a defn's name in that position … |
Two of those rows want reading carefully. A quoted symbol works where a
restart is named — (invoke-restart 'use-value 21) is the ordinary
spelling — and is refused only as a value in its own right, because there is no symbol
type to give it. And the type-variable row is about the sigil: bare
a in type position is not a type variable and never became one;
$t is, and generics is where it is written down.
Beyond that list, and just as true: there is no overloading; defer is
refused inside a loop or a branch (a let is fine — it has the function's
extent); find-restart and compute-restarts are blocked on a
Restart type rather than on effort; a restart with parameters cannot be
taken from the break loop, which aims at a frame by position and has nothing to fill them
with; there is no package-private marker other than main not being exported;
there are no threads in the language; and the managed class facility that
plan.org describes is a plan and not a feature.
One of these is settled rather than pending. There is no interpreter
and there is not going to be one: compiling is the only way a form is ever run. The
instrumentation-based step debugger that wanted one is cut, and compiled redefinition at
~19ms is perceptually instant for expression evaluation too. That is also how macros run
— a defmacro is compiled into a shared object and dlopened into the
compiler before the file that calls it is expanded, so there is no second evaluator to
disagree with the first.
Further reading
The repository's own documents, in the order they are worth reading:
plan.org— the design, the build sequence, and the open decisions.NEXT.md— the project's memory, and the authority on what is actually built.spec-memory.md— ownership, containers, places, generics, function values. Frozen.spec-conditions.md— conditions and restarts, operational semantics. Frozen.conditions.org— a cheatsheet for driving conditions.calc-me.flan,sand.flan,test/programs/— real programs that run.