20 KiB
Let's discuss
Open questions, raised and deliberately not answered yet. Nothing here is a decision or a task. Each entry is the question as asked, plus what is already known in this repo that bears on it — so the investigation starts from what exists rather than from scratch.
Settled decisions live in NEXT.md. Reasons for what already exists live in BUILT.md.
1. i, the inspector, and the frame it cannot see
Two things got conflated here and they should be separated.
What the inspector already does. Most of what was asked for is built. flan-inspect opens its own buffer, lays a
value's fields one per line, RET walks into one, l comes back, g re-reads. The renderer bounds its walk at depth 4
and span 8, and entering a field renders that field from depth 0 — so the elision moves with you rather than
truncating permanently. It is CIDER's inspector adapted, and the file says what the adaptation changed.
What is actually missing is detail on the leaves: a number shows in decimal only, with no hex and no binary, and a pointer does not show its address. Purely additive, small, and worth doing.
The real problem, and it is sharper than "a bug". The locals listing renders from each frame's own slot addresses,
so it is frame-accurate. The inspector is built on a stack of expressions — going into a field means sending a
different expression ((.pos b) where the last was b), and l works by popping back to the previous one. That design
is forced: a Flan value has no header, the thunk that rendered it is dlclosed as soon as it returns, and there is no
heap to retain anything in, so nothing can be held server-side the way CIDER holds a JVM object.
The consequence is that i evaluates a name wherever the evaluator stands, not in the frame being looked at. On the
innermost frame that happens to be right. On any other it may resolve to a global, to a different binding, or fail —
with nothing saying so.
An earlier suggestion in this conversation — "root the inspector at the slot's address" — does not work, and the
reason is worth keeping: an address is not an expression, so the first RET has nothing to build the next expression
from and navigation dies at step one. Recorded because it is the obvious fix and it is wrong.
So the options are genuinely three, and none is free:
- Teach the program to evaluate an expression relative to a frame. The most useful and the most work: the frame's
slots would have to be in scope for a compiled thunk, which means the daemon building a thunk whose free names bind
to that frame's addresses. It would also fix
C-x C-ewhile stopped, which has the same blindness. - Give the inspector a second rooting mode — an address root that can still walk, by carrying a type alongside the
address and stepping to a field's address rather than to a sub-expression. Navigation then works, but the two modes
have different capabilities and
lhas to cross between them. - Refuse
ioutside the innermost frame, honestly and by name. Cheapest, and it gives up the feature exactly where it is most wanted, since the innermost frame is the one already fully visible.
2. Annotating the IR and the disassembly with the source
The IR half is nearly free and should just be done. emit.ml writes .ll as text, so a comment costs nothing and
cannot break anything, and every typed IR node already carries a Loc.t.
The disassembly half, with the optimisation question settled. objdump already interleaves source into a listing
when DWARF is present (-S), and the daemon already shells out to objdump — so the -O0 case is close to a flag.
Settled in conversation: -O0 is expected to follow the source and gets the full annotation; -O2 is not expected to
and gets either nothing or whatever best-effort mapping falls out. That removes what looked like the blocking
question. --debug forcing -O0 is therefore fine and does not need decoupling for this.
How SBCL does it, since it came up. SBCL does not shell out. sb-disassem is its own disassembler, written in
Lisp, that knows the instruction encodings directly. What that buys is annotation from the inside: it labels constants
the function references, names the functions being called, marks entry points, and shows its own calling conventions —
because it compiled the code object and still holds the metadata.
The relevant observation is that this project is closer to SBCL's position than the objdump route suggests. The daemon owns the build and holds the metadata too; it simply is not feeding much of it into the listing yet. Naming the function behind an indirection cell, or a constant by its source name, needs no instruction decoder — only the information the daemon already has. Writing a disassembler is not the interesting part and should stay off the table; richer annotation of objdump's output is cheap and is where SBCL's advantage actually comes from.
3. def, defvar, defconst — three roles, currently two words
There is no def. defvar is a mutable global that is always re-initialised on redefinition; defconst is a
compile-time constant, folded into its use sites.
The problem in practice: C-c C-k re-evaluates every top-level form against a running program, so today it wipes
state you may have spent a session accumulating.
Settled in conversation — three roles, three words:
def— always re-initialised. The tweakable: some number you are adjusting and do not care about preserving.defvar— re-initialised only if it would come out different. For heavier initialisers: an N×M grid that should recompute only when the rows or columns actually changed.defconst— folded, and reserved for what genuinely cannot change for the life of the program.rows/colsinsand.flanare honestdefconsts: the program does not support resizing on the fly and changing them means resetting everything anyway.
Note this defvar is not Common Lisp's, which is "assign only if unbound". This one is value-dependent, which suits
a live-edited game better and is the harder of the two to implement.
The open question is what "different" means, and the grid example is what makes it sharp. If the grid is sized by
rows and cols, the initialising expression's text is unchanged when rows changes — only its value is. So:
- Compare the expression (textual or AST). Cheap, and does not catch the case above.
- Compare the expression plus everything it depends on. Catches it. Needs dependency tracking.
- Evaluate and compare the result. Correct, and defeats the entire purpose whenever the initialiser is expensive — which is the only case this feature exists for.
Option 2 is what is wanted.
The reason to do it: it is the same machinery defconst needs. A folded constant cannot be tuned live, because its
value is baked into every function that used it — so the values you most want to tweak while the game runs are exactly
the ones you cannot. Fixing that means knowing which functions depend on which constants, and rebuilding those. That is
the same dependency tracking defvar's "if different" requires. Built once, it buys live-tunable constants and the
conditional re-initialisation. Neither is worth building alone; together they are clearly worth it.
4. Structural typing, row polymorphism, anonymous structs
The goal as stated: make this feel Clojure-like while still being typed.
The representation is already structural. types.ml does structural equality on resolved types, and a Flan struct is
exactly its C layout with no header and no tag word. What is nominal is the checking, not the data — so this is a
front-end question, not a data-model change.
What it buys, in rough order of value here:
- Your destructuring already looks like this, and that is the strongest argument. Clojure's
{:keys [x y]}works in binding position today. Structural typing makes the same notation work in parameter position — the pattern you write to pull fields out becomes the signature saying what you accept. One notation, two places, no separate declaration. That is precisely the "Clojure-like but typed" feel being asked for, and it needs no new syntax, only a new meaning for syntax that exists. - Functions over "anything with these fields". Magnitude over anything with an
xand ay— a position, a velocity, an enemy, a bullet. In a game this recurs constantly because everything has a position. - Ad-hoc returns. Returning a hit flag and a point without declaring a type for the pair. The Clojure habit of returning a map, typed and free.
- The FFI stops needing conversions. raylib's
Vector2and a locally-defined vector are byte-identical and are today two nominal types. Structurally they are one.
Two costs, and the second is the real work:
- Not an "explicit over convenient" question — that framing was wrong and is recorded so it is not repeated. The standing rules about implicit numeric conversion and about a bare integer not becoming an enum are about lossy things happening silently. A structural match is not lossy; it is a different notion of type identity. The two were lumped together in conversation and the author rejected the pairing, correctly. Strong typing and nominal typing are separate choices and this only touches the second.
- Field order is layout, and that is the hard part.
{x f32, y f32}and{y f32, x f32}have the same fields and different memory. So either field order becomes significant — surprising, since nothing else about a structural type should be — or matching a differently-ordered type requires a copy, which breaks the zero-cost property the whole data model rests on. Accepted in conversation as a limitation to respect rather than fight.
Writability is what decides whether the copy escape exists. Fields would be writable on the same terms as any struct:
by value you get a copy and writes are local; through a (Ptr T) they reach the original. But that settles the layout
question rather than sitting beside it. Read-only structural access has an out — the compiler could gather the
requested fields into a temporary, and then field order would not matter at all. Writable structural access cannot:
it has to alias the real storage, so the layout must genuinely line up. Decide writability first; the layout answer
follows from it.
defclass may solve this, for the cases that can afford it. A class has an implementation-defined representation, so
the compiler owns the layout and access goes through metadata rather than a fixed offset — field order stops being a
user-visible fact. The limit is that a class carries identity and metadata, and a Vector2 should not pay for either.
So plain structs still need their own answer and classes only cover the cases where the overhead was already accepted.
See item 7.
Also unresolved: the shim generates a C typedef per named struct, and an anonymous struct has no name to generate one from. Either anonymous structs cannot cross the FFI, or the generator learns to name them.
5. A JS backend — for web apps, not for games
The goal, stated in conversation: calling JS libraries, and eventually a hiccup-style DSL for writing web apps in Flan. That is a different product from the wasm target and the two do not compete — games go to wasm, web apps go to JS.
This matters because the obvious objection does not apply. "A JS backend cannot call raylib, so the whole graphics layer would need reimplementing on canvas" is fatal for a game backend and irrelevant here: web apps do not use raylib.
The design fork, and it should be decided first:
- Linear memory — an
ArrayBufferstanding in for pointers and structs, with typed-array reads and writes. Fast arithmetic, and every value is opaque bytes, so every call into a JS library marshals both ways. This is asm.js, which wasm exists to replace, and it is precisely wrong for an interop-motivated backend. - Object mapping — a Flan struct becomes a plain JS object, a
Vecbecomes a JS array. Slower for tight numeric loops, natural for interop, readable output. For this goal this is clearly the right one, and performance is not the constraint people assume: DOM work is dominated by the browser, not by arithmetic.
The consequence to face early: the memory model does not come along. Object mapping means garbage collection, which
means no pointers, no manual free, no arena, no allocator. So this is a dialect, not merely a second backend —
some Flan programs will not compile to JS, and that should be named up front rather than discovered. Worth deciding
which subset is the JS-targetable one, and whether the checker enforces it per target.
The unexpected upside: the dev loop could work on the web. Conditions, restarts and a break loop all need the
ability to run new code in a stopped program. wasm cannot have this — --dev is refused there because reload is
dlopen and the browser has no sockets. JS has no such problem: it evaluates new code trivially. A JS target might be
the only place the live dev loop and the browser coexist.
Reader conditionals, decided in conversation. A cljc-style subset is accepted, and the author chose Clojure's
inline reader conditionals over file-level target naming. Note what that changes: this language has no conditional
compilation today, and that absence drove a decision earlier the same day — barf signals on web rather than being
compiled out, precisely because "this bit is desktop-only" is not expressible. Note also that the sand-on-web work
introduced file-level target replacement for C sources (foo.web.c replaces foo.c), so the repo now has a
file-level precedent that this decision deliberately does not follow. Both were put; inline was chosen.
Also wanted, and independent of all of the above: #_ to discard the next form, and repeated (#_#_) to discard
that many following forms, as in Clojure. Reader-only, small, and useful immediately rather than whenever a JS backend
happens.
One dependency: hiccup is a macro, and the macro expander is blocked on Form being a Flan union, which is union
values. The backend can start before that; the DSL cannot.
6. C interop as seamless as Zig's
Today: declare-c names one C function per line and the compiler generates the wrapper, the typedefs and the flattened
declaration — 175 lines for raylib. No header is ever read, deliberately, which means nothing verifies that a
declaration matches the real signature. That is written down as trusted rather than guaranteed.
The proposal: read the header, prefix a namespace, get rl/InitWindow for free, possibly kebab-cased to
rl/init-window.
The dependency is the crux, and this project already answered the same question once. Zig's @cImport runs clang as
a library. That is exactly the dependency rejected in plan.org's "Why LLVM IR as text": a version-pinned C++ library
breaks routinely on upgrade, while a binary on PATH does not. Linking libclang walks back into it.
The middle path that keeps the property: clang will dump a parsed header as JSON from the command line
(-Xclang -ast-dump=json). Still only clang on PATH, still no library linkage, and it yields real signatures
instead of hand-transcribed ones — which closes the "trusted, not guaranteed" gap that is the strongest argument for
doing this at all.
The design question is how much to import. Zig imports everything a header declares. For raylib that is several
hundred functions plus every struct and macro, nearly all unused. The current 175 lines are deliberate, and the
declaration site is also the checkpoint where the compiler refuses a signature it cannot safely flatten — an
aggregate return, a variadic, a string coming back. A wholesale import removes that checkpoint, or has to reproduce
it as a filter. Generating the list from the header while keeping it explicit in the source is a third option: generate
once, commit the result, regenerate when the library moves.
Kebab-casing is separable and small, with one constraint: it must be reversible, because the generated C wrapper needs the library's own spelling.
7. Where defclass stands
Specified in plan.org and deliberately not started — its own last line says nothing happens until ordinary
struct, Handle and reload semantics work. Those have largely landed since that was written, so the gate may be closer
than the document assumes.
What it is meant to add: identity, runtime shape metadata, an implementation-defined representation, generic-function dispatch, and live schema change with an explicit migration at a frame boundary — which is the answer to the one thing redefinition still cannot do, changing a struct's layout while instances exist.
Three findings from an earlier review, recorded in NEXT.md and not yet in plan.org:
- A generic function is a cell. Adding a method from a later module is the same problem indirection cells already solve, so the expensive half is built and tested.
- The pool is not one storage option among three.
migrate-instanceshas to enumerate live instances, which a pool behind a generationalHandlegives by construction and the other two options do not. - Every layout version must stay resolvable for as long as any instance holds it — the same rule as "nothing is ever
dlclosed".
Also open and related: whether a condition can be a class, which decides whether handler matching has one path or two.
NEXT.md records the argument; decision 4 there took the cheaper parent-link route for now and explicitly left real
inheritance possible later.
8. Watching variables
Raised while designing item 1, and deliberately separated from it.
Item 1 shows globals per frame, chosen by what the code in that frame references. That is the right default and it cannot cover the case where the thing you care about is not named by the frame you are standing in — stopped deep in a helper, still wanting to see the grid.
So: a way to say "always show me this", surviving a resume and the next break.
Questions it opens:
- What can be watched. A global is easy — it has a name and an address that does not move. A local is harder: it
belongs to a frame that is gone as soon as you resume, so "watch
y" either means a differentyevery time or means nothing. A watched expression —(at velocity 40 12)— is the most useful and the most expensive, since it has to be compiled and run in the program each time, which is whateval-expralready does. - Where the list lives. Per project, per session, or in the file. A watch list that vanishes when Emacs restarts is one you stop using; one in the repo is one you accidentally commit.
- When it updates. Only on entering a break is cheap and probably enough. Live-updating while the program runs is a different feature — closer to a HUD than a debugger — and worth not conflating.
- Whether it belongs in the break buffer at all, or in its own window that is useful while the program is running, which is arguably where a game developer wants it.
9. Indentation is wrong inside a binding vector
Reported against the let in sand.flan's settle: the second and later bindings indent one column too far.
Cause, found by reading flan-mode.el. flan-indent-function looks at the head of the enclosing form and, when
it is one of Flan's body forms (defn, let, if, while, until, dotimes, match, do, loop, defer),
indents one past the open paren. Inside a let's binding vector the enclosing open is the [, and the symbol after
it is the first binding's name — never a body form — so the check fails and it falls through to Emacs's generic
lisp-indent-function, which treats the vector as a function call and aligns continuation lines under the first
argument rather than under the first binding.
What it is built on: Emacs's built-in lisp-indent-function, with that one override. Not clojure-mode, which is
where the missing piece lives — clojure-mode special-cases binding vectors and aligns them as pairs.
The fix is a binding-vector case in flan-indent-function: when the enclosing open is [ and the form containing it is
a binding form, align to the column of the first binding. Worth checking the other bracket users at the same time —
defn parameter lists and restart-case clause parameters have the same shape.