1619 lines
129 KiB
Markdown
1619 lines
129 KiB
Markdown
# Where this is
|
||
|
||
## Start here — next session
|
||
|
||
**Branch `dev-loop`, 199 commits, working tree clean, `dune test` green.**
|
||
|
||
**[`DISCUSS.md`](DISCUSS.md) is what has been *asked* and not answered** — open questions with the repo context that
|
||
bears on each, so an investigation starts from what exists. Nothing in it is a decision or a task; when one becomes
|
||
either, it moves here.
|
||
|
||
**`NEXT.md` is what is left. [`BUILT.md`](BUILT.md) is why the existing parts are the shape they are** — the reload
|
||
primitive, cells, the agent, the session, the daemon, the Emacs client, conditions, the FFI shim, the layout, and the
|
||
order it all got built in. This file was half build log until it was split; do not let it become one again. When a
|
||
track here finishes, its explanation moves there and its entry here goes away.
|
||
|
||
The dev loop works end to end: `flan dev program.flan`, then `C-c C-c`, `C-x C-e` and `C-c C-r` in Emacs against the
|
||
running process.
|
||
|
||
**Conditions are three steps of four.** `(error c)` is the diverging variant — a handler that returns normally has not
|
||
answered it, so only a transfer gets past. The break loop is in, editor half included: an unhandled `error` stops the
|
||
program on the frame that erred, the daemon annotates every reply with `:stopped`/`:condition`, and `C-c C-b` lists the
|
||
restarts and resumes into the choice. A restart is chosen **by position**, off a snapshot taken when the break was
|
||
entered, because a name resolves to the innermost frame offering it and the stopped thread's stack does not hold still.
|
||
Restarts below the evaluation a break is inside are listed, marked, and refused with the reason.
|
||
|
||
**Restarts take parameters now** — §3's other half. `(use-value [v i32] ...)` binds them, `(invoke-restart 'use-value 21)`
|
||
supplies them, and what a clause takes against what was given is checked at run time and refused with both spellings,
|
||
because a restart is found by name on a dynamic stack and neither end of a transfer can see the other. The one path
|
||
that cannot yet supply a value is the break loop, which is item 2 below and is where the interesting half is.
|
||
|
||
Still open: **`handler-case`**, which §"What this does not settle" leaves open as possibly a macro over `handler-bind`
|
||
plus a transfer. **`find-restart` and `compute-restarts` are blocked on a type, not on effort** — §4 gives them
|
||
`(Option Restart)` and a list, and there is no `Restart` type and no list to return one in. The minibuffer prompt never
|
||
needed them; it reads the snapshot over the agent's socket. And a `restart-case` clause should still carry a **report
|
||
string**: `use-placeholder` is what `invoke-restart` needs, not what a person reading a list needs. §3 says to settle
|
||
that *before* parameters and it was not settled — the field is cheap and the accessor is cheap, but the only consumer
|
||
is the break loop's listing, which lives in the agent and the daemon, so it would have shipped as a field nothing
|
||
read. It belongs with item 2, where the listing is being changed anyway.
|
||
|
||
Read SBCL for what restarts should *mean* and ignore how it moves control: it transfers with `block`/`return-from`,
|
||
which §6 rules out.
|
||
|
||
### Landed — macros run, and `unless` is not a special form any more
|
||
|
||
The expander is written and the exit criterion plan.org set for milestone 5 is met: a conditional sugar moved out of
|
||
`parse.ml` and into `prelude.ml` as a `defmacro`, with the corpus that was written against the special form
|
||
unchanged. Running `test/programs/macro-unless.flan` means the compiler built a shared object, `dlopen`ed it into
|
||
itself and called a Flan function to find out what `(unless c a b)` means.
|
||
|
||
The full explanation is in [`BUILT.md`](BUILT.md), "Macros: the compiler dlopens the program". Four things worth
|
||
knowing before touching any of it, because each cost something to find:
|
||
|
||
- **A call inside a quasiquote is output, not a compile-order dependency.** A macro body that *calls* another macro
|
||
needs it compiled first; a macro body that *quasiquotes* a call to one needs nothing, because the call is part of
|
||
what it answers and the answer is expanded again. The first cycle test written for this got that wrong and was not
|
||
a cycle at all. The two non-termination failures are therefore different and are refused differently: a ring is
|
||
named, a macro that does not settle is bounded.
|
||
- **Quasiquote is desugared before the walk**, and that is load-bearing rather than tidy — with the quasiquote still
|
||
standing, the walk expands the call inside it against the wrong arguments.
|
||
- **`lib/dune` passes `-linkall`.** `lib/macro.ml` installs itself into `Parse.expander` and nothing references it, so
|
||
the linker would otherwise drop it from `bin/main.exe`. Installing by hand is not viable: `session.ml` parses for
|
||
`C-c C-c`, and `test_session.ml` drives the session library in-process.
|
||
- **Two parser bugs fell out of it**, both in the rule that tells a return type from the first form of a body. The
|
||
prelude's types were not in the set that rule consults, so `Form` in return position was read as a body form; and
|
||
adding them plainly made `(defn f [] (Rune {.code 65}) (bar))` a function returning a `Rune` with a one-form body,
|
||
silently, in every file in the language. Both are pinned in test_flan.ml's return-type section.
|
||
|
||
Costs: a build that names no macro is unchanged at 50ms; one that calls a macro is 310ms cold and 70ms warm, the
|
||
difference being a cached `.so`; and a hello-world carries eight bytes of it, because `Reach.link` drops the rest.
|
||
|
||
### Landed — a C header is read, so a binding is checked instead of trusted
|
||
|
||
`lib/cimport.ml`, `lib/cjson.ml`, a `headers` file beside `link`. Full reasoning in `BUILT.md`, "The header is read
|
||
now"; DISCUSS.md item 6 is rewritten down to the two decisions left, both the author's.
|
||
|
||
The gap closed is the one `BUILT.md` recorded as *trusted*: `declare-c` generates the wrapper, the typedefs and the
|
||
prototype from one declaration, so they agree with each other by construction and only the library could disagree —
|
||
and nothing had a second opinion to disagree with. Now clang is asked for a JSON AST dump of the header (shelled out,
|
||
never libclang — the dependency plan.org rejected; Zig has since left it too, for Aro) and both halves are compared
|
||
against it.
|
||
|
||
**The evidence.** Against raylib 5.5, the version whose `.so` `vendor/raylib/link` names: **all 16 `defstruct`s and
|
||
all 172 hand-written `declare-c` agree exactly.** Against the 5.1-dev header also installed on this machine, ten real
|
||
differences — nine functions that version lacks and one that gained a parameter — so picking the wrong header is
|
||
loud. Both comparisons run at build time and stop the build; verified by permuting `Texture2D` and by putting `f64`
|
||
where raylib says `float`, which is the hazard `BUILT.md` names and says only a test can catch.
|
||
|
||
**Costs, measured, because they decide the remaining question.** Release build +4ms warm — `Reach.link` already drops
|
||
a wrapper nothing reachable calls, confirmed on the wasm32 case it exists for with 256 extra declarations in play.
|
||
Redefinition 31.0ms → 46.5ms. Dev build +333ms cold, once per session, since `Build.shared` compiles no C. Reading the
|
||
header is cached (64ms → 17ms), keyed like the object cache; the cache was built against a measurement, not a guess.
|
||
|
||
**Re-measured, and the 15.5ms was misattributed** — see BUILT.md, "Where that 15.5ms actually is". A `C-c C-c` reads
|
||
no header: `Session.eval` puts the forms through `Load`, and forms with no `(import …)` in them touch no package. The
|
||
15.5ms is `flan reload`'s, and `flan reload` is a fresh process — ~14.5ms of it is session startup and ~4ms of *that*
|
||
is the header. What a redefinition really pays for an imported package is **+3.6ms per eval** in `Check` and in
|
||
`Emit.redefinition` declaring 256 more siblings, and no cache touches that; it is the number to attack next. The
|
||
header is now cached in the session as well as on disk, so a repeat import (a `C-c C-k` of a buffer carrying its own
|
||
`import` line) costs nothing, and a header edited mid-session is not picked up until the session restarts — the same
|
||
rule a changed `.c` file follows.
|
||
|
||
**Opt-in on purpose.** `vendor/raylib/headers` is `?${FLAN_RAYLIB_H}`. "A build needs libraylib linkable and not
|
||
raylib-devel installed" is a property chosen deliberately, and requiring a header would take it from everyone to give
|
||
the check to whoever has one. Unset means off; set-and-wrong is an error naming the path.
|
||
|
||
Worth knowing before touching it:
|
||
|
||
- **The import is bounded by the package's own `defstruct`s**, not by a curated list. A function mentioning a struct
|
||
the package has not described is refused with that reason. Of raylib's 581 functions, 256 import, 153 are refused,
|
||
172 are already bound by hand and left alone. Widening the binding is a `defstruct`, not a list edit.
|
||
- **No `defstruct` is generated, and that is load-bearing.** Generate them and the header becomes the authority on
|
||
layout, and checking the package's layouts against it would be comparing the header with itself — which is exactly
|
||
why `BUILT.md` rejected a `_Static_assert` as circular. Keeping them hand-written is what makes the check a second
|
||
source.
|
||
- **A refusal is a demotion, not a drop** — Zig's `failDecl`, which `Load.refuse_hidden` already implemented for
|
||
`main`. `rl/get-gamepad-name` is a name that exists, cannot be had, and says why at the use site.
|
||
- **`declare-c` and `declare` are untouched and still win.** A C symbol the package binds by hand is not imported, so
|
||
the escape hatch is the override.
|
||
- **`test/headers/sample.h`** is the importer's table — one function per decision, committed, no raylib needed. The
|
||
raylib acceptance case skips without `FLAN_RAYLIB_H`; that one does not.
|
||
|
||
Two things that are *not* done, and are 6a and 6b in DISCUSS.md: whether the header stays a build-time read or becomes
|
||
a committed generator (`flan import-c` already prints the lines, so it costs nothing more to switch), and whether the
|
||
172 hand-written lines migrate. Neither is blocked on correctness. The argument for the first is weaker than it
|
||
looked — committing the generated lines would save ~4ms of session startup and none of the +3.6ms per redefinition,
|
||
since that cost is the 256 declarations existing at all and not where they came from; needing the header at every build — vendoring raylib.h or requiring raylib-devel — is the argument on the
|
||
second.
|
||
|
||
One smaller thing found and worth not re-deriving: an enum parameter imports as `i32`, because the header says
|
||
`KeyboardKey` and nothing tells the importer the package calls that `Key`. The ABI is identical, the face is worse,
|
||
and it is why `(rl/key-down? :space)` keeps its hand-written line.
|
||
|
||
### Landed 2026-09-12 — six tracks, one session
|
||
|
||
Six agents in parallel worktrees. Kept short on purpose; the reasoning that outlives the change is in `BUILT.md` or in
|
||
the commit that made it.
|
||
|
||
1. **`nth` removed**, an alias of `at` that was asymmetric — `check.ml` aliased them but `parse.ml` and
|
||
`place_of_expr` matched only `at`, so `(set (nth a i) x)` and `(addr (nth a i))` were refused while the `at` forms
|
||
worked.
|
||
2. **`println` and `print`**, compiler-provided and structural. `Session.render` was already the compile-time walk
|
||
plan.org asks for; it moved to `lib/render.ml` parameterised on an emitter and a slot allocator, so the REPL and
|
||
stdout share one copy. Found doing it: `field_addr` in `emit.ml` accepted only `Types.Named`, so a field of an
|
||
`Option` threw at emit time and **the walk's Option arm had never run** — the inspector would have failed on the
|
||
first `(Option T)` pointed at it. prelude.ml's claim that this had to wait for milestone 5 and generics was wrong,
|
||
and is gone: a printer selected per *concrete* type has nothing to dispatch on and no type variable in it.
|
||
3. **`restart-at`** — a restart is taken by position now. See "Start here".
|
||
4. **Names in DWARF.** `Tast.fn` carries `snames` beside `slots`, so a let-bound local is its own name under lldb
|
||
instead of `s0`; a slot the compiler invented keeps `s<index>`, because inventing a name puts a variable in the
|
||
debugger that is not in the file. Shadowing had to be decided rather than assumed: every `!DILocalVariable` is
|
||
scoped to the subprogram — the typed IR has no block structure to build a `!DILexicalBlock` from — so two slots
|
||
called `v` left lldb answering `p v` with the outer one while the body computed with the inner, and not listing the
|
||
inner at all. A repeat gets a `~2` suffix, unspellable in source. That is a way of not lying rather than a way of
|
||
being right; see "One line away". Also `flan dev --debug`, one flag for host and every redefinition module, off by
|
||
default because a debug build is an `-O0` build.
|
||
5. **Ten raylib core examples** in `examples/`, plus seven bindings and the colour palette. The gap list they produced
|
||
is under "Unblocked now, and ranked"; the top item, that no number could reach `draw-text`, is fixed — `(string b)`
|
||
reinterprets a `[u8]` as a `string`, which costs no instructions because they are already the same 16 bytes.
|
||
|
||
6. **The `print-*` family is gone.** `print` and `println` are the whole printing surface; ~500 call sites across 47
|
||
files rewrote, and `web/index.html` gained a `#printing` section, the first documentation either has had. Two pinned
|
||
outputs moved and both are corrections: `sand-headless`'s hash is `15595743031174623232` rather than
|
||
`-2851001042534928384` — the same 64 bits, printed unsigned now that `hash-grid`'s `u64` no longer goes through an
|
||
`(i64 …)` cast — and a trap column shifted because the call it names got shorter.
|
||
|
||
### Landed — `(Map K V)`, and `defer` in a `let`
|
||
|
||
Step 4 of the container build order, following Odin: open-addressed Robin Hood hashing at a 75% load factor,
|
||
cache-line cell packing, pointer-width integers through the probe loop. Two deliberate departures from Odin — no
|
||
tombstones, because `spec-memory.md` defers removal, which deletes the backward-shift loop entirely; and no capacity
|
||
tagged into the data pointer, because this header has room for it and tagging would make correctness depend on an
|
||
alignment that is only ever requested.
|
||
|
||
**One amendment to a frozen `spec-memory.md`, and it is the `defer` half**: the spec says under "When storage is
|
||
released" that `(defer (free v))` for a `let`-bound `v` is "not expressible today" and that no idiom may depend on it.
|
||
It is expressible now. A `let` at the top level of a function body has exactly the function's extent — a `let` is not a
|
||
frame here, and nothing is released at scope exit — so a `defer` in one always registers. A loop body and a branch stay
|
||
refused, by name, for the reason that does apply to them.
|
||
|
||
**One restriction the spec does not have**: a fixed array is a map key only when its elements compare bytewise, so an
|
||
array of structs or of strings is refused by name. A struct key holding the array works, because a struct key is walked
|
||
field by field.
|
||
|
||
The measured answer to the author's "is this another Python dict": **six times quicker cache-resident and slower at a
|
||
million entries**. Python's algorithm is fine — what makes it slow is a separately allocated refcounted object per key
|
||
and value, and hashing through calls that cannot be inlined. The second half of that result is the interesting one and
|
||
is written down rather than left out; see the unsettled list under the build order.
|
||
|
||
### Landed — the allocator, the arena, `(Vec T)`, `StorageExhausted`
|
||
|
||
The critical path, and the thing NEXT.md said was the only one standing between this and writing a game. Steps 1, 2 and
|
||
3 of the build order below are struck; `Map` is step 4 and is untouched. `Allocator` is a builtin opaque type and
|
||
needed nothing from milestone 5, which was the whole bet. Three amendments to a **frozen** `spec-memory.md`, made
|
||
deliberately and stated as amendments in [`BUILT.md`](BUILT.md): `free-all` is retain-capacity with `arena-destroy`
|
||
beside it; `context/allocator` is a dynamic variable rather than a literal calling-convention parameter; and the `Vec`
|
||
header is six words in every build rather than four in release. One addition the spec does not have: a budget on the
|
||
allocator, because `retry` needs a handler that can make the *same* request succeed.
|
||
|
||
### Landed — the runtime under a sanitizer
|
||
|
||
`--sanitize` is a build flag beside `--debug`; `dune build --root . @sanitize` builds twenty-eight programs twice, plain
|
||
and sanitized, and compares output and exit status. Its own alias and not `dune test`, because the sweep is about nine
|
||
minutes. The checked sweep is **clean**. How ASan and UBSan reach a language whose IR is written by hand, and why the
|
||
flag does not force `-O0` when `--debug` does, is in [`BUILT.md`](BUILT.md).
|
||
|
||
Two defects came out of it, both found by reading rather than by the tools, both fixed with a regression case:
|
||
`flan_bytes_to_i64`/`flan_bytes_to_f64` clamped a slice length with `(size_t)n` and so read 63 or 511 bytes off the end
|
||
of a negative-length slice; and the three `snprintf` shims published snprintf's return as a slice length, which is what
|
||
it *would* have written.
|
||
|
||
**What is left, and it is most of what the sweep was meant to settle:**
|
||
|
||
1. **UBSan sees no Flan code and no flag changes that.** Its checks are branches clang's C frontend emits inline, not a
|
||
pass, so shift UB (`(<< 1 32)`, see Sharp edges), alignment, and the f32→i32 cast on NaN or an infinity — the things
|
||
`floor-f32` guards by hand and nothing else does — are unreached. Either `Emit` grows those checks behind the flag,
|
||
which is a compiler feature of the same shape the bounds checks already have, or they belong to the checker. Not
|
||
decided. `test_sanitize` pins the current answer with a control that must *not* report, so a future clang changing
|
||
this is a test failure rather than a discovery.
|
||
2. **Three of the four named buffers now have evidence; one still does not.** Two lanes closed different pairs and
|
||
they combine. The 4K result cap and `condition_name[128]` are driven over the agent's socket from `test_agent.ml`
|
||
— a 5000-byte value comes back as 4096 ending in the ellipsis, a 198-character condition class comes back from
|
||
`status` as 127. The 4K cap and the **dev registry overflow guard** are also run directly by `test/dev_limits.c`,
|
||
a C main beside `reload_host.c`, one process per limit because the name table never shrinks and the overflow case
|
||
aborts. Only `SNAP_MAX`/`SNAP_NAMES` is still read rather than tested: sixty-five nested `restart-case`s are a lot
|
||
of program for a clamp. `escaped[ESCAPE_MAX]` was already covered, because `println.flan`
|
||
drives a 1100-character string through it on purpose — 1019 bytes out against a worst case of 1021 into 1024.
|
||
`scratch[SCRATCH]` never sees more than 20 characters of 64.
|
||
3. **Valgrind over the headless corpus, done.** `dune build --root . @valgrind` runs forty-seven programs under
|
||
memcheck, twelve of them again with `--no-bounds-checks`, in 88 seconds including the compiles. Clean. It needs no
|
||
instrumentation at all — memcheck works on the binary, so `Emit`'s hand-written IR arrives on the same footing as
|
||
clang's C, which is why it was reachable where MSan was not. The uninitialised read ASan is blind to is now a
|
||
control that must report: index 3 of a `Vec` with len 2 and cap 4, with `--track-origins` naming the
|
||
`aligned_alloc` in `flan_vec_push`. Two more controls pin a heap overrun and a padded `Map` key. `test/valgrind.supp`
|
||
holds **no suppressions** — nothing false came up to suppress. Details, and the measured fact that memcheck catches
|
||
0 of `bounds.flan`'s 6 cases where ASan catches 3, in [`BUILT.md`](BUILT.md).
|
||
|
||
**The hole it leaves, and it is the arena.** `free-all` is retain-capacity, so the pages stay and memcheck is never
|
||
told the storage died: round two of a reset arena reads a byte it never wrote, prints round one's value, and
|
||
nothing reports. Interior overruns are invisible for the same structural reason — an arena is one `malloc`, and the
|
||
`Map`'s `keys | values | hashes | scratch` is one allocation too, so "probe overrun at high load" is not clean, it
|
||
is not observable. Closing the arena half means `VALGRIND_MAKE_MEM_UNDEFINED` in `flan_arena_proc`, a `runtime/`
|
||
change nobody has made. And both positive controls had to be written by hand: no corpus program reaches an
|
||
observable uninitialised read, so the sweep is a regression net from here rather than an audit that found the
|
||
runtime sound.
|
||
|
||
Two things the sweep structurally cannot cover: raylib and libm are uninstrumented, so the windowed examples are noise;
|
||
and a redefinition module is built by `llc` and `ld` rather than clang, so the reload path carries no instrumentation
|
||
whatever the flag says.
|
||
|
||
### Managed classes are planned. Do not start them.
|
||
|
||
plan.org grew a `class` facility beside `struct`: identity, runtime shape metadata, an implementation-defined
|
||
representation, generic-function dispatch, and live schema change with an explicit migration at a frame boundary. Its
|
||
own last line is the rule — nothing until ordinary `struct`, `Handle` and reload semantics are working. It is here so
|
||
that a session reading plan.org cold does not take it as the next task. Three things found while reviewing it, none of
|
||
them in plan.org yet:
|
||
|
||
- **A generic function is a cell.** "A later module can add `(defmethod draw ((e Enemy)) ...)` without editing the
|
||
original" means every compiled call site of `draw` has to find the new method — which is the problem the indirection
|
||
cells already solve. A generic function is a cell whose body is a dispatch table and a reload extends the table. The
|
||
expensive half of classes is therefore already built and tested.
|
||
- **The pool is not one storage option among three.** `migrate-instances` has to *enumerate* live instances. A pool
|
||
behind generational `(Handle T)` gives that by construction; a world arena and an owned region do not obviously.
|
||
plan.org presents the three as a free choice and they are not.
|
||
- **`Enemy@1` has to stay resolvable** for `migrate` to dispatch on it, so the session retains every layout version's
|
||
metadata for as long as any instance holds it. Same rule as "nothing is ever `dlclose`d", and worth stating as one.
|
||
|
||
### Open: can a condition be a class?
|
||
|
||
Unanswered, and it wants answering before `handler-case`, because it decides whether handler matching has one path or
|
||
two.
|
||
|
||
It would buy the thing conditions most lack: a **hierarchy**. §1 says flatly there is none, which is why nothing can say
|
||
"any condition" — no catch-all handler and nothing for a break loop to match on. Class inheritance gives it.
|
||
|
||
Three costs, one serious:
|
||
|
||
- **Signalling would allocate.** A struct condition is a stack value and `signal` takes its address; a class instance
|
||
needs a pool slot at the signal site. That is the failure path, sometimes the hot path, and sometimes the thing that
|
||
failed is allocation itself. plan.org also says no implicit allocation anywhere in the core.
|
||
- **§5's lifetime inverts.** Today the condition dies with the signalling frame and a handler that keeps it copies,
|
||
which is free for a value struct. A class instance survives the transfer — nicer, but now something owns and frees it.
|
||
- **Layout versions meet handler frames.** A struct condition cannot change layout; it is refused. A class can, and then
|
||
a frame pushed against `MyError@1` is on the stack while the signaller builds `MyError@2`.
|
||
|
||
The shape that probably wins is both: a struct condition stays exactly what it is — no allocation, matched by name hash,
|
||
dies with the frame — and a class condition is allocated, survives, and matches by walking its class chain. That is two
|
||
matching paths, which is the same bill the struct/class split already signs, so it is consistent rather than a new cost.
|
||
Either way it is an amendment to a **frozen** `spec-conditions.md`, not a gap in it.
|
||
|
||
|
||
**The dev loop is closed.** `C-c C-c` in Emacs recompiles the top-level form at point and installs it in a running
|
||
program, at that program's next frame boundary. Verified against sand: an unsaved buffer edit to `game-draw`, and 240
|
||
consecutive frames drew it.
|
||
|
||
Steps 1, 2 and 3 are done — see *The reload primitive* in `BUILT.md`. A list of top-level forms can be recompiled and installed
|
||
into a running process; call sites compiled before they existed follow them, and a `defn` or `defvar` the process was
|
||
never built with can be added and then redefined again. That is the whole of `C-c C-c`, minus an editor: sand.flan takes
|
||
a redefinition over a socket and installs it between frames.
|
||
|
||
What is left is the *session* — something that holds the checker environment between evaluations, tracks which names the
|
||
running process was built with, and speaks a protocol an editor can talk to.
|
||
|
||
Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its simulation has a headless acceptance case that
|
||
runs on the `dune test` path at `-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and runs; the
|
||
interpreter was dropped — open decision #7, settled — see "Why there is no interpreter" in `BUILT.md`).
|
||
|
||
```
|
||
reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅
|
||
```
|
||
|
||
| File | What it does |
|
||
|---|---|
|
||
| `lib/loc.ml` | source locations + `Loc.Error`, the frontend's one exception |
|
||
| `lib/form.ml` | reader output: `Sym Kw Int Float Str Byte List Vec Map` |
|
||
| `lib/reader.ml` | hand-written S-expression reader, no menhir/ocamllex |
|
||
| `lib/ast.ml` | AST: `texpr`, `expr`, `place`, `pattern`, `decl` |
|
||
| `lib/parse.ml` | forms → AST; special forms, desugaring, declarations |
|
||
| `lib/load.ml` | **imports: a package directory → qualified declarations** |
|
||
| `lib/types.ml` | resolved types; structural equality, `Never` fits anywhere |
|
||
| `lib/tast.ml` | the typed IR the backend consumes |
|
||
| `lib/check.ml` | AST → typed IR; two passes, bidirectional |
|
||
| `lib/session.ml` | **a live program: what the process was built from, plus every change since** |
|
||
| `lib/wire.ml` | **the editor protocol: one s-expression per message, length framed** |
|
||
| `lib/dev.ml` | **`flan dev`: a session, the program running beside it, and a socket** |
|
||
| `lib/prelude.ml` | printers + `rand-f32`, written in Flan |
|
||
| `lib/emit.ml` | typed IR → LLVM IR text |
|
||
| `lib/build.ml` | `.ll` + the shim + the packages' C → clang → executable |
|
||
| `runtime/flan_rt.c` | the host ABI: argv, stdout, exit, 4 conversions |
|
||
| `runtime/flan_dev.c` | **dev only: the by-name registry a run-time-new name needs** |
|
||
| `lib/shim.ml` | **`declare-c` -> the generated C that flattens a struct crossing** |
|
||
| `vendor/raylib/` | **the raylib package: `raylib.flan` and `link`, and no C at all** |
|
||
| `vendor/agent/` | **the dev agent: a socket, a loader thread, install at a frame boundary** |
|
||
| `emacs/` | **`flan-mode.el`, `flan-dev.el`, `flan-repl.el`: the editor half of the dev loop** |
|
||
| `bin/main.ml` | `flan read \| parse \| check \| emit \| shim \| build \| run \| reload \| dev` |
|
||
| `test/test_flan.ml` | reader, parser and checker |
|
||
| `test/test_acceptance.ml` | expression/result pairs + whole programs + the traps |
|
||
| `test/test_reload.ml` | **the reload primitive: recompile one function, load it, call it** |
|
||
| `test/test_agent.ml` | **a running program taking a redefinition over a socket** |
|
||
| `test/test_session.ml` | **what a running process cannot be told, and recovering from a typo** |
|
||
| `test/test_dev.ml` | **the daemon, driven the way an editor drives it** |
|
||
| `test/test_repl.ml` | **`C-x C-e`: an expression evaluated inside a running program** |
|
||
| `test/programs/conditions.flan` | **`handler-bind` and `signal`, the accumulation case** |
|
||
| `conditions.org` | **a cheatsheet for driving conditions: what works, the exact refusals, the gotchas** |
|
||
| `conditions-play.flan` | **a program to poke at them with, built to be attached to by `flan dev`** |
|
||
| `test/programs/restarts.flan` | **`restart-case` and `invoke-restart`: the transfer, across two frames** |
|
||
| `test/test_emacs.ml` | **the client, driven against a real daemon and a real program** |
|
||
| `test/reload_host.c` | the C host that loads and installs two rebuilds, in one process |
|
||
| `test/wasm-run.mjs` | **a WASI host in twenty lines of `node:wasi`, so the table can run a wasm32 build** |
|
||
|
||
```
|
||
$ flan run calc-me.flan "1 + 2 * (3 - 0.5) / 2"
|
||
3.5
|
||
$ flan run test/programs/sand-headless.flan
|
||
15595743031174623232
|
||
$ flan run sand.flan # a window, 120 fps, hold space
|
||
```
|
||
|
||
## Decided 2026-09-12, by the author, and not yet built
|
||
|
||
Five questions were put and answered in one sitting. Each is a decision, not a preference — build against them, and
|
||
reopen one only with a reason rather than a taste.
|
||
|
||
~~**1. Assets are embedded at compile time, one file or one directory.**~~ **Built** — `(embed "p")`, `(embed "p" string)`, `(embed-dir "d")`. See BUILT.md, "Assets are baked in". Odin's answer, and the reason it is the right
|
||
one here: it is a *compiler* feature, so it needs no build flags, no linker arguments and no per-target packaging, and
|
||
it works identically on desktop and web. That matters more here than it does for Odin, because `Load` gives link flags
|
||
only to directory packages — the single file doing `(rl/load-texture "brush.png")` is structurally the one file with
|
||
no link channel, which is what stopped the web lane from inventing a flag. Embedding has no such hole. Odin's
|
||
`#load` and `#load_directory` are the model (`src/parser.cpp:853`, `src/checker.cpp:3594`). emscripten's
|
||
`--preload-file` stays available later for assets that should load lazily rather than be baked in; the `@web` link
|
||
line already carries it if wanted.
|
||
|
||
~~**2. Reading a file works everywhere; writing is desktop-only and signals on web.**~~ **Built** — `barf` on the web signals `FileError` with reason `file-unsupported`, and `test/test_web.ml` runs it under node rather than asserting the artifact's shape. See BUILT.md, "slurp, barf, and the two ways they fail". Odin stubs its whole file API on
|
||
js/wasm — every operation returns `.Unsupported`, and `core/os/file_js.odin`'s own comment says the stubs exist only
|
||
so importing `core:os` "panics cleanly". Take the restriction and not the mechanism. **Flan has no conditional
|
||
compilation** — nothing in `parse.ml` or `check.ml` reads the target — so "isolate this code to desktop" is not
|
||
expressible in source, and a build-time refusal would therefore be unusable. A **silent no-op is worse than either**:
|
||
it is how a save file disappears with nothing said. So `barf` on web signals a condition under a restart and the
|
||
program decides. This is the language having something Odin does not; use it. Per-package target isolation, if a
|
||
whole desktop-only package is ever wanted, is the `@native`/`@wasi`/`@web` link-line tagging the web lane built.
|
||
|
||
**3. Build the shadow stack.** ~~Not yet built.~~ **Built**, both halves — see BUILT.md. Kept here as the decision it
|
||
was, with the measurement it asked for: +33% on call-heavy code over globals for the frames, +61% with the slot table,
|
||
and 0.06% of a 60fps frame.
|
||
|
||
plan.org:591 has specified it in the dev-build column since the beginning and nothing
|
||
has ever built it. It is the route to `(:op "backtrace")` *and* to locals, together, and it is dev-only so a shipped
|
||
game pays nothing. Chosen over the DWARF route deliberately: DWARF still owes a `!DILexicalBlock` per `Let` before
|
||
`p v` under shadowing is even honest, and that buys locals in lldb rather than in the break loop. The author's reason
|
||
is the one to keep in view — **the more a break loop can show, the less often a real debugger is needed** — which
|
||
makes this a dev-loop feature, not a debugger feature.
|
||
|
||
**4. Conditions get a parent link, not class inheritance.** A condition type may name a parent where it is declared;
|
||
matching walks that static chain. This buys the hierarchy §1 of `spec-conditions.md` says there is none of — a
|
||
catch-all handler, "any file error" — at compile-time cost only. **It is deliberately not the class answer** that the
|
||
"Open: can a condition be a class?" section below weighs: a class condition allocates at the signal site, which is
|
||
the failure path and sometimes the thing that failed; it inverts §5's lifetime, so something must own and free it;
|
||
and it lets a condition's layout change while a handler frame stands against the old one. A parent link has none of
|
||
those costs and leaves the frozen model otherwise intact. Real inheritance stays possible later if a case demands it;
|
||
this closes nothing off. That section stays open for the record but is no longer the blocking question for
|
||
`handler-case`.
|
||
|
||
~~**5. File I/O — `slurp` and `barf` — is the next stdlib work**~~ **Built.** It was the next stdlib work, after `Vec`, because `slurp` returns a string whose
|
||
length is not known until the file is read and therefore cannot exist before an allocator does.
|
||
|
||
## Decided later the same day, and queued
|
||
|
||
~~**6. A field label is written with a dot, not a colon, and the colon is reserved for keys.**~~ **Done.**
|
||
`{.x 1.0 .y 2.0}` is struct construction and `{inner .field}` is destructuring; the old spelling is refused, and the
|
||
refusal names the new one. `:keys` kept its colon — it names no field, so leaving it alone is what lets the dot mean
|
||
exactly one thing. 681 labels across 45 `.flan` files including `vendor/`, plus 94 more in the Flan embedded in
|
||
`lib/prelude.ml` and the tests. `Map` is now free to take `{:key value}` without colliding with struct literals. See
|
||
BUILT.md, "The colon belongs to keys".
|
||
|
||
**What it left for the Emacs lane, both verified.** `render.ml` still *prints* a struct with colons, deliberately:
|
||
`emacs/flan-inspect.el:165` parses that output and hard-codes the colon when it reads a field out, so the printer
|
||
has to move in the same commit as its reader. That half is still open and belongs with whoever next opens the
|
||
inspector. ~~And `flan-mode.el:61` font-locks `:name` as a constant with nothing matching `.name`, so a field label
|
||
is now unfontified where it used to be coloured.~~ **The font-lock half is done:** a field is drawn as a constant in
|
||
both of the spellings that exist while the corpus moves, so `{.x 1}` and the accessor `(.x v)` read alike, and the
|
||
keyword rule stayed where it was because the colon still means an enum member and a map key.
|
||
|
||
**7. `Map` follows Odin's implementation.** Read `base/runtime/dynamic_map_internal.odin` before writing any of it;
|
||
the checkout is at `~/Repositories/Odin`. Three properties are the ones worth copying, and they are stated in its own
|
||
header comment:
|
||
|
||
- **Open-addressed Robin Hood hashing at a 75% load factor.** No buckets, no per-entry allocation, and probe
|
||
distances stay even because a later arrival steals a slot from an earlier one.
|
||
- **Cache-line-aligned `Map_Cell` packing**, so no single key or value ever straddles a cache line and a linear probe
|
||
walks memory in a cache-friendly order. This is the part a hand-rolled open-addressed map usually gets wrong.
|
||
- **`uintptr` throughout** for sizes, masks and offsets, to keep sign-extension and masking instructions out of the
|
||
probe loop.
|
||
|
||
Its static/dynamic split is the same type-erasure this project already committed to: `Map_Info` carries size,
|
||
alignment and offsets, and the compiler emits the hash and equality pair per key type. `spec-memory.md`'s
|
||
structural-key restriction holds this to the built-in key set, so there is no dispatch to design.
|
||
|
||
**Why this will not be Python's dict.** Worth recording because it is the question that prompted the decision.
|
||
Python's dict algorithm is fine; what makes it slow is that every key and value is a separately allocated, reference-
|
||
counted object, and hashing and comparison go through indirect calls that cannot be inlined. Flan stores raw bytes
|
||
and compiles the hash and comparison concretely at each use. That difference is most of the gap before any
|
||
algorithmic cleverness. **jank is not the model** — it is Clojure, so its maps are persistent with structural
|
||
sharing, which plan.org rules out by name because shared structure destroys the clear ownership that is the whole
|
||
reason there is no collector.
|
||
|
||
## Decided in discussion, queued
|
||
|
||
**Globals in the break buffer — built.** One section under the stack, holding the union of the globals every frame on
|
||
the current stack references, each entry annotated with the frames that touch it and ordered by the innermost one. It
|
||
is `(:op "globals")` in `dev.ml` and `flan-cnr--insert-globals` in the break buffer. See BUILT.md, "Globals of a
|
||
stopped stack". ~~The one hole left open — the redefinition check is a fingerprint over a body's *slots*, so a new body
|
||
that names different globals while binding the same locals is not caught.~~ **Closed**: `Reach.ref_fingerprint` is a
|
||
second fingerprint over the set of globals a body names, carried beside the slot one in `%fninfo` and checked the same
|
||
way, and such a frame is now refused by name. Kept separate from the slot fingerprint deliberately, so `locals` still
|
||
reads a frame whose locals are fine and whose global attribution is not.
|
||
|
||
**The break buffer opens by itself when the program stops.** Today a condition stops the program and the buffer appears
|
||
only when `C-c C-b` is typed. `flan-dev--absorb` already inspects every reply for `:stopped` and a poll covers the case
|
||
where no reply is pending, so the client already knows the moment it happens and already moves the mode line from it —
|
||
this is a hook at a point that exists, not new plumbing.
|
||
|
||
Three things to settle while building it: whether it takes focus or only displays; whether `(pause)` should always take
|
||
the window, being a deliberate stop rather than a failure; and what it does when the program stops while point is
|
||
mid-edit in another buffer.
|
||
|
||
**`Handle` and the pool are the real gate on classes, and they are buildable now.** plan.org's rule is that nothing
|
||
starts on managed classes "until ordinary `struct`, `Handle`, and reload semantics are working". Checked against the
|
||
tree: structs work fully; reload works with one known hole (a changed signature is refused rather than versioned);
|
||
**`Handle` does not exist at all** — `check.ml:218` still refuses `(Handle T)` by name.
|
||
|
||
It is not an incidental precondition. `migrate-instances` has to *enumerate* live instances, and a pool behind a
|
||
generational `(Handle T)` gives that by construction while a world arena and an owned region do not. plan.org presents
|
||
the three storage strategies as a free choice and they are not: handles are the one that makes migration possible.
|
||
|
||
**The allocator and arena landing today are what unblock it** — a pool is built on them, so `Handle` is buildable now
|
||
where it was not this morning. Build `Handle` and the pool next and treat *that* as the gate. It earns its place
|
||
independently of classes: stable references to things that move or die is something any game wants.
|
||
|
||
Already banked, and it means classes are less work than plan.org implies: **a generic function is an indirection cell**
|
||
whose body is a dispatch table, which a reload extends. That is the expensive half of method dispatch, and it is built
|
||
and tested.
|
||
|
||
**Resource cleanup: `defer` stays the answer. `drop` is not built, and `with-cleanup` is not either.** Reached by
|
||
working the case through rather than by preference, so the reasoning is worth keeping.
|
||
|
||
`drop` was specified in `spec-memory.md` this morning. **This amends it: the hook is deferred, not built.** Three things
|
||
decided against it. It runs code somewhere the reader is not looking, which is the C++ behaviour the author explicitly
|
||
does not want. It would not even cover the motivating case — `Image` and `Texture2D` are *raylib's* types, and attaching
|
||
a hook to a foreign type is its own unsolved design question. And its one real advantage, cascading through a container,
|
||
is the case `Handle` is about to make rare: entities holding handles hold numbers, not resources.
|
||
|
||
`with-cleanup` / `unwind-protect` was also put and rejected: awkward with several resources, and it reads worse than
|
||
what already exists. The raylib begin/end pairs that seemed to motivate it are a macro problem, not a primitive one —
|
||
`with-drawing` and `with-mode-2d` are three-line macros once the expander lands.
|
||
|
||
**What to build instead is small: relax where `defer` may be written.** It is refused today inside a `let`, a loop or a
|
||
branch. The loop and branch refusals are right — `defer` is a *compile-time* construct, the cleanup copied into every
|
||
exit path, so "maybe registered" is not expressible and a loop body would fire once at function exit instead of once per
|
||
iteration. But **a `let` at the top level of a function body has exactly the function's extent** and always registers,
|
||
so it is as safe as function scope and is refused for a reason that does not apply to it. Relaxing it gives:
|
||
|
||
```
|
||
(defn load-brush []
|
||
(let [sheet (rl/load-image-from-memory ".png" brush-bytes)]
|
||
(defer (rl/unload-image sheet))
|
||
(set brush (rl/load-texture-from-image sheet))
|
||
(rl/image-flip-horizontal (addr sheet))
|
||
(set brush-mirrored (rl/load-texture-from-image sheet))))
|
||
```
|
||
|
||
Several resources are several defers, released in reverse, visible in acquisition order. A container of resources is an
|
||
ordinary loop inside the defer body — the manual cascade, three lines, at one level of nesting.
|
||
|
||
**Odin, for the record**, has no destructors, no drop and no finalizers: `delete` frees container memory and nothing
|
||
else, and resource release is `defer` at the acquisition site. That idiom does not transfer directly only because
|
||
Odin's `defer` is block-scoped; the relaxation above recovers most of it.
|
||
|
||
**The safety net, and the better use of effort: a debug tracking allocator.** ASan's leak detection covers memory
|
||
*instrumented* code allocated — the Flan allocator, and it is already wired up and clean. It does **not** cover a leaked
|
||
texture, because that memory belongs to uninstrumented raylib, which is the same reason the sanitizer sweep treats the
|
||
windowed examples as noise. But every raylib call goes through a *generated* wrapper, so a dev build can count
|
||
acquisitions against releases at that boundary and report what is still held at exit, by name. No hook, no type
|
||
annotation, nothing running at a distance — it does not change how code is written, it reports when something was
|
||
forgotten.
|
||
|
||
## Before the batch below: read `DISCUSS.md`'s "NEXT SESSION STARTS HERE"
|
||
|
||
An architectural question was raised at the end of 2026-09-12 and agreed as the next thing to investigate: **putting
|
||
the compiler inside the running program's process, instead of the two separate processes there are today.** It may
|
||
reopen several things recorded here as settled — the watch design, the 4K result cap and its seqlock, the snapshot
|
||
machinery, the render-thunk-per-inspection design for locals and globals, and whether an in-process JIT or a
|
||
hand-written backend is needed at all.
|
||
|
||
It is research first, not building. The batch below stays valid and none of it is blocked by the question.
|
||
|
||
## ~~Queued: a second tier of the standard library, after macros~~ — **landed**
|
||
|
||
See [`BUILT.md`](BUILT.md), "The prelude's second tier". The diagnosis here was right and the prelude had 44
|
||
allocation-free functions because there was nothing to allocate from; there are 24 more now, and the "Refused, by
|
||
name" block at the foot of `prelude.ml` is down from eight entries to four, each with a *different* reason rather
|
||
than the one shared sentence.
|
||
|
||
What landed: `append!`/`append-i64!`/`append-f64!` (the builder), `concat`, `join`, `split` returning a
|
||
`(Vec [u8])`, `repeat-bytes`, `replace-bytes`, `to-lower`, `to-upper`, `slices-new`; `format-f64` with a precision;
|
||
`atan2-f32` and `pow-f32`; `clamp` as a `defmacro`; and the slice family at two more element types —
|
||
`sort-f32!`, `reverse-f32!`, `swap-f32!`, `min-f32`, `max-f32`, `sum-f32`, `bytes<?`, `swap-bytes!`, `sort-bytes!`.
|
||
Tests: `programs/strings.flan`, `programs/format.flan`, `programs/algorithms.flan`, `programs/math2.flan`.
|
||
|
||
`string-from-bytes`, refused in that block, turned out to already exist: `string` is a builtin and
|
||
`(string (as-slice v))` is the round trip.
|
||
|
||
**What could not be built, and why each one could not.** All four want a compiler or runtime change, and none of
|
||
them wants a language decision.
|
||
|
||
- **`Map` keys and values.** The only item on the list above that could not be built at all. `len` reaches a Map and
|
||
`get`/`put`/`has-key?` address one entry, but nothing walks the block: `flan_map_len`, `_get`, `_put`, `_has`,
|
||
`_clone`, `_reserve` and `_free` is the runtime's whole map surface, with no iterator among them. It wants one
|
||
runtime function — `flan_map_next` over the open-addressed block, taking a cursor — and one builtin in `check.ml`
|
||
to emit the key and value sizes at the call site. It is *not* a generics problem, and it is a small job.
|
||
|
||
- **`map`, `filter`, `reduce`, and a sort taking a comparator.** Blocked on **function values**, not on generics,
|
||
which is the sharper statement than the one this list made. `Types.Fn` exists; `check.ml` refuses it with "a
|
||
function type is not implemented yet — milestone 5"; and there is nothing else in the language to pass. Generics on
|
||
top of that is what would make them one copy rather than one per element type, but without function values there is
|
||
nothing to be generic *over*. `sort-f32!` and `sort-bytes!` are the concrete answer in the meantime, and `sum-i32`
|
||
and `sum-f32` already are `reduce` with the `+` written in.
|
||
|
||
- **`(vec-new [u8])` is refused**, so a `(Vec [u8])` can only be made where the *context* names the type.
|
||
`check.ml`'s `vec_new_elem` accepts a single bare symbol naming a type and nothing else, and a `let` has no type
|
||
annotation to say it the other way round — so `split` needs a one-line `(defn slices-new [] (Vec [u8]) (vec-new))`
|
||
standing in as the place where the type is said. The fix is to let `vec_new_elem` take a type *expression* rather
|
||
than a name, which is the same parser that already reads `[u8]` in a parameter list.
|
||
|
||
- **An array literal cannot say it is `[f32]`.** A float literal defaults to `f64`, an array literal has no context,
|
||
and a `let` has no annotation, so `[3.5 -1.0]` is an `[f64]` and every element in `programs/algorithms.flan` is
|
||
written `(f32 3.5)`. Same shape of gap as the one above and probably the same fix.
|
||
|
||
Two smaller findings, both written down beside the code that ran into them:
|
||
|
||
- **The prelude is never macro-expanded.** `macro.ml`'s pass runs over the file being compiled; the prelude reaches
|
||
the checker through `Check.program`'s own prepend and never goes through the expander. So a prelude *function*
|
||
calling a prelude *macro* resolves the macro's underlying `defn` — the one taking a `[Form]` — and reports an arity
|
||
error. `format-f64` writes `(min 9 (max 0 prec))` where it wanted `clamp`. This is next to, and not the same as,
|
||
"a prelude macro may not call a macro" below.
|
||
|
||
- **A returned `Vec` is a move, and the dead set spans the function**, so an early `(return v)` on one branch kills
|
||
the binding for the `v` at the foot of another. `replace-bytes` guards its empty-needle case with an `if` rather
|
||
than a `when`/`return` for that reason. Probably correct as it stands — the analysis is not path-sensitive and
|
||
making it so is a real piece of work — but it is a shape that reads as though it should compile.
|
||
|
||
Already present and easy to miss: an **EDN parser**, at `vendor/edn/edn.flan`.
|
||
|
||
## Decided: the Clojure patterns we are deliberately not copying
|
||
|
||
This language borrows Clojure's shape and is **not** trying to be Clojure-compatible, so its known wrinkles are ours
|
||
to avoid rather than inherit. Four, with what to do instead.
|
||
|
||
**1. One argument-order rule, held everywhere.** Clojure's sequence functions take the collection *last*
|
||
(`(map f coll)`) and its collection functions take it *first* (`(assoc m k v)`). The split is deliberate there, and it
|
||
is why Clojure needs **two** threading macros instead of one. **Our rule: the thing being operated on comes first.**
|
||
That is already what the language does — `(at a i)`, `(len xs)`, `(push v x)`, `(as-slice v)` — and `into` follows it
|
||
with the source first. Hold it; do not ship two of anything to paper over a split.
|
||
|
||
**2. A membership test says which thing it tests.** Clojure's `contains?` checks *keys*, so `(contains? [1 2 3] 1)` is
|
||
true because index 1 exists — the most-cited confusion in the language, and there is no built-in for "is this value in
|
||
this list". Half of this is already right here: `has-key?` on a `Map` is named for what it does. If a value-membership
|
||
test is added for sequences, name it for values and never overload one name across both meanings.
|
||
|
||
**3. A predicate returns a boolean; a search returns what it found.** Clojure's `some` returns the *value*, so
|
||
`(some even? [1 2])` is `true` but `(some identity [nil false])` is `nil` — one name doing two jobs. Keep them apart:
|
||
a `?` name answers yes or no, a finder answers the thing or nothing, and neither pretends to be the other.
|
||
|
||
**4. Composition reads in the same direction as threading.** Clojure's `comp` is right-to-left while `->` is
|
||
left-to-right, so the two compose mentally in opposite directions. If anything here ever composes operations, it reads
|
||
left to right, the way `into` does.
|
||
|
||
Sources are community consensus rather than a specification; the `contains?` complaint is documented in *Getting
|
||
Clojure*. Recorded because these are cheap to honour now and expensive to unpick once a standard library depends on
|
||
them.
|
||
|
||
## Queued: `into`, fused transformation without transducers
|
||
|
||
Decided in conversation. **Not transducers, and not Rust's iterators — a macro that fuses the chain at compile time.**
|
||
|
||
```
|
||
(into xs (vec-new i32) (map double) (filter even?))
|
||
```
|
||
|
||
Argument order is **source, destination, then any number of transforms**, matching the `into->` macro the author
|
||
already uses in Clojure (`from to xform & xforms`). It reads as a sentence — take this, put it there, doing these —
|
||
and the variadic transforms have to trail anyway, which is the mechanical reason they cannot sit in the middle.
|
||
Clojure's own `into` composes them into one `xform` first, which is why that macro exists at all.
|
||
|
||
**Why a macro and not transducers.** Transducers compose at *runtime*: they need function values, closures and
|
||
allocation, and every element pays a chain of indirect calls. Rust has no transducers — it has iterators, which are
|
||
lazy but fuse into a single loop at compile time via monomorphisation and inlining, needing generics to do it. A
|
||
macro reaches the same destination with neither: `(map double xs)` expands to `(double x)` written straight into the
|
||
loop body, so the function name is *syntax* and never a value. **No intermediate collection at any step, no closure,
|
||
no generics, and nothing to inline.**
|
||
|
||
**It therefore does not need function values** and is independent of that work.
|
||
|
||
**What it gives up, and the author does not want it anyway:** you cannot build a transformation at runtime and pass it
|
||
around. That is transducers' actual selling point and it is close to useless in a game.
|
||
|
||
**Why the destination belongs in the form, and why this suits Flan better than `->>` would.** Every collecting
|
||
operation here allocates from an *explicit* allocator — that is a frozen rule in `spec-memory.md`. `->>` hides where
|
||
the result goes; `into` names it, so the macro knows the destination type, emits the right loop and the right
|
||
allocation, and the rule is honoured by construction. plan.org's `->>` threading over slices is the thing this
|
||
replaces for the collecting cases.
|
||
|
||
**Open, and worth settling when it is built:** whether reductions share the form. `(into xs 0 (map cost) (sum))`
|
||
reads oddly because zero is not a collection. A second macro with the same shape may be cleaner, so that the
|
||
destination is always honest about what it is.
|
||
|
||
Drop Clojure's `:eduction` branch — that is the pass-around case, and the one part that would need runtime machinery.
|
||
|
||
## The next batch, in order
|
||
|
||
Agreed at the end of 2026-09-12. Ordered by priority, not by size. Items 1-3 and 5-6 want the compiler core and should
|
||
run one lane at a time; item 4 is disjoint and runs alongside any of them.
|
||
|
||
~~1. **Fix the one failing test** — `the frame of a superseded body answered with the new body's names`.~~ **Done**, and
|
||
the handoff's diagnosis was wrong. Nothing was dropping the number: the fingerprint was emitted into `%fninfo` and
|
||
never read back. `flan_dev.c` called the field `spare`, there was no accessor, the agent never snapshotted it, the
|
||
backtrace line never carried it, and `Dev.locals` compared slot counts and nothing else — four of the five
|
||
hand-offs were never written, and printing both sides of the comparison could not have found it because there was
|
||
no comparison. The mechanism was sound and stayed: it hashes slot *names* as well as types, so it does see a
|
||
rename. See BUILT.md, "Locals of a stopped frame".
|
||
|
||
2. ~~**The colon-to-dot change.**~~ **Done**, and `Map` is unblocked: `{:key value}` is free. The sweep is
|
||
`tools/colon-to-dot.py`, kept rather than thrown away, because the lanes that branched before it wrote Flan in the
|
||
old spelling and their files want the same pass at merge — `python3 tools/colon-to-dot.py .` over the tree, and
|
||
`--in-strings` for a `test/*.ml` that embeds Flan.
|
||
|
||
3. ~~**`Map`, and the `defer` relaxation.**~~ **Both done.** See *`(Map K V)`, which is Odin's map* and *`defer` may be
|
||
written in a `let`* in [`BUILT.md`](BUILT.md). The `defer` relaxation **amends `spec-memory.md`**, which said
|
||
`(defer (free v))` for a `let`-bound `v` was not expressible; it is now. `Map` restricts one thing the spec does not:
|
||
a fixed array is a key only when its elements compare bytewise, so an array of structs or of strings is refused by
|
||
name. The measured answer to "is this another Python dict" is six times quicker cache-resident and *slower* at a
|
||
million entries, and the second half is the interesting one — see below.
|
||
|
||
4. **The Emacs batch. Disjoint from the compiler, so it runs in parallel with anything above.** Globals in the break
|
||
buffer; the buffer opening itself when the program stops; ~~the indentation rewrite with `clojure-mode` as the
|
||
reference~~; ~~`#_`~~; hex, binary and addresses on primitives in the inspector. **The indentation rewrite and `#_`
|
||
are done.** The indenter is ported from `clojure-mode`'s source rather than derived from it — `flan-mode` still
|
||
requires nothing outside stock Emacs — and it aligns a binding vector name-under-name, which is the bug that cost
|
||
friction on every keystroke. `defn` parameter lists and `restart-case` clause parameters were the same shape and
|
||
came with it. What remains in this batch is the break buffer and the inspector, and they are independent.
|
||
|
||
5. ~~**Union values**~~, then the macro expander, then `Result`/`try`. Promoted above `Handle` on the author's call —
|
||
macros are the thing most worth wanting, and unions are the only thing between here and them.
|
||
|
||
**Union values are done.** See *Unions, and the tag they carry* in [`BUILT.md`](BUILT.md). The diagnosis was right:
|
||
`Option` is a two-case union wearing a special coat, so `Tast.arm`'s `acase` and `binds` already were union shape
|
||
and `check_match` grew a second subject rather than a second path. A union is `Types.Named` exactly as a struct is,
|
||
so every path that merely carries a type learned nothing.
|
||
|
||
What the spec did not settle and this lane did: the tag is an `i32` and the payload a blob aligned to the widest
|
||
member of any case, so `%"U" = type { i32, [k x iA] }` is C's `struct { int tag; union {...} u; }` byte for byte —
|
||
checked against clang's answer for the same declaration. A value is `(U.C {.field value ...})` and construction is
|
||
**qualified**; a pattern is bare `(C x y)` and resolves against the scrutinee. Tags are declaration order from
|
||
zero, so **case order is part of a union's contract**: a zeroed union is the first declared case. A non-exhaustive
|
||
match is **refused**, never defaulted.
|
||
|
||
What is left for the macro lane, and it is one thing: **`load.ml:312` refuses an imported union outright**, so a
|
||
union is file-local. That is not a blocker for `Form` — the prelude is parsed and prepended into the same flat
|
||
namespace before `collect` runs, so a `defunion Form` in `prelude.ml` is an ordinary same-file declaration and
|
||
needs no import and no `load.ml` change. Verified by declaring one there and matching it from a program.
|
||
|
||
**Macros landed on top of this** and needed no `load.ml` change for `Form`, exactly as this said. See
|
||
[`BUILT.md`](BUILT.md), "Macros: the compiler dlopens the program", and the short list of what is left of them
|
||
below.
|
||
|
||
Macros are what buy `with-drawing` and `with-mode-2d` over raylib's begin/end pairs, the hiccup DSL if a JS backend
|
||
ever happens, and the removal of special forms from the compiler.
|
||
|
||
**`Result`/`try`** follows, being another union.
|
||
|
||
**Generics are deliberately NOT here.** They feel adjacent and are not urgent, and today is the evidence: `Vec` and
|
||
`Map` were the obvious customer and needed none — they are type-erased, with the compiler emitting sizes and the
|
||
hash/equality pair per call site, which is Odin's design. The remaining customers are user-written allocators and
|
||
escaping closures, and both actually want **function values**, which is a separate milestone-5 feature. Leave
|
||
generics until something concrete needs them.
|
||
|
||
6. **`Handle` and the pool.** A reference to something that can die, that reports that it died rather than silently
|
||
resolving to whatever reused the slot. Wanted on its own terms for entities referred to across frames, and it is the
|
||
real gate on classes. Buildable now that the allocator exists.
|
||
|
||
7. ~~**`break` and `continue`, with loop labels.**~~ **Built.** Labels are Odin's in the head position, both blockers
|
||
are answered, and the refusals name the construct they refuse for. See BUILT.md, "`break` and `continue`, and the
|
||
rule that replaced a blanket refusal". What was settled in conversation before it was built, kept:
|
||
|
||
**Labels, Odin-style but in the head position.** A keyword names a loop and `break` takes it:
|
||
|
||
```
|
||
(while :outer (< i n)
|
||
(while (< j m)
|
||
(when (hit? i j) (break :outer))))
|
||
```
|
||
|
||
A keyword there is unambiguous because a loop condition is never one. It is **not a goto**: control can only leave
|
||
a loop it is already inside, which is what keeps it safe and is the same restriction Odin's labelled `break` has.
|
||
|
||
The two known blockers stand and must be answered: `check.ml`'s `in_frames` rule refuses `return` inside
|
||
`handler-bind`/`restart-case` **because `return` always crosses**, while `break` crosses only sometimes — a loop
|
||
wholly inside a `restart-case` body has a legitimate local break — so that blanket refusal has to become a
|
||
loop-depth-relative-to-frame-entry rule. And `continue` forces a `Tast.While` signature change to carry a latch,
|
||
because `check_dotimes` folds the step into the body and a `continue` branching to the header would skip it and
|
||
hang.
|
||
|
||
8. **Errors: a structured value with spans and notes, and more than one per compile.** One piece of work, not two —
|
||
both need `Loc.Error` to stop being a single location plus a string.
|
||
|
||
**Today:** `lib/loc.ml` carries a point location and a message, and `Loc.Error` is the frontend's *one* exception, so
|
||
the first error aborts the run. The author's workflow is write everything, compile at the end, squash the list —
|
||
which cannot work when there is never a list. The *content* of the messages is already good; they state the reason
|
||
and name what to write instead. What is missing is structure and volume.
|
||
|
||
**jank is the model** (`~/Repositories/jank`, `compiler+runtime/include/cpp/jank/error.hpp`). It is a Lisp on LLVM
|
||
with unusually good diagnostics and three things worth taking:
|
||
|
||
- **A named `kind` per error** — roughly a hundred, `lex_unterminated_string`, `parse_odd_entries_in_map` — each with
|
||
a stable string id. Machine-readable classification with no JSON mode and no prose parsing.
|
||
- **A source *span*, not a point.** This is what draws Elm's squiggle: you underline a range. A column number cannot.
|
||
- **Notes: an error carries zero or more, each with its own span and its own severity** (info/warning/error), sorted
|
||
by position. **This is the actual secret of Elm-quality messages** — "this is wrong *here*" plus "because of *that*
|
||
over there", two places highlighted and each explained. One location and one string can never express it.
|
||
|
||
jank also carries the **macro expansion** an error came from, which this project will want once macros land, and it
|
||
is worth building the field now rather than retrofitting it.
|
||
|
||
**Then collect rather than raise:** finish the function, finish the file, report everything found. Error recovery in
|
||
a checker is real work — the hard part is resynchronising after a bad form without cascading nonsense — and it is
|
||
what the workflow actually needs.
|
||
|
||
**No editor work is required.** Flan already prints `file:line:col: message`, the GNU format Emacs's
|
||
`compilation-mode` parses with no configuration, so `M-x compile` gives a clickable list and `next-error` free.
|
||
Flycheck and a structured JSON report were both considered and are **not** wanted — the workflow is
|
||
compile-at-the-end, not live linting.
|
||
|
||
**Cannot run beside the current lanes**: it touches every file that raises, which is the whole frontend.
|
||
|
||
8b. **The old entry, kept for its one extra fact:** Raised by the author's workflow: write everything, compile at the end, squash
|
||
the list. That does not work today — `Loc.Error` is the frontend's **one** exception, so the first error aborts the
|
||
run and you get them one at a time, which is exactly the loop that workflow exists to avoid.
|
||
|
||
The fix is in the checker, not in tooling: collect errors and carry on — finish the function, finish the file,
|
||
report everything found. **No editor work is needed once that exists.** Flan already prints `file:line:col: message`,
|
||
which is the GNU format Emacs's `compilation-mode` parses with no configuration, so `M-x compile` gives a clickable
|
||
list and `next-error` for free. Flycheck and a structured JSON report were both considered and are **not** wanted:
|
||
the author's workflow is compile-at-the-end, not live linting.
|
||
|
||
9. **Signature generations and stale-caller warnings.** The biggest remaining hole in "you never restart the program" —
|
||
a changed signature is still refused rather than versioned. Last because it is the largest and nothing else waits on
|
||
it.
|
||
|
||
Deliberately not scheduled: the JS backend and header-based C interop, both large and neither blocking current work; a
|
||
debug tracking allocator, which is the leak safety net and a good candidate whenever it is wanted.
|
||
|
||
## Decided in discussion — the array constructor and the module system
|
||
|
||
**`(array 4 rl/Vector2)` makes a fixed array; `[4 T]` stays the type syntax. Built** — see BUILT.md, "`(array 4 rl/Vector2)`, and the one position with no type slot". The problem it solved: a `let`
|
||
binding takes no type, so `(let [pts [4 rl/Vector2]] ...)` reads `[4 rl/Vector2]` as a two-element array *literal* and
|
||
fails with *unknown name rl/Vector2*. It cost 32 hand-written `Vector2`s in one raylib example.
|
||
|
||
`[4 T]` is not a special syntax — it is the ordinary type syntax and already works everywhere a type is expected:
|
||
`(defvar points [4 rl/Vector2] ...)`, `(defn draw [pts [4 rl/Vector2]] ...)`. A `let` binding is the single position
|
||
with no type slot, which is the whole of the bug.
|
||
|
||
`(zeroed [4 rl/Vector2])` was proposed first and rejected on how it reads: in argument position the bracket form is
|
||
unambiguous to the *parser*, but it still looks like a two-element vector to a person. `(array 4 rl/Vector2)` says what
|
||
it does with the count and the type as plain arguments. `zeroed` keeps its existing job — an empty thing of whatever
|
||
type the destination wants — and `array` is the one that is told.
|
||
|
||
**The module system stays as it is: the directory name is the module name.** No `package foo` line at the top of each
|
||
file. Confirmed against Odin, which requires the declaration despite having the same one-package-per-directory rule —
|
||
`package os` appears in 85 files, all of them in `core/os` — so the line is ceremony that buys only the ability to
|
||
disagree with the directory name.
|
||
|
||
What the rule already gives, and what was checked in conversation: **several files in one directory are one module**,
|
||
which is the case directory-as-package exists for; **a loose file is a module of one**, so several modules can sit at
|
||
the same filesystem level without a directory each; and two modules cannot share a directory, which is also true of
|
||
Odin.
|
||
|
||
**Acyclic imports are kept deliberately, not inherited by accident.** Odin forbids import cycles and so should this:
|
||
a definite package order is what the macro expander will need later, since every `defmacro` must be compiled before
|
||
anything that calls it. Nested import paths not being real nesting — Odin's `core:math/bits` is a separate package
|
||
rather than a submodule of `math`, with no re-exporting — was reviewed and accepted as fine.
|
||
|
||
**A package importing a package has landed**, so a project is no longer an entry file plus one flat layer of
|
||
libraries. Four things were settled doing it:
|
||
|
||
- **A name imported *through* a package keeps the inner alias.** If `area/` imports `shape`, the type is `shape/Box`
|
||
in the finished program and never `area/shape/Box`. This is forced rather than chosen: a directory reached along two
|
||
routes has to arrive under one set of names, or the checker sees every declaration twice and two copies of one
|
||
struct fail to unify. It is also what makes the dedupe coherent, and what keeps a qualified name the resolvable
|
||
identity the `layout` op and the break loop depend on.
|
||
- **The same directory under two aliases is refused**, including when one of the two aliases is a package's own and
|
||
pages away from the other. That is the price of the rule above and the refusal names both aliases.
|
||
- **A diamond loads its bottom once**, keyed by the real path.
|
||
- **A ring is refused and named** — `a -> b -> c -> a`, not "there is a cycle". Tolerating one was the earlier
|
||
behaviour and looked like it worked; what it cost is a definite package order, which is the thing the macro expander
|
||
needs, since every `defmacro` must be compiled before anything that calls it.
|
||
|
||
`Load.t.pkgs` now comes back in topological order, dependencies first. The *declaration* list is deliberately not
|
||
sorted and does not need to be — `check.ml` collects every top-level name before it checks any body.
|
||
|
||
**The expander did not end up reading that order**, and it is worth saying so rather than leaving the paragraphs above
|
||
to imply otherwise. Macros are collected from the prelude and from the file being compiled; a `defmacro` in a package
|
||
is refused by name, because reaching one means resolving that package's own imports over `Form`s before `Load` runs.
|
||
The order is there and correct and is what package-level macros will read on the day they exist; nothing reads it
|
||
today.
|
||
|
||
**Still missing: package visibility.** `rl/get-color-raw` is callable. The blocker is surface syntax, not `load.ml`:
|
||
`exported` and the refusal machinery already exist and take a second rule in one line, but there is no way for a
|
||
package to *mark* a name private, and adding one means a parser change.
|
||
|
||
## Decided in discussion — three more, all approved and none started
|
||
|
||
**A watch window, ported from the author's Clojure one.** `~/Development/siam-farmer/watch.el` is the working
|
||
original; read it first. Its design, and the parts to keep:
|
||
|
||
- **The program defines what is shown.** Emacs polls one function — `(watch/render)` — and paints the string it
|
||
returns. There is no watch-expression machinery, no per-variable registration, no UI for building a query. The user
|
||
writes a function in the game.
|
||
- **Async, not synchronous.** A sync request on a 0.2s timer blocks Emacs's UI thread every tick. The original says so
|
||
in a comment, having evidently learned it.
|
||
- **`replace-buffer-contents`, not erase-and-insert.** It diffs, so point and scroll survive every tick; erasing yanks
|
||
the cursor to the top five times a second.
|
||
- Nothing is appended — the buffer is always the current snapshot.
|
||
|
||
**The one thing that does not port, and it decides the design.** In Clojure an eval is cheap. Here `eval-expr`
|
||
*compiles a module and `dlopen`s it* — tens of milliseconds and a new `.so` each time, in a directory nothing sweeps.
|
||
Polling at 5Hz would produce hundreds of shared objects a minute. So **the watch thunk must be compiled once and then
|
||
called repeatedly**, which makes this a daemon feature rather than something the Emacs side can do alone: a `watch`
|
||
op that compiles on first use and a cheap re-invoke per tick.
|
||
|
||
The author also raised **ghost text** as an alternative or addition to a dedicated buffer — values shown inline at the
|
||
code they belong to. Not designed; the buffer is the port, ghost text is a further question.
|
||
|
||
~~**The inspector gets a second way to start: an address and a type.**~~ **Built.** See `BUILT.md`, "Two ways to root
|
||
a walk, and why neither subsumes the other". It went in as a frame and a slot *index* rather than an address and a
|
||
type — the daemon holds both and an index is the thing the listing can hand back, while an address is not something an
|
||
editor should be holding. The one prediction that did not survive contact: `l` crossing between the two modes was
|
||
listed as a cost and is not one, because a stack entry carries its own root and a mixed stack cannot be built.
|
||
|
||
**Structural typing requires identical layout — same fields, same types, same order.** Settled by the author, and it
|
||
makes the feature simple rather than hard: structural compatibility becomes "the same memory", which costs nothing at
|
||
run time and needs no copy, no reordering and no adaptor. The motivating case is `{.x 1.0 .y 1.0}` and that order is
|
||
natural anyway.
|
||
|
||
**Flexible field order waits for classes, deliberately.** A class has an implementation-defined representation, so the
|
||
compiler owns the layout and field order stops being observable — any order can match. That is the right place to pay
|
||
for flexibility, because a class already carries identity and metadata, and a `Vector2` should pay for neither. See the
|
||
`defclass` entry: `Handle` is the gate.
|
||
|
||
Note what this settles from the earlier discussion: writability was the question that decided layout, and requiring
|
||
identical layout answers it — fields are writable on the ordinary terms, by value a copy and through a `(Ptr T)` the
|
||
original, with no special case.
|
||
|
||
## Blocked and unfinished
|
||
|
||
Everything below was found, decided or half-built and then stopped. Each says what blocks it. Nothing here is a
|
||
vague intention — if it is listed, someone has already established it is real.
|
||
|
||
### Unblocked now, and ranked
|
||
|
||
**0. Signature generations and stale-caller warnings — milestone 7's unfinished half.**
|
||
Promoted here on the author's correction, and `session.ml:146` already says the same thing at the refusal itself. A
|
||
changed signature is refused today and **that is a placeholder, not the design**. plan.org's open decision #6 says what
|
||
should happen: a signature change makes a new version of the function, new callers resolve it, existing callers and any
|
||
stored `Fn` value stay safely on the old one, and the session *warns* at each tracked stale caller site. Milestone 7
|
||
names it outright — "signature generations and stale-caller warnings".
|
||
|
||
**The thesis of this project is that you never restart the program.** Every refusal that ends in "restart to change it"
|
||
is a hole in that, and this is the biggest one. It needs three things that do not exist: function versions, a
|
||
trampoline per version, and caller tracking good enough to name the sites. The cell already gives the indirection; what
|
||
is missing is that a cell holds one bare pointer with no signature, so there is nowhere to put a second version.
|
||
|
||
A changed **struct layout** is the genuinely hard case and plan.org still specifies it as a rejection — storage already
|
||
allocated has the old shape and a new body reads its fields at the wrong offsets. Managed classes are the planned way
|
||
through, with an explicit migration at a frame boundary. Do not conflate the two: one is unbuilt, the other is decided.
|
||
|
||
|
||
**From porting ten raylib examples** — the first code the language was pushed by that it was not designed around.
|
||
Ranked by how often they were hit, top two first because they are walls rather than conveniences:
|
||
|
||
1. ~~No number reaches `draw-text`.~~ **Fixed** by `(string b)`.
|
||
2. ~~An enum parameter cannot be driven by a loop variable.~~ **Fixed** by explicit conversions in both directions:
|
||
`(i32 k)` takes an enum to its integer, `(GamepadAxis n)` takes an integer to an enum. Neither is an instruction —
|
||
an enum is an i32 at run time and `emit.ml`'s `cast` already reduced one to that before choosing an opcode — so the
|
||
change is a guard in `check.ml`'s cast arm and nothing in the backend. The rule the refusals came from is
|
||
deliberately *not* relaxed: a bare integer still does not fit an enum parameter, so `:spcae` is still an error at
|
||
the call site. The rule was "an integer must not arrive silently", and a written `(GamepadAxis i)` is not silent.
|
||
The other escape stays closed too — one `declare-c` per C function — and no longer needs to be open.
|
||
- **A value that is no declared member is allowed**, deliberately. raylib's gesture is a bitfield and an OR of
|
||
flags is a legal `Gesture` that is no single member; and `session.ml`'s printer already falls through to the
|
||
number for an out-of-range enum, on purpose, so refusing to construct one while agreeing to print it would be
|
||
incoherent. An `Option` would make every site unwrap for no safety bought, and a literal-only refusal would
|
||
catch nothing, because the bitfield case is a run-time value.
|
||
- **Only an integer converts *to* an enum.** Not a float, and not another enum — a cross-enum hop goes through
|
||
`(i32 x)` so both ends are written down. Enum → any numeric is always allowed: lossless to i32 by construction,
|
||
and a narrower target truncates by the rule every int→int cast already follows.
|
||
- **The comparisons needed nothing else.** `(> (i32 g) 255)` checks because `binary` takes the non-literal side
|
||
first; `binary` was deliberately left ignorant of enums, since teaching it would be the implicit conversion this
|
||
avoids.
|
||
- **A bit-set type later builds on this rather than replacing it.** It would be its own type with its own
|
||
operations and would still want a named escape to the underlying integer for the FFI, spelled the same way. If
|
||
`Gesture` becomes one, the `(i32 g)` calls stay valid and only the range tests migrate to a membership test.
|
||
- One parse fix came with it: `defenum` names were not in `parse.ml`'s type set, so a local enum could not be a
|
||
function's return type. They are in it now under a key of their own, admitted as a bare symbol and never as a
|
||
list head — because `(Key n)` is a *value* now, and putting `Key` in `types` would make a body starting with one
|
||
be eaten as a return type.
|
||
3. ~~**`break` is not implemented.**~~ **Built, with `continue` and loop labels.** Both blockers are answered: the
|
||
`in_frames` rule became a relative one rather than a blanket one, and `Tast.While` grew a latch. See BUILT.md,
|
||
"`break` and `continue`, and the rule that replaced a blanket refusal".
|
||
4. ~~**A `let` binding takes no type annotation**~~ — still true, and **no longer the blocker it was**: `(array 4
|
||
rl/Vector2)` is built and is the answer to the case that raised it. The reasoning below is kept because it is what
|
||
chose between the three surfaces, and the first of them is not what was taken — see BUILT.md, "`(array 4
|
||
rl/Vector2)`, and the one position with no type slot". The original entry:
|
||
|
||
A fixed array is either a top-level `defvar` or a literal with
|
||
every element spelled out. `(let [pts [4 rl/Vector2]] …)` parses as a two-element array literal and fails with
|
||
*unknown name rl/Vector2*. Cost: 32 hand-written `Vector2`s in one example. **Looked at and stopped — it is a
|
||
grammar question, not a missing feature.** Everything under the surface is already there: `Ast.binding` carries a
|
||
`bty`, `load.ml` renames through it, and `check.ml:723` consumes it as the `want` for the value. Only the way it
|
||
is written is open, and the parser says so where it refuses (`parse.ml:366`): `let` is a flat list of pairs, so it
|
||
cannot disambiguate by *count* the way `defvar` and `defconst` do — those read `[n t v]` as three arguments to a
|
||
form, and there is no such boundary between one pair and the next. Three surfaces, in the order they are worth
|
||
considering:
|
||
- **`(zeroed [4 rl/Vector2])` — `zeroed` takes its type as an argument.** Recommended. It is one extra branch in
|
||
the arity-0 `zeroed` case in `check.ml`, no parser change, no ambiguity, and it answers the actual complaint,
|
||
which is not "locals cannot be annotated" but "there is nothing here to infer *from*". It also reads as what it
|
||
does: the value is a zeroed thing of that type, not a name that has been told what it is.
|
||
- **A marker between the name and the type**, `(let [pts :- [4 rl/Vector2] …] …)` or similar. Unambiguous, and it
|
||
buys a general annotation rather than one form's escape hatch. The cost is a new piece of syntax in the binding
|
||
vector, which is the one place this language has kept looking exactly like Clojure's.
|
||
- **Bare `(let [pts [4 rl/Vector2] …])`.** The obvious spelling and the one that cannot work: `[4 rl/Vector2]` is
|
||
a well-formed two-element array literal, and telling the two apart needs types in the parser, which there are
|
||
none of by design.
|
||
Note that plan.org's rule is "annotate function signatures, infer locals", so the general annotation is a
|
||
deliberate absence and not an oversight — which is the other reason the `zeroed` route is the smaller answer.
|
||
5. ~~**Arithmetic is strictly binary** — *+ takes 2 arguments, given 5*.~~ **Fixed.** `+ - * /`, `min`/`max` and
|
||
`bit-and`/`bit-or`/`bit-xor` fold left over two operands or more. `%` and the shifts stay at two, and one operand
|
||
is refused with the form to write instead — there is no unary minus and no reciprocal.
|
||
6. ~~**No `sin`/`cos`/`abs` for floats.**~~ **Fixed.** `sin-f32` and `cos-f32` are `declare`s in the prelude now,
|
||
with the caveat written beside them: IEEE-754 makes `sqrt` correctly rounded and requires nothing of the kind for
|
||
`sinf`, so these are the one place in the prelude where native and wasm32 may disagree bit for bit. Float `abs` is
|
||
not wrapped, for the reason integer `abs` is not — it is `(max x (- 0.0 x))` over two builtins.
|
||
|
||
**A `string` cannot be returned from C at all**, which is what makes `GetGamepadName` unbindable: *a string only
|
||
crosses as a parameter — a C function that returns one returns something Flan has no owner for*. Same rule refuses
|
||
`TextFormat`, which is also variadic and so has no honest signature.
|
||
|
||
**The negative result is worth as much.** None of the gaps expected blocked anything — no generics, no allocator, no
|
||
`Vec`/`Map`, no escaping closures, and function-scoped `defer` never came up. Input-and-draw over fixed-size state is
|
||
the shape the language already has. Three constructs unexercised anywhere else in the repo worked first try: a fixed
|
||
array with a struct element, a 2-D struct array, and `[N string]` as both `defconst` and mutable `defvar`.
|
||
|
||
### The web target: what it does not reach yet
|
||
|
||
`flan build --target=web` works, a raylib example builds unchanged and `test/test_web.ml` is green — see BUILT.md,
|
||
"The browser is the third target", for the mechanism and why asyncify rather than `emscripten_set_main_loop`. Four
|
||
things it does not cover.
|
||
|
||
~~**1. `sand.flan` has no web build, and the cause is one missing `#include`.**~~ **Built. It opens.** See BUILT.md,
|
||
"sand.flan in a browser", for the whole of it. Three summary lines, because the diagnosis below was right about the
|
||
structure and wrong about the cause:
|
||
|
||
- The `#include` was never the fix. **The agent is a socket server and a browser has no sockets**, so an agent that
|
||
compiles there is an agent that can never accept a connection. `vendor/agent/flan_agent.web.c` is three no-ops, and
|
||
`Build` selects it over `flan_agent.c` on `--target=web` and nowhere else.
|
||
- **Refusing `vendor:agent` on web was the honest-looking option and is ruled out by arithmetic.** There is no
|
||
conditional compilation, `sand.flan` calls `agent/start` unconditionally, `Reach` cannot prune a package something
|
||
reachable calls into — so a refusal means the flagship program does not build for the browser at all. A refusal is
|
||
only honest when the caller has a way to not ask. This does **not** reverse decision 2 above: `barf`'s no-op loses a
|
||
file the program believed it wrote, and there is nothing for the agent to lose because `--dev` is already refused by
|
||
name on every wasm target. The argument is written out at the top of `flan_agent.web.c`.
|
||
- **A package's `.c` files can now be addressed to a target**, by a tag in the name before the extension, and a tagged
|
||
file *replaces* the untagged file of the same base name on that target. This is the C-source half of the
|
||
`@native`/`@wasi`/`@web` link-line mechanism decision 2 pointed at for per-package target isolation.
|
||
|
||
The brush is `(embed "brush.png")` decoded through a new `LoadImageFromMemory` binding. `load-texture` and
|
||
`load-image` now have no call site anywhere in this repository — deliberately, because a path-based load is the one
|
||
shape the browser cannot have, and said here so it is not read later as an accident.
|
||
|
||
The original entry follows.
|
||
|
||
**1. `sand.flan` has no web build, and the cause is one missing `#include`.** `vendor/agent/flan_agent.c` does not
|
||
compile under emcc: *variable has incomplete type 'struct timeval'* at line 426, because emscripten's headers do not
|
||
pull `<sys/time.h>` in transitively the way glibc's do. `sand.flan`'s `main` calls `(agent/start ...)`
|
||
unconditionally, so `Reach` cannot prune the package, so the flagship program stops at that error — even without
|
||
`--dev`. Beneath the include is a structural fact worth deciding rather than patching around: **the agent is a socket
|
||
server and the browser has no sockets**, which is the same family as the `--dev` refusal. So the two fixes are not
|
||
equivalent — add the include and the agent compiles into a web build that can never accept a connection, or refuse
|
||
`vendor:agent` by name on a web target the way `--dev` is refused. The second is the honest one. Neither was taken
|
||
here: `vendor/agent/` belonged to another lane this session.
|
||
|
||
~~**2. Assets are two questions and only one of them is about emscripten.**~~ **Answered by the embed above, and the
|
||
answer was the third option neither half here considered: make it a compiler feature and neither question arises.** The
|
||
hard half below is exactly right about the problem — the file that needs the asset is structurally the one file that
|
||
cannot declare it — and the conclusion drawn from it, that the fix must be a link channel or a new declaration, was
|
||
the wrong one. `(embed "brush.png")` needs no channel, because there is nothing to tell the linker. What is *not* done
|
||
is `sand.flan` itself: `(rl/load-texture "brush.png")` takes a path and raylib opens it, so pointing raylib at embedded
|
||
bytes needs `LoadTextureFromImage` over `LoadImageFromMemory`, which is a raylib binding question and not this one.
|
||
The original text follows. `sand.flan` does `(rl/load-texture "brush.png")` against a bare relative path.
|
||
|
||
- The easy half: a bare relative path has no meaning on a target with no filesystem. emscripten's answer is
|
||
`--embed-file` or `--preload-file` into MEMFS, and both are *linker arguments*, so they are already expressible as an
|
||
`@web` line in a package's `link` file. No new mechanism is needed for a package.
|
||
- The hard half, and the actual design question: **the file that needs the asset is structurally the one file that
|
||
cannot declare it.** `Load` hands out `lflags` only for a directory package (`one_file` → `[]`), and `main` is not
|
||
exported, so a program can never be a package. The program doing the `load-texture` therefore has no link channel at
|
||
all. Answering this means either giving a single-file program a way to carry build arguments, or making assets their
|
||
own declaration rather than a linker flag. No flag was invented for it here.
|
||
|
||
**3. Nothing has been opened in a browser.** *Still true, and now it is the only thing left between here and
|
||
"someone played with it".* `sand.flan` builds for the web, the module carries asyncify, raylib's GL imports and
|
||
brush.png's own bytes whole, and `node sand.js` gets as far as `glfwInit` before dying on `window is not defined` —
|
||
which proves the module is live and proves nothing about the canvas. BUILT.md carries the exact commands to serve and
|
||
open it, and the list of what only a human will discover: whether it paints, whether the audio round trip through
|
||
MEMFS survives, and the canvas size. The `until` loop never exits on the web, so none of `main`'s `defer`s run —
|
||
expected, and worth knowing before reading anything into it.
|
||
|
||
The original entry follows.
|
||
|
||
**3. Nothing has been opened in a browser.** The test is headless and permanently so: it asserts the artifact's shape,
|
||
the `asyncify_start_unwind` export and the `glViewport` import, and that node runs the emitted JS. Whether the canvas
|
||
actually paints is unverified by anything in CI, and a human should look once.
|
||
|
||
**4. Unmeasured and untested.** Asyncify's cost is quoted from emscripten's documentation (roughly a doubling of code
|
||
size) and not measured here, and no frame time on web has been taken at all. raylib's audio and any use of threads on
|
||
the web target are untried. And a **wasi** build that reaches raylib now fails on undefined symbols rather than on a
|
||
missing `-l:libraylib.so.550`, because that line is tagged `@native` — the same error one step later, and a worse
|
||
message.
|
||
|
||
### `break`, and why it was not built — **it is built now**
|
||
|
||
Kept as written, because everything in it held and the two blockers at the end are the two things the build had to
|
||
rule on. Both are ruled on in BUILT.md, "`break` and `continue`, and the rule that replaced a blanket refusal":
|
||
the `in_frames` precedent was replaced by a barrier on the loop stack, which refuses a *crossing* rather than
|
||
everything, and `Tast.While` grew the latch. The original note:
|
||
|
||
Settled, so the next attempt is cheap rather than a rediscovery:
|
||
|
||
- **`dotimes` gets it free** — it desugars to `Tast.While`, so one implementation covers both loop forms.
|
||
- **`defer` is a non-question.** It is function-scoped, `break` does not leave the function, nothing fires. No refusal
|
||
needed and no interaction to design.
|
||
- **Type it `Never`**, as `exit` and `return` already are.
|
||
- **`pads` is the structural model.** `emit_while` already makes an `endloop` label; break is a push/pop of that around
|
||
the body plus a `br`. `return` is a direct terminator with no context threading, so there is nothing else to mirror.
|
||
|
||
What stopped it, and neither is small:
|
||
|
||
- **`check.ml`'s `in_frames` rule does not extend.** It refuses `return` inside `handler-bind`/`restart-case` because
|
||
those frames are popped on the way out, and that refusal is blanket because `return` *always* crosses. `break`
|
||
crosses only sometimes — a loop wholly inside a `restart-case` body has a legitimate local break — so the precedent
|
||
has to be replaced by a loop-depth-relative-to-frame-entry rule nobody has ruled on.
|
||
- **`continue` forces a `Tast.While` signature change.** `check_dotimes` folds the step into the body as
|
||
`While (cond, body @ [step])`, so a `continue` branching to the header skips the increment and hangs. It needs a
|
||
latch — `While of expr * expr list * expr list` — across `check.ml` and `emit.ml`. plan.org settles break and
|
||
continue as one item and `parse.ml` refuses them in one case, so building break against today's `While` is exactly
|
||
the thing that would have to be undone.
|
||
|
||
plan.org's single line on it (831) names a `for` the language does not have and gives no mechanism.
|
||
|
||
|
||
1. ~~**Allocators, then `Vec` and `Map`.**~~ **Steps 1, 2 and 3 are done** — the allocator, the arena, `(Vec T)`,
|
||
`StorageExhausted` and `retry`. `Map` is step 4 and is what is left of this item. See *Allocators, `(Vec T)` and
|
||
`StorageExhausted`* in [`BUILT.md`](BUILT.md) for the shape, the three amendments to a frozen `spec-memory.md` and
|
||
the one addition. The claim below held: **`Vec` does not need generics** — that was wrong and is worth un-learning: Odin's containers are compiler
|
||
builtins over a *type-erased* runtime (`base/runtime/dynamic_array_internal.odin`), where `$T` appears only in thin
|
||
wrappers producing `size_of`/`align_of` at the call site, and per-key hash and equality are compiler-emitted
|
||
procedures passed as a runtime argument (`Map_Info`, `base/runtime/core.odin:369`). That runtime is what
|
||
`spec-memory.md` specifies.
|
||
|
||
**The four questions that used to sit here are answered**, in `spec-memory.md`'s "Allocators" section, which is
|
||
frozen along with the rest of that file: when storage is released, the `drop` hook, alignment, and allocation
|
||
failure. Read them there rather than in a second copy here. The one consequence the build order below turns on is
|
||
that no allocating operation returns an error — a failure signals `StorageExhausted` under a `retry` restart — so
|
||
`push` and `put` are `Unit`, `clone` returns the container, and no signature grows a `Result`. One question is left
|
||
open in that section on purpose; it does not block the build.
|
||
|
||
2. **The editor half of a typed restart.** The language half is in (see "Landed"): `(use-value [v i32] ...)` and
|
||
`(invoke-restart 'use-value 21)` work, and a mismatch is refused at run time with both signatures in the message.
|
||
What is missing is the half only an editor can do — the leverage SBCL lacks. `eval` already compiles and runs an
|
||
expression inside the live program and the daemon already holds the struct layouts, so "ask the human, type-check
|
||
the answer, hand it over" is a short hop, and it is the one path the runtime today *refuses*: a restart with
|
||
parameters taken from the break loop traps, because `flan_break_resume` and `flan_restart_take` aim the channel at
|
||
a frame and have nothing to fill its buffer with. What it needs, end to end:
|
||
- the frame already carries the arity and the signature as a string — `flan_restart_arity` and `flan_restart_sig`
|
||
beside `flan_restart_name`, the same walk, so `restarts` can say what each one takes;
|
||
- `:restarts` on the wire carries the signature per entry, so the minibuffer can show `use-value (i32)` rather
|
||
than a bare name, and `restart-at` grows an `:args` form — a list of *expressions*, since the answer is a Flan
|
||
expression and there is already something that compiles one;
|
||
- the daemon compiles each argument against the declared type with the session's layouts (the same path `C-x C-e`
|
||
takes), refuses it there if it does not fit, and otherwise writes the values into the frame's buffer and marks it
|
||
filled before aiming the channel. That last store is what `flan_restart_take` cannot do today and is the whole
|
||
of the remaining work; the marking exists so this cannot be forgotten silently.
|
||
|
||
3. **`handler-case`.** Not a convenience — it is the fix for the loudest gotcha in `conditions.org`. A handler closes
|
||
over nothing *only because* a `handler-bind` clause runs at the signal point; a `handler-case` clause runs in the
|
||
establishing frame, which is ordinary in-frame code exactly like a `restart-case` clause. SBCL's is `handler-bind`
|
||
plus a transfer and nothing more (`src/code/error.lisp:196-268`). Every piece exists.
|
||
|
||
### `Vec` and `Map` — the order to build them in
|
||
|
||
**Steps 1, 2 and 3 are built; 4 to 8 are what is left.** The reasoning is kept because it is what the remaining steps
|
||
rest on, and because the escape it describes was tested rather than assumed — see *Allocators, `(Vec T)` and
|
||
`StorageExhausted`* in [`BUILT.md`](BUILT.md).
|
||
|
||
The dependency nobody had written down, and the reason it looked worse than it is. `spec-memory.md` defines an
|
||
allocator as "a procedure plus an opaque data pointer" — a function value. `check.ml` refuses function values four
|
||
ways, and all four say milestone 5: a written `(Fn ...)` annotation (`Ast.Tfn`), a written `fn` literal (`Ast.Fn`), a
|
||
`defn`'s name used as a value, and calling anything other than a named function. `(Map K V)`, `(Result T E)` and
|
||
`(Handle T)` are still refused beside those as milestone 6; `(Vec T)` is not, any more. Read straight off those lines,
|
||
milestone 6's allocators need milestone 5's function values and the work doubles.
|
||
|
||
**The escape is real and the work did not double** — this is the claim the built thing confirms. All four refusals are about *surface syntax*, and a value the
|
||
compiler builds that no surface form names trips none of them. The compiler already does exactly this, twice:
|
||
|
||
- A `handler-bind` clause is lowered to a function whose address goes into a `flan_handler` and is called back through
|
||
`h->fn(condition, xfer)` (`runtime/flan_rt.c:38` and `:66`). `check.ml` builds that body as its own `Tast.fn`
|
||
(`:619`, `:654`), not as an `Ast.Fn`, so line 458 never sees it, and no Flan type names the result.
|
||
- In a dev build, `emit.ml`'s `call` loads a pointer out of an indirection cell and calls through it
|
||
(`lib/emit.ml:781`–`793`). That is the indirect call line 1023 refuses in source, emitted routinely.
|
||
|
||
It is also what `spec-memory.md` already assumes for `Map`: the hash and equality pair is compiler-emitted and passed
|
||
as a runtime argument. Odin's `Map_Info` is two contextless `proc` fields (`base/runtime/core.odin:369`), and Odin's
|
||
`Allocator` is a `procedure` plus a `data: rawptr` (`:422`) — the same shape, reached the same way. If the hash pair is
|
||
expressible with no function type in the surface language, so is the allocator's procedure.
|
||
|
||
So: **`Allocator` is a builtin opaque type, the way `string` is a builtin ptr+len.** It is a `Types.t` case with no
|
||
user-writable constructor. Its procedure is an ordinary top-level function resolved to a symbol at the emit site, and
|
||
`vec-new`, `push`, `put`, `clone`, `free` and `free-all` are named calls, which `check_call` already routes through
|
||
`named_call` (`check.ml:1021`). **The built-in allocators need nothing from milestone 5.**
|
||
|
||
What *does* need milestone 5 is a **user-written** allocator: the moment a program says "here is my proc, make an
|
||
`Allocator` from it", it needs a `defn`'s name in value position, which is `check.ml:571` verbatim. That is a real
|
||
limit and not a fatal one — Odin ships arena, general-purpose, stack, pool and scratch in its own std, and most
|
||
programs write none. Ship the built-in set; user allocators arrive with function values.
|
||
|
||
**5 and 6 interleave rather than nest.** plan.org orders generics and macros (5) before allocators and containers (6),
|
||
and that order cannot hold: the macro expander is blocked on `Form` being a Flan union and union *values* are milestone
|
||
6 (see "Macros" below). Conditions and restarts, also listed under 6, are already three steps of four. The milestone
|
||
numbers are a topological hint, not a sequence. Take 6's container half first, 5's generics half second, and 5's
|
||
expander last, on 6's unions.
|
||
|
||
~~1. **`Allocator` and the arena.**~~ **Done.** The builtin opaque type, the four operations with `size` and `align`,
|
||
the capability set read off the allocator value, `with-allocator`, `context/allocator`, `context/temp` and the epoch
|
||
counter. `free-all` was decided as retain-capacity with `arena-destroy` beside it; the context is a dynamic variable
|
||
rather than a literal calling-convention parameter; both are stated as amendments in `BUILT.md`. A user-written
|
||
allocator is refused by name with milestone 5 as the reason.
|
||
|
||
~~2. **`(Vec T)`**~~ **Done**, over the type-erased runtime, with `push`, `reserve`, `at`, `len`, `as-slice`, `free` and
|
||
`clone`, and with move-only enforced by a dead set that unions at an `if` or a `match` join. `at` and `len` were
|
||
extended rather than duplicated. The header is six words in *every* build, not four in release — a layout that
|
||
changes with a build flag can disagree silently across the reload boundary — and that is the third amendment.
|
||
Ownership is not transitive yet, so a struct field of `Vec` type, a global `Vec` and a `(Vec (Vec T))` are each
|
||
refused where they are declared, naming `drop` as what they wait on.
|
||
|
||
The note below still stands and is now the only thing between `Vec` and the accumulation pattern: **capture does not
|
||
exist at all.** Nothing about it changed.
|
||
|
||
~~3. **`StorageExhausted` and `retry`**~~ **Done, with step 2 and not after**, exactly for the reason given. It is a
|
||
`while` around a `restart-case` around the attempt, built in the checker out of nodes that already existed, so the
|
||
backend learned nothing about allocation. `test/programs/exhausted.flan` exhausts an allocator for real and takes the
|
||
restart; `exhausted-unhandled.flan` is the same failure with nothing handling it.
|
||
|
||
~~4. **`(Map K V)`**~~ **Done**, following Odin: open-addressed Robin Hood hashing at a 75% load factor, cache-line
|
||
cell packing, and pointer-width integers through the probe loop. `map-new`, `put`, `get`, `has-key?`, and `len`,
|
||
`reserve`, `clone` and `free` extended rather than duplicated. Two departures from Odin, both deliberate: no
|
||
tombstones, because the spec defers removal, which deletes the backward-shift loop entirely; and no capacity tagged
|
||
into the data pointer, because this header has room and tagging would make correctness depend on an alignment that
|
||
is only requested. `Tast.FnAddr` carries the emitted hash and equality pair and is not a function value — the same
|
||
escape the allocator used.
|
||
5. **`drop`.** The hook, the transitive move-only and non-`clone`able rules, and the refusal to construct a
|
||
`drop`-carrying value against an allocator without `can-free`. It is additive — no type in the repo has a hook today
|
||
— but the `can-free` refusal has to land with the construction path it guards, before any arena-allocated container
|
||
of a user struct is trusted.
|
||
6. **`(Result T E)` and `try`, then the rest of union values.** Unions are what `Form` needs, and `Form` is what the
|
||
macro expander needs.
|
||
7. **Generics and monomorphisation**, then function values. User-written allocators and escaping closures both fall out
|
||
of the second.
|
||
8. **The macro expander**, last, on 6's unions.
|
||
|
||
**What is genuinely unsettled.**
|
||
|
||
- **The Map is slower than CPython's dict at a million entries** (1.41s against 1.16s on the same workload), while
|
||
being six times quicker cache-resident (21ns against 132ns per lookup at 10k entries). Both are memory-bound at the
|
||
larger size and this layout waits longer: keys, values and hashes are three separate runs, so a lookup that misses
|
||
everything costs three cache misses where a compact dict costs two, and the hash run is a full eight bytes a slot.
|
||
Cell packing buys probe locality, which is a win while the hash run is resident and a loss once nothing is. One byte
|
||
of metadata a slot — the Swiss-table arrangement — is the known answer and is not built. Worth measuring before
|
||
building: the crossover is somewhere between 10k and 1M and nobody has found it.
|
||
- **What is left at 18ns cache-resident is the type erasure itself** — one non-inlinable call into the runtime and two
|
||
non-inlinable indirect calls to the hash and equality pair. That is the trade `spec-memory.md` chose on purpose, and
|
||
monomorphisation is what would buy it back. It is a reason to want generics, not a reason to regret the choice.
|
||
- **A fixed array of structs or of strings is not a map key**, which is narrower than `spec-memory.md`'s key set. It
|
||
needs the per-element walk a struct key gets, driven by a loop rather than a field list. Refused by name rather than
|
||
written untested; a struct holding the array works today.
|
||
- **Map removal is not built**, which is what keeps the implementation free of tombstones and of Odin's backward-shift
|
||
loop. The spec defers it deliberately. When it arrives, that loop is the cost.
|
||
|
||
- `spec-memory.md`'s "Open: catching a use-after-release statically" is still open, and it is now open with evidence
|
||
available for the first time: the epoch trap is built and `test/programs/stale-region.flan` is the case it catches.
|
||
What the spec says would settle it — real Flan programs using arenas, to show whether the escapes that actually occur
|
||
are lexical — is now *producible*, because there is a `Vec` to write them with. That is the next thing to look at,
|
||
not the next thing to build.
|
||
- ~~**The operation table may be one operation short.**~~ **Decided.** `free-all` is retain-capacity and
|
||
`arena-destroy` hands the pages back — two names rather than the mode parameter, so the table the spec froze at four
|
||
operations did not grow. `BUILT.md` states it as the amendment it is.
|
||
- **The `Vec` header is six words in release too, and should not stay that way.** The 32-byte layout the spec fixes is
|
||
blocked on one thing: a redefinition module is built by `llc` and `ld` against a host built separately, and nothing
|
||
makes the two agree on a struct size. Give the reload path a way to carry the build flags and this falls out.
|
||
- **The generation word has no reader.** It is bumped on every reallocation as specified, and the stale-slice trap it
|
||
exists for needs a slice that can carry the Vec's identity — a slice is ptr+len. Either slices grow a word in a dev
|
||
build or the trap does not exist; today it does not.
|
||
- **The allocator grew a budget** (`alloc-budget` / `set-alloc-budget`), which `spec-memory.md` does not have. It is
|
||
there because `retry` is only answerable by a handler that can make the *same* request succeed, and for a fixed
|
||
backing store that handler is the one that raises the ceiling — releasing the region the container lives in
|
||
invalidates the container. Worth folding into the spec or replacing with a growable arena.
|
||
- Escaping closures are still deferred (`spec-memory.md`, "Function values", case 3), and a user-written allocator is
|
||
not one — its procedure is a top-level `defn` with no captured environment. The two should not be conflated when
|
||
function values arrive.
|
||
|
||
### Bugs found and not yet fixed
|
||
|
||
- **Two citations in `spec-memory.md`'s Allocators section do not land where they say.** Checked against Odin
|
||
`819fdc7a8` and Carp `ea121b5a`, every other one is exact — `Map_Info` at `base/runtime/core.odin:369`,
|
||
`Allocator_Proc` at `:422`, the arena answering `.Free` with `.Mode_Not_Implemented` at
|
||
`core/mem/allocators.odin:307`–`308`, `#optional_allocator_error` on `append_elem` at
|
||
`base/runtime/core_builtin.odin:767`, and `// TODO(bill): Better error handling for failed reservation` at
|
||
`base/runtime/dynamic_array_internal.odin:107` and `:128`. The two that miss: `Map_Cell_Info` is at `core.odin:351`,
|
||
not `:350`; and `check.ml:1670` is the FFI `Declare` arm, not the `defer` registration — the claim it is offered for,
|
||
that a top-level `defer` is checked in a scope holding only parameters and globals, is true and lives in `check_fn`
|
||
at `check.ml:1800`–`1812`. `check.ml:505` is the `defer` refusal exactly as cited, and the Carp citations are right:
|
||
`getDropFunc` is `Memory.hs:804`, the drop-before-delete emit is `Emit.hs:1044`, and `docs/Drop.md` says outright
|
||
that `A.drop` "will be run ... when the `let` scope ends".
|
||
|
||
- ~~`web/examples/breakdemo.out` is stale and `check.sh` fails on it.~~ **Fixed.** Commit `4a6a8fa` made the break
|
||
banner number its restarts and the `.out` was never repinned. Nothing had to drive the socket in the end:
|
||
`check.sh` already builds this one `--dev` and runs it under `timeout 5`, keeping what it printed before it
|
||
stopped, so the repin was the `.out` plus the two prose copies of the banner — `web/index.html` and `BUILT.md` —
|
||
and a sentence on the page saying what the numbers are for, since a restart is taken by position.
|
||
|
||
- ~~A shadowed restart is offered and cannot be taken.~~ **Fixed.** A restart is taken by *position* now:
|
||
`(:op "restart-at" :index N :name NAME)` on the daemon, `restart-at N NAME` on the agent, and a numbered
|
||
`completing-read` in `C-c C-b`. `:name` is a receipt, not the lookup — it is checked against the name the snapshot
|
||
holds at that index and refused if the two have drifted, so a bare integer can be wrong out loud. `restart <name>`
|
||
survives for a raw socket and is now defined as `restart-at` on the first index offering the name, so the two verbs
|
||
cannot disagree. `break.flan` grew the shadowed pair and asserts 900, which is the only value in that file no by-name
|
||
lookup can produce. The C&R buffer still marks the shadowed row by name and could now offer it instead — small, and
|
||
not done here.
|
||
|
||
- ~~A restart chosen at a break inside a thunk is accepted, announced, and silently not taken.~~ **Fixed by refusing
|
||
it, with the reason.** Not by the depth NEXT.md proposed: recording the restart-stack depth on *entering the break
|
||
loop* counts the frames a `restart-case` inside the thunk pushed before it erred, and those are above the boundary
|
||
and work. The boundary is where it is made — `restart_floor` is set to `flan_restart_count()` around `j.call()` in
|
||
`flan_agent_poll`, saved and restored so thunks nest — and the outermost `floor` entries of the snapshot are marked
|
||
unreachable. They are listed and marked rather than hidden, refused by the listener before the reply, and carried to
|
||
the editor as `:unreachable (2 3)`. `test_dev.ml` breaks a stopped program a second time from inside `C-x C-e` and
|
||
asserts both halves: index 2 refused, index 0 taken.
|
||
|
||
- ~~Restart names are served from a stack that is being mutated.~~ **Fixed, and it was a precondition rather than a
|
||
separate bug.** Index-based resume is wrong by construction against a moving stack: unlike a name, an index carries
|
||
no evidence of what it meant. The agent copies the list on entering `break_loop` — names into its own buffer, frames
|
||
as the addresses a transfer carries — one snapshot per nested break, and every verb answers from it. Caps are
|
||
`SNAP_MAX` 64 restarts and `SNAP_NAMES` 4096 bytes; past either, the listing says how many it did not show. Neither
|
||
cap has a test; the 4K result cap that shared that blind spot now does.
|
||
|
||
- **A snapshot generation has no test, and the window is a race.** A choice is validated against the snapshot on top
|
||
when the request arrives and resolved against the snapshot on top when the game thread next looks. Between those,
|
||
an evaluation the break loop is running can error and push a break of its own, whose loop would otherwise reach
|
||
[chosen_ready] first and take *its* index 2 for the one someone chose from the outer list. Each snapshot now carries
|
||
a generation, a choice is stamped with the one it was validated against, and a loop claims only what is addressed to
|
||
it — a mismatch is left set rather than discarded, because the listener already answered ok for it. Depth would not
|
||
do: an outer break resuming and a new one starting reuses the number. None of this is tested, because arranging the
|
||
window means landing a request inside a two-millisecond poll from outside the process. It wants a hook the test can
|
||
drive, not a sleep.
|
||
|
||
- ~~The job ring has no fullness check.~~ **Fixed by refusing, at the sender.** Dropping loses a reload the sender was
|
||
told was ok; blocking stalls the accept loop, which serves connections inline, so a program that had stopped polling
|
||
would also stop answering `status` and `abort`. The refusal happens before the `dlopen`, so a module there is no room
|
||
for is never relocated and no handle is taken for it. `programs/agent-queue.flan` blocks on stdin so the window is
|
||
held open by the test rather than by a timer: 64 queued, the 65th refused with a reason, 64 installed when it finally
|
||
polls.
|
||
|
||
- ~~`flan_dev_result_get` is not the seqlock its comment claims.~~ **Fixed by making it one**, rather than by writing
|
||
the honest comment — what it guaranteed was nothing, and the daemon has no other way to read a result. The counter is
|
||
odd while a value is being written, `flan_dev_result_read` copies into the caller's buffer and checks the counter
|
||
either side of the copy, and a reader that loses the race reports the last complete generation and no bytes. The
|
||
count handed out is the number of complete values, so `lib/dev.ml`'s "has it moved" still means what it meant. The
|
||
race itself has no test, for the same reason the snapshot generation above has none.
|
||
|
||
- ~~Smaller: `exit(134)` from the break loop with the listener inside `dlopen`; a `dlopen` handle leaked when a module
|
||
has no installer.~~ **Both fixed.** `exit` runs the atexit chain and the ELF destructors, which want the loader lock
|
||
the listener may be holding — a program asked to abort would hang instead of dying; `_exit`, with the streams flushed
|
||
by hand. The leak was the handle *value* and not the mapping: a module with no installer published nothing, so
|
||
nothing can point into it, and it is closed. The deadlock is read rather than tested; the exit status is tested.
|
||
- **`rt_die` in `flan_rt.c` still calls `exit(134)`**, which is the shape just fixed in the break loop: a trap on the
|
||
game thread runs the atexit chain and the ELF destructors, which want the loader lock the agent's listener thread may
|
||
be holding inside `dlopen`, so a program that should die could hang. Found while fixing the break loop and not fixed
|
||
with it — `rt_die` is the non-dev path too, where there is no listener and nothing to deadlock against, so whether it
|
||
should be `_exit` unconditionally or only under `--dev` is a decision rather than a typo.
|
||
|
||
- ~~**`(A {.x 1})` on a union variant says "unknown struct A"**~~ **Fixed** with union values. `env` now carries a
|
||
case table keyed both by the full spelling `U.C`, which is how a value of it is written, and by the bare `C`, which
|
||
is how a mistake spells a constructor; the bare entry exists only to say *"A is a case of the union U, not a struct
|
||
— a union value names both, as `(U.A {.field value ...})`"*. Two unions may share a case name and that is not
|
||
refused: construction is qualified and a pattern resolves against the scrutinee, so both are unambiguous.
|
||
|
||
### Test blind spots, from a mutation pass
|
||
|
||
Sixty mutations, nineteen left the whole suite green. The severe cluster was closed first (`cleanup.flan`,
|
||
`signedness.flan`); the rest are closed now. Every one below was re-planted, watched leave the suite green, and then
|
||
watched fail against the new test before the mutation was reverted — a test nobody saw fail is not evidence.
|
||
|
||
- **`Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies.** `programs/reach-walk.flan`
|
||
calls three functions from three places that are each the only route to them. The failure is not a wrong answer:
|
||
the function is not emitted and the program stops linking, so the case catches the build exception rather than
|
||
comparing output. The `addr` case goes through a `deref` place deliberately, so the index case cannot stand in for
|
||
it.
|
||
- **`flan_dev_global`'s size-change guard.** `programs/reload-v5.flan` is v4 with `extra` as an `i32`, loaded on top
|
||
of v3 in a host run of its own, because what it does is abort. The message is asserted next to the exit status: a
|
||
process that died for another reason is not this guard firing.
|
||
- **A local shadowing an imported name.** `programs/shadow-pkg.flan` binds locals over its own constant and var;
|
||
`pkg-shadow.flan` prints four numbers that separate the expression renamer from the place renamer. Nothing refuses
|
||
a renamer that qualifies through a binding — it reads the top-level name instead and runs — so only the number says
|
||
so.
|
||
- **The 4K result cap and the registry overflow guard.** `test/dev_limits.c`, a second C main beside `reload_host.c`,
|
||
drives them directly: neither has a Flan spelling and no corpus program reaches either. One process per mode — the
|
||
name table never shrinks and the overflow case aborts.
|
||
- **The reader's unknown string escape, and `+5`.** Rows in the reader table, with the escapes it *does* know
|
||
asserted on their decoded bytes rather than through `Form.to_string`, which escapes them again and would compare
|
||
the source with itself.
|
||
- **The hang.** A reader branch that forgets to advance loops for ever, and `dune test` waits as long as it is left
|
||
to; in CI that is a job the runner kills with nothing named. `test/watchdog.ml` arms an alarm on every test binary
|
||
— generous, because an alarm that fires on a slow machine is a flake and a flake is how a watchdog gets deleted —
|
||
and a five-second one around every read in `test_flan`. The first read that does not return wedges the rest, so a
|
||
looping reader costs five seconds and names the row instead of never finishing.
|
||
|
||
What is still open here: the mutation pass has not been re-run since, so the count of nineteen is the old one. The
|
||
sanitized sweep (`@sanitize`) is under the same watchdog but has never been observed to fire it, and so is the
|
||
memcheck sweep (`@valgrind`), whose alarm is looser at 5400s because memcheck is 20-50x on execution.
|
||
|
||
### Asked for by the editor lanes
|
||
|
||
- **`(:op "condition")` → the stopped program's condition, rendered.** Two steps: `break_loop` currently does
|
||
`(void)condition;` and *discards the pointer*, so stash it beside `condition_name`; then the daemon builds a render
|
||
thunk aimed at that address, which is `Session.render` rooted at a `Ptr` instead of an expression. **The second step
|
||
now exists** — `Session.render_locals` is exactly that thunk, rooted at an address the program supplies — so what is
|
||
left is the first: keep the pointer, and give the agent a verb that hands it back. The type is already known: it is
|
||
the `condition_name` the break loop reports, which `layout` already resolves.
|
||
- **The type identity is settled, and it is the qualified name** — `layout` is in, see BUILT.md. `Load` qualifies
|
||
every declaration at import, so the names in `Tast.structs` are a flat namespace where two packages' `Missing` are
|
||
`a/Missing` and `b/Missing`; a bare name is refused with the candidates rather than resolved. `condition` inherits
|
||
it for free: the string the break loop already reports *is* that name, because `Emit.struct_name_of` writes
|
||
`Types.Named` into `flan_error`. It is still open for **locals**, where DWARF gives a name and the name a debugger
|
||
reads is not qualified by anything.
|
||
- ~~**`(:op "backtrace")` is blocked** on frame metadata.~~ **Built**, and not out of DWARF: decision 3's shadow
|
||
stack carries the name and the location on the frame itself, so a backtrace needs no debug information at all. See
|
||
BUILT.md, "The shadow stack, and `backtrace`", for what it costs. **Locals landed with it** — the pointer-rooted
|
||
render thunk turned out to be `Render.render` over a `Deref` of a slot's address, and one new arm in the backend.
|
||
See "Locals of a stopped frame" for the four things it refuses. Restart source locations and arity are still blocked — `flan_restart` carries `prev`, `name_id`, `name` and `namelen`, so both need a new
|
||
field in the frame, which means the compiler emitting it.
|
||
|
||
### One line away
|
||
|
||
- **`match` over enums.** Fully desugarable, wanted, and blocked only by `Ast.pattern` needing a keyword case, which
|
||
`load.ml` matches exhaustively.
|
||
- **`Build.executable` returns only `out`**, so the daemon recovers the host `.ll` by recomputing `Build.workdir ()`.
|
||
- **A `!DILexicalBlock` per `Let`.** Not one line, but the one thing left in the DWARF work: every `!DILocalVariable`
|
||
is currently scoped to the subprogram, so inside `(let [v 22] …)` nested in `(let [v 11] …)` lldb still answers
|
||
`p v` with **11**. The `~2` suffix makes both *visible*, which is not the same as making the answer right. It needs
|
||
block structure the typed IR does not carry, and the `llvm.dbg.declare`s moved out of the entry block.
|
||
|
||
### Deferred with a reason
|
||
|
||
- **Writing through a string literal** — see Sharp edges. Needs provenance, which is open decision #3.
|
||
- **`cstring` as a type.** Odin has no `string → cstring` conversion at all; it pays the same copy our shim already
|
||
makes. The one thing it buys is the *return* direction, and nothing in `vendor/raylib` returns a string.
|
||
- **`rune`.** Odin's is a 4-byte integer distinguished by a flag, so `i32` is the same thing. Non-ASCII text is
|
||
blocked on font loading, not on the string layer — and fonts are now bound.
|
||
- **Macro expansion.** The reader and the declaration are in. Running a macro means compiling it and `dlopen`ing it
|
||
into the compiler, which is the reload primitive pointed at ourselves — but a macro is `[Form] -> Form`, so `Form`
|
||
has to be a Flan union whose layout the compiler and the loaded macro agree on, and union *values* are milestone 6.
|
||
|
||
### Documents that contradict the code
|
||
|
||
- **`plan.org`'s jank #947 citation is wrong in its mechanism.** jank does not relink (it calls through vars, which
|
||
are already indirection cells) and never unloads (`remove_symbol` has no callers). The real cause was a
|
||
process-teardown race. We are safe from the repro — because we compile out of process, not because of cells. A
|
||
normative document citing the wrong mechanism protects the wrong invariant.
|
||
- **`plan.org` still lists open decision #7 as open** and the interpreter as a backend. It was settled the other way;
|
||
`NEXT.md` records the consequences as "already applied" to `plan.org`, and they never were.
|
||
- **nREPL's `eval` does carry `file`, `line` and `column`** — jank reads all three. The choice of s-expressions still
|
||
stands on its other grounds; the stated reason does not.
|
||
|
||
## Sharp edges
|
||
|
||
- **Two formatted numbers cannot be held at once.** `flan_i64_to_bytes`, `flan_f64_to_bytes` and `flan_u64_to_bytes`
|
||
all write into one `static char scratch[64]` — "rendered text lives here until the next call", flan_rt.c:184 — and
|
||
`(string b)` does not copy. So
|
||
|
||
```
|
||
(let [a (string (i64->bytes 11))
|
||
b (string (i64->bytes 22))]
|
||
(print a) (print " ") (println b)) ; => 22 22
|
||
```
|
||
|
||
`a` is 11 and prints 22. No crash and no diagnostic. This is not new — the `[u8]` already aliased — but a `string`
|
||
reads as more value-like and invites exactly this. Format, draw, measure, then format the next one; `digits.flan`
|
||
sequences itself strictly for this reason. `rl/draw-text` is safe because the shim's `flan_shim_cstr` copies out of
|
||
ptr+len before the call.
|
||
|
||
**The prelude now has the shape that does not have this problem**, and it is the reason that shape exists.
|
||
`append-i64!` and `append-f64!` copy out of the scratch buffer into a `(Vec u8)` before returning, so a builder
|
||
holds as many rendered numbers as it likes, and `format-f64` answers a `Vec` rather than a view. The hazard is
|
||
unchanged for anyone calling `i64->bytes` directly — nothing was taken away — but a caller assembling a line of
|
||
text has a way not to meet it.
|
||
|
||
- **Writing through a string literal is undefined, and the two build modes
|
||
disagree about how.** `(let [s (bytes "Hi")] (set (at s 0) \h))` stores into
|
||
a `private unnamed_addr constant`. At `-O0` that is a store to read-only
|
||
memory and the program takes SIGSEGV; at `-O2` LLVM deletes it as undefined
|
||
and the program prints `Hi` and exits 0. Same source, and which way it fails
|
||
depends on a flag — the worst shape available, and worse than either outcome
|
||
alone.
|
||
|
||
Nothing refuses it. `bytes` turns a `string` into a `[u8]`, the language lets
|
||
you write through a slice, and by then nothing records that the bytes came
|
||
from a constant. The honest fix is provenance — knowing a slice's origin —
|
||
which is plan.org open decision #3 and deliberately deferred. A cheaper one
|
||
that is *not* a fix: emitting literals as mutable globals only moves which
|
||
flag misbehaves, and costs their read-only placement.
|
||
|
||
Found by the string lane while deciding whether `lower-ascii` should mutate
|
||
in place. It ships the copying version for exactly this reason, and that is
|
||
the rule to follow until provenance exists: **a function over a `string` must
|
||
not write through it.**
|
||
|
||
|
||
Most of these are edges the language keeps and you should know about. Two — the top-level namespace and the shift count,
|
||
both found by review after milestone 4 — were bugs that reached LLVM or ran wrong, and are **fixed**; each says so. They
|
||
stay written down because each one is now a rule the checker enforces, and a later change could quietly drop it.
|
||
|
||
- **An index converts from a narrower integer and never from a wider one.** `(at colors current-color)` with a `u32`
|
||
index works — anything above 2³¹ truncates to a negative `i32` and the unsigned bounds check rejects it. An `i64` index
|
||
is refused with the reason: 2³²+5 truncates to 5 and would read the wrong element with no trap at all.
|
||
- **There is one top-level namespace, and `check.ml` now enforces it.** The environment's tables are per-kind — structs,
|
||
unions, aliases, enums, functions, externs and globals each have their own — so only a function was ever checked for a
|
||
duplicate. `(defn item …)` beside `(defvar item …)` type checked and then died in LLVM as `redefinition of function
|
||
'@flan.item'`, a message about an emitted symbol with no source location left, and two colliding *type* declarations
|
||
were not caught anywhere. One pass over `Ast.declared_name` now runs before every other collection pass and rejects the
|
||
second declaration of a name whatever kind either one is. `declared_name` lives in `ast.ml` because `Load` needs exactly
|
||
the same set — the names an import renames — and two copies of that list would drift.
|
||
- **A shift count is bounded, two different ways.** A shift by the operand's own width or more is *poison* in LLVM, not
|
||
a wrong number: `(defn main [] i32 (<< 1 32))` compiled at -O2 to a bare `retq`, returning an undefined value. A literal
|
||
count out of range is now rejected in `check.ml` — that is the typo case — and `emit.ml` masks a computed count to
|
||
`width - 1`, which is what the hardware does anyway and which LLVM folds away whenever the count is constant. The
|
||
prelude's rotate masks its own count; that is now redundant but harmless.
|
||
- **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real `u64` and not an error. The cost is that
|
||
a negative *decimal* literal is accepted as a `u64` too, because the reader records the value and not how it was
|
||
written. Narrower unsigned types keep the strict check, which is where a typo like `300` for a `u8` actually shows up.
|
||
- **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted from the folding pass's value, because a
|
||
global's initialiser has to be a compile-time constant and only that pass knows this one is. Its range check is
|
||
therefore its own call to `in_range`; there is a regression test.
|
||
- A `let` binding takes no type annotation, which is why `sand.flan` names its FNV constants instead of writing them
|
||
inline.
|
||
- `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean f64*: with a single body form the parser
|
||
cannot tell a return type from the first expression. Only the parameter position and `(Option …)` are unambiguous.
|
||
|
||
## Loose ends from milestone 4
|
||
|
||
None of them blocking: block-scoped `defer`; package visibility, so `rl/get-color-raw` is not callable; imported
|
||
unions.
|
||
|
||
A package importing a package was on this list and is off it. It loads, a diamond shares one copy of the bottom
|
||
package, the alias clash is refused through a chain as well as inside one file, and a ring is refused by name. What is
|
||
left of the item is visibility, which is listed above and needs a marker the parser does not have.
|
||
|
||
## Macros — landed; what is left of them
|
||
|
||
**The expander works and `unless` is a prelude `defmacro`.** How all of it fits together is in
|
||
[`BUILT.md`](BUILT.md), "Macros: the compiler dlopens the program" — the image format, the thunk ABI, why quasiquote
|
||
runs before the walk, the two different ways expansion fails to terminate, `-linkall`, and the three cost numbers.
|
||
What follows is only the part that is still missing.
|
||
|
||
- **Four special forms left**, and two of them are the hard ones. `until` and `cond` are free to move whenever
|
||
somebody wants them. `when` and `dotimes` are not: the prelude itself uses them 29 and 12 times, so moving either
|
||
makes the prelude depend on the macro that the macro module has to compile the prelude to get. Breaking that needs
|
||
either a prelude that stops using them, or a two-stage prelude where the macro module is built from a subset. The
|
||
first is a mechanical edit of `prelude.ml` and is probably the answer.
|
||
|
||
`cond` has its own snag, and it is the reason `unless` went first: `parse.ml` refuses `(cond a)` with "cond clause
|
||
has no body", and a macro cannot produce that (see the next item), so moving `cond` changes an existing test.
|
||
|
||
- **A macro has no error facility**, and this is the biggest gap. A macro runs inside the compiler; anything it
|
||
signals aborts the compile with no location. So the prelude's `unless` answers `(unless-takes-a-test-and-a-body)`
|
||
when it is handed fewer than two forms, and the report is "unknown name unless-takes-a-test-and-a-body" at the call
|
||
site — right place, wrong sentence. What a macro wants is a way to say *this is wrong and here is why*, reported at
|
||
the call site. The queued structured-error rewrite is where that belongs, and the call site's `Loc.t` is already
|
||
stamped onto everything a macro returns, so the location half is done.
|
||
|
||
- **Macros are not imported.** A `defmacro` in a package is refused by name in `load.ml`. Reaching one would mean
|
||
resolving that package's own imports over `Form`s, before `Load` runs — a second import resolver. `programs/pkg-macro.flan`.
|
||
|
||
- **A prelude macro may not call a macro.** The prelude is in every macro module by construction, so there is no
|
||
round it could be compiled in after something else. It would fail with an unknown name rather than with a reason,
|
||
which is worth fixing the day the prelude wants one.
|
||
|
||
- **A prelude *function* may not call a prelude macro either**, which is the neighbouring gap and was found by
|
||
walking into it. `Macro.program` runs over the file being compiled; the prelude arrives at the checker through
|
||
`Check.program`'s own prepend and is never handed to the expander at all. A `defmacro` is an ordinary `defn` taking
|
||
one `[Form]` by the time the checker sees it, so the call resolves to that and the report is "clamp takes 1
|
||
argument, given 3" — pointing at the prelude, about a call the author wrote as a macro use. `format-f64` writes
|
||
`(min 9 (max 0 prec))` in place of `(clamp prec 0 9)` because of it. The fix is not obviously cheap: expanding the
|
||
prelude means building a macro module to compile the prelude that the macro module is built from, which is the same
|
||
bootstrap the `when`/`dotimes` item above describes.
|
||
|
||
- **A quasiquote inside a quasiquote is refused.** Nothing counts nesting levels — not the reader, deliberately, and
|
||
not the desugaring. Only a macro that writes a macro wants one.
|
||
|
||
- **`gensym`'s counter restarts in a second module.** It lives in the loaded module, and a module is dlopened once per
|
||
compiler process, so it is process-wide in practice. The rounds already build more than one module for a program
|
||
whose macros call macros, and the fix that day is to seed the counter from the module's index.
|
||
|
||
- **The macro programs are not in the sanitizer sweep.** `test_sanitize.ml` runs an explicit list, not a glob, so
|
||
`macros.flan` and `macro-unless.flan` were not added to it by landing them. `dune build --root . @sanitize` is
|
||
clean as it stands; adding the two is a one-line edit in a file this lane did not own.
|
||
|
||
- **No `&rest` sugar.** A macro takes one parameter, the slice of forms at its call site, and `(len args)` is the
|
||
arity. That is deliberate — it is where variadics come from — but a `when` written against it reads worse than
|
||
`parse.ml`'s version did.
|
||
|
||
## Watch for
|
||
|
||
The rule that caught the two misparse bugs applies unchanged: **anything that binds a name, alters control flow, or is
|
||
not yet implemented must be recognised explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`,
|
||
`Result`/`try`, union values, closures, quoted symbols, generics and function values *by name*, each with the milestone
|
||
it belongs to; `load.ml` rejects the package shapes it does not handle; and the FFI boundary rejects an aggregate. The
|
||
tests assert on the reason, not just on the failure.
|
||
|
||
## Untracked on purpose
|
||
|
||
`calc-me` and `sand`, the executables `flan build` drops beside their sources, are now in `.gitignore` — anchored
|
||
(`/calc-me`, `/sand`) so the patterns cannot match anything nested.
|
||
|
||
`old-ocaml/` — the pre-rewrite menhir/ocamllex frontend, kept as reference and excluded from the build by the root
|
||
`dune` file. Its contents are also in git history at `2c232dd`.
|
||
|
||
## Handoff: the shadow stack lane, stopped mid-repair
|
||
|
||
Two commits landed and are green: the shadow stack with `(:op "backtrace")`, and `(:op "locals" :frame N)`. See
|
||
BUILT.md's two new sections for the design and the measurements. A third commit was **half-built and its own test
|
||
left red** on purpose; **it is finished now** — see the struck item 1 above — and the rest of this section is kept
|
||
because the parts of it that were true are still worth having.
|
||
|
||
**What is broken, exactly — and this paragraph was wrong; kept for what it cost.** It said `locals` compares the
|
||
frame on the stack against the body this session holds and the comparison is not firing, that every piece of the
|
||
fingerprint was written, and that one of five hand-offs was dropping the number. Four of the five were never written at
|
||
all: `Emit.fninfo` stored the fingerprint and nothing else touched it. The first step it recommended — printing both
|
||
sides of the comparison in `Dev.locals` — could not have worked, because `Dev.locals` had no comparison to print. The
|
||
lesson is the ordinary one: a lane that stops mid-repair should say which pieces it *ran*, not which it believes it
|
||
wrote.
|
||
|
||
**Not obvious from the diff.** Two things cost a day between them. The linked-list frame beat an array-with-a-stack-
|
||
pointer on both benchmarks, which is the opposite of what the escaping-alloca argument predicts, and the measurement
|
||
that first said otherwise was comparing a 40-frame binary with a 600-frame one; every number in BUILT.md is now a
|
||
minimum of nine runs for that reason. And `redefinition`'s transient rule (`m.nstr = 0`) silently stops every module
|
||
carrying a string literal from ever being unloaded — the frame descriptors go through their own counter, `m.nfi`, for
|
||
that reason, and a locals thunk passes `~retains:false` because everything it emits is memcpy'd into the result buffer.
|
||
|
||
**No Emacs surface.** `backtrace` and `locals` are daemon ops; nothing in `emacs/` calls them yet. One command showing
|
||
the backtrace with the selected frame's locals is the whole of what is missing, and `flan-cnr.el`'s
|
||
fixture-driven shape is the model.
|