1210 lines
96 KiB
Markdown
1210 lines
96 KiB
Markdown
# Where this is
|
||
|
||
## Start here — next session
|
||
|
||
**Branch `dev-loop`, 199 commits, working tree clean, `dune test` green.**
|
||
|
||
**[`DISCUSS.md`](DISCUSS.md) is what has been *asked* and not answered** — open questions with the repo context that
|
||
bears on each, so an investigation starts from what exists. Nothing in it is a decision or a task; when one becomes
|
||
either, it moves here.
|
||
|
||
**`NEXT.md` is what is left. [`BUILT.md`](BUILT.md) is why the existing parts are the shape they are** — the reload
|
||
primitive, cells, the agent, the session, the daemon, the Emacs client, conditions, the FFI shim, the layout, and the
|
||
order it all got built in. This file was half build log until it was split; do not let it become one again. When a
|
||
track here finishes, its explanation moves there and its entry here goes away.
|
||
|
||
The dev loop works end to end: `flan dev program.flan`, then `C-c C-c`, `C-x C-e` and `C-c C-r` in Emacs against the
|
||
running process.
|
||
|
||
**Conditions are three steps of four.** `(error c)` is the diverging variant — a handler that returns normally has not
|
||
answered it, so only a transfer gets past. The break loop is in, editor half included: an unhandled `error` stops the
|
||
program on the frame that erred, the daemon annotates every reply with `:stopped`/`:condition`, and `C-c C-b` lists the
|
||
restarts and resumes into the choice. A restart is chosen **by position**, off a snapshot taken when the break was
|
||
entered, because a name resolves to the innermost frame offering it and the stopped thread's stack does not hold still.
|
||
Restarts below the evaluation a break is inside are listed, marked, and refused with the reason.
|
||
|
||
**Restarts take parameters now** — §3's other half. `(use-value [v i32] ...)` binds them, `(invoke-restart 'use-value 21)`
|
||
supplies them, and what a clause takes against what was given is checked at run time and refused with both spellings,
|
||
because a restart is found by name on a dynamic stack and neither end of a transfer can see the other. The one path
|
||
that cannot yet supply a value is the break loop, which is item 2 below and is where the interesting half is.
|
||
|
||
Still open: **`handler-case`**, which §"What this does not settle" leaves open as possibly a macro over `handler-bind`
|
||
plus a transfer. **`find-restart` and `compute-restarts` are blocked on a type, not on effort** — §4 gives them
|
||
`(Option Restart)` and a list, and there is no `Restart` type and no list to return one in. The minibuffer prompt never
|
||
needed them; it reads the snapshot over the agent's socket. And a `restart-case` clause should still carry a **report
|
||
string**: `use-placeholder` is what `invoke-restart` needs, not what a person reading a list needs. §3 says to settle
|
||
that *before* parameters and it was not settled — the field is cheap and the accessor is cheap, but the only consumer
|
||
is the break loop's listing, which lives in the agent and the daemon, so it would have shipped as a field nothing
|
||
read. It belongs with item 2, where the listing is being changed anyway.
|
||
|
||
Read SBCL for what restarts should *mean* and ignore how it moves control: it transfers with `block`/`return-from`,
|
||
which §6 rules out.
|
||
|
||
### Landed 2026-09-12 — six tracks, one session
|
||
|
||
Six agents in parallel worktrees. Kept short on purpose; the reasoning that outlives the change is in `BUILT.md` or in
|
||
the commit that made it.
|
||
|
||
1. **`nth` removed**, an alias of `at` that was asymmetric — `check.ml` aliased them but `parse.ml` and
|
||
`place_of_expr` matched only `at`, so `(set (nth a i) x)` and `(addr (nth a i))` were refused while the `at` forms
|
||
worked.
|
||
2. **`println` and `print`**, compiler-provided and structural. `Session.render` was already the compile-time walk
|
||
plan.org asks for; it moved to `lib/render.ml` parameterised on an emitter and a slot allocator, so the REPL and
|
||
stdout share one copy. Found doing it: `field_addr` in `emit.ml` accepted only `Types.Named`, so a field of an
|
||
`Option` threw at emit time and **the walk's Option arm had never run** — the inspector would have failed on the
|
||
first `(Option T)` pointed at it. prelude.ml's claim that this had to wait for milestone 5 and generics was wrong,
|
||
and is gone: a printer selected per *concrete* type has nothing to dispatch on and no type variable in it.
|
||
3. **`restart-at`** — a restart is taken by position now. See "Start here".
|
||
4. **Names in DWARF.** `Tast.fn` carries `snames` beside `slots`, so a let-bound local is its own name under lldb
|
||
instead of `s0`; a slot the compiler invented keeps `s<index>`, because inventing a name puts a variable in the
|
||
debugger that is not in the file. Shadowing had to be decided rather than assumed: every `!DILocalVariable` is
|
||
scoped to the subprogram — the typed IR has no block structure to build a `!DILexicalBlock` from — so two slots
|
||
called `v` left lldb answering `p v` with the outer one while the body computed with the inner, and not listing the
|
||
inner at all. A repeat gets a `~2` suffix, unspellable in source. That is a way of not lying rather than a way of
|
||
being right; see "One line away". Also `flan dev --debug`, one flag for host and every redefinition module, off by
|
||
default because a debug build is an `-O0` build.
|
||
5. **Ten raylib core examples** in `examples/`, plus seven bindings and the colour palette. The gap list they produced
|
||
is under "Unblocked now, and ranked"; the top item, that no number could reach `draw-text`, is fixed — `(string b)`
|
||
reinterprets a `[u8]` as a `string`, which costs no instructions because they are already the same 16 bytes.
|
||
|
||
6. **The `print-*` family is gone.** `print` and `println` are the whole printing surface; ~500 call sites across 47
|
||
files rewrote, and `web/index.html` gained a `#printing` section, the first documentation either has had. Two pinned
|
||
outputs moved and both are corrections: `sand-headless`'s hash is `15595743031174623232` rather than
|
||
`-2851001042534928384` — the same 64 bits, printed unsigned now that `hash-grid`'s `u64` no longer goes through an
|
||
`(i64 …)` cast — and a trap column shifted because the call it names got shorter.
|
||
|
||
### Landed — the allocator, the arena, `(Vec T)`, `StorageExhausted`
|
||
|
||
The critical path, and the thing NEXT.md said was the only one standing between this and writing a game. Steps 1, 2 and
|
||
3 of the build order below are struck; `Map` is step 4 and is untouched. `Allocator` is a builtin opaque type and
|
||
needed nothing from milestone 5, which was the whole bet. Three amendments to a **frozen** `spec-memory.md`, made
|
||
deliberately and stated as amendments in [`BUILT.md`](BUILT.md): `free-all` is retain-capacity with `arena-destroy`
|
||
beside it; `context/allocator` is a dynamic variable rather than a literal calling-convention parameter; and the `Vec`
|
||
header is six words in every build rather than four in release. One addition the spec does not have: a budget on the
|
||
allocator, because `retry` needs a handler that can make the *same* request succeed.
|
||
|
||
### Landed — the runtime under a sanitizer
|
||
|
||
`--sanitize` is a build flag beside `--debug`; `dune build --root . @sanitize` builds twenty-eight programs twice, plain
|
||
and sanitized, and compares output and exit status. Its own alias and not `dune test`, because the sweep is about nine
|
||
minutes. The checked sweep is **clean**. How ASan and UBSan reach a language whose IR is written by hand, and why the
|
||
flag does not force `-O0` when `--debug` does, is in [`BUILT.md`](BUILT.md).
|
||
|
||
Two defects came out of it, both found by reading rather than by the tools, both fixed with a regression case:
|
||
`flan_bytes_to_i64`/`flan_bytes_to_f64` clamped a slice length with `(size_t)n` and so read 63 or 511 bytes off the end
|
||
of a negative-length slice; and the three `snprintf` shims published snprintf's return as a slice length, which is what
|
||
it *would* have written.
|
||
|
||
**What is left, and it is most of what the sweep was meant to settle:**
|
||
|
||
1. **UBSan sees no Flan code and no flag changes that.** Its checks are branches clang's C frontend emits inline, not a
|
||
pass, so shift UB (`(<< 1 32)`, see Sharp edges), alignment, and the f32→i32 cast on NaN or an infinity — the things
|
||
`floor-f32` guards by hand and nothing else does — are unreached. Either `Emit` grows those checks behind the flag,
|
||
which is a compiler feature of the same shape the bounds checks already have, or they belong to the checker. Not
|
||
decided. `test_sanitize` pins the current answer with a control that must *not* report, so a future clang changing
|
||
this is a test failure rather than a discovery.
|
||
2. **Three of the four named buffers now have evidence; one still does not.** Two lanes closed different pairs and
|
||
they combine. The 4K result cap and `condition_name[128]` are driven over the agent's socket from `test_agent.ml`
|
||
— a 5000-byte value comes back as 4096 ending in the ellipsis, a 198-character condition class comes back from
|
||
`status` as 127. The 4K cap and the **dev registry overflow guard** are also run directly by `test/dev_limits.c`,
|
||
a C main beside `reload_host.c`, one process per limit because the name table never shrinks and the overflow case
|
||
aborts. Only `SNAP_MAX`/`SNAP_NAMES` is still read rather than tested: sixty-five nested `restart-case`s are a lot
|
||
of program for a clamp. `escaped[ESCAPE_MAX]` was already covered, because `println.flan`
|
||
drives a 1100-character string through it on purpose — 1019 bytes out against a worst case of 1021 into 1024.
|
||
`scratch[SCRATCH]` never sees more than 20 characters of 64.
|
||
3. **Valgrind over the headless corpus, not done.** ASan does not see uninitialised reads, which is where `zeroed` and
|
||
struct padding live. MSan is out: it needs every dependency instrumented and raylib settles that.
|
||
|
||
Two things the sweep structurally cannot cover: raylib and libm are uninstrumented, so the windowed examples are noise;
|
||
and a redefinition module is built by `llc` and `ld` rather than clang, so the reload path carries no instrumentation
|
||
whatever the flag says.
|
||
|
||
### Managed classes are planned. Do not start them.
|
||
|
||
plan.org grew a `class` facility beside `struct`: identity, runtime shape metadata, an implementation-defined
|
||
representation, generic-function dispatch, and live schema change with an explicit migration at a frame boundary. Its
|
||
own last line is the rule — nothing until ordinary `struct`, `Handle` and reload semantics are working. It is here so
|
||
that a session reading plan.org cold does not take it as the next task. Three things found while reviewing it, none of
|
||
them in plan.org yet:
|
||
|
||
- **A generic function is a cell.** "A later module can add `(defmethod draw ((e Enemy)) ...)` without editing the
|
||
original" means every compiled call site of `draw` has to find the new method — which is the problem the indirection
|
||
cells already solve. A generic function is a cell whose body is a dispatch table and a reload extends the table. The
|
||
expensive half of classes is therefore already built and tested.
|
||
- **The pool is not one storage option among three.** `migrate-instances` has to *enumerate* live instances. A pool
|
||
behind generational `(Handle T)` gives that by construction; a world arena and an owned region do not obviously.
|
||
plan.org presents the three as a free choice and they are not.
|
||
- **`Enemy@1` has to stay resolvable** for `migrate` to dispatch on it, so the session retains every layout version's
|
||
metadata for as long as any instance holds it. Same rule as "nothing is ever `dlclose`d", and worth stating as one.
|
||
|
||
### Open: can a condition be a class?
|
||
|
||
Unanswered, and it wants answering before `handler-case`, because it decides whether handler matching has one path or
|
||
two.
|
||
|
||
It would buy the thing conditions most lack: a **hierarchy**. §1 says flatly there is none, which is why nothing can say
|
||
"any condition" — no catch-all handler and nothing for a break loop to match on. Class inheritance gives it.
|
||
|
||
Three costs, one serious:
|
||
|
||
- **Signalling would allocate.** A struct condition is a stack value and `signal` takes its address; a class instance
|
||
needs a pool slot at the signal site. That is the failure path, sometimes the hot path, and sometimes the thing that
|
||
failed is allocation itself. plan.org also says no implicit allocation anywhere in the core.
|
||
- **§5's lifetime inverts.** Today the condition dies with the signalling frame and a handler that keeps it copies,
|
||
which is free for a value struct. A class instance survives the transfer — nicer, but now something owns and frees it.
|
||
- **Layout versions meet handler frames.** A struct condition cannot change layout; it is refused. A class can, and then
|
||
a frame pushed against `MyError@1` is on the stack while the signaller builds `MyError@2`.
|
||
|
||
The shape that probably wins is both: a struct condition stays exactly what it is — no allocation, matched by name hash,
|
||
dies with the frame — and a class condition is allocated, survives, and matches by walking its class chain. That is two
|
||
matching paths, which is the same bill the struct/class split already signs, so it is consistent rather than a new cost.
|
||
Either way it is an amendment to a **frozen** `spec-conditions.md`, not a gap in it.
|
||
|
||
|
||
**The dev loop is closed.** `C-c C-c` in Emacs recompiles the top-level form at point and installs it in a running
|
||
program, at that program's next frame boundary. Verified against sand: an unsaved buffer edit to `game-draw`, and 240
|
||
consecutive frames drew it.
|
||
|
||
Steps 1, 2 and 3 are done — see *The reload primitive* in `BUILT.md`. A list of top-level forms can be recompiled and installed
|
||
into a running process; call sites compiled before they existed follow them, and a `defn` or `defvar` the process was
|
||
never built with can be added and then redefined again. That is the whole of `C-c C-c`, minus an editor: sand.flan takes
|
||
a redefinition over a socket and installs it between frames.
|
||
|
||
What is left is the *session* — something that holds the checker environment between evaluations, tracks which names the
|
||
running process was built with, and speaks a protocol an editor can talk to.
|
||
|
||
Milestone 4 is done: **sand.flan builds, links raylib and runs**, and its simulation has a headless acceptance case that
|
||
runs on the `dune test` path at `-O0` and `-O2`. Milestones 2 and 3 are behind it (`calc-me.flan` compiles and runs; the
|
||
interpreter was dropped — open decision #7, settled — see "Why there is no interpreter" in `BUILT.md`).
|
||
|
||
```
|
||
reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅
|
||
```
|
||
|
||
| File | What it does |
|
||
|---|---|
|
||
| `lib/loc.ml` | source locations + `Loc.Error`, the frontend's one exception |
|
||
| `lib/form.ml` | reader output: `Sym Kw Int Float Str Byte List Vec Map` |
|
||
| `lib/reader.ml` | hand-written S-expression reader, no menhir/ocamllex |
|
||
| `lib/ast.ml` | AST: `texpr`, `expr`, `place`, `pattern`, `decl` |
|
||
| `lib/parse.ml` | forms → AST; special forms, desugaring, declarations |
|
||
| `lib/load.ml` | **imports: a package directory → qualified declarations** |
|
||
| `lib/types.ml` | resolved types; structural equality, `Never` fits anywhere |
|
||
| `lib/tast.ml` | the typed IR the backend consumes |
|
||
| `lib/check.ml` | AST → typed IR; two passes, bidirectional |
|
||
| `lib/session.ml` | **a live program: what the process was built from, plus every change since** |
|
||
| `lib/wire.ml` | **the editor protocol: one s-expression per message, length framed** |
|
||
| `lib/dev.ml` | **`flan dev`: a session, the program running beside it, and a socket** |
|
||
| `lib/prelude.ml` | printers + `rand-f32`, written in Flan |
|
||
| `lib/emit.ml` | typed IR → LLVM IR text |
|
||
| `lib/build.ml` | `.ll` + the shim + the packages' C → clang → executable |
|
||
| `runtime/flan_rt.c` | the host ABI: argv, stdout, exit, 4 conversions |
|
||
| `runtime/flan_dev.c` | **dev only: the by-name registry a run-time-new name needs** |
|
||
| `lib/shim.ml` | **`declare-c` -> the generated C that flattens a struct crossing** |
|
||
| `vendor/raylib/` | **the raylib package: `raylib.flan` and `link`, and no C at all** |
|
||
| `vendor/agent/` | **the dev agent: a socket, a loader thread, install at a frame boundary** |
|
||
| `emacs/` | **`flan-mode.el`, `flan-dev.el`, `flan-repl.el`: the editor half of the dev loop** |
|
||
| `bin/main.ml` | `flan read \| parse \| check \| emit \| shim \| build \| run \| reload \| dev` |
|
||
| `test/test_flan.ml` | reader, parser and checker |
|
||
| `test/test_acceptance.ml` | expression/result pairs + whole programs + the traps |
|
||
| `test/test_reload.ml` | **the reload primitive: recompile one function, load it, call it** |
|
||
| `test/test_agent.ml` | **a running program taking a redefinition over a socket** |
|
||
| `test/test_session.ml` | **what a running process cannot be told, and recovering from a typo** |
|
||
| `test/test_dev.ml` | **the daemon, driven the way an editor drives it** |
|
||
| `test/test_repl.ml` | **`C-x C-e`: an expression evaluated inside a running program** |
|
||
| `test/programs/conditions.flan` | **`handler-bind` and `signal`, the accumulation case** |
|
||
| `conditions.org` | **a cheatsheet for driving conditions: what works, the exact refusals, the gotchas** |
|
||
| `conditions-play.flan` | **a program to poke at them with, built to be attached to by `flan dev`** |
|
||
| `test/programs/restarts.flan` | **`restart-case` and `invoke-restart`: the transfer, across two frames** |
|
||
| `test/test_emacs.ml` | **the client, driven against a real daemon and a real program** |
|
||
| `test/reload_host.c` | the C host that loads and installs two rebuilds, in one process |
|
||
| `test/wasm-run.mjs` | **a WASI host in twenty lines of `node:wasi`, so the table can run a wasm32 build** |
|
||
|
||
```
|
||
$ flan run calc-me.flan "1 + 2 * (3 - 0.5) / 2"
|
||
3.5
|
||
$ flan run test/programs/sand-headless.flan
|
||
15595743031174623232
|
||
$ flan run sand.flan # a window, 120 fps, hold space
|
||
```
|
||
|
||
## Decided 2026-09-12, by the author, and not yet built
|
||
|
||
Five questions were put and answered in one sitting. Each is a decision, not a preference — build against them, and
|
||
reopen one only with a reason rather than a taste.
|
||
|
||
~~**1. Assets are embedded at compile time, one file or one directory.**~~ **Built** — `(embed "p")`, `(embed "p" string)`, `(embed-dir "d")`. See BUILT.md, "Assets are baked in". Odin's answer, and the reason it is the right
|
||
one here: it is a *compiler* feature, so it needs no build flags, no linker arguments and no per-target packaging, and
|
||
it works identically on desktop and web. That matters more here than it does for Odin, because `Load` gives link flags
|
||
only to directory packages — the single file doing `(rl/load-texture "brush.png")` is structurally the one file with
|
||
no link channel, which is what stopped the web lane from inventing a flag. Embedding has no such hole. Odin's
|
||
`#load` and `#load_directory` are the model (`src/parser.cpp:853`, `src/checker.cpp:3594`). emscripten's
|
||
`--preload-file` stays available later for assets that should load lazily rather than be baked in; the `@web` link
|
||
line already carries it if wanted.
|
||
|
||
~~**2. Reading a file works everywhere; writing is desktop-only and signals on web.**~~ **Built** — `barf` on the web signals `FileError` with reason `file-unsupported`, and `test/test_web.ml` runs it under node rather than asserting the artifact's shape. See BUILT.md, "slurp, barf, and the two ways they fail". Odin stubs its whole file API on
|
||
js/wasm — every operation returns `.Unsupported`, and `core/os/file_js.odin`'s own comment says the stubs exist only
|
||
so importing `core:os` "panics cleanly". Take the restriction and not the mechanism. **Flan has no conditional
|
||
compilation** — nothing in `parse.ml` or `check.ml` reads the target — so "isolate this code to desktop" is not
|
||
expressible in source, and a build-time refusal would therefore be unusable. A **silent no-op is worse than either**:
|
||
it is how a save file disappears with nothing said. So `barf` on web signals a condition under a restart and the
|
||
program decides. This is the language having something Odin does not; use it. Per-package target isolation, if a
|
||
whole desktop-only package is ever wanted, is the `@native`/`@wasi`/`@web` link-line tagging the web lane built.
|
||
|
||
**3. Build the shadow stack.** ~~Not yet built.~~ **Built**, both halves — see BUILT.md. Kept here as the decision it
|
||
was, with the measurement it asked for: +33% on call-heavy code over globals for the frames, +61% with the slot table,
|
||
and 0.06% of a 60fps frame.
|
||
|
||
plan.org:591 has specified it in the dev-build column since the beginning and nothing
|
||
has ever built it. It is the route to `(:op "backtrace")` *and* to locals, together, and it is dev-only so a shipped
|
||
game pays nothing. Chosen over the DWARF route deliberately: DWARF still owes a `!DILexicalBlock` per `Let` before
|
||
`p v` under shadowing is even honest, and that buys locals in lldb rather than in the break loop. The author's reason
|
||
is the one to keep in view — **the more a break loop can show, the less often a real debugger is needed** — which
|
||
makes this a dev-loop feature, not a debugger feature.
|
||
|
||
**4. Conditions get a parent link, not class inheritance.** A condition type may name a parent where it is declared;
|
||
matching walks that static chain. This buys the hierarchy §1 of `spec-conditions.md` says there is none of — a
|
||
catch-all handler, "any file error" — at compile-time cost only. **It is deliberately not the class answer** that the
|
||
"Open: can a condition be a class?" section below weighs: a class condition allocates at the signal site, which is
|
||
the failure path and sometimes the thing that failed; it inverts §5's lifetime, so something must own and free it;
|
||
and it lets a condition's layout change while a handler frame stands against the old one. A parent link has none of
|
||
those costs and leaves the frozen model otherwise intact. Real inheritance stays possible later if a case demands it;
|
||
this closes nothing off. That section stays open for the record but is no longer the blocking question for
|
||
`handler-case`.
|
||
|
||
~~**5. File I/O — `slurp` and `barf` — is the next stdlib work**~~ **Built.** It was the next stdlib work, after `Vec`, because `slurp` returns a string whose
|
||
length is not known until the file is read and therefore cannot exist before an allocator does.
|
||
|
||
## Decided later the same day, and queued
|
||
|
||
~~**6. A field label is written with a dot, not a colon, and the colon is reserved for keys.**~~ **Done.**
|
||
`{.x 1.0 .y 2.0}` is struct construction and `{inner .field}` is destructuring; the old spelling is refused, and the
|
||
refusal names the new one. `:keys` kept its colon — it names no field, so leaving it alone is what lets the dot mean
|
||
exactly one thing. 681 labels across 45 `.flan` files including `vendor/`, plus 94 more in the Flan embedded in
|
||
`lib/prelude.ml` and the tests. `Map` is now free to take `{:key value}` without colliding with struct literals. See
|
||
BUILT.md, "The colon belongs to keys".
|
||
|
||
**What it left for the Emacs lane, both verified.** `render.ml` still *prints* a struct with colons, deliberately:
|
||
`emacs/flan-inspect.el:165` parses that output and hard-codes the colon when it reads a field out, so the printer
|
||
has to move in the same commit as its reader. And `flan-mode.el:61` font-locks `:name` as a constant with nothing
|
||
matching `.name`, so a field label is now unfontified where it used to be coloured. Neither is urgent; both belong
|
||
with whoever next opens `emacs/`.
|
||
|
||
**7. `Map` follows Odin's implementation.** Read `base/runtime/dynamic_map_internal.odin` before writing any of it;
|
||
the checkout is at `~/Repositories/Odin`. Three properties are the ones worth copying, and they are stated in its own
|
||
header comment:
|
||
|
||
- **Open-addressed Robin Hood hashing at a 75% load factor.** No buckets, no per-entry allocation, and probe
|
||
distances stay even because a later arrival steals a slot from an earlier one.
|
||
- **Cache-line-aligned `Map_Cell` packing**, so no single key or value ever straddles a cache line and a linear probe
|
||
walks memory in a cache-friendly order. This is the part a hand-rolled open-addressed map usually gets wrong.
|
||
- **`uintptr` throughout** for sizes, masks and offsets, to keep sign-extension and masking instructions out of the
|
||
probe loop.
|
||
|
||
Its static/dynamic split is the same type-erasure this project already committed to: `Map_Info` carries size,
|
||
alignment and offsets, and the compiler emits the hash and equality pair per key type. `spec-memory.md`'s
|
||
structural-key restriction holds this to the built-in key set, so there is no dispatch to design.
|
||
|
||
**Why this will not be Python's dict.** Worth recording because it is the question that prompted the decision.
|
||
Python's dict algorithm is fine; what makes it slow is that every key and value is a separately allocated, reference-
|
||
counted object, and hashing and comparison go through indirect calls that cannot be inlined. Flan stores raw bytes
|
||
and compiles the hash and comparison concretely at each use. That difference is most of the gap before any
|
||
algorithmic cleverness. **jank is not the model** — it is Clojure, so its maps are persistent with structural
|
||
sharing, which plan.org rules out by name because shared structure destroys the clear ownership that is the whole
|
||
reason there is no collector.
|
||
|
||
## Decided in discussion, queued
|
||
|
||
**Globals in the break buffer — one section, scoped to the stack.** Locals are readable; globals are not shown anywhere,
|
||
and in this language they are arguably the more useful half: a game keeps most of its state in top-level `defvar`s and
|
||
`sand.flan` holds its entire grid that way.
|
||
|
||
Not per frame. A global is not part of a frame — it is program state the frame happened to touch — so nesting it under
|
||
one implies an ownership that is not there and repeats the name once per frame that reads it. Instead: its own section,
|
||
whose contents are the **union of the globals every frame on the current stack references**, which keeps the compiler
|
||
doing the choosing (the per-function reference set is already known) without listing all of a program's globals.
|
||
|
||
Two refinements decided with it: **annotate each entry with which frames touch it**, which recovers what per-frame would
|
||
have told you at no cost in duplication; and **order by the innermost frame that touches it**, since a deep stack makes
|
||
the union large and proximity to the error is the ordering that puts the likely culprit on top.
|
||
|
||
The daemon already has the pieces — `describe` returns the globals, `Session.render` walks a concrete type to a printed
|
||
value, and the `layout` op established that a qualified name is an identity the daemon can resolve.
|
||
|
||
**The break buffer opens by itself when the program stops.** Today a condition stops the program and the buffer appears
|
||
only when `C-c C-b` is typed. `flan-dev--absorb` already inspects every reply for `:stopped` and a poll covers the case
|
||
where no reply is pending, so the client already knows the moment it happens and already moves the mode line from it —
|
||
this is a hook at a point that exists, not new plumbing.
|
||
|
||
Three things to settle while building it: whether it takes focus or only displays; whether `(pause)` should always take
|
||
the window, being a deliberate stop rather than a failure; and what it does when the program stops while point is
|
||
mid-edit in another buffer.
|
||
|
||
**`Handle` and the pool are the real gate on classes, and they are buildable now.** plan.org's rule is that nothing
|
||
starts on managed classes "until ordinary `struct`, `Handle`, and reload semantics are working". Checked against the
|
||
tree: structs work fully; reload works with one known hole (a changed signature is refused rather than versioned);
|
||
**`Handle` does not exist at all** — `check.ml:218` still refuses `(Handle T)` by name.
|
||
|
||
It is not an incidental precondition. `migrate-instances` has to *enumerate* live instances, and a pool behind a
|
||
generational `(Handle T)` gives that by construction while a world arena and an owned region do not. plan.org presents
|
||
the three storage strategies as a free choice and they are not: handles are the one that makes migration possible.
|
||
|
||
**The allocator and arena landing today are what unblock it** — a pool is built on them, so `Handle` is buildable now
|
||
where it was not this morning. Build `Handle` and the pool next and treat *that* as the gate. It earns its place
|
||
independently of classes: stable references to things that move or die is something any game wants.
|
||
|
||
Already banked, and it means classes are less work than plan.org implies: **a generic function is an indirection cell**
|
||
whose body is a dispatch table, which a reload extends. That is the expensive half of method dispatch, and it is built
|
||
and tested.
|
||
|
||
**Resource cleanup: `defer` stays the answer. `drop` is not built, and `with-cleanup` is not either.** Reached by
|
||
working the case through rather than by preference, so the reasoning is worth keeping.
|
||
|
||
`drop` was specified in `spec-memory.md` this morning. **This amends it: the hook is deferred, not built.** Three things
|
||
decided against it. It runs code somewhere the reader is not looking, which is the C++ behaviour the author explicitly
|
||
does not want. It would not even cover the motivating case — `Image` and `Texture2D` are *raylib's* types, and attaching
|
||
a hook to a foreign type is its own unsolved design question. And its one real advantage, cascading through a container,
|
||
is the case `Handle` is about to make rare: entities holding handles hold numbers, not resources.
|
||
|
||
`with-cleanup` / `unwind-protect` was also put and rejected: awkward with several resources, and it reads worse than
|
||
what already exists. The raylib begin/end pairs that seemed to motivate it are a macro problem, not a primitive one —
|
||
`with-drawing` and `with-mode-2d` are three-line macros once the expander lands.
|
||
|
||
**What to build instead is small: relax where `defer` may be written.** It is refused today inside a `let`, a loop or a
|
||
branch. The loop and branch refusals are right — `defer` is a *compile-time* construct, the cleanup copied into every
|
||
exit path, so "maybe registered" is not expressible and a loop body would fire once at function exit instead of once per
|
||
iteration. But **a `let` at the top level of a function body has exactly the function's extent** and always registers,
|
||
so it is as safe as function scope and is refused for a reason that does not apply to it. Relaxing it gives:
|
||
|
||
```
|
||
(defn load-brush []
|
||
(let [sheet (rl/load-image-from-memory ".png" brush-bytes)]
|
||
(defer (rl/unload-image sheet))
|
||
(set brush (rl/load-texture-from-image sheet))
|
||
(rl/image-flip-horizontal (addr sheet))
|
||
(set brush-mirrored (rl/load-texture-from-image sheet))))
|
||
```
|
||
|
||
Several resources are several defers, released in reverse, visible in acquisition order. A container of resources is an
|
||
ordinary loop inside the defer body — the manual cascade, three lines, at one level of nesting.
|
||
|
||
**Odin, for the record**, has no destructors, no drop and no finalizers: `delete` frees container memory and nothing
|
||
else, and resource release is `defer` at the acquisition site. That idiom does not transfer directly only because
|
||
Odin's `defer` is block-scoped; the relaxation above recovers most of it.
|
||
|
||
**The safety net, and the better use of effort: a debug tracking allocator.** ASan's leak detection covers memory
|
||
*instrumented* code allocated — the Flan allocator, and it is already wired up and clean. It does **not** cover a leaked
|
||
texture, because that memory belongs to uninstrumented raylib, which is the same reason the sanitizer sweep treats the
|
||
windowed examples as noise. But every raylib call goes through a *generated* wrapper, so a dev build can count
|
||
acquisitions against releases at that boundary and report what is still held at exit, by name. No hook, no type
|
||
annotation, nothing running at a distance — it does not change how code is written, it reports when something was
|
||
forgotten.
|
||
|
||
## The next batch, in order
|
||
|
||
Agreed at the end of 2026-09-12. Ordered by priority, not by size. Items 1-3 and 5-6 want the compiler core and should
|
||
run one lane at a time; item 4 is disjoint and runs alongside any of them.
|
||
|
||
~~1. **Fix the one failing test** — `the frame of a superseded body answered with the new body's names`.~~ **Done**, and
|
||
the handoff's diagnosis was wrong. Nothing was dropping the number: the fingerprint was emitted into `%fninfo` and
|
||
never read back. `flan_dev.c` called the field `spare`, there was no accessor, the agent never snapshotted it, the
|
||
backtrace line never carried it, and `Dev.locals` compared slot counts and nothing else — four of the five
|
||
hand-offs were never written, and printing both sides of the comparison could not have found it because there was
|
||
no comparison. The mechanism was sound and stayed: it hashes slot *names* as well as types, so it does see a
|
||
rename. See BUILT.md, "Locals of a stopped frame".
|
||
|
||
2. ~~**The colon-to-dot change.**~~ **Done**, and `Map` is unblocked: `{:key value}` is free. The sweep is
|
||
`tools/colon-to-dot.py`, kept rather than thrown away, because the lanes that branched before it wrote Flan in the
|
||
old spelling and their files want the same pass at merge — `python3 tools/colon-to-dot.py .` over the tree, and
|
||
`--in-strings` for a `test/*.ml` that embeds Flan.
|
||
|
||
3. **`Map`, and the `defer` relaxation.** `Map` is step 4 of the container build order and finishes what `Vec` started.
|
||
The `defer` change — permitting it in a `let` whose extent is the function body — is small, independent, and is the
|
||
whole of the resource-cleanup answer.
|
||
|
||
4. **The Emacs batch. Disjoint from the compiler, so it runs in parallel with anything above.** Globals in the break
|
||
buffer; the buffer opening itself when the program stops; the indentation rewrite with `clojure-mode` as the
|
||
reference; `#_`; hex, binary and addresses on primitives in the inspector. The indentation one is worth doing first
|
||
within this batch — it costs friction on every keystroke today.
|
||
|
||
5. **Union values, then the macro expander, then `Result`/`try`.** Promoted above `Handle` on the author's call —
|
||
macros are the thing most worth wanting, and unions are the only thing between here and them.
|
||
|
||
**Unions are closer than the milestone number suggests.** `Tast.arm` already carries `acase` (a case name) and
|
||
`binds` (the slots a payload binds to) — that is union shape, built and exercised, because `Option` is a two-case
|
||
union wearing a special coat and `match` over it works today. What is missing is the declared layout (a tag plus the
|
||
largest variant), construction, and letting `match` bind payloads from a user-declared union. `defunion` already
|
||
parses and its shape is already checked; `check.ml:312` and `:1075` are the two refusals to remove.
|
||
|
||
**Then macros**, which are blocked on exactly this and nothing else: a macro is `[Form] -> Form`, so `Form` has to be
|
||
a Flan union whose *layout* the compiler and the `dlopen`ed macro agree on byte for byte. `NEXT.md`'s macro section
|
||
has the expander design — a pre-pass fixpoint before `Parse`, `gensym` as a compiler-side counter, quasiquote as a
|
||
desugaring over `Form`. The exit criterion is already written: move `when` or `cond` out of `parse.ml` into the
|
||
prelude as a `defmacro` with the existing tests unchanged and still green.
|
||
|
||
Macros are what buy `with-drawing` and `with-mode-2d` over raylib's begin/end pairs, the hiccup DSL if a JS backend
|
||
ever happens, and the removal of special forms from the compiler.
|
||
|
||
**`Result`/`try`** follows, being another union.
|
||
|
||
**Generics are deliberately NOT here.** They feel adjacent and are not urgent, and today is the evidence: `Vec` and
|
||
`Map` were the obvious customer and needed none — they are type-erased, with the compiler emitting sizes and the
|
||
hash/equality pair per call site, which is Odin's design. The remaining customers are user-written allocators and
|
||
escaping closures, and both actually want **function values**, which is a separate milestone-5 feature. Leave
|
||
generics until something concrete needs them.
|
||
|
||
6. **`Handle` and the pool.** A reference to something that can die, that reports that it died rather than silently
|
||
resolving to whatever reused the slot. Wanted on its own terms for entities referred to across frames, and it is the
|
||
real gate on classes. Buildable now that the allocator exists.
|
||
|
||
7. **`break` and `continue`, with loop labels.** Declined once deliberately — see "`break`, and why it was not built"
|
||
— but a game loop wants it and the author has asked for it. Two things settled in conversation:
|
||
|
||
**Labels, Odin-style but in the head position.** A keyword names a loop and `break` takes it:
|
||
|
||
```
|
||
(while :outer (< i n)
|
||
(while (< j m)
|
||
(when (hit? i j) (break :outer))))
|
||
```
|
||
|
||
A keyword there is unambiguous because a loop condition is never one. It is **not a goto**: control can only leave
|
||
a loop it is already inside, which is what keeps it safe and is the same restriction Odin's labelled `break` has.
|
||
|
||
The two known blockers stand and must be answered: `check.ml`'s `in_frames` rule refuses `return` inside
|
||
`handler-bind`/`restart-case` **because `return` always crosses**, while `break` crosses only sometimes — a loop
|
||
wholly inside a `restart-case` body has a legitimate local break — so that blanket refusal has to become a
|
||
loop-depth-relative-to-frame-entry rule. And `continue` forces a `Tast.While` signature change to carry a latch,
|
||
because `check_dotimes` folds the step into the body and a `continue` branching to the header would skip it and
|
||
hang.
|
||
|
||
8. **More than one error per compile.** Raised by the author's workflow: write everything, compile at the end, squash
|
||
the list. That does not work today — `Loc.Error` is the frontend's **one** exception, so the first error aborts the
|
||
run and you get them one at a time, which is exactly the loop that workflow exists to avoid.
|
||
|
||
The fix is in the checker, not in tooling: collect errors and carry on — finish the function, finish the file,
|
||
report everything found. **No editor work is needed once that exists.** Flan already prints `file:line:col: message`,
|
||
which is the GNU format Emacs's `compilation-mode` parses with no configuration, so `M-x compile` gives a clickable
|
||
list and `next-error` for free. Flycheck and a structured JSON report were both considered and are **not** wanted:
|
||
the author's workflow is compile-at-the-end, not live linting.
|
||
|
||
9. **Signature generations and stale-caller warnings.** The biggest remaining hole in "you never restart the program" —
|
||
a changed signature is still refused rather than versioned. Last because it is the largest and nothing else waits on
|
||
it.
|
||
|
||
Deliberately not scheduled: the JS backend and header-based C interop, both large and neither blocking current work; a
|
||
debug tracking allocator, which is the leak safety net and a good candidate whenever it is wanted.
|
||
|
||
## Decided in discussion — the array constructor and the module system
|
||
|
||
**`(array 4 rl/Vector2)` makes a fixed array; `[4 T]` stays the type syntax.** The problem this solves: a `let`
|
||
binding takes no type, so `(let [pts [4 rl/Vector2]] ...)` reads `[4 rl/Vector2]` as a two-element array *literal* and
|
||
fails with *unknown name rl/Vector2*. It cost 32 hand-written `Vector2`s in one raylib example.
|
||
|
||
`[4 T]` is not a special syntax — it is the ordinary type syntax and already works everywhere a type is expected:
|
||
`(defvar points [4 rl/Vector2] ...)`, `(defn draw [pts [4 rl/Vector2]] ...)`. A `let` binding is the single position
|
||
with no type slot, which is the whole of the bug.
|
||
|
||
`(zeroed [4 rl/Vector2])` was proposed first and rejected on how it reads: in argument position the bracket form is
|
||
unambiguous to the *parser*, but it still looks like a two-element vector to a person. `(array 4 rl/Vector2)` says what
|
||
it does with the count and the type as plain arguments. `zeroed` keeps its existing job — an empty thing of whatever
|
||
type the destination wants — and `array` is the one that is told.
|
||
|
||
**The module system stays as it is: the directory name is the module name.** No `package foo` line at the top of each
|
||
file. Confirmed against Odin, which requires the declaration despite having the same one-package-per-directory rule —
|
||
`package os` appears in 85 files, all of them in `core/os` — so the line is ceremony that buys only the ability to
|
||
disagree with the directory name.
|
||
|
||
What the rule already gives, and what was checked in conversation: **several files in one directory are one module**,
|
||
which is the case directory-as-package exists for; **a loose file is a module of one**, so several modules can sit at
|
||
the same filesystem level without a directory each; and two modules cannot share a directory, which is also true of
|
||
Odin.
|
||
|
||
**Acyclic imports are kept deliberately, not inherited by accident.** Odin forbids import cycles and so should this:
|
||
a definite package order is what the macro expander will need later, since every `defmacro` must be compiled before
|
||
anything that calls it. Nested import paths not being real nesting — Odin's `core:math/bits` is a separate package
|
||
rather than a submodule of `math`, with no re-exporting — was reviewed and accepted as fine.
|
||
|
||
**Still missing, and this is the real gap:** `load.ml` does not support **a package importing a package**. It is
|
||
listed among the milestone-4 loose ends. Until it lands a project is an entry file plus one flat layer of libraries,
|
||
with no library depending on another.
|
||
|
||
## Blocked and unfinished
|
||
|
||
Everything below was found, decided or half-built and then stopped. Each says what blocks it. Nothing here is a
|
||
vague intention — if it is listed, someone has already established it is real.
|
||
|
||
### Unblocked now, and ranked
|
||
|
||
**0. Signature generations and stale-caller warnings — milestone 7's unfinished half.**
|
||
Promoted here on the author's correction, and `session.ml:146` already says the same thing at the refusal itself. A
|
||
changed signature is refused today and **that is a placeholder, not the design**. plan.org's open decision #6 says what
|
||
should happen: a signature change makes a new version of the function, new callers resolve it, existing callers and any
|
||
stored `Fn` value stay safely on the old one, and the session *warns* at each tracked stale caller site. Milestone 7
|
||
names it outright — "signature generations and stale-caller warnings".
|
||
|
||
**The thesis of this project is that you never restart the program.** Every refusal that ends in "restart to change it"
|
||
is a hole in that, and this is the biggest one. It needs three things that do not exist: function versions, a
|
||
trampoline per version, and caller tracking good enough to name the sites. The cell already gives the indirection; what
|
||
is missing is that a cell holds one bare pointer with no signature, so there is nowhere to put a second version.
|
||
|
||
A changed **struct layout** is the genuinely hard case and plan.org still specifies it as a rejection — storage already
|
||
allocated has the old shape and a new body reads its fields at the wrong offsets. Managed classes are the planned way
|
||
through, with an explicit migration at a frame boundary. Do not conflate the two: one is unbuilt, the other is decided.
|
||
|
||
|
||
**From porting ten raylib examples** — the first code the language was pushed by that it was not designed around.
|
||
Ranked by how often they were hit, top two first because they are walls rather than conveniences:
|
||
|
||
1. ~~No number reaches `draw-text`.~~ **Fixed** by `(string b)`.
|
||
2. ~~An enum parameter cannot be driven by a loop variable.~~ **Fixed** by explicit conversions in both directions:
|
||
`(i32 k)` takes an enum to its integer, `(GamepadAxis n)` takes an integer to an enum. Neither is an instruction —
|
||
an enum is an i32 at run time and `emit.ml`'s `cast` already reduced one to that before choosing an opcode — so the
|
||
change is a guard in `check.ml`'s cast arm and nothing in the backend. The rule the refusals came from is
|
||
deliberately *not* relaxed: a bare integer still does not fit an enum parameter, so `:spcae` is still an error at
|
||
the call site. The rule was "an integer must not arrive silently", and a written `(GamepadAxis i)` is not silent.
|
||
The other escape stays closed too — one `declare-c` per C function — and no longer needs to be open.
|
||
- **A value that is no declared member is allowed**, deliberately. raylib's gesture is a bitfield and an OR of
|
||
flags is a legal `Gesture` that is no single member; and `session.ml`'s printer already falls through to the
|
||
number for an out-of-range enum, on purpose, so refusing to construct one while agreeing to print it would be
|
||
incoherent. An `Option` would make every site unwrap for no safety bought, and a literal-only refusal would
|
||
catch nothing, because the bitfield case is a run-time value.
|
||
- **Only an integer converts *to* an enum.** Not a float, and not another enum — a cross-enum hop goes through
|
||
`(i32 x)` so both ends are written down. Enum → any numeric is always allowed: lossless to i32 by construction,
|
||
and a narrower target truncates by the rule every int→int cast already follows.
|
||
- **The comparisons needed nothing else.** `(> (i32 g) 255)` checks because `binary` takes the non-literal side
|
||
first; `binary` was deliberately left ignorant of enums, since teaching it would be the implicit conversion this
|
||
avoids.
|
||
- **A bit-set type later builds on this rather than replacing it.** It would be its own type with its own
|
||
operations and would still want a named escape to the underlying integer for the FFI, spelled the same way. If
|
||
`Gesture` becomes one, the `(i32 g)` calls stay valid and only the range tests migrate to a membership test.
|
||
- One parse fix came with it: `defenum` names were not in `parse.ml`'s type set, so a local enum could not be a
|
||
function's return type. They are in it now under a key of their own, admitted as a bare symbol and never as a
|
||
list head — because `(Key n)` is a *value* now, and putting `Key` in `types` would make a body starting with one
|
||
be eaten as a return type.
|
||
3. **`break` is not implemented.** Declined deliberately rather than built — see below.
|
||
4. **A `let` binding takes no type annotation**, so a fixed array is either a top-level `defvar` or a literal with
|
||
every element spelled out. `(let [pts [4 rl/Vector2]] …)` parses as a two-element array literal and fails with
|
||
*unknown name rl/Vector2*. Cost: 32 hand-written `Vector2`s in one example. **Looked at and stopped — it is a
|
||
grammar question, not a missing feature.** Everything under the surface is already there: `Ast.binding` carries a
|
||
`bty`, `load.ml` renames through it, and `check.ml:723` consumes it as the `want` for the value. Only the way it
|
||
is written is open, and the parser says so where it refuses (`parse.ml:366`): `let` is a flat list of pairs, so it
|
||
cannot disambiguate by *count* the way `defvar` and `defconst` do — those read `[n t v]` as three arguments to a
|
||
form, and there is no such boundary between one pair and the next. Three surfaces, in the order they are worth
|
||
considering:
|
||
- **`(zeroed [4 rl/Vector2])` — `zeroed` takes its type as an argument.** Recommended. It is one extra branch in
|
||
the arity-0 `zeroed` case in `check.ml`, no parser change, no ambiguity, and it answers the actual complaint,
|
||
which is not "locals cannot be annotated" but "there is nothing here to infer *from*". It also reads as what it
|
||
does: the value is a zeroed thing of that type, not a name that has been told what it is.
|
||
- **A marker between the name and the type**, `(let [pts :- [4 rl/Vector2] …] …)` or similar. Unambiguous, and it
|
||
buys a general annotation rather than one form's escape hatch. The cost is a new piece of syntax in the binding
|
||
vector, which is the one place this language has kept looking exactly like Clojure's.
|
||
- **Bare `(let [pts [4 rl/Vector2] …])`.** The obvious spelling and the one that cannot work: `[4 rl/Vector2]` is
|
||
a well-formed two-element array literal, and telling the two apart needs types in the parser, which there are
|
||
none of by design.
|
||
Note that plan.org's rule is "annotate function signatures, infer locals", so the general annotation is a
|
||
deliberate absence and not an oversight — which is the other reason the `zeroed` route is the smaller answer.
|
||
5. ~~**Arithmetic is strictly binary** — *+ takes 2 arguments, given 5*.~~ **Fixed.** `+ - * /`, `min`/`max` and
|
||
`bit-and`/`bit-or`/`bit-xor` fold left over two operands or more. `%` and the shifts stay at two, and one operand
|
||
is refused with the form to write instead — there is no unary minus and no reciprocal.
|
||
6. ~~**No `sin`/`cos`/`abs` for floats.**~~ **Fixed.** `sin-f32` and `cos-f32` are `declare`s in the prelude now,
|
||
with the caveat written beside them: IEEE-754 makes `sqrt` correctly rounded and requires nothing of the kind for
|
||
`sinf`, so these are the one place in the prelude where native and wasm32 may disagree bit for bit. Float `abs` is
|
||
not wrapped, for the reason integer `abs` is not — it is `(max x (- 0.0 x))` over two builtins.
|
||
|
||
**A `string` cannot be returned from C at all**, which is what makes `GetGamepadName` unbindable: *a string only
|
||
crosses as a parameter — a C function that returns one returns something Flan has no owner for*. Same rule refuses
|
||
`TextFormat`, which is also variadic and so has no honest signature.
|
||
|
||
**The negative result is worth as much.** None of the gaps expected blocked anything — no generics, no allocator, no
|
||
`Vec`/`Map`, no escaping closures, and function-scoped `defer` never came up. Input-and-draw over fixed-size state is
|
||
the shape the language already has. Three constructs unexercised anywhere else in the repo worked first try: a fixed
|
||
array with a struct element, a 2-D struct array, and `[N string]` as both `defconst` and mutable `defvar`.
|
||
|
||
### The web target: what it does not reach yet
|
||
|
||
`flan build --target=web` works, a raylib example builds unchanged and `test/test_web.ml` is green — see BUILT.md,
|
||
"The browser is the third target", for the mechanism and why asyncify rather than `emscripten_set_main_loop`. Four
|
||
things it does not cover.
|
||
|
||
~~**1. `sand.flan` has no web build, and the cause is one missing `#include`.**~~ **Built. It opens.** See BUILT.md,
|
||
"sand.flan in a browser", for the whole of it. Three summary lines, because the diagnosis below was right about the
|
||
structure and wrong about the cause:
|
||
|
||
- The `#include` was never the fix. **The agent is a socket server and a browser has no sockets**, so an agent that
|
||
compiles there is an agent that can never accept a connection. `vendor/agent/flan_agent.web.c` is three no-ops, and
|
||
`Build` selects it over `flan_agent.c` on `--target=web` and nowhere else.
|
||
- **Refusing `vendor:agent` on web was the honest-looking option and is ruled out by arithmetic.** There is no
|
||
conditional compilation, `sand.flan` calls `agent/start` unconditionally, `Reach` cannot prune a package something
|
||
reachable calls into — so a refusal means the flagship program does not build for the browser at all. A refusal is
|
||
only honest when the caller has a way to not ask. This does **not** reverse decision 2 above: `barf`'s no-op loses a
|
||
file the program believed it wrote, and there is nothing for the agent to lose because `--dev` is already refused by
|
||
name on every wasm target. The argument is written out at the top of `flan_agent.web.c`.
|
||
- **A package's `.c` files can now be addressed to a target**, by a tag in the name before the extension, and a tagged
|
||
file *replaces* the untagged file of the same base name on that target. This is the C-source half of the
|
||
`@native`/`@wasi`/`@web` link-line mechanism decision 2 pointed at for per-package target isolation.
|
||
|
||
The brush is `(embed "brush.png")` decoded through a new `LoadImageFromMemory` binding. `load-texture` and
|
||
`load-image` now have no call site anywhere in this repository — deliberately, because a path-based load is the one
|
||
shape the browser cannot have, and said here so it is not read later as an accident.
|
||
|
||
The original entry follows.
|
||
|
||
**1. `sand.flan` has no web build, and the cause is one missing `#include`.** `vendor/agent/flan_agent.c` does not
|
||
compile under emcc: *variable has incomplete type 'struct timeval'* at line 426, because emscripten's headers do not
|
||
pull `<sys/time.h>` in transitively the way glibc's do. `sand.flan`'s `main` calls `(agent/start ...)`
|
||
unconditionally, so `Reach` cannot prune the package, so the flagship program stops at that error — even without
|
||
`--dev`. Beneath the include is a structural fact worth deciding rather than patching around: **the agent is a socket
|
||
server and the browser has no sockets**, which is the same family as the `--dev` refusal. So the two fixes are not
|
||
equivalent — add the include and the agent compiles into a web build that can never accept a connection, or refuse
|
||
`vendor:agent` by name on a web target the way `--dev` is refused. The second is the honest one. Neither was taken
|
||
here: `vendor/agent/` belonged to another lane this session.
|
||
|
||
~~**2. Assets are two questions and only one of them is about emscripten.**~~ **Answered by the embed above, and the
|
||
answer was the third option neither half here considered: make it a compiler feature and neither question arises.** The
|
||
hard half below is exactly right about the problem — the file that needs the asset is structurally the one file that
|
||
cannot declare it — and the conclusion drawn from it, that the fix must be a link channel or a new declaration, was
|
||
the wrong one. `(embed "brush.png")` needs no channel, because there is nothing to tell the linker. What is *not* done
|
||
is `sand.flan` itself: `(rl/load-texture "brush.png")` takes a path and raylib opens it, so pointing raylib at embedded
|
||
bytes needs `LoadTextureFromImage` over `LoadImageFromMemory`, which is a raylib binding question and not this one.
|
||
The original text follows. `sand.flan` does `(rl/load-texture "brush.png")` against a bare relative path.
|
||
|
||
- The easy half: a bare relative path has no meaning on a target with no filesystem. emscripten's answer is
|
||
`--embed-file` or `--preload-file` into MEMFS, and both are *linker arguments*, so they are already expressible as an
|
||
`@web` line in a package's `link` file. No new mechanism is needed for a package.
|
||
- The hard half, and the actual design question: **the file that needs the asset is structurally the one file that
|
||
cannot declare it.** `Load` hands out `lflags` only for a directory package (`one_file` → `[]`), and `main` is not
|
||
exported, so a program can never be a package. The program doing the `load-texture` therefore has no link channel at
|
||
all. Answering this means either giving a single-file program a way to carry build arguments, or making assets their
|
||
own declaration rather than a linker flag. No flag was invented for it here.
|
||
|
||
**3. Nothing has been opened in a browser.** *Still true, and now it is the only thing left between here and
|
||
"someone played with it".* `sand.flan` builds for the web, the module carries asyncify, raylib's GL imports and
|
||
brush.png's own bytes whole, and `node sand.js` gets as far as `glfwInit` before dying on `window is not defined` —
|
||
which proves the module is live and proves nothing about the canvas. BUILT.md carries the exact commands to serve and
|
||
open it, and the list of what only a human will discover: whether it paints, whether the audio round trip through
|
||
MEMFS survives, and the canvas size. The `until` loop never exits on the web, so none of `main`'s `defer`s run —
|
||
expected, and worth knowing before reading anything into it.
|
||
|
||
The original entry follows.
|
||
|
||
**3. Nothing has been opened in a browser.** The test is headless and permanently so: it asserts the artifact's shape,
|
||
the `asyncify_start_unwind` export and the `glViewport` import, and that node runs the emitted JS. Whether the canvas
|
||
actually paints is unverified by anything in CI, and a human should look once.
|
||
|
||
**4. Unmeasured and untested.** Asyncify's cost is quoted from emscripten's documentation (roughly a doubling of code
|
||
size) and not measured here, and no frame time on web has been taken at all. raylib's audio and any use of threads on
|
||
the web target are untried. And a **wasi** build that reaches raylib now fails on undefined symbols rather than on a
|
||
missing `-l:libraylib.so.550`, because that line is tagged `@native` — the same error one step later, and a worse
|
||
message.
|
||
|
||
### `break`, and why it was not built
|
||
|
||
Settled, so the next attempt is cheap rather than a rediscovery:
|
||
|
||
- **`dotimes` gets it free** — it desugars to `Tast.While`, so one implementation covers both loop forms.
|
||
- **`defer` is a non-question.** It is function-scoped, `break` does not leave the function, nothing fires. No refusal
|
||
needed and no interaction to design.
|
||
- **Type it `Never`**, as `exit` and `return` already are.
|
||
- **`pads` is the structural model.** `emit_while` already makes an `endloop` label; break is a push/pop of that around
|
||
the body plus a `br`. `return` is a direct terminator with no context threading, so there is nothing else to mirror.
|
||
|
||
What stopped it, and neither is small:
|
||
|
||
- **`check.ml`'s `in_frames` rule does not extend.** It refuses `return` inside `handler-bind`/`restart-case` because
|
||
those frames are popped on the way out, and that refusal is blanket because `return` *always* crosses. `break`
|
||
crosses only sometimes — a loop wholly inside a `restart-case` body has a legitimate local break — so the precedent
|
||
has to be replaced by a loop-depth-relative-to-frame-entry rule nobody has ruled on.
|
||
- **`continue` forces a `Tast.While` signature change.** `check_dotimes` folds the step into the body as
|
||
`While (cond, body @ [step])`, so a `continue` branching to the header skips the increment and hangs. It needs a
|
||
latch — `While of expr * expr list * expr list` — across `check.ml` and `emit.ml`. plan.org settles break and
|
||
continue as one item and `parse.ml` refuses them in one case, so building break against today's `While` is exactly
|
||
the thing that would have to be undone.
|
||
|
||
plan.org's single line on it (831) names a `for` the language does not have and gives no mechanism.
|
||
|
||
|
||
1. ~~**Allocators, then `Vec` and `Map`.**~~ **Steps 1, 2 and 3 are done** — the allocator, the arena, `(Vec T)`,
|
||
`StorageExhausted` and `retry`. `Map` is step 4 and is what is left of this item. See *Allocators, `(Vec T)` and
|
||
`StorageExhausted`* in [`BUILT.md`](BUILT.md) for the shape, the three amendments to a frozen `spec-memory.md` and
|
||
the one addition. The claim below held: **`Vec` does not need generics** — that was wrong and is worth un-learning: Odin's containers are compiler
|
||
builtins over a *type-erased* runtime (`base/runtime/dynamic_array_internal.odin`), where `$T` appears only in thin
|
||
wrappers producing `size_of`/`align_of` at the call site, and per-key hash and equality are compiler-emitted
|
||
procedures passed as a runtime argument (`Map_Info`, `base/runtime/core.odin:369`). That runtime is what
|
||
`spec-memory.md` specifies.
|
||
|
||
**The four questions that used to sit here are answered**, in `spec-memory.md`'s "Allocators" section, which is
|
||
frozen along with the rest of that file: when storage is released, the `drop` hook, alignment, and allocation
|
||
failure. Read them there rather than in a second copy here. The one consequence the build order below turns on is
|
||
that no allocating operation returns an error — a failure signals `StorageExhausted` under a `retry` restart — so
|
||
`push` and `put` are `Unit`, `clone` returns the container, and no signature grows a `Result`. One question is left
|
||
open in that section on purpose; it does not block the build.
|
||
|
||
2. **The editor half of a typed restart.** The language half is in (see "Landed"): `(use-value [v i32] ...)` and
|
||
`(invoke-restart 'use-value 21)` work, and a mismatch is refused at run time with both signatures in the message.
|
||
What is missing is the half only an editor can do — the leverage SBCL lacks. `eval` already compiles and runs an
|
||
expression inside the live program and the daemon already holds the struct layouts, so "ask the human, type-check
|
||
the answer, hand it over" is a short hop, and it is the one path the runtime today *refuses*: a restart with
|
||
parameters taken from the break loop traps, because `flan_break_resume` and `flan_restart_take` aim the channel at
|
||
a frame and have nothing to fill its buffer with. What it needs, end to end:
|
||
- the frame already carries the arity and the signature as a string — `flan_restart_arity` and `flan_restart_sig`
|
||
beside `flan_restart_name`, the same walk, so `restarts` can say what each one takes;
|
||
- `:restarts` on the wire carries the signature per entry, so the minibuffer can show `use-value (i32)` rather
|
||
than a bare name, and `restart-at` grows an `:args` form — a list of *expressions*, since the answer is a Flan
|
||
expression and there is already something that compiles one;
|
||
- the daemon compiles each argument against the declared type with the session's layouts (the same path `C-x C-e`
|
||
takes), refuses it there if it does not fit, and otherwise writes the values into the frame's buffer and marks it
|
||
filled before aiming the channel. That last store is what `flan_restart_take` cannot do today and is the whole
|
||
of the remaining work; the marking exists so this cannot be forgotten silently.
|
||
|
||
3. **`handler-case`.** Not a convenience — it is the fix for the loudest gotcha in `conditions.org`. A handler closes
|
||
over nothing *only because* a `handler-bind` clause runs at the signal point; a `handler-case` clause runs in the
|
||
establishing frame, which is ordinary in-frame code exactly like a `restart-case` clause. SBCL's is `handler-bind`
|
||
plus a transfer and nothing more (`src/code/error.lisp:196-268`). Every piece exists.
|
||
|
||
### `Vec` and `Map` — the order to build them in
|
||
|
||
**Steps 1, 2 and 3 are built; 4 to 8 are what is left.** The reasoning is kept because it is what the remaining steps
|
||
rest on, and because the escape it describes was tested rather than assumed — see *Allocators, `(Vec T)` and
|
||
`StorageExhausted`* in [`BUILT.md`](BUILT.md).
|
||
|
||
The dependency nobody had written down, and the reason it looked worse than it is. `spec-memory.md` defines an
|
||
allocator as "a procedure plus an opaque data pointer" — a function value. `check.ml` refuses function values four
|
||
ways, and all four say milestone 5: a written `(Fn ...)` annotation (`Ast.Tfn`), a written `fn` literal (`Ast.Fn`), a
|
||
`defn`'s name used as a value, and calling anything other than a named function. `(Map K V)`, `(Result T E)` and
|
||
`(Handle T)` are still refused beside those as milestone 6; `(Vec T)` is not, any more. Read straight off those lines,
|
||
milestone 6's allocators need milestone 5's function values and the work doubles.
|
||
|
||
**The escape is real and the work did not double** — this is the claim the built thing confirms. All four refusals are about *surface syntax*, and a value the
|
||
compiler builds that no surface form names trips none of them. The compiler already does exactly this, twice:
|
||
|
||
- A `handler-bind` clause is lowered to a function whose address goes into a `flan_handler` and is called back through
|
||
`h->fn(condition, xfer)` (`runtime/flan_rt.c:38` and `:66`). `check.ml` builds that body as its own `Tast.fn`
|
||
(`:619`, `:654`), not as an `Ast.Fn`, so line 458 never sees it, and no Flan type names the result.
|
||
- In a dev build, `emit.ml`'s `call` loads a pointer out of an indirection cell and calls through it
|
||
(`lib/emit.ml:781`–`793`). That is the indirect call line 1023 refuses in source, emitted routinely.
|
||
|
||
It is also what `spec-memory.md` already assumes for `Map`: the hash and equality pair is compiler-emitted and passed
|
||
as a runtime argument. Odin's `Map_Info` is two contextless `proc` fields (`base/runtime/core.odin:369`), and Odin's
|
||
`Allocator` is a `procedure` plus a `data: rawptr` (`:422`) — the same shape, reached the same way. If the hash pair is
|
||
expressible with no function type in the surface language, so is the allocator's procedure.
|
||
|
||
So: **`Allocator` is a builtin opaque type, the way `string` is a builtin ptr+len.** It is a `Types.t` case with no
|
||
user-writable constructor. Its procedure is an ordinary top-level function resolved to a symbol at the emit site, and
|
||
`vec-new`, `push`, `put`, `clone`, `free` and `free-all` are named calls, which `check_call` already routes through
|
||
`named_call` (`check.ml:1021`). **The built-in allocators need nothing from milestone 5.**
|
||
|
||
What *does* need milestone 5 is a **user-written** allocator: the moment a program says "here is my proc, make an
|
||
`Allocator` from it", it needs a `defn`'s name in value position, which is `check.ml:571` verbatim. That is a real
|
||
limit and not a fatal one — Odin ships arena, general-purpose, stack, pool and scratch in its own std, and most
|
||
programs write none. Ship the built-in set; user allocators arrive with function values.
|
||
|
||
**5 and 6 interleave rather than nest.** plan.org orders generics and macros (5) before allocators and containers (6),
|
||
and that order cannot hold: the macro expander is blocked on `Form` being a Flan union and union *values* are milestone
|
||
6 (see "Macros" below). Conditions and restarts, also listed under 6, are already three steps of four. The milestone
|
||
numbers are a topological hint, not a sequence. Take 6's container half first, 5's generics half second, and 5's
|
||
expander last, on 6's unions.
|
||
|
||
~~1. **`Allocator` and the arena.**~~ **Done.** The builtin opaque type, the four operations with `size` and `align`,
|
||
the capability set read off the allocator value, `with-allocator`, `context/allocator`, `context/temp` and the epoch
|
||
counter. `free-all` was decided as retain-capacity with `arena-destroy` beside it; the context is a dynamic variable
|
||
rather than a literal calling-convention parameter; both are stated as amendments in `BUILT.md`. A user-written
|
||
allocator is refused by name with milestone 5 as the reason.
|
||
|
||
~~2. **`(Vec T)`**~~ **Done**, over the type-erased runtime, with `push`, `reserve`, `at`, `len`, `as-slice`, `free` and
|
||
`clone`, and with move-only enforced by a dead set that unions at an `if` or a `match` join. `at` and `len` were
|
||
extended rather than duplicated. The header is six words in *every* build, not four in release — a layout that
|
||
changes with a build flag can disagree silently across the reload boundary — and that is the third amendment.
|
||
Ownership is not transitive yet, so a struct field of `Vec` type, a global `Vec` and a `(Vec (Vec T))` are each
|
||
refused where they are declared, naming `drop` as what they wait on.
|
||
|
||
The note below still stands and is now the only thing between `Vec` and the accumulation pattern: **capture does not
|
||
exist at all.** Nothing about it changed.
|
||
|
||
~~3. **`StorageExhausted` and `retry`**~~ **Done, with step 2 and not after**, exactly for the reason given. It is a
|
||
`while` around a `restart-case` around the attempt, built in the checker out of nodes that already existed, so the
|
||
backend learned nothing about allocation. `test/programs/exhausted.flan` exhausts an allocator for real and takes the
|
||
restart; `exhausted-unhandled.flan` is the same failure with nothing handling it.
|
||
|
||
4. **`(Map K V)`** — flat open-addressed key and value arrays, with a compiler-emitted hash and equality pair per key
|
||
type passed as arguments. `spec-memory.md`'s structural-key restriction holds this to the built-in key set, so there
|
||
is no dispatch to design.
|
||
5. **`drop`.** The hook, the transitive move-only and non-`clone`able rules, and the refusal to construct a
|
||
`drop`-carrying value against an allocator without `can-free`. It is additive — no type in the repo has a hook today
|
||
— but the `can-free` refusal has to land with the construction path it guards, before any arena-allocated container
|
||
of a user struct is trusted.
|
||
6. **`(Result T E)` and `try`, then the rest of union values.** Unions are what `Form` needs, and `Form` is what the
|
||
macro expander needs.
|
||
7. **Generics and monomorphisation**, then function values. User-written allocators and escaping closures both fall out
|
||
of the second.
|
||
8. **The macro expander**, last, on 6's unions.
|
||
|
||
**What is genuinely unsettled.**
|
||
|
||
- `spec-memory.md`'s "Open: catching a use-after-release statically" is still open, and it is now open with evidence
|
||
available for the first time: the epoch trap is built and `test/programs/stale-region.flan` is the case it catches.
|
||
What the spec says would settle it — real Flan programs using arenas, to show whether the escapes that actually occur
|
||
are lexical — is now *producible*, because there is a `Vec` to write them with. That is the next thing to look at,
|
||
not the next thing to build.
|
||
- ~~**The operation table may be one operation short.**~~ **Decided.** `free-all` is retain-capacity and
|
||
`arena-destroy` hands the pages back — two names rather than the mode parameter, so the table the spec froze at four
|
||
operations did not grow. `BUILT.md` states it as the amendment it is.
|
||
- **The `Vec` header is six words in release too, and should not stay that way.** The 32-byte layout the spec fixes is
|
||
blocked on one thing: a redefinition module is built by `llc` and `ld` against a host built separately, and nothing
|
||
makes the two agree on a struct size. Give the reload path a way to carry the build flags and this falls out.
|
||
- **The generation word has no reader.** It is bumped on every reallocation as specified, and the stale-slice trap it
|
||
exists for needs a slice that can carry the Vec's identity — a slice is ptr+len. Either slices grow a word in a dev
|
||
build or the trap does not exist; today it does not.
|
||
- **The allocator grew a budget** (`alloc-budget` / `set-alloc-budget`), which `spec-memory.md` does not have. It is
|
||
there because `retry` is only answerable by a handler that can make the *same* request succeed, and for a fixed
|
||
backing store that handler is the one that raises the ceiling — releasing the region the container lives in
|
||
invalidates the container. Worth folding into the spec or replacing with a growable arena.
|
||
- Escaping closures are still deferred (`spec-memory.md`, "Function values", case 3), and a user-written allocator is
|
||
not one — its procedure is a top-level `defn` with no captured environment. The two should not be conflated when
|
||
function values arrive.
|
||
|
||
### Bugs found and not yet fixed
|
||
|
||
- **Two citations in `spec-memory.md`'s Allocators section do not land where they say.** Checked against Odin
|
||
`819fdc7a8` and Carp `ea121b5a`, every other one is exact — `Map_Info` at `base/runtime/core.odin:369`,
|
||
`Allocator_Proc` at `:422`, the arena answering `.Free` with `.Mode_Not_Implemented` at
|
||
`core/mem/allocators.odin:307`–`308`, `#optional_allocator_error` on `append_elem` at
|
||
`base/runtime/core_builtin.odin:767`, and `// TODO(bill): Better error handling for failed reservation` at
|
||
`base/runtime/dynamic_array_internal.odin:107` and `:128`. The two that miss: `Map_Cell_Info` is at `core.odin:351`,
|
||
not `:350`; and `check.ml:1670` is the FFI `Declare` arm, not the `defer` registration — the claim it is offered for,
|
||
that a top-level `defer` is checked in a scope holding only parameters and globals, is true and lives in `check_fn`
|
||
at `check.ml:1800`–`1812`. `check.ml:505` is the `defer` refusal exactly as cited, and the Carp citations are right:
|
||
`getDropFunc` is `Memory.hs:804`, the drop-before-delete emit is `Emit.hs:1044`, and `docs/Drop.md` says outright
|
||
that `A.drop` "will be run ... when the `let` scope ends".
|
||
|
||
- ~~`web/examples/breakdemo.out` is stale and `check.sh` fails on it.~~ **Fixed.** Commit `4a6a8fa` made the break
|
||
banner number its restarts and the `.out` was never repinned. Nothing had to drive the socket in the end:
|
||
`check.sh` already builds this one `--dev` and runs it under `timeout 5`, keeping what it printed before it
|
||
stopped, so the repin was the `.out` plus the two prose copies of the banner — `web/index.html` and `BUILT.md` —
|
||
and a sentence on the page saying what the numbers are for, since a restart is taken by position.
|
||
|
||
- ~~A shadowed restart is offered and cannot be taken.~~ **Fixed.** A restart is taken by *position* now:
|
||
`(:op "restart-at" :index N :name NAME)` on the daemon, `restart-at N NAME` on the agent, and a numbered
|
||
`completing-read` in `C-c C-b`. `:name` is a receipt, not the lookup — it is checked against the name the snapshot
|
||
holds at that index and refused if the two have drifted, so a bare integer can be wrong out loud. `restart <name>`
|
||
survives for a raw socket and is now defined as `restart-at` on the first index offering the name, so the two verbs
|
||
cannot disagree. `break.flan` grew the shadowed pair and asserts 900, which is the only value in that file no by-name
|
||
lookup can produce. The C&R buffer still marks the shadowed row by name and could now offer it instead — small, and
|
||
not done here.
|
||
|
||
- ~~A restart chosen at a break inside a thunk is accepted, announced, and silently not taken.~~ **Fixed by refusing
|
||
it, with the reason.** Not by the depth NEXT.md proposed: recording the restart-stack depth on *entering the break
|
||
loop* counts the frames a `restart-case` inside the thunk pushed before it erred, and those are above the boundary
|
||
and work. The boundary is where it is made — `restart_floor` is set to `flan_restart_count()` around `j.call()` in
|
||
`flan_agent_poll`, saved and restored so thunks nest — and the outermost `floor` entries of the snapshot are marked
|
||
unreachable. They are listed and marked rather than hidden, refused by the listener before the reply, and carried to
|
||
the editor as `:unreachable (2 3)`. `test_dev.ml` breaks a stopped program a second time from inside `C-x C-e` and
|
||
asserts both halves: index 2 refused, index 0 taken.
|
||
|
||
- ~~Restart names are served from a stack that is being mutated.~~ **Fixed, and it was a precondition rather than a
|
||
separate bug.** Index-based resume is wrong by construction against a moving stack: unlike a name, an index carries
|
||
no evidence of what it meant. The agent copies the list on entering `break_loop` — names into its own buffer, frames
|
||
as the addresses a transfer carries — one snapshot per nested break, and every verb answers from it. Caps are
|
||
`SNAP_MAX` 64 restarts and `SNAP_NAMES` 4096 bytes; past either, the listing says how many it did not show. Neither
|
||
cap has a test; the 4K result cap that shared that blind spot now does.
|
||
|
||
- **A snapshot generation has no test, and the window is a race.** A choice is validated against the snapshot on top
|
||
when the request arrives and resolved against the snapshot on top when the game thread next looks. Between those,
|
||
an evaluation the break loop is running can error and push a break of its own, whose loop would otherwise reach
|
||
[chosen_ready] first and take *its* index 2 for the one someone chose from the outer list. Each snapshot now carries
|
||
a generation, a choice is stamped with the one it was validated against, and a loop claims only what is addressed to
|
||
it — a mismatch is left set rather than discarded, because the listener already answered ok for it. Depth would not
|
||
do: an outer break resuming and a new one starting reuses the number. None of this is tested, because arranging the
|
||
window means landing a request inside a two-millisecond poll from outside the process. It wants a hook the test can
|
||
drive, not a sleep.
|
||
|
||
- ~~The job ring has no fullness check.~~ **Fixed by refusing, at the sender.** Dropping loses a reload the sender was
|
||
told was ok; blocking stalls the accept loop, which serves connections inline, so a program that had stopped polling
|
||
would also stop answering `status` and `abort`. The refusal happens before the `dlopen`, so a module there is no room
|
||
for is never relocated and no handle is taken for it. `programs/agent-queue.flan` blocks on stdin so the window is
|
||
held open by the test rather than by a timer: 64 queued, the 65th refused with a reason, 64 installed when it finally
|
||
polls.
|
||
|
||
- ~~`flan_dev_result_get` is not the seqlock its comment claims.~~ **Fixed by making it one**, rather than by writing
|
||
the honest comment — what it guaranteed was nothing, and the daemon has no other way to read a result. The counter is
|
||
odd while a value is being written, `flan_dev_result_read` copies into the caller's buffer and checks the counter
|
||
either side of the copy, and a reader that loses the race reports the last complete generation and no bytes. The
|
||
count handed out is the number of complete values, so `lib/dev.ml`'s "has it moved" still means what it meant. The
|
||
race itself has no test, for the same reason the snapshot generation above has none.
|
||
|
||
- ~~Smaller: `exit(134)` from the break loop with the listener inside `dlopen`; a `dlopen` handle leaked when a module
|
||
has no installer.~~ **Both fixed.** `exit` runs the atexit chain and the ELF destructors, which want the loader lock
|
||
the listener may be holding — a program asked to abort would hang instead of dying; `_exit`, with the streams flushed
|
||
by hand. The leak was the handle *value* and not the mapping: a module with no installer published nothing, so
|
||
nothing can point into it, and it is closed. The deadlock is read rather than tested; the exit status is tested.
|
||
- **`rt_die` in `flan_rt.c` still calls `exit(134)`**, which is the shape just fixed in the break loop: a trap on the
|
||
game thread runs the atexit chain and the ELF destructors, which want the loader lock the agent's listener thread may
|
||
be holding inside `dlopen`, so a program that should die could hang. Found while fixing the break loop and not fixed
|
||
with it — `rt_die` is the non-dev path too, where there is no listener and nothing to deadlock against, so whether it
|
||
should be `_exit` unconditionally or only under `--dev` is a decision rather than a typo.
|
||
|
||
- **`(A {.x 1})` on a union variant says "unknown struct A"** rather than the union refusal `check_struct` plainly
|
||
intends — `env` has no table of variant names. A diagnostics bug, not a backend death.
|
||
|
||
### Test blind spots, from a mutation pass
|
||
|
||
Sixty mutations, nineteen left the whole suite green. The severe cluster was closed first (`cleanup.flan`,
|
||
`signedness.flan`); the rest are closed now. Every one below was re-planted, watched leave the suite green, and then
|
||
watched fail against the new test before the mutation was reverted — a test nobody saw fail is not evidence.
|
||
|
||
- **`Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies.** `programs/reach-walk.flan`
|
||
calls three functions from three places that are each the only route to them. The failure is not a wrong answer:
|
||
the function is not emitted and the program stops linking, so the case catches the build exception rather than
|
||
comparing output. The `addr` case goes through a `deref` place deliberately, so the index case cannot stand in for
|
||
it.
|
||
- **`flan_dev_global`'s size-change guard.** `programs/reload-v5.flan` is v4 with `extra` as an `i32`, loaded on top
|
||
of v3 in a host run of its own, because what it does is abort. The message is asserted next to the exit status: a
|
||
process that died for another reason is not this guard firing.
|
||
- **A local shadowing an imported name.** `programs/shadow-pkg.flan` binds locals over its own constant and var;
|
||
`pkg-shadow.flan` prints four numbers that separate the expression renamer from the place renamer. Nothing refuses
|
||
a renamer that qualifies through a binding — it reads the top-level name instead and runs — so only the number says
|
||
so.
|
||
- **The 4K result cap and the registry overflow guard.** `test/dev_limits.c`, a second C main beside `reload_host.c`,
|
||
drives them directly: neither has a Flan spelling and no corpus program reaches either. One process per mode — the
|
||
name table never shrinks and the overflow case aborts.
|
||
- **The reader's unknown string escape, and `+5`.** Rows in the reader table, with the escapes it *does* know
|
||
asserted on their decoded bytes rather than through `Form.to_string`, which escapes them again and would compare
|
||
the source with itself.
|
||
- **The hang.** A reader branch that forgets to advance loops for ever, and `dune test` waits as long as it is left
|
||
to; in CI that is a job the runner kills with nothing named. `test/watchdog.ml` arms an alarm on every test binary
|
||
— generous, because an alarm that fires on a slow machine is a flake and a flake is how a watchdog gets deleted —
|
||
and a five-second one around every read in `test_flan`. The first read that does not return wedges the rest, so a
|
||
looping reader costs five seconds and names the row instead of never finishing.
|
||
|
||
What is still open here: the mutation pass has not been re-run since, so the count of nineteen is the old one. The
|
||
sanitized sweep (`@sanitize`) is under the same watchdog but has never been observed to fire it.
|
||
|
||
### Asked for by the editor lanes
|
||
|
||
- **`(:op "condition")` → the stopped program's condition, rendered.** Two steps: `break_loop` currently does
|
||
`(void)condition;` and *discards the pointer*, so stash it beside `condition_name`; then the daemon builds a render
|
||
thunk aimed at that address, which is `Session.render` rooted at a `Ptr` instead of an expression. **The second step
|
||
now exists** — `Session.render_locals` is exactly that thunk, rooted at an address the program supplies — so what is
|
||
left is the first: keep the pointer, and give the agent a verb that hands it back. The type is already known: it is
|
||
the `condition_name` the break loop reports, which `layout` already resolves.
|
||
- **The type identity is settled, and it is the qualified name** — `layout` is in, see BUILT.md. `Load` qualifies
|
||
every declaration at import, so the names in `Tast.structs` are a flat namespace where two packages' `Missing` are
|
||
`a/Missing` and `b/Missing`; a bare name is refused with the candidates rather than resolved. `condition` inherits
|
||
it for free: the string the break loop already reports *is* that name, because `Emit.struct_name_of` writes
|
||
`Types.Named` into `flan_error`. It is still open for **locals**, where DWARF gives a name and the name a debugger
|
||
reads is not qualified by anything.
|
||
- ~~**`(:op "backtrace")` is blocked** on frame metadata.~~ **Built**, and not out of DWARF: decision 3's shadow
|
||
stack carries the name and the location on the frame itself, so a backtrace needs no debug information at all. See
|
||
BUILT.md, "The shadow stack, and `backtrace`", for what it costs. **Locals landed with it** — the pointer-rooted
|
||
render thunk turned out to be `Render.render` over a `Deref` of a slot's address, and one new arm in the backend.
|
||
See "Locals of a stopped frame" for the four things it refuses. Restart source locations and arity are still blocked — `flan_restart` carries `prev`, `name_id`, `name` and `namelen`, so both need a new
|
||
field in the frame, which means the compiler emitting it.
|
||
|
||
### One line away
|
||
|
||
- **`match` over enums.** Fully desugarable, wanted, and blocked only by `Ast.pattern` needing a keyword case, which
|
||
`load.ml` matches exhaustively.
|
||
- **`Build.executable` returns only `out`**, so the daemon recovers the host `.ll` by recomputing `Build.workdir ()`.
|
||
- **A `!DILexicalBlock` per `Let`.** Not one line, but the one thing left in the DWARF work: every `!DILocalVariable`
|
||
is currently scoped to the subprogram, so inside `(let [v 22] …)` nested in `(let [v 11] …)` lldb still answers
|
||
`p v` with **11**. The `~2` suffix makes both *visible*, which is not the same as making the answer right. It needs
|
||
block structure the typed IR does not carry, and the `llvm.dbg.declare`s moved out of the entry block.
|
||
|
||
### Deferred with a reason
|
||
|
||
- **Writing through a string literal** — see Sharp edges. Needs provenance, which is open decision #3.
|
||
- **`cstring` as a type.** Odin has no `string → cstring` conversion at all; it pays the same copy our shim already
|
||
makes. The one thing it buys is the *return* direction, and nothing in `vendor/raylib` returns a string.
|
||
- **`rune`.** Odin's is a 4-byte integer distinguished by a flag, so `i32` is the same thing. Non-ASCII text is
|
||
blocked on font loading, not on the string layer — and fonts are now bound.
|
||
- **Macro expansion.** The reader and the declaration are in. Running a macro means compiling it and `dlopen`ing it
|
||
into the compiler, which is the reload primitive pointed at ourselves — but a macro is `[Form] -> Form`, so `Form`
|
||
has to be a Flan union whose layout the compiler and the loaded macro agree on, and union *values* are milestone 6.
|
||
|
||
### Documents that contradict the code
|
||
|
||
- **`plan.org`'s jank #947 citation is wrong in its mechanism.** jank does not relink (it calls through vars, which
|
||
are already indirection cells) and never unloads (`remove_symbol` has no callers). The real cause was a
|
||
process-teardown race. We are safe from the repro — because we compile out of process, not because of cells. A
|
||
normative document citing the wrong mechanism protects the wrong invariant.
|
||
- **`plan.org` still lists open decision #7 as open** and the interpreter as a backend. It was settled the other way;
|
||
`NEXT.md` records the consequences as "already applied" to `plan.org`, and they never were.
|
||
- **nREPL's `eval` does carry `file`, `line` and `column`** — jank reads all three. The choice of s-expressions still
|
||
stands on its other grounds; the stated reason does not.
|
||
|
||
## Sharp edges
|
||
|
||
- **Two formatted numbers cannot be held at once.** `flan_i64_to_bytes`, `flan_f64_to_bytes` and `flan_u64_to_bytes`
|
||
all write into one `static char scratch[64]` — "rendered text lives here until the next call", flan_rt.c:184 — and
|
||
`(string b)` does not copy. So
|
||
|
||
```
|
||
(let [a (string (i64->bytes 11))
|
||
b (string (i64->bytes 22))]
|
||
(print a) (print " ") (println b)) ; => 22 22
|
||
```
|
||
|
||
`a` is 11 and prints 22. No crash and no diagnostic. This is not new — the `[u8]` already aliased — but a `string`
|
||
reads as more value-like and invites exactly this. Format, draw, measure, then format the next one; `digits.flan`
|
||
sequences itself strictly for this reason. `rl/draw-text` is safe because the shim's `flan_shim_cstr` copies out of
|
||
ptr+len before the call.
|
||
|
||
- **Writing through a string literal is undefined, and the two build modes
|
||
disagree about how.** `(let [s (bytes "Hi")] (set (at s 0) \h))` stores into
|
||
a `private unnamed_addr constant`. At `-O0` that is a store to read-only
|
||
memory and the program takes SIGSEGV; at `-O2` LLVM deletes it as undefined
|
||
and the program prints `Hi` and exits 0. Same source, and which way it fails
|
||
depends on a flag — the worst shape available, and worse than either outcome
|
||
alone.
|
||
|
||
Nothing refuses it. `bytes` turns a `string` into a `[u8]`, the language lets
|
||
you write through a slice, and by then nothing records that the bytes came
|
||
from a constant. The honest fix is provenance — knowing a slice's origin —
|
||
which is plan.org open decision #3 and deliberately deferred. A cheaper one
|
||
that is *not* a fix: emitting literals as mutable globals only moves which
|
||
flag misbehaves, and costs their read-only placement.
|
||
|
||
Found by the string lane while deciding whether `lower-ascii` should mutate
|
||
in place. It ships the copying version for exactly this reason, and that is
|
||
the rule to follow until provenance exists: **a function over a `string` must
|
||
not write through it.**
|
||
|
||
|
||
Most of these are edges the language keeps and you should know about. Two — the top-level namespace and the shift count,
|
||
both found by review after milestone 4 — were bugs that reached LLVM or ran wrong, and are **fixed**; each says so. They
|
||
stay written down because each one is now a rule the checker enforces, and a later change could quietly drop it.
|
||
|
||
- **An index converts from a narrower integer and never from a wider one.** `(at colors current-color)` with a `u32`
|
||
index works — anything above 2³¹ truncates to a negative `i32` and the unsigned bounds check rejects it. An `i64` index
|
||
is refused with the reason: 2³²+5 truncates to 5 and would read the wrong element with no trap at all.
|
||
- **There is one top-level namespace, and `check.ml` now enforces it.** The environment's tables are per-kind — structs,
|
||
unions, aliases, enums, functions, externs and globals each have their own — so only a function was ever checked for a
|
||
duplicate. `(defn item …)` beside `(defvar item …)` type checked and then died in LLVM as `redefinition of function
|
||
'@flan.item'`, a message about an emitted symbol with no source location left, and two colliding *type* declarations
|
||
were not caught anywhere. One pass over `Ast.declared_name` now runs before every other collection pass and rejects the
|
||
second declaration of a name whatever kind either one is. `declared_name` lives in `ast.ml` because `Load` needs exactly
|
||
the same set — the names an import renames — and two copies of that list would drift.
|
||
- **A shift count is bounded, two different ways.** A shift by the operand's own width or more is *poison* in LLVM, not
|
||
a wrong number: `(defn main [] i32 (<< 1 32))` compiled at -O2 to a bare `retq`, returning an undefined value. A literal
|
||
count out of range is now rejected in `check.ml` — that is the typo case — and `emit.ml` masks a computed count to
|
||
`width - 1`, which is what the hardware does anyway and which LLVM folds away whenever the count is constant. The
|
||
prelude's rotate masks its own count; that is now redundant but harmless.
|
||
- **A `u64` literal is its 64-bit pattern**, so `0xcbf29ce484222325` is a real `u64` and not an error. The cost is that
|
||
a negative *decimal* literal is accepted as a `u64` too, because the reader records the value and not how it was
|
||
written. Narrower unsigned types keep the strict check, which is where a typo like `300` for a `u8` actually shows up.
|
||
- **A folded constant skips `check`.** `(defconst rows (/ h c))` is emitted from the folding pass's value, because a
|
||
global's initialiser has to be a compile-time constant and only that pass knows this one is. Its range check is
|
||
therefore its own call to `in_range`; there is a regression test.
|
||
- A `let` binding takes no type annotation, which is why `sand.flan` names its FNV constants instead of writing them
|
||
inline.
|
||
- `(defn f [] f65 0.0)` still says *unknown name* rather than *did you mean f64*: with a single body form the parser
|
||
cannot tell a return type from the first expression. Only the parameter position and `(Option …)` are unambiguous.
|
||
|
||
## Loose ends from milestone 4
|
||
|
||
None of them blocking: block-scoped `defer`; package visibility, so `rl/get-color-raw` is not callable; a package
|
||
importing a package; imported unions.
|
||
|
||
## Macros — the reader and the declaration are in, the expander is not
|
||
|
||
The front half landed. What exists:
|
||
|
||
- **The reader** reads `` `x ``, `~x` and `~@x` as `(quasiquote x)`, `(unquote x)` and `(unquote-splicing x)`, exactly
|
||
as `'x` reads as `(quote x)`. It stays dumb: it does not count nesting levels, does not know whether an unquote is
|
||
inside a quasiquote, and attaches no meaning to the three names. Clojure's spelling, not Common Lisp's, because a comma
|
||
is whitespace in `is_delimiter` and every binding vector in the corpus relies on that. Backtick and tilde are delimiters
|
||
now, so `a~b` is two things.
|
||
- **`parse.ml` refuses all four by name.** `quasiquote` and `gensym` say expansion is not wired up; `unquote` and
|
||
`unquote-splicing` say they mean nothing outside a quasiquote, which is a mistake rather than a missing feature.
|
||
`(defmacro name [params] body ...)` at the top level is checked for shape and *then* refused — a malformed defmacro and
|
||
an unimplemented one get different reasons, so the shape rule is enforced before the feature exists.
|
||
|
||
Nothing is stored. There is deliberately no macro table and no `Ast.Defmacro`, because a table nothing reads is a place
|
||
for a design to rot, and the storage shape is the expander author's first decision, not a decision to inherit.
|
||
|
||
### How the expander should work
|
||
|
||
**There is no interpreter** (see "Why there is no interpreter" in `BUILT.md`) and there is not going to be one, so running a macro at
|
||
compile time means *compiling it and loading it into the compiler*. That machinery already exists and is measured:
|
||
`Emit.redefinition` → `Build.shared` → `dlopen` is ~19ms end to end, with the load itself at 0.04ms (see "The reload
|
||
primitive"). A macro is that pipeline pointed at the compiler's own process instead of the program's.
|
||
|
||
The shape it wants:
|
||
|
||
1. **A macro is a function `[Form] -> Form`.** Its parameters are forms and its result is a form, which means `Form.t`
|
||
has to exist on the Flan side — a `defunion` mirroring `lib/form.ml`, in the prelude, plus constructors and accessors.
|
||
That is the real work, and it is bigger than the expander itself: the compiler and the compiled macro have to agree on
|
||
the *layout* of a `Form`, not merely its shape, so whatever the checker does for unions has to be exact here. Until
|
||
unions are values this cannot start — `check.ml` puts union values and `match` on a union at **milestone 6**, so that is
|
||
milestone 6 work landing before milestone 5's.
|
||
2. **Expansion runs over `Form`, before `Parse`.** Not a pass over `Ast`: there is no `Ast.Defmacro` and `Parse` refuses
|
||
`defmacro` outright, so an `Ast`-level pass would have nothing to work with. That refusal is not a dead end, it is the
|
||
ordering — the expander runs first and `Parse` never sees a macro call at all. It is also the Clojure ordering, and the
|
||
reason a macro expanding to a special form is ordinary rather than a special case.
|
||
3. **Order matters and files do not have one.** Top-level names in a package are order-independent everywhere else
|
||
(`declared_types`, the constant fixpoint in `check.ml`). Macros cannot be: a macro must be compiled and loaded before a
|
||
call to it is expanded. Either collect every `defmacro` in a pre-pass and compile them as one module, or require
|
||
definition-before-use for macros specifically and say so in the error. The pre-pass is better and matches how the rest
|
||
of the frontend already behaves.
|
||
4. **A macro's own body may call macros**, so the pre-pass is a fixpoint, not a single sweep, and a cycle has to be
|
||
detected and named rather than looping.
|
||
5. **`gensym` is a runtime function of the compiler**, called by the loaded macro while it runs. It needs a counter that
|
||
lives in the compiler process and a name that cannot collide with a reader-produced symbol — the usual trick is a
|
||
character no symbol may contain, and this reader now has two new ones it could reserve. Hygiene is settled (plan.org,
|
||
open decision 2): deliberately non-hygienic, Common Lisp/Clojure style, explicit `gensym`, no `macrolet` until a
|
||
concrete use case appears.
|
||
6. **Quasiquote itself is a macro-shaped desugaring**, not a compiler feature: `` `(a ~b) `` becomes list-construction
|
||
over quoted pieces, with `~@` splicing. Written once, in the expander, over `Form`.
|
||
|
||
The four files this touches — `build.ml`, `check.ml`, `emit.ml`, `load.ml` — were owned by other lanes when the front
|
||
half landed, which is the only reason the expander is not here too.
|
||
|
||
### What would tell you it works
|
||
|
||
`when`, `unless`, `until`, `cond` and `dotimes` are special forms in `parse.ml` today, and plan.org milestone 5 says
|
||
they are special forms *only until macros land*. Moving one of them out of the compiler and into the prelude as a
|
||
`defmacro`, with the existing tests unchanged and still green, is the exit criterion — it proves expansion, quasiquote,
|
||
`gensym` and the ordering pre-pass at once, against a test suite written before any of them existed.
|
||
|
||
## Watch for
|
||
|
||
The rule that caught the two misparse bugs applies unchanged: **anything that binds a name, alters control flow, or is
|
||
not yet implemented must be recognised explicitly and rejected if unsupported.** `check.ml` rejects `Vec`, `Map`,
|
||
`Result`/`try`, union values, closures, quoted symbols, generics and function values *by name*, each with the milestone
|
||
it belongs to; `load.ml` rejects the package shapes it does not handle; and the FFI boundary rejects an aggregate. The
|
||
tests assert on the reason, not just on the failure.
|
||
|
||
## Untracked on purpose
|
||
|
||
`calc-me` and `sand`, the executables `flan build` drops beside their sources, are now in `.gitignore` — anchored
|
||
(`/calc-me`, `/sand`) so the patterns cannot match anything nested.
|
||
|
||
`old-ocaml/` — the pre-rewrite menhir/ocamllex frontend, kept as reference and excluded from the build by the root
|
||
`dune` file. Its contents are also in git history at `2c232dd`.
|
||
|
||
## Handoff: the shadow stack lane, stopped mid-repair
|
||
|
||
Two commits landed and are green: the shadow stack with `(:op "backtrace")`, and `(:op "locals" :frame N)`. See
|
||
BUILT.md's two new sections for the design and the measurements. A third commit was **half-built and its own test
|
||
left red** on purpose; **it is finished now** — see the struck item 1 above — and the rest of this section is kept
|
||
because the parts of it that were true are still worth having.
|
||
|
||
**What is broken, exactly — and this paragraph was wrong; kept for what it cost.** It said `locals` compares the
|
||
frame on the stack against the body this session holds and the comparison is not firing, that every piece of the
|
||
fingerprint was written, and that one of five hand-offs was dropping the number. Four of the five were never written at
|
||
all: `Emit.fninfo` stored the fingerprint and nothing else touched it. The first step it recommended — printing both
|
||
sides of the comparison in `Dev.locals` — could not have worked, because `Dev.locals` had no comparison to print. The
|
||
lesson is the ordinary one: a lane that stops mid-repair should say which pieces it *ran*, not which it believes it
|
||
wrote.
|
||
|
||
**Not obvious from the diff.** Two things cost a day between them. The linked-list frame beat an array-with-a-stack-
|
||
pointer on both benchmarks, which is the opposite of what the escaping-alloca argument predicts, and the measurement
|
||
that first said otherwise was comparing a 40-frame binary with a 600-frame one; every number in BUILT.md is now a
|
||
minimum of nine runs for that reason. And `redefinition`'s transient rule (`m.nstr = 0`) silently stops every module
|
||
carrying a string literal from ever being unloaded — the frame descriptors go through their own counter, `m.nfi`, for
|
||
that reason, and a locals thunk passes `~retains:false` because everything it emits is memcpy'd into the result buffer.
|
||
|
||
**No Emacs surface.** `backtrace` and `locals` are daemon ops; nothing in `emacs/` calls them yet. One command showing
|
||
the backtrace with the selected frame's locals is the whole of what is missing, and `flan-cnr.el`'s
|
||
fixture-driven shape is the model.
|