flan/FIX.org
Joseph Ferano cc2cfa175b Sets are read, and the reader that answers a Value is the package's
The tokenizer refused #{} because "it needs a hash set to even
represent" — which is a claim about a reader, and a tokenizer represents
nothing. #{ now pushes } on the same balance stack { does, there is one
new token kind and no new closer, and err-set is gone rather than kept
with a message it no longer earns. skip-value needed nothing: it is
written against the depth and not against the kinds.

The dynamic reader moves out of test/programs/arena-edn.flan and into
vendor/edn/read.flan as (edn/read bytes), answering an (Option Value)
against whichever allocator the caller bound. Two decisions are written
down where they are made:

  * a set is a Value.Set holding a deduplicated (Vec Value), because
    (Map Value bool) does not typecheck — keyable refuses a key holding
    a Vec or a Map — and restricting elements to keyable Values would
    refuse #{[0 0] [1 0]}, which is the file this was built for. Insert
    is O(n) against a structural value=?, so building the tileset's 54
    pairs is 1458 comparisons, once.
  * a Value copies every string into the allocator where a Token stays
    a view. A view handed back out of the function that owns the buffer
    is a dangling pointer, and free-all would not even take it. Odin's
    json parser clones for the same reason.

An imported defdata was a refusal in load.ml — "not implemented yet
(milestone 4)" — and it had to go first. It is the type's name plus the
Type. half of a constructor symbol, which arrives as a Var node when the
case has no fields and a Struct node when it has; a match pattern needed
nothing, because a case resolves against the scrutinee's type and was
never a top-level name. programs/pkg-data.flan is that on its own.

programs/edn-read.flan reads assets/edn/tileset.edn, which is the
editor's real output: :texture-path and a :selected-cells of 54 integer
pairs, with no type declared for any of it. It also overwrites the
source buffer in place after reading and prints the document back, which
is the copy contract asserted rather than described.
2026-09-18 22:50:54 +07:00

342 lines
19 KiB
Org Mode
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

* Stuff I've found
** Why do I need to call flan-dev to open another window?
let: flan: the program exited; restart flan dev
** I can't eval a toplevel defvar, need to eval-defun (C-c C-c)
For Flans intended live-program workflow, C-x C-e on any complete top-level form should do the natural thing:
- expression → compile/run temporary thunk; print its value
- defn, defvar, defmacro, etc. → compile/install it; report what changed
The compiler already has both paths. The current split is an editor/UI artifact: C-x C-e is wired directly to eval-expr, while C-c C-c is wired to declaration reload. It is not a fundamental limitation.
A good fix would make C-x C-e context-aware: if the enclosing form is top-level, send it through the declaration evaluator; otherwise use expression evaluation. Then C-c C-c can remain a convenient explicit “reload this definition” alias, but not the only way defvar works.
** I can't eval a top level Vec
slurp returns (Vec u8), an owning, move-only buffer. Flan currently forbids every move-only global because it has no global ownership/lifetime model: any function could read and free it, while ownership tracking only exists within one function.
For data that is fixed at build time, use an embedded immutable array instead:
(defconst the-data (embed "game-data.edn"))
That produces a fixed [u8], not a heap-owning Vec, so it can live globally. It also resolves relative to sand.flan.
If game-data.edn genuinely must be loaded at runtime, then today it has to be owned by a local—typically load it in main and pass it through the functions that need it. For a game-wide runtime-owned data asset, that is a missing language/runtime feature, not a bad use case on your part.
** The edn module seems to need a struct declaration, it should do both; go into a struct but also return a Map with Vecs and Sets when we don't provide a type
** defenum needs optional autoincrementing discriminants
** We need a javascript backend so we can reach the world
** We need to have C-style unions, maybe those are called defunion, and then sum types are defdata or deftype
* Decisions, 2026-09-17
** 1. Re-runnable main — DISPATCHED
The process does not actually die. [flan_exit_hook] is [flan_merged_exit]
(lib/dev.ml:2669): it flushes, reclaims fd 1, and parks in [for (;;) pause()].
What is missing is a way to wake it. The park becomes a condvar wait, a daemon
op signals it, and the main thread — not a new one, because raylib wants the
main thread — re-enters [flan_program_main]. [alive] gains a third state,
parked-and-re-runnable, and each of the ten guard sites decides for itself
whether it accepts one.
Globals are NOT reset between runs. That is the CL/Clojure semantics asked for:
the process never died, so a second (main) sees what the first one left.
** 2. C-x C-e on a top-level form — QUEUED behind 1
Same file as 1 (emacs/flan-dev.el), so it waits rather than merging by hand.
No design questions; the note specifies it.
** 3. Runtime-loaded owning globals — DISPATCHED
Not a missing global ownership model. One rule: a move-only global is legal,
and reading one is always a borrow, never a move. Nothing takes ownership,
nothing frees it, its lifetime is the process's. Sound precisely because the
lifetime question has a constant answer.
Mutable in place as well — a global Vec can be pushed to. Aliasing follows
whatever locals already do; no new borrow regime for globals that locals lack.
[embed] (lib/check.ml:4265) still covers build-time data and is untouched.
** 4. edn both typed and dynamic — REDIRECTED to arenas, drop parked
Two projects, not one. (read-edn T bytes) does not exist — vendor/edn/edn.flan
is only a tokenizer, and the compile-time struct walk is NEXT.md item 9.
[drop] was dispatched to unblock the dynamic half and is being PARKED unmerged
on its branch, not reverted, because the premise was wrong. The refusal at
check.ml:599 is about *teardown*, not ownership: the type-erased runtime
releases slots bytewise and cannot walk a move-only element. An arena never
releases a slot — [free-all] takes the whole region — so the premise does not
hold there.
That is also what Odin does, which NEXT.md:1620 already recorded: no
destructors, no drop, no finalizers; [delete] frees container memory and
nothing else. core:encoding/json ships a hand-written recursive
[destroy_value] in the *library*, and the idiomatic alternative is to parse
against temp_allocator and [free_all]. Neither is a language feature. Building
[drop] was a departure from NEXT.md:1589's settled "defer stays the answer",
taken on the assistant's prompting and withdrawn.
So: lift check.ml:599 for arena-allocated containers, and let read-edn take an
allocator — which is already the idiom, since spec-memory.md:283 makes the
allocator part of the calling convention with an explicit override.
The real cost, stated because it is not free: [can-free] is a RUNTIME
capability on the allocator value while check.ml:599 is a COMPILE-TIME refusal,
and the compiler cannot generally know statically that a construction site's
allocator is an arena. The spec's answer for the analogous drop case is a check
at the point of construction, one branch per container — a runtime branch. This
likely becomes a runtime trap rather than a static guarantee.
Ownership tracking itself is untouched. Moves are still tracked; what is given
up is freeing one element individually, which is the point of an arena.
** 5. defenum autoincrement — DISPATCHED
C's rule: no value means previous+1, the first is 0, explicit and implicit mix.
Duplicates: an explicitly written one is an intended alias and is allowed. One
produced by autoincrement walking into a value another member holds is an
accident and is refused, naming both members.
** 6. JavaScript backend — HELD
wasm32 already works: test/wasm-run.mjs is a WASI host, the test table runs
wasm32 builds, web/index.html is in the tree. A second backend beside emit.ml
and x86.ml is the largest item here and the dev loop comes first.
** 7. defdata and defunion — QUEUED last
Today's [defunion] is already the tagged sum type. It is renamed [defdata],
and [defunion] becomes the C-style untagged one. Serves both FFI and type
punning, and cimport verifies it against the header where one exists —
cimport.ml:295 currently skips any record holding an anonymous union, leaving
the defstruct beside it unchecked.
Last, because the rename sweeps parse/check/emit/prelude/docs and every .flan
file, and would conflict with everything above.
* Open, found while working the list
** A transient signal -11 on the globals daemon
Seen once, in one of three consecutive test runs, by the agent doing the
C-x C-e work; the runs either side of it were clean. Not reproduced since —
three forced full runs (dune test --force) are green, 232 checks, 0 failures.
Worth remembering rather than chasing, because the daemon it appeared on is
one the move-only-global work (c124df3) changed: test_reload's fixture gained
a host global Vec and a run-time-new one. A teardown or a reload module that
defines rather than declares a global Vec would strand the block the live
process is using, which is exactly the shape a rare SIGSEGV takes. If it comes
back, start there.
* Status, end of 2026-09-17
Six of seven items are merged on dev-loop and green (dune build, dune test
--force, @x86, @page). One agent is still running: the arena work for item 4.
| item | what | state |
|------+------+-------|
| 1 | re-runnable main after the window closes | merged |
| 2 | C-x C-e installs a top-level form | merged |
| 3 | move-only globals, borrowed never moved | merged |
| 4 | edn dynamic value | merged (arena route) |
| 5 | defenum autoincrement | merged |
| 6 | javascript backend | first lane IN FLIGHT, 2026-09-17 evening |
| 7 | defdata rename + C-style defunion | merged |
Also merged, not from the list: macro-module symbol visibility, which
unblocked [flan dev --x86] in one process. sand.flan --x86 builds in 292ms
against 985ms on LLVM.
** Parked branches, kept deliberately
- worktree-agent-a18e9e62485eaedb5 — [drop] and recursive teardown. Finished
and green, not merged. See docs/handoffs/HANDOFF-drop.md, which is the part
worth keeping. Withdrawn because the refusal it answered is about teardown,
and an arena has none; see item 4 above.
** Open, carried forward
- A transient signal -11 on the globals daemon, seen once, not reproduced.
Recorded below.
- [drop]'s handoff flags that the [clone] / [get] / [map-next!] refusals are
needed by the arena route too — they are about a copy of a header, which an
arena does not make safe — and that a Map has no operation answering *where*
a value lives, which is what reading an arena-parsed EDN document back would
need. Both were relayed to the arena agent.
- [Tast.Addr (Tast.Pfield ...)] on an Option: closed, and closed as
unreachable rather than fixed. Nothing in the source language builds it.
[.field] goes through [struct_target], which admits a struct and a pointer
to one and refuses everything else by name with a location — "(Option Point)
is not a struct, so it has no fields" — so [(addr (.x o))] never reaches a
place for [addr] to take. The node the drop lane hit was one the compiler
built for itself. Two rows in test_flan.ml pin the refusal, on the bare field
and on the address of one.
What is still asymmetric, and is a note rather than a bug: [x86.ml]'s
[field_loc] does lay out an Option's tag and value, and [emit.ml]'s [place]
admits only a named struct. Neither is reachable, so neither is tested, and
growing the LLVM side to match would be untestable code written to balance a
path nothing takes.
- Re-run still does not work under --two-process: a finished child is
genuinely gone. It now works under --x86 because --x86 runs merged.
- sand.flan still holds an uncommitted experiment line that is refused with a
message naming the fix: use (defvar the-data (Vec u8)) and fill it in a
function.
* Landed on dev-loop
Items 1, 2, 3 and 5 are merged and green (dune test --force, 232 elisp checks,
0 failures). Items 4 (drop) and 7 (defdata) are still being written. Item 6 is
held.
** Re-run is merged, and does not work under --x86
Park and re-run live in the merged entry point's main(), and --x86 refuses the
merged daemon by design: a merged host exports every flan.* body for -rdynamic
and so interposes the prelude bodies of the LLVM-built macro module the
compiler loads into itself. --x86 therefore runs --two-process, where the
program is a child, and a child that finishes is genuinely Gone — there is
nothing to wake. [Program.rerun] answers with the two-process refusal rather
than the merged one, and a test pins it.
Re-run on x86 needs the merged daemon to accept --x86 first. Separate work.
** The headline complaint is verified fixed on sand.flan
Window opened, closed, the daemon reported parked, (:op "rerun") returned ok,
and xdotool found a live window from the second run.
The earlier claim that [flan dev sand.flan] failed with "unknown function
begin-drawing" was true only on the stale base the work started from, and was
retracted after a re-test. Nothing to chase.
* Evening of 2026-09-17 — the review, and four lanes off it
docs/REVIEW-production-readiness.md is the production-readiness review, written to be
implemented from. Item 4 (arena EDN) merged green before it was written, so the FIX list
proper is six of seven done and one in flight.
Four agent lanes are running off the review, each in its own worktree:
- Tier 1 + the runtime half of Tier 4: overflow guards, map removal (Odin's
backward-shift), the registry race, the scratch buffer, Addr-on-Option, the dev-runtime
aborts. One lane because they share runtime/*.c.
- Tier 3: clock, math, getenv, basic file ops. Appends to flan_rt.c in its own section so
the merge with the lane above stays clean.
- Tier 4 without the runtime: CLI error arms, flan run flags and -O, CI, the stale
DISCUSS.md x86 table, README's missing subcommands and env-var table.
- The JS backend's first slice, per docs/DISCUSS.md §5's settled decisions: #_ first, then
one function under node, then the corpus with a MATCH/DIFFER/REFUSED survey. New
reference clones for it are recorded in docs/REFERENCES.md ("Compiling to JavaScript").
Tier 2 (install and shipping) was explicitly passed over. Package visibility is skipped in
every lane — it needs a syntax decision from the author.
** All four lanes merged, end of 2026-09-17 evening
Tier 1 (runtime correctness + the runtime strays of Tier 4), Tier 3 (clock,
libm, getenv, file verbs), Tier 4 (CLI arms, flags, CI, docs) and the JS
dialect's first slice are all on dev-loop. Verified together: dune test
--force 232/0, @x86 116 MATCH / 0 DIFFER, @js 0 DIFFER, @sanitize clean.
Left for the author, recorded where each lives:
- Package visibility needs a syntax decision (review Tier 4 item 5).
- The v->gen word: spec-memory.md mandates it, nothing reads it; delete or
implement is a spec amendment (BUILT.md records the two options).
- sand.flan:167 still holds the refused defconst experiment; the diagnostic
now prints in full and names the fix.
- Tier 2 (install and shipping) deliberately not started.
* The repeal, 2026-09-18
The ownership flow analysis is removed: the per-function dead set, the borrow
flag, the loop-iteration diff, and the borrowed-never-moved rule for globals.
Use-after-move and double-free are no longer compile errors. What stands:
move-only as a type property (assignment hands over the header, clone is the
only copy), the struct/union/pool ownership rules, defconst-vs-defvar for
move-only globals, defer, all allocator capabilities, and the dev build's
generation checks — now the primary net, which is the Odin position the
memory design came from.
Decided after the bug hunt put four of its ten lanes inside this machinery.
An unsound checker is worse than none, because it is believed. The door back
is spec-memory.md's provenance pass: removal widened acceptance without
changing any accepted program's meaning, so a stricter pass can return
additively. spec-memory.md "The repeal" is the amendment; BUILT.md and
NEXT.md are annotated at their live claims.
Two of the day's fix lanes were cancelled with this (borrowed-flag, region
element); the while-condition fix merged in the morning is deleted again by
the repeal, and its pin with it.
* The second round, 2026-09-18 — Pool, move-only, and the gen word
Ordered by the author after the ownership repeal, on the same argument: the
Odin position, full stop.
- Pool and (Handle T) are gone — types, checker arms, runtime section,
fixtures. Two containers are enough; a slab with generational handles is a
library over a Vec when a program wants one.
- The move-only concept is gone: everything copies as its header, copyable?
left the predicate list (four remain), and the struct/defdata/defunion
owning-field refusals are lifted. The region rule stands untouched — a
container of owning elements is still built against a region and released
by one free-all.
- The gen word left both container headers (read by nothing since it was
written). A Vec is ptr len cap allocator epoch; the epoch trap stays.
- Container globals keep both declaration rules, reworded: they start zeroed
(a global initialiser is a compile-time constant, and a container's only
constant is the empty one), and a defconst container is refused since a
constant is not an assignable place. sand.flan's experiment line would now
be refused with the reworded sentence.
Everything verified together: dune test --force 232/0 with zero suite FAILs,
@x86 122 MATCH / 0 DIFFER, @sanitize clean, and the whole-repo check sweep
against the parent differs only where it should: the two negative fixtures
now accepted (vec-in-struct, the clause-less generics corpus), the two pool
fixtures deleted.
* End of 2026-09-18 — the hunt closed out
Ten bug lanes and three demolitions, all on dev-loop and verified together:
232 checks / 0 failures, @x86 122 MATCH / 0 DIFFER, @sanitize clean.
- Fixed: while-condition move (then repealed with the machinery), defenum i32
range, the Emacs client's framing/poll/point-min/quit bugs, the session
NULL-cell rollback, the stdout pipe drains, x86 shift masking, the reversed
slice traps in every build, NaN prints unsigned, the registry answers
honestly under churn, and reg at's stopped-only race.
- Removed by decision: ownership flow tracking, Pool and Handle, the
move-only concept, the v->gen word. spec-memory.md carries the repeal.
- Still recorded, not scheduled: trap paths that bypass the break loop,
parked orphans outliving dead daemons, emit.ml's transient test and
globals, map-grow's stale quote, x86's slice-from-ptr sentence, the
float->int UB divergence, the narrowed-buffer C-x C-e quirk, and the JS
backend's items (deprioritised).
* Late 2026-09-18 — the last two lanes
- Six trap paths park instead of killing the session — merged (e6d85c8).
All six park for inspection; transfer-fail and restart-unarmed refuse the
resume with the trap's sentence. New flan_trap_hook beside flan_break_hook,
whose contract could not carry these. Standalone builds die as before.
- Parked orphans exiting with their daemon — merged. PDEATHSIG on the
two-process child, armed in the agent, with a spawn-SIGKILL-reap test.
Found in passing: the eight orphans split 4/4 — four are MERGED daemons
whose editor vanished, a separate defect (accept_loop has no client
liveness), recorded here, not fixed. The eight are still alive and the
author decides their fate.
* The dynamic half of item 4, finished
Item 4 above is the complaint at line 23: the edn module should read into a
struct *and* answer a dynamic value when no type is given. The dynamic half is
now the package's rather than a test program's.
- [#{}] is read, not refused. The tokenizer's stated reason ("needs a hash set
to even represent") was a claim about a reader, and a tokenizer represents
nothing; [#{] pushes [}] on the same balance stack [{] does, one new token
kind, and [err-set] is gone rather than kept with a new message.
- [vendor/edn/read.flan] holds the [Value] data type and [(edn/read bytes)],
which answers an [(Option Value)] against the calling convention's
allocator. A set is [Value.Set] holding a deduplicated [(Vec Value)] —
[(Map Value bool)] does not typecheck, because [keyable] refuses a key
holding a Vec or a Map, and restricting set elements to keyable Values would
refuse [#{[0 0] [1 0]}], which is the file this was built for.
- A Value COPIES every string into the allocator; a Token stays a view. The
two layers diverge deliberately and both headers say so. A view handed out
of the function that owns the buffer is a dangling pointer no free-all would
even take back.
- Needed one compiler change to be possible at all: an imported [defdata] was
a refusal in load.ml ("not implemented yet, milestone 4"). It is a rename of
the type's name plus the [Type.Case] half of a constructor symbol; a match
pattern resolves its case against the scrutinee's type and never needed one.
Still not built, still item 9 on docs/PORTING.md's list: [(read-edn T bytes)],
the typed half. It wants a compile-time walk over a struct's fields and there
is no run-time type information to do it with at run time.