Clojure's spelling and Clojure's semantics. Repeated — #_#_ a b c — discards
that many following forms, and that falls out of the recursion rather than
being counted: the discard reads *a form*, and the form it reads may itself
begin with a discard, so the outer one throws away what the inner one already
stepped past.
It belongs to read_form rather than to the sequence readers, which is what makes
it work in every position a form can appear — top level, inside a list or a
vector or a map, before or after a quote. The two loops that look for a closer
or for end of input skip it as well, because a discard is not an element and a
file ending in one has read everything there is to read.
A trailing #_ with nothing after it is an error, and it is the same error an
unterminated form already gives.
Nothing in dune test exercised cimport.ml or cjson.ml. The raylib case is the
better evidence and the worse coverage: it needs raylib installed, at the
version whose .so is linked, with FLAN_RAYLIB_H set, so as the only test of
this it would skip everywhere and cover nothing.
test/headers/sample.h is one function per decision the importer makes, and the
table asserts on the reasons rather than the counts — a refusal that fires for
the wrong cause still refuses, and a count still matches. Accepted: an
aggregate in and out, const char * as a string, a pointer parameter, a second
typedef name for a record described once, a C enum against a defenum. Refused,
each by reason: a returned char *, a non-const char * C may write through, a
variadic, a callback, a long, a struct with no defstruct, and a kebab
collision. Plus that nothing is in both lists, which is the bug the collision
case found.
check_structs and diff_bound get a row each for agreeing, for a permuted field
order, for a widened field, and for a symbol the header does not have — the
last being how a package pinned to the wrong release announces itself. The
name rule and the JSON reader get their own rows.
Checked by breaking two of them on purpose and watching both fail.
test/programs/raylib-imported.flan is the end-to-end evidence, back and in the
new struct-literal spelling: four bindings the package does not bind by hand.
ColorToInt of {17,34,51,68} is 0x11223344 and ColorTint by white hands the four
bytes back separately, so field order is pinned by arithmetic and not by a
round trip, which is the trap BUILT.md records.
maps.flan and map-exhausted.flan as fixed-output cases, the six refusals
by name, and map-stale-region.flan beside stale-region.flan.
The last one is not a line in the Vec's program because the two reach
the check by different routes. A Vec's operations check on the way in
and stop there. A map's get goes on to call a hash and an equality
function through pointers into the block, so a missing check there is
not a wrong number — it is a probe loop walking released memory. It
traps naming the site and exits 134, as the Vec's does.
{K V} resolves now, so the test that asserted it was milestone 6 is
replaced by the one that still holds: the arity, refused for the reason
Vec's arity is refused, because a near-miss would otherwise resolve to a
type variable and come back as generics.
render.ml's output is parsed by emacs/flan-inspect.el, which hard-codes
the colon when it reads a field out of a rendered struct. Moving the
printer on its own would break inspection in the dev loop without
breaking a test that says so, so the printer waits and moves with its
reader, in the Emacs lane.
The sweep could not tell a rendered *expectation* from a Flan *source*
snippet -- both are strings in a test -- so it converted both. The suite
named every one it got wrong, and those are back.
emacs/test-flan-dev.el:415 is the one edit inside emacs/: Flan source sent
to the daemon for eval, which the parser now refuses in the old spelling.
One label, in a fixture.
defer is a compile-time construct: the cleanup is copied into every exit
path of the function. That is why a loop body and a branch are refused —
a loop body's would fire once at function exit rather than once per
iteration, and a branch would have to express "maybe registered", which
a form copied into every exit path or into none cannot say.
A let is neither. It is not a frame here: its bindings are function slots
like any other and nothing is released at scope exit, so a let at the top
level of a function body has exactly the function's extent and a defer
written in it always registers. It was refused for a reason that does not
apply to it. A let nested inside such a let has the same extent and the
same permission; a let inside a while or an if has the loop's or the
arm's, and inherits the refusal.
The permission is granted again before every form of a body, never once
around the body: check withdraws it as it starts, so granting it once
would let the first defer through and refuse the second — and two
resources acquired in one let is the case this exists for. defer-let.flan
covers that one specifically, along with nesting, interleaved
registration order across the let boundary, and an early return.
The two refusals that stay now name what blocks them.
The delimiter is what disambiguates: (.x v) is a call and therefore an
access, {.x 1.0} is a brace form and therefore a construction. The colon
kept two jobs -- field label and enum member -- and this leaves it with
one, keys, which is what a map literal will want.
The old spelling is refused rather than quietly accepted, and the refusal
names the new one. Two accepted spellings is how two spellings become
permanent, and this repo rejects what it does not support and says why.
:keys keeps its colon. It names no field -- it is an instruction to the
compiler that happens to sit in the same brace -- so leaving it alone is
what lets the dot mean exactly one thing.
render.ml prints the dot too, or a struct the daemon shows would not be
Flan anyone could paste back.
The script is in tools/ rather than thrown away, because two lanes are
writing Flan in the old spelling right now and their files need the same
pass at merge.
It works on forms, not on text: a keyword becomes a dot only where it sits
in a field-label position inside a brace, so an enum member in value
position, a map key inside an EDN string and a type-position {K V} are all
left alone. :keys keeps its colon -- it names no field.
Two element types, one runtime, and the element type appears nowhere below
the call site: size_of and align_of are produced where the concrete type is
known, which without generics is simply the concrete call site. That is
Odin's arrangement and it is what spec-memory.md specifies. `at` and `len`
were already the names for a fixed array and a slice, so a Vec extends them
rather than adding a parallel pair — the asymmetry `nth` was removed for —
and the value form and the place form go through one helper so they cannot
drift apart.
StorageExhausted lands with step 2 rather than after it, because the
signatures depend on it: `push` and `reserve` are Unit, `clone` is the
container, and nothing grows a Result. It is built out of nodes that already
existed — a while, a restart-case and an error — so the backend learned
nothing about allocation. The restart is established at the failing
allocation, which spec-memory.md names as the exception to "restarts go at
the resync point, once", and the element a push was given is bound to a slot
before the loop so a retry re-attempts the allocation and not the expression.
Move-only is a dead set on the checker context, and it is flow-sensitive at
an `if`: both arms start from the same set and the union survives the join,
so `(if c (free v) (free v))` is legal and a one-armed free still kills the
binding. The case a dead set cannot answer is a move inside a loop — merged
once at the end of the body it counts one move, not two — so that is a rule,
refused with its reason.
Four decisions the spec did not settle:
The Vec header is six words in every build, not four in release. A layout
that changes with a build flag can disagree across the reload boundary
silently: a redefinition module is built by llc and ld against a host built
separately, and nothing makes the two agree on a struct size. The 32-byte
release layout is deferred on that.
A zeroed Vec has a null allocator, and the first operation needing storage
adopts the context allocator. Odin's behaviour. The alternative was refusing a
Vec-typed struct field until drop lands; shipping the null was a null deref on
the first push.
A Vec's length and index are i32, like every other length here. Widening
indices is one change across all the containers, not a Vec question.
`let` has no type annotation, so a local Vec has nowhere to say what it holds
and the element type is written at the call: `(vec-new i32)`. This is not the
explicit instantiation syntax the generics section rules out — nothing here is
generic and the name resolves as an ordinary type. Where the context says, it
may be left out.
The allocator grew a budget: a ceiling on live bytes, 0 for none. The retry
restart is only answerable by a handler that can make the *same* request
succeed, and for a fixed backing store the handler that works is the one that
raises the ceiling — releasing the region a container lives in invalidates
the container, which is what the epoch check catches. The spec's "grows the
arena and then invokes retry" needed something to grow.
The generation word is bumped on every reallocation and read by nothing. The
stale-slice trap it is for needs a slice that can carry the Vec's identity,
and a slice is ptr+len. Said plainly rather than implied by the word's
presence.
The mutation pass turned up one defect that did not make the suite go
red: a reader branch that forgets to advance reads the same character
for ever, and dune test waits as long as it is left to. In CI that is a
job killed by the runner with nothing named and no output to read.
watchdog.ml puts an alarm on every test binary — generous, because an
alarm that fires on a slow machine is a flake — and a five-second one
around each read in test_flan, where the budget really is small. The
first read that does not return wedges the rest, so a looping reader
costs five seconds and names the row instead of costing eight minutes
or never finishing. Both were watched: the string-escape loop now fails
in five seconds with the case named, and the per-binary backstop was
armed short and observed to fire.
spec-conditions.md §3's remaining half: a clause binds parameters, an
invoke-restart supplies them, and what a restart takes is compared at run
time because a restart is found by name on a dynamic stack — neither end
of the transfer can see the other.
The parameters live in a buffer the restart-case owns, not the invoker's
frame. A clause runs after every frame between the two has returned (§5),
so anything on the invoking side is gone by then; the invoker stores into
the target frame while both are still alive, which is the one moment they
are.
The frame carries the parameter count and a hash of how the types are
spelled, and every frame carries them whether it takes parameters or not:
a clause taking none has to refuse arguments as loudly as one taking two
of the wrong type. The count is not redundant with the hash — it is what
makes a 32-bit collision between two different signatures harmless — and
the spelling itself rides along so that a mismatch can say what was
wanted and what was given, which neither end alone knows.
The arguments are evaluated into slots before the invoke node rather than
hanging off it. An argument that transfers on its own is then guarded
before anything aims the channel, and a call written in an argument is on
the ordinary walk Reach and Load already do — a node they treat as a leaf
would have dropped the function and failed to link.
The other way a transfer starts is the break loop, which chooses by
position and has nothing to fill parameters in with. It reaches a clause
through the same channel, so nothing downstream could tell the two apart:
the frame is pushed with the buffer marked unfilled and a clause with
parameters checks that mark before reading it. Refused with the reason
rather than run on values no one supplied.
runtime/flan_rt.c gains two message functions and nothing else; the
restart frame's first four fields, which are the ones C declares, do not
move.
Two mutations the reader survived: dropping '+' from the number
dispatch, so +5 reads as a symbol nobody defined, and accepting an
unknown string escape as the character after the backslash, so a typo
silently reads a different string. Both now have a row, and the known
escapes are asserted on the decoded bytes rather than through
Form.to_string, which escapes them again and would compare the source
with itself.
print-str, print-i64, print-f64, print-bytes, print-line and newline leave
the prelude. print and println are the whole printing surface now, and print
is the better call at every one of the sites that used them: it is the same
structural walk without the newline, so the no-newline case the family was
kept for is covered, and it takes the value as it is. The old print-i64
forced an explicit (i64 x) at every call site, because this language widens
nothing implicitly; that cast is gone from 127 places.
Dropping it moves one answer. hash-grid returns u64, and the cast through
the signed printer showed sand-headless's hash as -2851001042534928384.
print routes a u64 through flan_u64_to_bytes, so it now prints
15595743031174623232 — the same 64 bits, read as the unsigned number they
are. The pinned expectation follows the correction.
test-flan-dev.el and test_session.ml both reached for print-line as "a name
the prelude has"; they reach for rand-seed instead.
Two ways to write a match over an enum and two different refusals, neither
of them true. (match k :lo ...) died in the parser with "expected a pattern,
found :hi" — which arm it named depended on cons evaluation order, and it
never mentioned enums. (match k lo ...) died in the checker blaming milestone
2, which is not what stands in the way.
What stands in the way is worth writing down, because the feature is close.
An enum is an i32 at run time and its members are all known, so the arms are
a chain of (= k :member) and the exhaustiveness check falls out of env.enums
— a desugaring, no new IR node, the same shape as everything else this lane
landed. What is missing is a case in Ast.pattern for a keyword, and load.ml
matches that type exhaustively with no wildcard, so the variant cannot be
added from a session that does not own the file. One line, for whoever does.
That is also why destructuring went through a call to an unspellable name
instead: a name in call position is an open namespace check.ml already owns,
whereas tagging Pctor with ":lo" would put a second meaning into a field
another file destructures as a constructor.
The struct-tail case in the acceptance program is unrelated housekeeping: the
corpus slices arrays of i32, u8 and f32 and nothing wider, so nothing else
proves the desugared (slice xs n (len xs)) gets a struct's stride right.
The checker tests pin the reason rather than the failure: an array pattern
over a slice has to fail *because a slice's length is a runtime value*, not
because something went wrong. The four map-destructuring keys Clojure has and
this does not are each named individually, because "unexpected form" leaves
the author guessing which of the four they wrote is the missing one.
The acceptance program exists for the case none of the above can see. A
pattern is desugared away entirely, so there is nothing in the typed IR to
inspect; the only way to tell that the value was bound once is to destructure
something with a side effect and print how often it ran. Four names, two
calls. A desugaring that re-evaluated the initialiser per name prints 4, and
every other line in the program stays green through the mistake.
Every sigil in the corpus test sat in leading position, where read_form
handles it before is_delimiter is ever consulted — so reverting the
is_delimiter line alone left every case green. a`b now has the case a~b
already had, and the corpus carries both, which is what makes the class
guard cover the delimiter change rather than only the read branches.
The handoff note hedged on the one thing it exists to decide: expansion
runs over Form before Parse, not over Ast. There is no Ast.Defmacro, so
an Ast pass would have nothing to read. Says milestone 6 for union
values because that is the number check.ml itself gives.
(defmacro m [x] x) answered "unknown top-level form (defmacro ...)" —
refused, but not by name and with no reason, because the refusal list
only covered expressions. Now it checks the shape and then refuses,
which are two different mistakes and get two different reasons: a
defmacro with no body is a typo, a defmacro with a body is a feature
that is not here.
The reader's new sigils made this urgent rather than tidy. quasiquote,
unquote and unquote-splicing are now real heads arriving at the parser,
and without a case each they would fall through to Call and come back
from the checker as "unknown name quasiquote" — which tells you nothing
about what is missing. unquote and unquote-splicing are refused as
mistakes rather than as milestones: they mean nothing outside a
quasiquote and the reader cannot notice, because it does not track
where it is.
gensym is neither a reader token nor a special form — it is a function a
macro body calls while the macro runs, and there is nowhere for it to
run. Refused by name so it does not arrive as an unknown one.
`(a b) came back as the unknown name "`" — precisely the failure the
reader's own header warns about for the apostrophe, one sigil over and
still open. Same fix: the sigil reads as a wrapper and the reader stays
dumb about what it means.
Clojure's ` ~ ~@ rather than Common Lisp's ` , ,@ because a comma is
already whitespace here, and every binding vector in the corpus is
written assuming that. Changing is_delim to free up the comma would
rewrite more of the language than macros are worth.
Backtick and tilde join is_delimiter so that a~b is two things and can
never be one name. No symbol in the corpus contains either character, so
closing the class costs nothing now and would cost a migration later.
spec-memory.md drops (set (get m k) v) from the assignable forms: a map has an
upsert of its own, put, which either inserts or replaces, so there is no store
into a lookup - and an absent entry has no location to store into anyway.
The compiler still parsed it into an Ast.Pkey and refused it downstream as
unimplemented, milestone 6, which is the wrong reason for something that is
never arriving. The place form is gone from ast, tast, load, check and emit,
and the parser refuses the shape where it is written, with the reason and a
pointer to put.
spec-conditions.md §2. The same lookup as signal, and the difference is
entirely what happens when the walk ends: signal returns Unit and the
signalling function carries on, error has type Never and the program stops.
Only a transfer gets past it, so emit puts a guard after the call and then
unreachable - and flan_error cannot be marked noreturn for the same reason, it
does return, on exactly one path.
Being Never is what lets it stand where a value was expected, which is the
fall-through shape §1's load-texture example needs and the reason it is worth
having before the break loop rather than after. An unhandled one names the
condition on stderr and dies the way every other trap does; flan_error is where
the dev-build break loop will go.
The two spellings share one AST and IR node with a kind beside them, the same
shape Ast.unwrap already uses for some and try, because they differ in one
decision and nothing else. test/programs/error.flan is the unhandled case,
asserted on the exit code and the reason rather than through the outputs table,
which only has room for a program that exits 0.
error, find-restart and compute-restarts are named in spec-conditions.md and had
no case in parse.ml, so each fell through to Call and came back as unknown name
- the very shape the house rule exists to prevent, and the one that makes a
missing feature look like a typo.
Three message strings from the previous commit had their line continuations
collapsed into runs of spaces. Rewrapped; no change to what they say.
spec-conditions.md §3 to §6. A handler runs where the signal was, decides, and
control resumes at a restart-case further out - so unlike step 1 this one does
alter control flow, and it is lowered explicitly rather than through platform
unwinding, because wasm32 cannot unwind and because a cmp/jne after a call
reads like ordinary code.
The channel is the out-parameter §6 settled on: one ptr appended to every Flan
signature, written by an invoke-restart and checked after every call. The
return type stays what the source says, one pointer threads down the whole
chain, and a frame that sees the channel set just returns early - which reuses
the existing return path and with it §5's defers for free. Emit.signature was
already the one place a signature is spelled, which is what made that part
small.
Every function is transfer-transparent, release included. §6's escape analysis
is an optimisation; in a dev build a cell can hold anything, so the honest
answer to what a call can reach is anything, and uniform means redefinition
acquires no new refusal class.
The transfer target is the restart frame's own address and not a static clause
id, which corrects what the handoff note had settled. An id has to be unique
against every module a running program may later load, and a hash is only
probably unique - two restart-cases colliding means the inner one silently
catches a transfer aimed at the outer. The frame is an alloca in the function
that offers it, so the address is exact and it also says which clause, which is
how clause ids disappeared. Re-entering a restart-case then needs nothing
extra, since each activation allocates its own frames.
Cleanup is landing blocks, one per region rather than one per function: a
restart-case's pops its frames and either dispatches or forwards, a
handler-bind's pops the handler frames on the way past, and the function's own
runs its defers and returns. One function-wide block would have jumped straight
past the very restart-case that was meant to catch the transfer. The channel is
cleared before any cleanup runs and put back after, or a defer's first call
would branch straight back into the block it came from.
flan_signal takes the channel and passes it to each handler, stopping once one
writes to it. That makes the one C frame every handler is reached through
transparent to a transfer, which it has to be; it is also the only one, since
extern is Flan-to-C only and there are no function values yet.
Refused by name with the reason, each with a test on the reason: restarts with
parameters, return inside a restart-case body, one restart-case offering a name
twice, and invoke-restart inside a defer - a defer is the cleanup a transfer
already runs, so starting one there leaves the defers half run with two targets
and no way to choose. The lexical case is the checker's and the one that
reaches a function through a call is trapped at run time. No restart of that
name is a located runtime error at the invoke site, because there is nowhere to
resume.
Two things found on the way. `{ ctx with in_handler = true }` was a latent bug:
ctx.slots is mutable, so a copy allocated the body's slots into a record the
function never saw again - harmless only because no handler-bind body in the
tests had a let in it. And test/reload_host.c calls flan.outer through an asm
label, which does not fail at link time when the prototype is a parameter
short; it reads garbage as the channel and dies somewhere else.
test/programs/restarts.flan runs at -O2, at -O0 and as a dev build. -O0 is not
redundant: the guard after every call is control flow the optimiser would
otherwise launder, and the dev build is where each of those calls goes through
a cell.
spec-conditions.md §1 and §2 and nothing else, because those two are worth
having alone: signal returns Unit whatever it finds, a handler that returns
normally leaves the signalling function to carry on, and with nothing matching
it is a no-op. So none of §6's transfer machinery exists yet and no signature
changed - which is the whole reason to do this step first.
The runtime is a linked list. Establishing a handler is two stores and a push
onto a frame on the establishing function's own stack, and signal with an empty
stack is a null check, which is what §2 asks for. Popping is by frame rather
than by count, so restoring what this one displaced is right even if something
below it left the stack out of step.
A condition's type is a hash of its name and not an index: an index would shift
the moment a struct were added, and every handler a running program had already
pushed would match the wrong type. The condition crosses as a pointer, since a
handler runs while the signalling frame is alive and there is nothing to copy -
but what the clause binds is the condition itself, the pointer being a hidden
parameter and the name a slot loaded from it, so a handler passing c to
something expecting the struct is not handed an address.
A clause is lifted into a function of its own, because a handler runs from
wherever the signal was and cannot be a branch in the function that wrote it.
That gives two refusals, both by the house rule. A handler cannot see the
establishing function's locals - that is a closure with an explicit
environment, so a reference to one is refused for that reason rather than
reported as an unknown name. And return inside a handler-bind body is refused,
since the frames are popped on the way out and an early exit would leave them
pointing into a function that has gone.
Settled in advance for the next step: in a dev build every function is
transfer-transparent, because a cell can hold anything and the honest answer to
what it can call is anything. Same bargain as the indirect call, and it means
redefinition acquires no new refusal class. Still open is whether the
discriminated result is returned by value or through an out-parameter.
A shift by the operand's own width or more is poison in LLVM, not a wrong
number: (<< 1 32) at -O2 compiled to a bare retq. A literal count out of range
is now rejected in check.ml, and emit.ml masks a computed one to width - 1,
which is what the hardware does and which LLVM folds away for a constant.
There is one top-level namespace, but the environment's tables are per-kind, so
only a function was ever checked for a duplicate. (defn item ...) beside
(defvar item ...) type checked and then died in LLVM as a redefinition of
'@flan.item'; two colliding type declarations were not caught anywhere. One
pass over Ast.declared_name now runs before every other collection pass. That
function lives in ast.ml because Load needs the same set - the names an import
renames - and two copies would drift.
Second stage of the milestone-2 frontend. calc-me.flan (12 decls) and
sand.flan (20 decls) both parse end to end, and both are test deps so a
regression fails `dune test` rather than surfacing at the CLI.
Three silent-misparse bugs fixed along the way -- all cases that read
cleanly and meant something else:
- dotimes/defer/some/try/fn fell through to Call, discarding their
binding and control-flow meaning. Now special forms. Forms from later
milestones (handler-bind, restart-case, loop/recur, defmacro, signal,
with-allocator, errdefer, await) are rejected outright rather than
parsed as calls.
- (Some 1) in first body position was read as a return type, because
(Option f64) and (Some 1) are identical s-expressions and the
heuristic was capitalisation. Now decided by the set of names actually
declared as types, collected in a pre-pass -- exact, and
order-independent so a type declared below its user still resolves.
- Array literals in value position were rejected outright.
Also adds NEXT.md with the handoff for the checker.
