The arm was written with the others and through the same deferral, and neither
the generics row in test_flan.ml nor the paragraph in BUILT.md that enumerates
what defers had it. Its placeholder is get's, for get's reason: it answers an
(Option V), so the match around it still has to check while the key is a
variable.
FIX.org carried Addr(Pfield ...) on an Option as a hole in both backends. It is
not reachable from the language: a field access goes through struct_target,
which admits a struct or a pointer to one and refuses everything else by name
with a location, so (addr (.x o)) is refused at the field and never reaches a
place. The node that failed was one the compiler built for itself.
The refusal is pinned on the bare field and on the address of one, and FIX.org
now records the finding, including the asymmetry that stays: the x86 backend
lays out an Option's tag and value as fields and the LLVM backend does not.
Neither path is reachable, so matching them would be untestable code written to
balance a road nobody drives on.
check_main raised against Loc.unknown, so both of its refusals opened with
<unknown>:0:0. env.locs is the table of where each type was declared and a
function is not in it, so the location comes from the declaration list the
caller already holds. A main that arrived without a defn keeps the unknown
span rather than being given an invented one.
A (Vec Value) where a Value may itself hold a (Vec Value) — the recursive
dynamic value an EDN reader has to answer with when nobody hands it a target
struct type — was refused five different ways, and every one of the five gave
the same reason: the container runtime is type-erased, so it copies and
releases slots bytewise and cannot reach inside a slot. A free would release
the slots and leave every block they point at stranded.
That reason is about teardown, and it does not hold for a region. free-all
never releases an individual slot; it takes the whole arena, and every block
the elements own is in it, because they came out of it. The refusals were
over-broad, and what they were guarding was never ownership — ownership
tracking is untouched here, moves are still moves, and Types.is_move_only is
the same function it was.
So the question moved rather than disappeared. It could not stay at the type,
because can-free is a capability on an allocator value and with-allocator
rebinds a dynamic variable: which tier a (vec-new) will meet is not a property
of the place its type is written. What is decided at compile time is only
whether to ask, which is a property of the element type; the answer is a
run-time branch on the allocator, one per container and never per element,
because the alternative is a walk at release and a walk at release is the
registry of destructors the frame tier's reset exists to not have. It is
emitted at every growth and not only at the construction, because ZII means a
container can exist without ever passing through (vec-new) — a case field left
out of a literal, a global that starts zeroed — and those adopt the context on
their first push.
free on such a container is refused rather than made quietly shallow. It cannot
recurse, which is the whole premise, and releasing the outer block alone would
be "I freed it" written over a program that stranded everything inside; this
runtime refuses that collapse everywhere else. The message names free-all,
which is reachable by construction. clone stays refused for a reason the region
does not dissolve, and the old message had bundled the two failures under one
sentence: what disqualifies clone is not that it copies a header — so do at and
get, and they are fine, because they promise nothing — it is that clone
allocates a new block and promises independence, and a bytewise copy hands back
elements still pointing into the original's region.
A struct or union field is admitted only where the field's container holds
owning elements, because that container can only have been built against a
region. A field holding a plain (Vec u8) stays refused: nothing would force
that one into a region, and two copies of the aggregate would be two headers
over one heap block. vec-in-struct.flan still pins that.
The epoch already covered use after free-all, including the case this makes
reachable — an inner header copied out of an arena-held element into a local
still traps, because an Allocator is a pointer and a copied-by-value one would
carry its own epoch.
arena-value.flan builds the value by hand; arena-edn.flan reads a real document
through the tokenizer, and its reader takes no allocator and names none,
because spec-memory.md already puts the allocator in the calling convention.
arena-region.flan is the branch itself: run 0 is the (Vec (Vec i32)) control
that must not trap, and runs 1 and 2 are the two ways this dies.
The refusal was on the defvar path alone, so (defconst c U (U {.i 1}))
still came back from the emitter as "a global's value must be a
compile-time constant — this one is computed", which is true and says
nothing about unions. Both kinds of global reach the same encoder, so
both get the same message.
It is decided on the checked value rather than on the declared type now,
which is what lets the one initialiser that *is* a constant through: (U
{}) is all-bytes-zero, the same value a declaration with no value gets,
and refusing it would have been telling someone to write the thing they
had written.
The name freed up by the rename now means what C means by it: the members
overlay one storage, the size is the largest of them, the alignment the
strictest, and nothing anywhere records which one was written. It serves
two things that wanted it. Binding a C header means holding the union the
library holds and reading whichever member the library's own tag says is
live -- a tag Flan cannot see, because the rule relating them is prose in
a manual. Overlaying an f32 on a u32 to look at its bits is the other,
and it is the same read.
So that read is defined rather than refused. This is the one place in the
checker where bytes win over safety on purpose, and the alternative was
not a safer language, it was no feature: type punning *is* reading the
member that was not written. The promise is the one C's implementations
make and C's standard does not -- the layout is the target's, the bytes
are the bytes, a read is a reinterpretation of them -- and what is not
promised is anything about bytes nobody wrote, where a member wider than
the one last stored reads a tail that is indeterminate exactly as a
struct's padding is. ZII narrows that to almost nothing: a union starts
all-bytes-zero unless uninit says otherwise.
uninit on one is allowed, unlike on a defdata. The refusal there was
never about garbage; it is that a tag steers, and a tag no case names
falls past every comparison in a match into a block LLVM may treat as
unreachable. An untagged union steers nothing.
Which is also why three things are refused, each for a reason that does
not expire with a milestone. No move-only member: nothing knows which
member is live, so nothing can tear one down, and unlike the struct and
defdata refusals this is not waiting on recursive teardown -- there is no
fact for teardown to read. No bool at any depth: an i1 loaded from a byte
that is neither 0 nor 1 is a value the optimiser may assume cannot exist,
and a union is the only type that can produce one. No defdata at any
depth, for the reason uninit gives, arriving the other way round. An
Option member is fine and the walk says why: its match is a tag test and
a branch, not a chain with an unreachable tail.
Two members in one literal, a match on a union, a union map key and a
member written into a global initialiser are each refused by name.
A union is a field list whose every offset is zero, so it travels as a
Tast.structure and the checker, the emitter and the x86 backend each grow
one table rather than one shape. A value is a zeroed temporary and a
store -- Set over Pfield, which every backend already has -- so there is
no new IR node and no layout rule spelled out a second time per backend.
The LLVM type is the blob clang gives a union, the DWARF is
DW_TAG_union_type with every member at zero, and the printer names the
type and does not walk it: it cannot know which member is live, and one
of them may be a pointer.
cimport can now check what it could not. A C record holding a union
member was not recorded at all, so the defstruct beside it went unchecked
rather than checked wrongly; a named union member resolves to a defunion
now and the whole record is compared field by field. The defunion itself
is compared against the header's union as a set and not in order --
every member is at offset zero, so a permuted one is the same type and
reporting it would be a finding that is not one -- while a member the
header has and Flan lacks is reported, because that is what changes the
size. A defunion against a C struct, or a defstruct against a C union,
is reported in both directions. An anonymous union member is still
skipped, and the comment now says that the gap is on the Flan side:
there is nothing to declare.
Flan's tagged sum has been spelled defunion since it landed, which was
accurate right up until the language wanted C's untagged union as well.
Both cannot be called the same thing, and the tagged one is the one with
an alternative name that says what it is: a case, its fields, and a tag
that steers which case is live is a data type, not a union.
So the form is defdata everywhere -- the parser, the AST, the checker,
both backends, the prelude's Form, the editor's font-locking and imenu,
the docs and every .flan file in the tree. The internal vocabulary moves
with it: Tast.union is Tast.data, uname is dname, the tables the checker
and the emitter keep are datas. Leaving them would have inverted the
words permanently, with surface defunion meaning one thing and
env.unions meaning the other, which is exactly the kind of drift the
comments in those files exist to prevent. What did not move is case,
variant and vfields: a tagged sum still has cases, and it still has one
live at a time.
defunion is not kept as an alias. An alias would compile the day the
untagged form lands and mean the opposite of what it used to -- the same
silent misparse that made defn's return type mandatory, and worse,
because the reader would have no reason to look. The old spelling is a
named refusal instead, parse/defunion-renamed, which says what it is now
called and that the name is reserved for something else. It fires on the
head alone, so (defunion U [A B]) -- which would otherwise have parsed
cleanly as one field A of type B -- is refused with the rest.
A program that wants to load its data once and keep it could not say so. Every
move-only global was refused where it was declared, on an argument about the
dead set being per function: two functions each freeing the same global would
be a double free nothing could see. The argument was sound and the conclusion
was too strong. It assumed a global has an owner. It does not.
Reading a move-only global is now always a borrow. Nothing may take ownership
of one, so nothing may free one, and with no owner to hand over there is no
double free left to catch. This is not a general ownership model for globals
and is not meant to grow into one: it is sound precisely because the lifetime
question that model would exist to answer has a constant answer here, the
process's. The refusal lands at the read, which is where a move would have been
recorded for a local -- passing the global to something that owns its
parameter, binding it to a local, returning it and freeing it all reach the
same place, and each is told to borrow instead, or to clone if it really wants
something of its own.
Such a global is mutable where it stands. push, put, reserve and set already
take their target through the borrow path, so a global (Vec u8) is filled and
grown in place, and the aliasing that raises is the one every Vec has:
spec-memory.md's explicit Zig/Odin contract, where a push that reallocates
invalidates a slice taken before it and the dev build's generation word traps
on the stale one. Globals get no borrow rule locals do not have, because the
hazard is not new and the trap lives on the Vec rather than on the binding.
What a move-only global may not do is carry a computed initialiser. A global's
initialiser is a link-time constant -- there is no init-at-startup path in the
LLVM backend by design, and the x86 backend that has one deliberately leaves it
out of a reload module, because re-running an initialiser wipes the live state
reloading exists to preserve. So the global starts zeroed, which for a Vec is
an empty Vec and therefore a value rather than a placeholder, and the load is
an ordinary assignment in whichever function loads it. That is also what makes
the data survive: nothing runs between one entry to main and the next, so a
re-entered main finds the global as it left it. A defconst cannot be one at
all, since a constant is not an assignable place and nothing could ever load
it; both refusals name the (defvar g (Vec u8)) that works.
The reload fixture gains a global Vec in the host and another that arrives at
run time, because that is where declaring instead of defining has teeth: a
module that defined the host's Vec would take a zeroed header of its own and
strand the block the process is still using, which a re-zeroed i64 cannot
demonstrate.
Every defenum member had to carry a literal integer, so an enum of twenty keys
was twenty numbers typed by hand and renumbered by hand the first time a member
was inserted in the middle. A value may now be left out, and then it is the one
above it plus one, starting at 0 -- C's rule, because the enums written here are
as often a transcription of a header as they are original.
Autoincrement brings its own silent failure with it. Renumber a member, or slip
one into the middle, and the member below can land on a value some other member
already holds: two names for one number, the program still compiles, and one of
the two is now unreachable through a match on the other, with nothing in the
source saying so. So a duplicate that was *written* is kept -- a Count or a Last
pointing at an existing value is a real idiom and is somebody's decision -- and
a duplicate autoincrement walked into is refused, naming both members and the
number they collide on, and saying that writing the value out is how the alias
is declared to be intended.
The rule lives in the parser rather than beside the duplicate-name check in the
checker because it is a question about the source text. Ast.Defenum holds
resolved numbers and no per-member locations, so by the time the checker has an
enum in hand it can no longer tell which of the values were typed, nor point at
the other member. All members are resolved before any of them is checked: the
value collided with is as often below as above, and (defenum E [A B 0]) has to
refuse A.
`indexed` took an Array or a Slice, so a `(Ptr T)` that came back
from C was readable at element 0 through `deref` and nowhere else.
The length is not missing from the world — for `font.recs` it is in
the struct, one field over — it was missing from the language.
`(slice-from-ptr p n)` is the form that says it. No marker on the
name: `!` here means mutates and `?` means asks, and `zeroed`, the
nearest neighbour, carries neither; `ptr` is the marker, because a
`(Ptr T)` only ever arrives from a `declare-c`.
Nothing new in the representation. A slice is already {ptr, i64} in
both backends, so this is two insertvalues; `x86.ml` takes the new
constructor on its existing `unsupported` arm.
It refuses a first argument that is not a pointer, a negative literal
length at check time, and a negative computed one at run time — that
last through `signal_block` and `@flan_slice_error`, reused rather
than growing the runtime a function, and *signed*, because
`check_slice` compares unsigned and a negative i32 sign-extends to a
huge u64 that walks through it. Behind `f.md.checks` like the other
two: on at -O0 and -O2, off only when checks were asked off.
It owns nothing and needed no analysis to say so — a slice is not
move-only and carries no allocator, so `free` refuses it by the rule
that already refuses `(as-slice v)`.
`rl/font-recs` and `rl/font-glyphs` are where the promise is written,
beside raylib's own invariant rather than at every call site, and
they are the shape a count-naming binding directive could never have
covered. `examples/text-rectangle-bounds.flan` is the port that
motivated this and it runs; `test/programs/slice-from-ptr.flan`
covers the form with no raylib and no window.
hashable? gated the type and not the operations: a generic could take and
return a (Map $t V) and could not get or put into one. The hash and the
equality are emitted as concrete symbols chosen from the key type, and
while $t is a variable there is no symbol to name.
The five arms that reach the pair - put, get, has-key?, reserve, clone -
now check their arguments and return a placeholder of the operation's own
type when the key is a type variable: Unit for put and reserve, None for
get so the (Option V) around it still checks, false for has-key?, a zeroed
map for clone. The node is thrown away with the rest of the abstract pass
and the real one is built in the copy, exactly as println's is.
What makes that different from print's free ride is the clause. A map
operation can fail at a concrete type; it is deferred anyway because
{:where (hashable? $t)} is in the signature, so the refusal lands at the
call that asked for the type, against a requirement the author wrote down.
A generic that declares nothing gets no deferral - deferred_key checks
first, and map_type has usually refused the signature already. So the rule
for the allow-list is not a headcount: either the operation cannot fail
after substituting, or a declared predicate gives its failure somewhere to
land. The comment at the print arm says that now instead of "stays two
long".
The instantiation-time refusal names the call site, the type it asked for,
the predicate and the clause, rather than repeating the generic's name
twice.
A map keyed by a type variable cannot be put into inside a generic body:
the hash and the equality are concrete symbols chosen from the concrete
key type, and there is none until the copy exists. The refusal now says
that, and says what hashable? does buy - taking and returning a
(Map $t V) - rather than leaving the reader to infer it.
Closing the hole means adding the map operations to the list of forms
the abstract pass defers to instantiation. That list is print and
println and nothing else, and every member is a place where a refusal
moves from the definition to a call site, which is what the abstract
pass exists to prevent. Two is short enough to hold in your head.
Also written down: four of the prelude's copyable? declarations are
convention rather than checker-enforced. The move analysis tracks
locals, not reads out of a slice, so swap! and friends check without it
- and would still duplicate a header at [(Vec i32)].
The author's decision, and it removes the one syntax question generics
had. A return type can no longer be written in braces, so a {...} after
the signature is unambiguously the constraint map and there is no
structural rule to explain.
The reasons for the record: the brace's value meaning and its type
meaning do not correspond the way the bracket's do - [1 2 3] is a value
whose type is [3 i32], but {.x 1} is a value whose type is a name, and a
map value is built by map-new with no braces anywhere - and dropping it
reserves {} in type position for anonymous struct types.
Braces in a type are refused with the surviving spelling named rather
than falling through to "expected a type". Types.to_string and
Cimport's source printer both print (Map K V) now, and Shim refuses the
application spelling where it used to refuse only Ast.Tmap.
swap!, reverse!, sort!, sort-by!, index-of, min-of, max-of, map!,
reduce and filter, each written once over $t. Every call site in the
corpus moves with them.
min-of and max-of are not min and max because min and max are builtins
over two or more numbers and nothing shadows a builtin. These reduce a
slice, which is a different operation at a different arity.
sort-bytes! did not collapse into sort!, and the reason is the point of
the predicates: a [u8] is not ordered? and cannot be, because < is an
instruction and comparing two slices lexicographically is a loop. It is
sort-by! with bytes<? written in, one line, keeping its name and its
stability note. sum-i32/sum-f32 and append-i64!/append-f64! stay for the
reasons the spike gave.
Not what the notes predicted: none of the ten collapses on a signature
change alone. filter and reduce need copyable? because the checker
demands it - reduce's accumulator at (Vec i32) is a double move - and
the rest declare it because a slice of owning elements would have them
duplicating headers.
check_constants makes two kinds of finding and they were treated alike.
A value that does not match, or a C name the header does not have, is
the library contradicting the package and stops a build the way a
permuted defstruct does. An enum nobody mapped and a rule that reaches
nothing are about the package's own bindings file -- real, and worth
fixing, but telling a lane that added a defenum to go and edit a config
in a message shaped like "your layout is wrong" is the wrong thing to
fail a build with. Those gate generate-c, where that file is edited.
Also: a const prefix now counts as reaching a name before an explicit
constant line is consulted, so a rule whose every match is also spelled
out by hand is not reported as matching nothing.
Two gaps the raylib examples hit.
The layout check compared a Flan enum against the header's `int` and
called it a disagreement. It is not one: Shim.cty lowers a defenum to
int32_t in a struct field exactly as it does in a parameter, which is
what the signature check already knew and the layout check did not. One
predicate now serves both, symmetric, and tolerant of a 32-bit integer
and nothing else -- f64 against the library's float still fails, in the
very struct whose other field is an enum. Camera3D.projection is a
CameraProjection again and rl/camera-projection is gone with it, so
`.projection :perspective` resolves at the construction site.
And generate-c's claim said nothing about a defconst or a defenum
member, so a wrong flag bit was completely silent. `bindings` gained
`enum`, `const` and `constant` lines saying what a Flan constant is
called in C -- the prefix is nowhere in the Flan name, so it is declared
rather than guessed. Nothing goes quiet in either direction: a name the
rule builds and the header lacks is reported, a rule that reaches
nothing is reported, and a defenum with no line is itself a finding,
because otherwise the silence just moves up one level.
clang's dump gives anonymous EnumDecls for every raylib enum and no
value at all for an enumerator written without `= n`, so the constants
are one flat table and the values are counted the way C counts them.
cache_format bumped with the dump type.
(into xs (vec-new i32) (map double) (filter even?)). The function name is
syntax and never a value, so (map double) is (double x) written into the
loop body: no intermediate collection, no closure, no generics, nothing
to inline. Transducers would compose at run time and Rust's iterators
need monomorphisation; a macro needs neither. into.flan counts the pulls,
which is the assertion a unit test cannot make.
The destination is in the form because the destination is the allocation,
and that is what makes spec-memory.md's explicit-allocator rule true by
construction rather than by convention. Which also settles the open
question: reductions do not share the form. A seed is not an allocation,
so (into xs 0 (map cost) (sum)) would be a second form wearing the same
spelling, and the destination would stop being honest about what it is.
A source that is already a name is used as it is, not bound. A (Vec T) is
move-only, so binding it would take the caller's ownership for something
that only reads; a fixed array would be copied once per into. A source
that is anything else is still bound once, which is what a call needs.
The wart is that an owning temporary there leaks, because the binding has
a name the caller cannot reach — a call in that position should borrow,
and drop is what would close it.
All four of the prelude's macro limits bit and none blocked anything. The
three refusals are names nothing defines, which is the only error
facility a macro has. into-wrap is a defn using only special forms, so
Macro.reduce does not drop it, and it is the first thing in the prelude
written as a loop/recur.
There is no TCO here and recur is not a cheaper substitute for one: the
compiler verifies the call is in the loop body's tail position, so the
mistake is a compile error where it was written rather than a stack
overflow somewhere else. A loop is a let, a While whose condition is
true, and two jumps — emit.ml is untouched, and the barrier question
recur asks is the one labelled break already answered.
Tail position is a permission that is withdrawn at the top of check, the
same read-and-withdraw defer_ok does, handed back only by a block's last
form, both arms of an if and a match arm. So nothing enumerates the forms
that are not tails, which a pre-pass over the Ast would have had to, and
would have had to keep doing.
loop is also a barrier for break and continue, which is added rather than
inherited: a loop answers with the value of its body and a jump out has
no value to give. That is also why it takes no label. A while inside a
loop keeps its own break.
Two things the shape forced. A loop binding is a plain name, because
destructuring would make recur's argument count unreadable off the
binding vector. And in_loop's "moves a value bound outside the loop"
rule had to be told about the loop's own names, or (loop [v (vec-new
i32)] ...) would have been refused for doing the ordinary thing.
Exclusion says why rather than going quiet, a pattern matches by prefix, an
override changes the Flan face and leaves the C symbol verbatim, and a rename
dissolves a kebab collision instead of leaving both halves refused — that last
one is why the kebab rule is consulted in exactly one place.
The file itself too: a line that is neither directive is an error, because a
typo in a name override would otherwise land a binding under the wrong name.
generated.flan carries the 253 declarations the importer reads out of raylib's
header, so a build needs libraylib linkable and no header at all. The opt-in
no longer decides how many bindings a package has — every build now gets all
425, they are greppable, and they diff when raylib moves.
What that gives up is the build-time check, so `flan generate-c` is the only
thing that writes the file and it compares first: every defstruct against the
header's record, every hand-written declare-c against the header's signature,
and it writes nothing when they disagree. Against the 5.1-dev header on this
machine that is ten real differences and no write.
The 172 hand-written lines stay, and not out of caution. Everything the
generator emits agrees with the header by construction, so diffing generated
output against its own source is a tautology; the hand-written lines were
transcribed by a person, so they are the only thing here a header can
contradict. All ten of those differences came from them.
`bindings` beside `headers` is what survives regeneration, because a hand-edit
to a committed generated file does not. Two directives: `exclude` drops
raylib's three allocator entry points, and `name` gives the 19 generated
predicates the `?` spelling the hand-written ones already use.
Loc.Errors is a second exception, and the handlers in the session and the
daemon name only Loc.Error — so a list reaching them is an unhandled
exception and a dead session, which is the one thing the dev loop exists to
prevent. A flag on the function the session already calls left that one
label away from happening. Parse.program_all and Check.program_all are
separate names, so the session's call site has to be edited by a person for
its behaviour to change, and the guarantee stops being a default argument.
Placeless diagnostics now sort last rather than first. A wrong main signature
is raised against unknown, which is line 0, and sorting on the number alone
put it above every error that can actually be clicked. It is a real error and
it is not anywhere, so it goes after the ones that are.
A kind is a stable id per error, so a test can assert which error this is
without matching on prose and a message can be reworded without breaking
anything. The reader's fourteen refusals all have one; in the checker they
go on the errors a test names and the handful that are common enough to be
worth classifying. Not a hundred of them, because jank has a hundred from
being mature and the number is not the feature.
The notes are the part that could not be said before. A duplicate definition
now points at the second and notes the first; a duplicate parameter and a
duplicate field do the same; an unknown field, an unknown struct and a
non-exhaustive match all note the declaration and list what is actually
there, so the reader's next move arrives with the question instead of after
it. The reader's unclosed bracket is the clearest case — the error sits on
the bracket, because that is where the fix goes, and the note sits where the
file ran out, because that is the surprise.
No message text changed, so every existing needle still means what it meant.
The new assertions are on kinds and on note positions, which is the house
rule about asserting the reason, made stable.
The refusals: the arities, a pool of an owning element, ordering handles,
free of a handle, and clone of a pool. Each names what it would cost — a
cloned pool duplicates the generation counters with the slots, so one handle
would resolve in both copies and name two different things.
pool-stale-region.flan is the other failure, and the point of it is that it
is not the first. A stale handle is an answer and resolve says None; a pool
whose region was released has no slot array left to ask, so it traps. Same
rule that keeps a Vec's generation word and its epoch word apart.
Loc.Error now carries a diagnostic: a stable kind, a span, notes that each
have their own span and severity, and the macro expansion it came from. The
notes are the part that was actually missing — "this is wrong here" plus
"because of that, over there" is two places and two explanations, and a
single string can state only one of them.
The compatibility story for the daemon, which was the open question: the
single-diagnostic exception stays the single-diagnostic exception. Session
and dev evaluate one form and have one failure to report, so they take a
location and a message out of it with Loc.summary and are otherwise
unchanged. A second exception carries a list, and only a driver that
compiles a whole file raises it, so nothing interactive has to know it is
there.
No message text changed.
Loc.t grows an exclusive end, defaulting to the start, so a location nobody
widened is a zero-width span at a point and every existing call site keeps
its old meaning. Only the reader knows where a form ends, so only the reader
fills them in — one helper in the one place that holds both ends, which is
why nothing above Reader had to learn a span exists.
The width assertion is the point of the tests: the field could exist, nothing
could fill it, and every underline would be one character long while the
feature looked finished.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
A dotimes whose every iteration continues still counts to its trip count. The
existing case fails by hanging if the latch is wrong; this one fails by
counting wrong, which is the off-by-one the four-block layout could have.
A restart-case clause was made a barrier on reasoning alone and nothing
observed it. Now something does.
And say what a labelled continue means, which is the half that is not obvious:
it advances the named loop's counter and skips the rest of its body, not just
the rest of the innermost one.
return is refused inside handler-bind and restart-case blanketly, and rightly:
a return always crosses the frames they pushed. A break does not. A loop
written wholly inside a restart-case body has a perfectly good local break, so
the rule is a barrier on the loop stack rather than a flag — a jump is refused
exactly when a barrier stands between it and the loop it names, and the message
says which construct. handler-bind and restart-case bodies are barriers, so is
a restart clause, so are a defer's forms; a handler clause is lifted into its
own function and needs no rule at all. in_frames is untouched: a return is the
special case where the target is always outside every barrier.
continue wanted the other blocker. check_dotimes folded its step onto the end
of the body, which a continue would jump past, so the counter would never
advance and the loop would hang. Tast.While carries a latch now — condition,
body, latch — the step goes there, and emit_while emits four blocks. A while's
latch is empty and folds away.
Labels are Odin's, in the head position: (while :outer c ...) and (break
:outer). A keyword there is unambiguous because a loop condition is never one,
so one label function serves while, until, dotimes, break and continue. It is
not a goto — the checker resolves a label against the loops the form is
lexically inside, so control can only leave a loop it is already in.
Break and Continue carry a relative depth rather than a name, because that is
what a backend already has: emit keeps one entry per While the way it keeps
one pad per frame, and indexes it.
Nothing in the prelude wants either. Every early exit there is a return from
the function, which break cannot replace; the sentinel-flag loop break exists
to remove does not appear in it. The two the compiler emits are that shape and
are the one place it cannot help — their sentinel is set inside a restart-case.
reach.ml and render.ml take the While arity change and nothing else.
[4 T] is the type syntax and is unchanged; it already works in a defvar, a
parameter, a field and a return. A let binding is the one position with no
type slot, and there the brackets are an array literal of two elements whose
second is a type name — which came back as "unknown name rl/Vector2" and cost
32 hand-written Vector2s in one raylib example.
(array COUNT TYPE) is a parser form rather than a builtin call, because the
second argument is a type and the parser's callers have none. Parse assembles
the Tarray itself, so the count takes a constant's name for free and a value
in the type position is refused by the type reader's own message. The checker
resolves it to Tast.Zero — no new backend node and no new type.
(zeroed [4 T]) was proposed first and rejected: the parser can tell, a person
cannot. zeroed keeps its job of being inferred; array is the one that is told.
plan.org milestone 5 says when, unless, until, cond and dotimes are special
forms only until macros land. This is the first one to stop being one, and
running test/programs/macro-unless.flan means the compiler built a shared
object, dlopened it into itself and called a Flan function to find out what
(unless c a b) means.
unless is the one that moved because it is the one nothing else needs: zero
uses in the prelude, so moving it cannot make the prelude depend on the
expander that compiles it. Its coverage is sand.flan, seven calls, compiled
through Session in test_session -- which is the in-process path and the reason
lib/dune now passes -linkall. Say plainly what that coverage is not: nothing
in test/programs used unless before today, so macro-unless.flan is a test
written after the feature. The corpus that was written before it is sand.flan
and web/examples/control.flan, and both compile unchanged.
lib/macro.ml is the half of expansion that has to compile something. Expand is
the image format and the quasiquote desugaring and depends on nothing above
Form; this needs Check, Build and Emit, so it sits above the parser it feeds
and arrives through Parse.expander.
What it does, in order:
- Collects every defmacro from the prelude and from the file. Not from an
imported package: Load learns a package's imports by parsing it, so
collecting from one means a second import resolver over Forms, and that is a
bigger thing than this.
- Builds them in rounds, because a macro's body may call a macro and a body
with an unexpanded call in it will not compile at all -- the call is a name
nothing defines. Round 0 takes every macro that names no macro still
waiting; round 1 expands the rest against round 0's module. A round that
takes nothing while macros remain is a ring and is named. macros.flan has
the round-1 case and macro-cycle.flan has the ring, and the distinction
between them is the one thing here that is easy to get wrong: a call inside
a quasiquote is *not* a compile-order dependency. It is part of what the
macro answers, and the answer is expanded again after it returns. The first
macro-cycle.flan written for this commit quasiquoted, and it was not a cycle
at all -- it hit the fuel instead, correctly.
- Walks bottom up, so a macro never sees a call to another macro in what it is
handed, and re-expands what comes back, so a macro that expands into a call
to itself keeps going. That loop is bounded at 200 and says which macro ran
out: macro-spin.flan.
- Skips all of it when the file names no macro, which is nearly every file.
Otherwise every build in the suite would pay a clang driver to answer a
question nobody asked. When it does build, the module is cached under the
object cache and keyed by the prelude's source plus the file's defmacros, so
a second process pays a dlopen.
lib/dune passes -linkall, which is the one line in another lane's file. The
module installs itself into Parse.expander at initialisation and nothing
references it, so without -linkall the linker drops it from every executable
that does not name the module -- bin/main.exe among them -- and a program
calling a macro fails with an unknown name. The alternative was an install
call at every entry point, including ones in files this lane must not touch.
The one thing a macro cannot do that parse.ml could is give a reason. A macro
runs inside the compiler and anything it signals aborts the compile with no
location, so a malformed (unless) answers a name nothing defines and the
report is "unknown name unless-takes-a-test-and-a-body" at the call site --
right place, wrong sentence. NEXT.md says so.
test_flan.ml's "unless -> if(not)" assertion is gone, because it asserted a
desugaring in a file that no longer does one. Nothing else in the suite
changed.
Two things that look like plumbing and are the frontend half of expansion.
A quasiquote becomes calls to the prelude's three form-building functions and
nothing else: form-nil, form-cons for an item, form-append for a splice. It is
pure, it needs nothing loaded, and it runs over every form on the way into
Parse.program and Parse.decl, which is what lets the prelude's own macros parse
in a process that has not built a macro module yet.
Running it *before* the expander's walk is not an ordering preference. A cond
macro's body contains a quasiquoted (cond ...) for its own tail; with the
quasiquote still standing, the walk would see that head and expand it then and
there, against the wrong arguments. Desugared first, that subform is a
(Form.Sym {.s "cond"}) and there is no head left to mistake. So the walk needs
no idea that quoting exists, which is the whole reason this runs first.
Nesting levels are not counted -- not by the reader, which was written that way
deliberately, and not here. A quasiquote inside a quasiquote is refused by
name. Only a macro that writes a macro wants one, nothing in the corpus does,
and CL's level arithmetic costs more than the use case is worth so far.
A defmacro is now an Ast.Defn: (defmacro m [args] body) is
(defn m [args [Form]] Form body). There is no Ast.Defmacro and there is not
going to be one -- a macro is [Form] -> Form, compiled by the same backend as
everything else, and the only thing that makes it a macro is that the expander
calls it at compile time. One parameter, the slice of forms at the call site,
so variadics come free in a language with no &rest; two parameters is a
misunderstanding rather than an arity error and says so.
Parse.expander is the hook the walk arrives through, because expanding a macro
means compiling and dlopening it, so the expander sits above Check and Build
and Parse sits below them. Nothing fills it in yet.
The quasiquote refusal stays as a backstop: it now means a form reached the
parser without coming through program or decl. gensym's refusal is gone -- it
is an ordinary prelude function returning a Form, and a macro body calls it
like any other.
The last commit put the prelude's types into the set the parser uses to tell a
return type from the first form of a body, and put them in plainly. That set is
read by two arms: a bare symbol, and a list head. The list-head arm is why
(defn f [] (Some 1) (bar)) does not lose its body, and the comment above it has
warned about this since it was written -- so adding Rune plainly made
(defn f [] (Rune {.code 65}) (bar)) a function returning a Rune with a
one-form body, silently, in every file in the language. Confirmed before
fixing: it failed with "a map type is {K V}", which is the misparse arriving a
step later wearing someone else's error.
The enums already solve this one comment up, under their own key, for the same
reason. The prelude's types go in the same way. No prelude type takes
arguments, so a bare symbol is the only type position any of them can occupy.
Both halves are pinned in test_flan.ml's return-type section: Form is a return
type, and a prelude struct literal opening a body is not.
The acceptance table runs unions.flan at -O2, at -O0 and as a dev build. -O0
because a union value is built in an alloca and mem2reg is exactly what would
hide a store to the wrong half of it; dev because every body goes behind an
indirection cell there and a union crosses one both as a parameter and as a
return value.
The layout goes through the oracle the DWARF section already had: LLVM's own
answer for the emitted type, read back as a folded ptrtoint. Two unions, one
whose widest case is a pair of f64 and one whose cases are all i32, so the
payload size and alignment are not constants the test could have agreed with
by accident.
Eleven refusals, each by name. The first is the diagnostics bug NEXT.md
listed: a case name written as if it were a struct said "unknown struct A",
because nothing in the environment could tell a case from a misspelling.
Non-exhaustive matches are refused rather than defaulted. A match that fell
through would have to produce a value of the match's type out of nothing, and
the case a union grows tomorrow is the one a reader wants to be told about
today; _ is how to say "the rest", written where it can be seen.
A case pattern binds all of a case's fields or none, positionally: binding
some of them reads the wrong field the moment one is inserted above it.
test_flan's 'match works on an Option at milestone 2' assertion moved with the
message, which no longer blames a milestone that has arrived.
Clojure's spelling and Clojure's semantics. Repeated — #_#_ a b c — discards
that many following forms, and that falls out of the recursion rather than
being counted: the discard reads *a form*, and the form it reads may itself
begin with a discard, so the outer one throws away what the inner one already
stepped past.
It belongs to read_form rather than to the sequence readers, which is what makes
it work in every position a form can appear — top level, inside a list or a
vector or a map, before or after a quote. The two loops that look for a closer
or for end of input skip it as well, because a discard is not an element and a
file ending in one has read everything there is to read.
A trailing #_ with nothing after it is an error, and it is the same error an
unterminated form already gives.
Nothing in dune test exercised cimport.ml or cjson.ml. The raylib case is the
better evidence and the worse coverage: it needs raylib installed, at the
version whose .so is linked, with FLAN_RAYLIB_H set, so as the only test of
this it would skip everywhere and cover nothing.
test/headers/sample.h is one function per decision the importer makes, and the
table asserts on the reasons rather than the counts — a refusal that fires for
the wrong cause still refuses, and a count still matches. Accepted: an
aggregate in and out, const char * as a string, a pointer parameter, a second
typedef name for a record described once, a C enum against a defenum. Refused,
each by reason: a returned char *, a non-const char * C may write through, a
variadic, a callback, a long, a struct with no defstruct, and a kebab
collision. Plus that nothing is in both lists, which is the bug the collision
case found.
check_structs and diff_bound get a row each for agreeing, for a permuted field
order, for a widened field, and for a symbol the header does not have — the
last being how a package pinned to the wrong release announces itself. The
name rule and the JSON reader get their own rows.
Checked by breaking two of them on purpose and watching both fail.
test/programs/raylib-imported.flan is the end-to-end evidence, back and in the
new struct-literal spelling: four bindings the package does not bind by hand.
ColorToInt of {17,34,51,68} is 0x11223344 and ColorTint by white hands the four
bytes back separately, so field order is pinned by arithmetic and not by a
round trip, which is the trap BUILT.md records.
maps.flan and map-exhausted.flan as fixed-output cases, the six refusals
by name, and map-stale-region.flan beside stale-region.flan.
The last one is not a line in the Vec's program because the two reach
the check by different routes. A Vec's operations check on the way in
and stop there. A map's get goes on to call a hash and an equality
function through pointers into the block, so a missing check there is
not a wrong number — it is a probe loop walking released memory. It
traps naming the site and exits 134, as the Vec's does.
{K V} resolves now, so the test that asserted it was milestone 6 is
replaced by the one that still holds: the arity, refused for the reason
Vec's arity is refused, because a near-miss would otherwise resolve to a
type variable and come back as generics.
render.ml's output is parsed by emacs/flan-inspect.el, which hard-codes
the colon when it reads a field out of a rendered struct. Moving the
printer on its own would break inspection in the dev loop without
breaking a test that says so, so the printer waits and moves with its
reader, in the Emacs lane.
The sweep could not tell a rendered *expectation* from a Flan *source*
snippet -- both are strings in a test -- so it converted both. The suite
named every one it got wrong, and those are back.
emacs/test-flan-dev.el:415 is the one edit inside emacs/: Flan source sent
to the daemon for eval, which the parser now refuses in the old spelling.
One label, in a fixture.
