check_finite already walked a union's cases, so a union containing itself by
value was refused before the emitter could try to lay it out -- which it would
have done forever, since payload_lay calls lay calls payload_lay. Asserted
both ways round: directly, and two unions through each other.
Through a pointer it works, and that is the shape a Form has, so it is in the
program rather than only in the prose: a Tree with a (Ptr Tree) field, matched
through a deref, summed recursively.
BUILT.md also records why match's fall-through is still unreachable rather
than a trap. It is only sound because no reachable program can hold a tag no
case names: Zero is tag 0, every construction writes a tag the checker
resolved, and uninit -- the one way to get bytes nobody wrote -- is refused on
a union for exactly this reason. The refusal is what pays for the unreachable.
BUILT.md gets the section and NEXT.md's item 5 and its diagnostics bug are
struck through.
The decisions worth recording are the ones nothing upstream had made: an i32
tag, a payload aligned to the widest member of any case, qualified
construction and bare patterns, declaration-order tags -- so case order is
part of a union's contract the way field order is a struct's -- and a
non-exhaustive match refused rather than defaulted.
And the one finding the macro lane needs: an imported union is still refused
at load.ml:312, but the prelude is prepended into the same flat namespace
before collect runs, so a defunion Form in prelude.ml needs no import and no
load.ml change. Verified by declaring one there and matching it.
The globals section attributed a frame by its slot fingerprint, which is the
wrong cut for it: a redefined body can name entirely different globals while
binding identical locals, so the check saw no change and the new body's
reference set went into the union under the old body's frame, with the frame
numbers beside an entry saying so.
So a second fingerprint. Reach.ref_fingerprint hashes the set of globals a body
names — sorted and deduplicated, because a reference set is not ordered, where
slot indices make the slot fingerprint order-sensitive on purpose — and it
travels the path the first one already cut: %fninfo, flan_dev_frame_refsig, the
agent's snapshot, the backtrace line, Dev.globals_op. Different means the frame
is skipped by name with its reason, and the rest of the stack still contributes.
Two numbers rather than one, because they are two facts. A frame whose slots
match and whose globals do not has locals that are perfectly readable and
attribution that is not, and a combined hash would make locals refuse a frame
with nothing wrong with it. locals still checks the slot fingerprint alone.
It lives in reach.ml because expr_refs is already the walk that answers what a
body refers to, and is the walk the union itself is built from. One consequence:
emit now reaches reach, which closes a cycle through Load if cimport calls
Build.cachedir, so the header cache spells the object cache directory itself.
test_dev.ml drives the exact case — a body that binds identical locals and names
untouched where the stopped frame names pressure. With the check disabled it
fails twice: the missing refusal, and untouched appearing under frame 0.
Everywhere else uninit is an opt-out from ZII and the bytes are whatever they
were: a garbage f64 is a garbage number. A union is the one type where that
is qualitatively worse. Its tag steers control flow, a tag no case names falls
past every comparison in a match, and the block after those comparisons is
unreachable -- which LLVM is entitled to assume cannot happen. So the one
place where garbage becomes "the optimiser may do anything" is refused by
name, with the zeroed form, which is a real case, named beside it.
(.x u) on a union said "Shape is not a struct, so it has no fields", which
is true and unhelpful. A union's fields belong to a case and which case is
being held is what the tag says, so they are reached by match, whose arms bind
the fields of the case they matched. The message says that.
The acceptance table runs unions.flan at -O2, at -O0 and as a dev build. -O0
because a union value is built in an alloca and mem2reg is exactly what would
hide a store to the wrong half of it; dev because every body goes behind an
indirection cell there and a union crosses one both as a parameter and as a
return value.
The layout goes through the oracle the DWARF section already had: LLVM's own
answer for the emitted type, read back as a folded ptrtoint. Two unions, one
whose widest case is a pair of f64 and one whose cases are all i32, so the
payload size and alignment are not constants the test could have agreed with
by accident.
Eleven refusals, each by name. The first is the diagnostics bug NEXT.md
listed: a case name written as if it were a struct said "unknown struct A",
because nothing in the environment could tell a case from a misspelling.
Non-exhaustive matches are refused rather than defaulted. A match that fell
through would have to produce a value of the match's type out of nothing, and
the case a union grows tomorrow is the one a reader wants to be told about
today; _ is how to say "the rest", written where it can be seen.
A case pattern binds all of a case's fields or none, positionally: binding
some of them reads the wrong field the moment one is inserted above it.
test_flan's 'match works on an Option at milestone 2' assertion moved with the
message, which no longer blames a milestone that has arrived.
print, the REPL inspector and the break buffer's locals all walk a concrete
type through render.ml, and a union fell through its Named arm to <Shape>.
It now recovers the case from the tag by a chain of comparisons -- the same
shape the enum arm already had, and for the same reason: the name is erased
before any backend sees it -- and reads the fields of that case only. Reading
the others would be reading a payload that is not there.
It prints (Shape.Dot {.x 1.5 .y -2.5}), which is what the source would write.
The union table has to reach the walk, so Render.ctx grew a field and its
three construction sites in session.ml and one in check.ml pass it. That is
the whole of the session.ml change.
test/programs/unions.flan is the program: a case with no fields, a case wider
than another, a case holding a string, a union in a struct, a union through a
call in both directions, ZII, reassignment, and printing. Its layout was
checked against clang's for the same declaration -- 32 bytes aligned 8 with
the payload at offset 8, and 40/8 for the struct holding it.
The 15.5ms attributed to re-reading the header on every reload is not that.
A timer around each stage says the cached dump reads in 0.33ms, the extraction
takes 3.3ms and the checks 0.55ms — about 4ms, once, in Session.create. The
rest of flan reload's delta is Load and Check over 256 more declarations, and
the +3.6ms a redefinition really pays is Check and Emit.redefinition against a
bigger program. A C-c C-c reads no header at all: eval's forms carry no import,
so no package is read.
Both cache levels anyway, because a long-lived process should pay nothing
twice. In the session, two tables: the dump by header, the declarations by
header and by what the package already declares. On disk, the existing cache
moved into the object cache directory beside the .o files. The in-memory key
is the path and the flags with no mtime, so a header edited mid-session is not
picked up until the session restarts — the rule a changed .c file follows, and
the rule that keeps new signatures from being checked against a process still
running the old layouts.
Measured: repeat import 3.65ms to nothing; flan reload unchanged, as it must
be, since it imports once per process.
It does not get to. Which of the two imports is refused is whichever arrived
second, which follows the entry file's textual order — reverse the two lines and
the message moves from the package's import to the program's. Both refusals are
correct and the needle matches either, so the test was green while its comment
was wrong.
The comment now says what the case actually tests: that a clash is caught when
its two halves are a directory apart, rather than side by side as in
pkg-two-aliases.
defunion parsed and its shape checked; naming the type and constructing a
value were both refused as milestone 6. They are not any more.
A union is Types.Named, exactly as a struct is, so every path that carries a
type -- a field, a parameter, a slot, a copy -- learns nothing about unions.
Which table the name is in is the only thing that tells the two apart.
The layout is a tag then room for the largest case, with the alignment the
widest member of any case needs: %"U" = type { i32, [k x iA] }, and one
named %"U.C" per case laid over the blob. That is C's
struct { int tag; union { ... } u; } byte for byte, which is the requirement
the macro expander's Form will arrive with.
A value is (U.C {.field value ...}), or U.C on its own when the case has no
fields. Construction goes through the struct-literal syntax already there, so
parse.ml is untouched: the dot is a symbol constituent and U.C reads as one
name.
Tags are declaration order from zero, so an all-bytes-zero union is the first
declared case with a zeroed payload -- the same rule that makes an Option's
zero a None, and it makes case order part of a union's contract.
A move-only field in a case is refused in the same words a struct's is, and a
union is refused as a map key: the payload past the case in hand is
indeterminate, so hashing the blob would make two equal values hash
differently.
BUILT.md described a tolerated cycle as a property — "mutually dependent
packages simply work" — and NEXT.md still listed a package importing a package
as the real gap, which it stopped being some commits ago. Both now say what the
code does.
Written down with them: what a name imported through an intermediate package is
called, and why the inner alias is forced rather than chosen; that the diamond
is proven by the numbers pkg-diamond prints rather than by its compiling; and
that pkgs is topologically ordered while the declaration list deliberately is
not.
Package visibility stays on the list. The gap is that a package has no way to
mark a name private, which is surface syntax; the predicate and the refusal it
would hang off are already there.
Loading a package kept one table, keyed by real path, and used it for two
different questions. Already loaded meant "skip", which is right for the second
route of a diamond and wrong for a ring: a package that imported itself round a
chain met its own entry, contributed nothing, and appeared to work. The comment
said so and called it a feature.
It is not one. A ring has no package order, and a definite package order is what
the macro expander needs — every defmacro has to be compiled before anything
that calls it. So the chain currently being read is now carried separately from
the set already finished. A directory found in the first is a cycle and is
refused; a directory found only in the second is still the diamond's second
route and still a no-op.
The refusal names the ring — a -> b -> c -> a — and only the ring, not the route
that led to it. "There is a cycle" leaves the reader to find which three imports
it was.
pkgs now comes back dependencies-first, which is the topological order the
acyclic rule buys. The declaration list is left alone: check.ml collects every
top-level name before it checks any body, so declarations are order-independent
by construction and sorting them would be churn in the field every test reads.
The tests are a real tree rather than a second copy of pkg-shared. pkg-diamond
builds a shape/Box inside area/ and hands it to a function declared inside
draw/, which only type-checks if the bottom package was read once — two copies
of one struct are two types. What proves it is the numbers, not the compile.
Two gaps nothing in the suite reached.
A dev build, because the hash and equality pair emitted for a struct key
is a function nobody wrote, and the only other inhabitant of the lifted
list — a handler-bind clause — carries a parent this one cannot: the
pair is shared by every function that maps that key type, so it has no
single parent. A dev build puts every body behind an indirection cell
and is the build that would notice. It does not; maps.flan answers the
same nineteen ways at --dev as it does at -O2 and -O0.
And a map crossing a function boundary in both directions. Everything
else in the file lives and dies inside one let, so nothing would have
noticed if the 48-byte header travelled wrongly by value while every
runtime operation takes its address. Returning one and passing one are
both moves, which is the rule a Vec already follows — verified against a
Vec rather than assumed, since a refusal that fired for the wrong reason
would look the same.
has-key? is flagged in BUILT.md as what it is: an addition, not
something spec-memory.md names.
BUILT.md gets the Map and the defer relaxation; NEXT.md strikes step 4
and item 3, and records four things that are genuinely open rather than
finished.
The one worth reading is that the Map is slower than CPython's dict at a
million entries while being six times quicker cache-resident. Both are
memory-bound at that size and this layout waits longer: keys, values and
hashes are three separate runs, so a lookup that misses everything costs
three cache misses where a compact dict costs two, and the hash run is a
full eight bytes a slot. Cell packing buys probe locality, which is a
win while the hash run is resident and a loss once nothing is. One byte
of metadata a slot is the known answer and is not built, and the
crossover between the two results is somewhere nobody has looked.
Also recorded: the defer change amends a frozen spec-memory.md, which
said a defer for a let-bound value was not expressible; and the Map is
narrower than the spec on one point, a fixed array being a key only when
its elements compare bytewise.
Measured rather than guessed, and the guesses were wrong twice: the
per-slot cell division and the block-size divisions were each replaced
first, and neither moved the number. A profile named the four that did.
The hash was FNV one byte at a time, a serial multiply chain per byte
and a quarter of the operation. It is eight bytes at a time now, and a
key that is one machine word — every integer, every enum, every bool,
so very nearly every key — is one load and one mix with no loop at all.
This is where "the hash is compiled concretely per key type" stops
describing the arrangement and starts being the reason it is quick.
Equality on eight bytes was a call into libc's vectorised memcmp, an
eighth of the operation, and copying a value out was a call into
memmove. Both are a load and a compare now for the sizes that are one
word.
The block geometry was recomputed five times over inside one function,
and that function ran twice per lookup — once in the probe and once
again in get. It is one struct built once and handed back. The seed was
a five-multiply avalanche on the critical path of every probe, for
mixing the hasher does again immediately afterwards; one multiply is
all it has to do. And 64/size is a table, which is Odin's Map_Cell_Info
by another route — Odin precomputes it per type because the probe loop
must not divide, and the sizes reach this runtime as plain arguments.
Numbers, on this machine, i64 to i64, against CPython 3.13's dict on
the same workload. Cache-resident, 10k entries, 10M lookups: 21ns
against 132ns, so about six times quicker. That is the answer to "is
this another Python dict", and it is the one the design predicted.
At a million entries it loses, 1.41s to 1.16s, and that is worth
writing down rather than leaving out. Both are waiting on memory there,
and this layout waits longer: keys, values and hashes are three
separate runs, so a lookup that misses everything takes three cache
misses where a compact dict takes two, and the hash run is a full eight
bytes a slot. The layout buys probe locality, which is a win while the
hash run is resident and a loss once nothing is.
A global is program state a frame happened to touch, not part of it, so
nesting it under one implies an ownership that is not there and repeats the
name once per frame that reads it. One section instead, holding the union of
the globals every frame on the stack references — the compiler does the
choosing, since Reach.expr_refs already answers a body's reference set, and
listing every global a program has would bury the one that matters under the
prelude's PRNG state.
Each entry says which frames touch it, by the index the stack section already
numbers them with, which recovers what per-frame nesting would have told you
at no cost in duplication. Ordered by the innermost frame that touches it:
a deep stack makes the union large and proximity to the error is what puts
the likely culprit on top.
Simpler than locals, because a global is reached by name rather than by
address. Emit.redefinition writes a global the host has as external, so the
thunk binds to the program's own storage and nothing is asked of the stopped
thread — no dev-slot round trip and no not-yet-bound case to refuse.
A frame that cannot be attributed contributes nothing and is named in
:skipped; the union being incomplete and the union being complete are
different answers. The hole in that is stated rather than papered over:
slot_fingerprint hashes a body's slots, which is the right cut for locals and
not for this, so a body that names different globals while binding the same
locals is not caught. The test drives the case that is.
MANUAL.md also loses a stale paragraph claiming the fingerprint check never
fires with a failing test pinned to it. It fires, and test_dev covers it.
The client already knew the moment: flan-dev--absorb reads :stopped off every
reply and the poll covers the case where no reply is coming. This is a hook at
that point, not new plumbing.
Deferred through a zero-delay timer, which is the part that is not optional.
absorb notices the stop in the middle of reading a reply on the socket, with
flan-dev--busy bound, and showing the buffer asks the daemon three more
questions — break, layout, backtrace. Issuing those from inside the read they
were triggered by would interleave two conversations on one connection. The
deferred call re-checks the state rather than trusting the edge that scheduled
it, because by then the edge has been consumed and the program may have been
resumed.
Three decisions, settled and written down beside the code.
It displays and does not select. A program stops on its own clock, not the
editor's, and the likeliest moment is a frame of its own game loop while
someone is typing somewhere else. Taking the window would send the next
keystrokes where they were not aimed. `focus' is there for anyone who
disagrees, and nil goes back to the mode line alone.
(pause) is not a special case, though it was worth asking: it is deliberate at
the moment it is *written*, and the frame it fires on still arrives whenever the
program gets there, which is no more expected than an error. What it does get is
an honest headline. (pause) is `error' under a `restart-case', so nothing in the
compiler knows a breakpoint from a failure and this buffer is the first place
that can — calling it unhandled is a small lie at the top of the one buffer that
exists to say what happened.
A stop mid-edit disturbs nothing, which falls out of displaying rather than
selecting. Two guards go past that: nothing happens under an active minibuffer,
because a prompt is modal and rearranging windows under one is hostile; and
nothing happens inside a keyboard macro, because a macro that behaves
differently depending on whether the program stopped cannot be trusted. In both
cases the mode line still says stopped and C-c C-b still works.
render.ml's output and emacs/flan-inspect.el's parser are the two ends of one
wire format, which is why the printer was left on the colon when the rest of
the corpus moved: shifting it alone would have broken inspection in the dev
loop without breaking a test that said so. They move together here.
The field list in the inspector is labelled with the dot too, which is the
spelling flan-inspect-step-expr already used to build `(.x b)' — the label and
the expression it stands for now read the same.
One case needed a guard the colon never did: `...' also begins with a dot and
is the renderer saying it stopped, not a field called `..'. A field name never
starts with a second dot, so one character of lookahead separates them.
The colon is not gone from the rendered grammar. An enum member is `:green' and
is a *value*, so the two are now told apart by the character alone, which is
the only thing that distinguishes them.
Also font lock, handed over with the same change: `:name' was the rule that
drew field labels, and with the colon belonging to keywords every label in the
corpus was left unfontified. `.name' is drawn as a constant, in both the places
it appears — the label in `{.x 1.0}' and the accessor in `(.x v)', which are
the same name.
Decimal is what the value is and stays first; hex and binary go beside it. It
is the wrong base for about half the numbers anyone opens this buffer for — a
colour is 0x303030FF, a gesture is an OR of flags, a mask is read a bit at a
time — and reading those out of a decimal is arithmetic done by hand.
In two places: under the header of a value opened on its own, and on each
numeric row of a field list. The second is the one that matters, because a leaf
cannot be stepped into, so the field list is the only place most numbers are
ever seen.
Nothing is asked of the program. It is arithmetic on text the renderer already
wrote, so it works on a stopped program and costs no round trip. Binary is
grouped in nibbles because a mask is read in nibbles. A negative is shown as
the 64-bit two's complement it is in memory and says the width out loud: the
rendered value carries none, and every Flan integer comes back through i64.
A float is left alone rather than answered wrongly — its bits are an IEEE
layout, reinterpreting them is a different question, and the rendered text does
not carry the width to answer it.
The pointer half of this is not done and the refusal now says why. Render.render
writes the bare word <ptr> for every pointer on purpose: it is the same renderer
print uses, an address is not stable across runs, and test_acceptance pins the
current text for that reason. Showing one is a decision about the language's
printer, not about this buffer.
BUILT.md gains "The header is read now", directly under the section whose last
paragraph promised that reading a header was what would convert the trusted
half into a checked one and that it was not built. That sentence is replaced by
a pointer to the one below it, in BUILT.md and in shim.ml's docstring both.
It records the things worth not re-deriving: why the dump and not libclang (and
that Zig left libclang too, which strengthens the argument rather than weakening
it), why the import is bounded by the package's own defstructs, why generating
defstructs would make the check circular in exactly the way a _Static_assert
was rejected for, refusal-by-demotion from Zig's failDecl, the naming rule and
what it must actually guarantee, and both const-vs-non-const char * and the
target-varying widths.
The diff and the costs are stated as measurements, with the table: 16 of 16
defstructs and 172 of 172 declare-c agree against 5.5, ten real differences
against 5.1-dev, release +4ms warm, redefinition 31.0 -> 46.5ms.
DISCUSS.md item 6 is rewritten rather than removed. The mechanism question is
settled and is now in BUILT.md; what is left is narrower and is two decisions
that are the author's — whether the header stays a build-time read or becomes a
committed generator, and whether the 172 hand-written lines migrate. Both have
the argument on each side written out, including what migration would lose:
key-pressed? is a better name than is-key-pressed, and an enum parameter
imports as i32 because nothing tells the importer the package calls KeyboardKey
"Key".
The indentation rules were written and tested but never described anywhere a
user would look. MANUAL.md had no section on editing at all — it starts at
`C-c C-c' and assumes the file is already written — so the rule that cost the
friction, a binding vector lining up name under name, was only visible by
trying it.
What is written down is what was checked, not what the port was aimed at: the
call fallback, the `handler-bind' clause vector, `defn' parameter alignment
with a return type after it, and `restart-case' clause bodies were each
reindented from scratch and the manual quotes the result.
NEXT.md keeps the half of the field-label handover that is still open. The
printer in render.ml has to move in the same commit as the inspector that
parses it, and that is the inspector lane's; the font-lock half is done here,
so only that half is struck.
The same boundary the raylib FFI case covers, reached from declarations
generated out of the header instead of transcribed into raylib.flan. The
package binds none of the four functions by hand, so the program running at all
is the claim.
What it prints pins more than that, by the argument the GetColor case already
makes: handing a struct over and reading it back proves nothing, since storing
and returning is symmetric and a permuted layout comes back permuted the same
way. ColorToInt of {17,34,51,68} is 0x11223344, so exchanging any two fields
changes the number, and ColorTint by white hands the four bytes back
separately. TextLength of "hello" is 5 only if the wrapper NUL-terminated the
copy.
At -O0 as well, for the reason the rest of the table is: every struct here
crosses as (addr v) on a local, which is the alloca mem2reg would launder
before anyone noticed it was wrong.
Skipped without FLAN_RAYLIB_H, since the import is opt-in. The importer's own
table does not skip — it runs against test/headers/sample.h, which is
committed.
Clojure's spelling and Clojure's semantics. Repeated — #_#_ a b c — discards
that many following forms, and that falls out of the recursion rather than
being counted: the discard reads *a form*, and the form it reads may itself
begin with a discard, so the outer one throws away what the inner one already
stepped past.
It belongs to read_form rather than to the sequence readers, which is what makes
it work in every position a form can appear — top level, inside a list or a
vector or a map, before or after a quote. The two loops that look for a closer
or for end of input skip it as well, because a discard is not an element and a
file ending in one has read everything there is to read.
A trailing #_ with nothing after it is an error, and it is the same error an
unterminated form already gives.
Nothing in dune test exercised cimport.ml or cjson.ml. The raylib case is the
better evidence and the worse coverage: it needs raylib installed, at the
version whose .so is linked, with FLAN_RAYLIB_H set, so as the only test of
this it would skip everywhere and cover nothing.
test/headers/sample.h is one function per decision the importer makes, and the
table asserts on the reasons rather than the counts — a refusal that fires for
the wrong cause still refuses, and a count still matches. Accepted: an
aggregate in and out, const char * as a string, a pointer parameter, a second
typedef name for a record described once, a C enum against a defenum. Refused,
each by reason: a returned char *, a non-const char * C may write through, a
variadic, a callback, a long, a struct with no defstruct, and a kebab
collision. Plus that nothing is in both lists, which is the bug the collision
case found.
check_structs and diff_bound get a row each for agreeing, for a permuted field
order, for a widened field, and for a symbol the header does not have — the
last being how a package pinned to the wrong release announces itself. The
name rule and the JSON reader get their own rows.
Checked by breaking two of them on purpose and watching both fail.
test/programs/raylib-imported.flan is the end-to-end evidence, back and in the
new struct-literal spelling: four bindings the package does not bind by hand.
ColorToInt of {17,34,51,68} is 0x11223344 and ColorTint by white hands the four
bytes back separately, so field order is pinned by arithmetic and not by a
round trip, which is the trap BUILT.md records.
maps.flan and map-exhausted.flan as fixed-output cases, the six refusals
by name, and map-stale-region.flan beside stale-region.flan.
The last one is not a line in the Vec's program because the two reach
the check by different routes. A Vec's operations check on the way in
and stop there. A map's get goes on to call a hash and an equality
function through pointers into the block, so a missing check there is
not a wrong number — it is a probe loop walking released memory. It
traps naming the site and exits 134, as the Vec's does.
{K V} resolves now, so the test that asserted it was milestone 6 is
replaced by the one that still holds: the arity, refused for the reason
Vec's arity is refused, because a near-miss would otherwise resolve to a
type variable and come back as generics.
The reported bug — the second and later bindings of a let one column too far —
was never one missing rule. `flan-indent-function' checked the head of the
enclosing form, and inside a binding vector the enclosing open is `[' and the
symbol after it is the first binding's name, so it fell through to Emacs's
`lisp-indent-function', which treats the vector as a call and aligns under the
first argument instead of the first binding.
Emacs Lisp is the wrong reference. It has no vectors-as-bindings, no maps and
no bracket variety, so every rule Flan needs has to be added by hand and the
binding vector is simply the first one hit. The indenter is rewritten from
clojure-mode's source instead: `clojure-mode' is neither an ancestor nor a
dependency — flan-mode still needs nothing beyond stock Emacs — it is the file
whose rules were read and written out again.
A bracket aligns under its first element, and that one rule fixes the binding
vector, `defn' parameter lists, `restart-case' and `handler-bind' clause
parameters and both spellings of a struct literal at once. `{:x 1}' and
`{.x 1}' indent identically because nothing here looks at the key, which is
what the colon-to-dot lane needs of it.
Where Flan diverges it is handled on purpose. `defn' is `:defn' rather than a
count because the return type between the parameters and the body is optional.
A clause — `(Name [params] body)' — is recognised by its shape, since its head
is a condition class or a restart name and can never be in a table; clojure-mode
reaches the same clauses by backtracking out to the enclosing form, which buys
generality this language has no other use for. Special arguments indent by one
body rather than Clojure's two, and a call whose head is alone on its line
indents its arguments by a body rather than aligning them under the head,
because that is how the whole corpus is written.
Checked by reindenting every .flan file in the tree: the only lines that move
are sand.flan's reported bug, raylib.flan's hand-wrapped parameter vectors —
which is the fix — and lone-`;' comment continuations, which stock
`lisp-indent-line' has always moved.
test-flan-mode.el is loaded from test-flan-cider.el rather than given a stanza
of its own, because emacs/*.el is already a dependency of that test.
Two failures in test-flan-cider.el that predate this: fixture frames lacked
`:fetched', so folding one open went looking for a daemon, and `layout' was
identified by being the last request when `flan-cnr-show' now makes three.
One rule over every allocating operation, so it has to hold for map-new,
put, reserve and clone exactly as it holds for vec-new, push, reserve
and clone. put stays Unit and clone stays the container; nothing grows a
Result.
A map is the harder of the two and that is why it gets its own program.
A Vec's failing allocation leaves the Vec untouched, whereas a map's
growth allocates a whole new block, rehashes into it and only then
releases the old one — so a failure partway has to leave the map exactly
as it was or the retry re-attempts against a half-moved map. 300 entries
through several grows against a ceiling that is raised each time, then
every one of them read back: no entry lost, none doubled.
Two C functions whose names kebab to one Flan name used to resolve by order:
the first won the name, the second was refused. Which one that is depends on
the order the header happens to declare them in, so moving two lines in
somebody else's header would silently rebind a name a Flan program is already
calling — and the winner was left in the hidden list too, so using the name it
did get reported that it could not be had.
Neither takes it now. There is no reading of spin-2d that is obviously right
when the header offers both Spin2D and spin2d, so both are refused and both say
why; the author binds the one they want with a hand-written declare-c, which is
what that form is for. Found by test/headers/sample.h, which is why it is a
fixture rather than a raylib case.
raylib is unaffected: its 581 names are injective under the rule.
Reading the header produced declarations and nothing else, so the gap the whole
thing exists to close — that nothing verifies a declaration against the library
— was closed by a command somebody could run rather than by a property the
build had. Now `import` runs both comparisons whenever a header resolves.
Build-stopping, not a note. The package named the header, so the header is the
package's own claim about what it binds; a defstruct that disagrees lays fields
out in the wrong order and reads as five plausible numbers rather than as a
link error. Continuing past a known-wrong layout to produce a program that will
read garbage is the shape the house rule against swallowing things exists to
prevent. Both messages point at the line in raylib.flan, not at the header.
Verified by breaking it on purpose: a permuted Texture2D stops the build naming
the field that moved, and `f64` where raylib says `float` stops it naming the
parameter — which is the hazard BUILT.md calls out by name and says only a test
can catch.
A set-but-wrong FLAN_RAYLIB_H used to be indistinguishable from not opting in:
the line was skipped and nothing was said. Unset still means off and silent; a
path that is not there is now an error naming it. That is the difference
between an opt-in and a trap.
test/headers/sample.h is one function per decision the importer makes. The
raylib case needs raylib installed, at the right version, with a variable set,
so it would skip everywhere and cover nothing; this one does not move. It also
found a bug, fixed next.
Reach still prunes with 256 extra declarations in play: a wasm32-wasi build of
a program that imports raylib and calls none of it links without libraylib,
which is the case Reach.link exists for.
`headers` beside `link`, read the same way: a path, any clang flags that header
needs, ${NAME} expanded from the environment. What comes back is ordinary
declare-c declarations, generated before the package's names are qualified, so
they arrive as rl/… exactly like the hand-written ones and nothing downstream
can tell which is which. No new form, no new decl_kind, no reader or parser
change.
A leading `?` makes a line optional. vendor/raylib uses it, because "a build
needs libraylib linkable and not raylib-devel installed" is a property worth
keeping — requiring a header would take it from everyone to give the check to
whoever has one. Unset FLAN_RAYLIB_H and the build is exactly what it was; set
it and every signature is checked against raylib's own header.
A C symbol the package already binds by hand is left alone, so declare-c
remains the escape hatch and stays the thing that wins. A refused function
becomes a hidden name through Load.refuse_hidden, so writing rl/get-gamepad-name
says "GetGamepadName returns char *, and a string only crosses as a parameter"
rather than "unknown name".
Measured, because the cost is the whole argument for how much to import:
release build +14ms cold, +4ms warm — Reach prunes the wrappers
redefinition 31ms -> 46.5ms
dev build +333ms cold — dev does not prune, 428 wrappers
Reach.link already drops a generated wrapper whose declaration nothing
reachable calls, and that is what makes a wholesale import cost nothing in a
release build. It does not prune dev builds, on purpose, so a dev build
compiles every wrapper once at session start; Build.shared compiles no C, so
redefinition does not pay that again.
Reading the header is cached — 64ms of a 72ms check, against 8ms for the whole
program without it. Keyed like the object cache, on everything that could
change the answer: the header's path, size and mtime, the full flag list, and a
format version, since the cached value is a marshalled dump. The extracted
signatures are cached rather than clang's JSON, because the parse is half the
cost. That takes the delta to 17ms.
Verified end to end and headless, using only imported declarations:
ColorToInt of {17,34,51,68} is 0x11223344 and ColorTint hands the four bytes
back separately, so field order is pinned by arithmetic rather than by a
round trip. TextLength of "hello" is 5, so the string crossing works.
declare-c generates the wrapper, the typedefs and the prototype from one
declaration, so they cannot disagree with each other. What nothing checked was
whether the declaration matched the library — BUILT.md records that as trusted
rather than guaranteed, because no header was ever read.
This reads one. clang is asked for a JSON AST dump of the header and shelled
out to, not linked: -Xclang -ast-dump=json is the same binary on PATH that
every build already runs, which is plan.org's "Why LLVM IR as text" applied a
second time. Zig's old @cImport linked clang as a library and that is precisely
the dependency plan.org rejected.
cjson.ml is enough JSON to read the dump and no more, so this adds no opam
package to parse it.
What comes out of the header is signatures and nothing else — not structs, not
enums, not macros. The bound on how much is imported is the package's own
defstructs: a function whose signature mentions a struct the package has not
described is refused with that reason, so vendor/raylib describing thirteen
structs is what makes the import thirteen structs wide. Keeping the layouts
hand-written is also what makes checking them against the header's records
worth doing — a _Static_assert was rejected in BUILT.md as circular, and this
is not, because the two sides have different authors.
Refusals are demotions, taken from Zig's translator: it never drops a
declaration it cannot handle, it binds the name to a @compileError carrying the
reason so the failure lands at the use site. Load.refuse_hidden is already that
mechanism. So a returned char * does not kill the header — it makes one name
unavailable, with the reason attached.
flan import-c prints what it would produce, what it refused, how the package's
defstructs compare with the header's records, and how the hand-written
declare-c lines compare with the header's signatures.
Against raylib 5.5, the version whose .so vendor/raylib/link names: all 16
defstructs and all 172 hand-written declare-c agree exactly. Against the 5.1-dev
header installed in /usr/local it reports ten differences, nine functions that
version does not have and one that gained a parameter — so the check has teeth
and the clean run is not a vacuous one.
test/programs/maps.flan is seven claims over the Map, each one a
plausible wrong version gets wrong, with the numbers differing per
failure so a single wrong answer names its own cause: an integer key
past eight grows, a struct key whose padding must never be hashed, a
struct key holding a string, an enum key, clone's independence, upsert
not growing the length, and a map living in an arena.
The move refusal said "a Vec is move-only" whatever had been moved, so
moving a Map was reported as a fact about Vecs. It names the type now.
The checker half. {K V} and (Map K V) resolve, and map-new, put, get,
has-key?, len, reserve, clone and free are named calls over the
type-erased runtime, with the two sizes and the key's hash and equality
pair produced at the site because the site is where the concrete types
are known. len, reserve, clone and free were extended rather than given
map-shaped names of their own, which is what at and len already did for
Vec: one question, one word.
The key's pair is resolved per key type and mostly is not emitted at
all. Every integer, enum, bool and fixed array of those is compared
bytewise and served by one runtime pair over (pointer, size). A string
is not, because its bytes are elsewhere and two equal strings at
different addresses must hash alike. A struct is not, because its
padding bytes are indeterminate — two structs equal field by field can
differ bytewise — and because it may hold a string. So a struct gets a
pair emitted for it, walking its fields in declaration order and
addressing nothing but fields, and that is the only case that does. Two
maps with the same key type share one pair, and a struct reached twice
through two fields emits one.
get returns (Option V) and builds it here rather than in the runtime,
which has no idea what an Option's layout is — keeping it that way is
what lets one entry point serve every value type. put is upsert
returning Unit. Both bind their arguments to slots before the guard, so
a retry re-attempts the allocation and not the expressions that produced
the key and the value.
Refusals, each by name: a float key has no usable equality at all, which
is not a milestone question; a Ptr, slice, Vec or Map key would hash an
address rather than what it points at; a move-only value would have its
header duplicated by clone, which is the refusal (Vec (Vec T)) already
carries; Unit as a value has no bytes to store, and it is the natural
spelling of a set, so it is refused by name rather than by dividing a
cache line by zero.
Work in progress: it builds and the runtime is exercised and green, but
no Flan program can reach it yet — the checker half is not written, so
(Map K V) is still refused where it is resolved.
runtime/flan_rt.c is Odin's map, followed deliberately: open-addressed
Robin Hood hashing at a 75% load factor, cache-line cell packing so no
key or value straddles a line, and the probe loop kept to pointer-width
integers. One type-erased runtime over (key size, value size) plus a
hash and equality pair, the same arrangement the Vec runtime has over
(size, align).
Two departures from Odin, both deliberate and both commented where they
are made. There are no tombstones, because removal is deferred by
spec-memory.md, and that deletes the backward-shift loop entirely — it is
the single largest reason this is shorter than the original. And the
header does not stuff log2cap into the low bits of the data pointer:
Odin does that because Raw_Map must be three words, whereas this header
already carries an allocator, a generation and an epoch, so the tagging
would buy nothing, cost a mask on every access, and make correctness
depend on the block being 64-byte aligned rather than merely faster
when it is.
The scaffolding around it: a Map is 48 bytes and six words like a Vec,
it crosses to the runtime by address because it is move-only and must be
mutated in place, and it has a DWARF type showing all six fields.
Tast.FnAddr is new — the address of a function, either one this compiler
emitted or a runtime C symbol. It is not a function value: nothing in
the surface language can produce one, name its type or call through it.
Odin's Map_Info reaches its hash and equality pair exactly this way.
reach.ml learns that edge, because a function reached only by address is
invisible to the reachability walk otherwise, which is the same hazard
handler-bind clauses already had.
The hash and equality pair carries the transfer channel as its last
parameter, because a pair emitted for a struct key is an ordinary Flan
function and every Flan function's signature ends with one.
