web/index.html had no section on the FFI's generated half at all; it has one
now, with the command, the config, and the reason committing the output is what
makes the no-header property honest rather than a caveat.
BUILT.md gets why the 172 stay, which is the part that is easy to get wrong:
136 of them are exactly what the rule produces and the rest are expressible as
overrides, so the superset argument is sound and still leads somewhere bad —
deleting them reduces the signature check to a tautology.
DISCUSS.md 6a and 6b are answered rather than left open, and 6b's own point
about enums turns out to be live in the tree: key-down? keeps its Key
parameter because it is hand-written, and the generated key-up? beside it
takes an i32.
compile.el puts note in the same capture group as info — group 7, level 0 —
while warning is group 6, level 1, and compilation-skip-threshold defaults to
1. So next-error walks the errors with no configuration, which is the claim
M-x compile rests on and it holds; it steps over the notes until the
threshold is 0. They are still parsed, coloured and clickable.
Labelling notes warning: would make them navigable at the default and is
refused. A note is not a warning, and a compile whose only complaint is an
error would start reporting warnings that are not warnings.
The macro expansion field gets the test it was missing, through a real
expansion rather than a unit test on either half: the tag is put on by Macro
and defaulted into the diagnostic by Loc, and either half alone would pass
with the other broken. clamp misused expands into a call to a name that does
not exist, so the checker refuses something the author never wrote, which is
the case the field is for.
Exclusion says why rather than going quiet, a pattern matches by prefix, an
override changes the Flan face and leaves the C symbol verbatim, and a rename
dissolves a kebab collision instead of leaving both halves refused — that last
one is why the kebab rule is consulted in exactly one place.
The file itself too: a line that is neither directive is an error, because a
typo in a name override would otherwise land a binding under the wrong name.
BUILT.md gets the section: the 32/32 split, live-is-odd and the two things
that fall out of it, wrapping retiring the slot, why resolve answers
(Option (Ptr T)) and where the spec already said so, why len is the slot
high-water and not the live count, and why a slot is released through the
pool rather than through free.
Two amendments to a frozen spec, both deferrals: .field and at do not
auto-deref a handle, and deref is not overloaded on one. Neither can answer
"gone", which is the whole job, and the spec's own worked example resolves
first and matches.
The pointer hole is written down rather than implied: a (Ptr T) from resolve
dies on any insert that grows, which is the slice contract one level down.
NEXT.md swept, not just struck — five places beyond item 6 were still
asserting that Handle did not exist.
generated.flan carries the 253 declarations the importer reads out of raylib's
header, so a build needs libraylib linkable and no header at all. The opt-in
no longer decides how many bindings a package has — every build now gets all
425, they are greppable, and they diff when raylib moves.
What that gives up is the build-time check, so `flan generate-c` is the only
thing that writes the file and it compares first: every defstruct against the
header's record, every hand-written declare-c against the header's signature,
and it writes nothing when they disagree. Against the 5.1-dev header on this
machine that is ten real differences and no write.
The 172 hand-written lines stay, and not out of caution. Everything the
generator emits agrees with the header by construction, so diffing generated
output against its own source is a tautology; the hand-written lines were
transcribed by a person, so they are the only thing here a header can
contradict. All ten of those differences came from them.
`bindings` beside `headers` is what survives regeneration, because a hand-edit
to a committed generated file does not. Two directives: `exclude` drops
raylib's three allocator entry points, and `name` gives the 19 generated
predicates the `?` spelling the hand-written ones already use.
BUILT.md gets the design: why the span went into Loc.t rather than beside it,
why macro provenance went the same way, why the first line of a report is
still the GNU format, and what the daemon sees.
NEXT.md item 8 is struck through, with the parts that were not built stated
plainly so they do not read as oversights — the reader does not collect,
because a paren stream cannot be resynchronised; pass one of the checker does
not collect, because thirty unknown-name lines under one wrong signature are
the same error thirty times; and there are not a hundred kinds, because the
count was never the feature.
handles.flan joins the ASan and memcheck corpora; pool-stale-region.flan
joins memcheck as a seventh program that aborts by design, for the reason
the other six are kept — a trap that stopped firing would be silent. Clean
both ways.
Loc.Errors is a second exception, and the handlers in the session and the
daemon name only Loc.Error — so a list reaching them is an unhandled
exception and a dead session, which is the one thing the dev loop exists to
prevent. A flag on the function the session already calls left that one
label away from happening. Parse.program_all and Check.program_all are
separate names, so the session's call site has to be edited by a person for
its behaviour to change, and the guarantee stops being a default argument.
Placeless diagnostics now sort last rather than first. A wrong main signature
is raised against unknown, which is line 0, and sorting on the number alone
put it above every error that can actually be clicked. It is a real error and
it is not anywhere, so it goes after the ones that are.
A kind is a stable id per error, so a test can assert which error this is
without matching on prose and a message can be reworded without breaking
anything. The reader's fourteen refusals all have one; in the checker they
go on the errors a test names and the handful that are common enough to be
worth classifying. Not a hundred of them, because jank has a hundred from
being mature and the number is not the feature.
The notes are the part that could not be said before. A duplicate definition
now points at the second and notes the first; a duplicate parameter and a
duplicate field do the same; an unknown field, an unknown struct and a
non-exhaustive match all note the declaration and list what is actually
there, so the reader's next move arrives with the question instead of after
it. The reader's unclosed bracket is the clearest case — the error sits on
the bracket, because that is where the fix goes, and the note sits where the
file ran out, because that is the surprise.
No message text changed, so every existing needle still means what it meant.
The new assertions are on kinds and on note positions, which is the house
rule about asserting the reason, made stable.
The refusals: the arities, a pool of an owning element, ordering handles,
free of a handle, and clone of a pool. Each names what it would cost — a
cloned pool duplicates the generation counters with the slots, so one handle
would resolve in both copies and name two different things.
pool-stale-region.flan is the other failure, and the point of it is that it
is not the first. A stale handle is an answer and resolve says None; a pool
whose region was released has no slot array left to ask, so it traps. Same
rule that keeps a Vec's generation word and its epoch word apart.
The provenance rides on the location, not on the form, because the location
is the thing that already travels: Expand.unmarshal stamps the call site onto
every node a macro answers with, and that stamp goes on through the AST and
the typed IR untouched. Tagging it there means an error raised anywhere
downstream can name the macro with no field added to Form, to Ast or to Tast.
Outermost wins. The macro the author wrote is the one worth naming, not
whatever it expanded into on the way down.
The honest limit, since it would otherwise read as a claim: a macro's
expansion has no source of its own to point at, so the note lands on the call
site along with the error. What it buys is the reader knowing the code being
refused is not the code they wrote.
A sink collects what a pass found so the pass can go on to the next thing.
It is switched on by the caller, not by the code that raises, which is what
leaves the interactive path untouched: the daemon checks one form, asks for
a sink that is off, and still gets one exception.
Two resync points, and both are places the work already had a boundary. In
the parser it is a top-level form — the reader found where each declaration
ends, so skipping a bad one cannot lose its place, while inside a
declaration there is no such landmark and one bad defn stays one error. In
the checker it is the two passes: pass one, which builds every name and
signature, still stops at the first refusal, because a signature it could
not make sense of leaves a hole that pass two would report once per mention.
Thirty unknown-name lines under one wrong signature are not thirty errors.
Pass two is where the volume is and where collecting pays, and by then every
signature is sound, so a body that fails cannot make the next body fail.
That is what makes a declaration a resync point needing no resynchronising.
The surface: (pool-new T), (insert p x) answering a handle, (resolve p h)
answering (Option (Ptr T)), (release p h) answering whether this call was
the one that released it, (len p) and (live p), and (pool-handle p i) for
enumeration. free extends to the pool and refuses a handle by name, because
a handle owns nothing and consuming one copy would say nothing about the
others.
resolve answers a pointer rather than a value because spec-memory.md's own
worked example does, and says why a line above it: a pattern binding binds
a value, and a copy cannot be written back.
test/programs/handles.flan prints <handle 1:1> and <handle 1:3> for the same
slot before and after a death, and the projectile still holding the first
gets -1 rather than the newcomer's 99.
The first line of an entry is still exactly file:line:col: message, because
that is the GNU format compilation-mode already parses and the whole of the
editor story. Everything under it is indented, which compilation-mode
ignores, so the underline is free. A note gets an entry of its own rather
than being folded into the error's block — that is what makes the second
place somewhere next-error can go, and is the reason notes carry locations.
Every part of it degrades to the bare first line: a location the checker
invented has line 0, the prelude and the REPL have names that are not paths,
and a file can change under us between being read and being blamed. An error
printer that can raise is worse than one that prints less.
(Handle T) and (Pool T) land as types and as a runtime. A handle is one
int64_t — slot index low, generation high — so it copies, zeroes and
compares like the integer it is and owns nothing. A live slot's generation
is odd, which makes a zeroed handle resolve to nothing rather than to slot
zero, and makes iteration free. Wrapping retires the slot rather than
reissuing it: 2^31 reuses is rare, and rare is not an answer when the
failure is the silent wrong one the type exists to prevent.
No surface yet — the checker still has no names for any of it.
Loc.Error now carries a diagnostic: a stable kind, a span, notes that each
have their own span and severity, and the macro expansion it came from. The
notes are the part that was actually missing — "this is wrong here" plus
"because of that, over there" is two places and two explanations, and a
single string can state only one of them.
The compatibility story for the daemon, which was the open question: the
single-diagnostic exception stays the single-diagnostic exception. Session
and dev evaluate one form and have one failure to report, so they take a
location and a message out of it with Loc.summary and are otherwise
unchanged. A second exception carries a list, and only a driver that
compiles a whole file raises it, so nothing interactive has to know it is
there.
No message text changed.
Loc.t grows an exclusive end, defaulting to the start, so a location nobody
widened is a zero-width span at a point and every existing call site keeps
its old meaning. Only the reader knows where a form ends, so only the reader
fills them in — one helper in the one place that holds both ends, which is
why nothing above Reader had to learn a span exists.
The width assertion is the point of the tests: the field could exist, nothing
could fill it, and every underline would be one character long while the
feature looked finished.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
The slot after a defn's parameters is about to become mandatory, and a void
function has to have something to write there. () is ML's spelling and it
cannot collide: an empty call is not a valid expression, so () has no reading
in value position for a body form to be confused with.
Additive on its own. Internally it stays Tname "Unit" -- the resolver, the
shim and the emitter all speak that name and none of them change -- so this is
two arms in parse.ml: texpr reads () as the unit type, and is_type_form says
that a leading () is a return type rather than the first form of a body.
The prose still said --two-process measured faster in every column, which was
true of the one number taken before item 1 and not of the four retaken back to
back. In the after pair the merged build is 0.9ms ahead; in the before pair
two-process was 1.3ms ahead. A difference that changes sign between runs is a
difference made of noise, and that is a better argument than a direction would
have been.
NEXT.md's file table gains lib/agent.ml and stops describing dev.ml and the
agent as though the compiler reached the program over a socket. It is the index
a next lane reads first.
Third on the list to delete, and nothing goes. "The compiler can read the
stopped frame's memory directly, so copying it is ceremony" is the right
instinct and the wrong diagnosis: the snapshot was never about two address
spaces.
A stopped program is not holding still. The break loop polls, flan_agent_poll
runs whatever was delivered, and a C-x C-e thunk is arbitrary Flan that pushes
and pops the one global restart list and the shadow stack while it runs. The
compiler is a thread beside it either way. Restart names are copied because
serving them off the live list hands the reader a pointer into a frame the
loop's own poll may already have popped; a pointer is meaningful to the
compiler now, and the frame it points into is no more alive for that. The
fingerprints have a sharper reason still, already written down: the module a
frame's description lives in can be unloaded before the comparison happens.
The generation stamp is about nested breaks. A thunk this loop runs can error,
push a break of its own, and reach chosen_ready first, claiming an index someone
chose from the outer list. Depth cannot tell those apart because a resumed
outer break and a new one reuse the number. One process changes none of that.
So of the three things the merge was expected to make deletable, one was. The
socket was transport. The result cap and the snapshot are concurrency, and were
only mistaken for transport because the socket was in front of them.
What the merge does unlock here is item 4, deliberately not touched:
flan_agent_frame_slot already hands back an address, and in one process the
compiler could read the value there instead of compiling a thunk to print it.
RESULT_MAX was written down twice, once in flan_dev.c and once in the agent,
with a run-time check that the two had not drifted. That second copy was a
buffer sized to be sent through a socket, and it is gone: the agent asks
flan_dev_result_cap() and allocates, which it can do because it is never the
game thread.
The bound itself stays, and the reason is the rule everything else here is
built around. result is what the game thread writes into, from a render thunk
at a frame boundary; a growable one is the frame thread calling realloc, which
is an allocation in the one place this design exists to keep allocation out of.
It would also break the seqlock, which is a protocol about torn contents and
assumes the address it copies from neither moves nor goes away underneath the
reader. Growing on the writer's side is a use-after-free the counter cannot
see.
So it is a render budget and not a wire size, and it only looked like one
because the agent had a copy of it. Removing the bound is a redesign of the
read -- probe, allocate, re-read, validate, retry -- and belongs with moving
the read to a frame boundary.
Also: flan_agent_request answers "err path too long" where the socket does,
instead of nothing. One verb table is only worth having if the two callers
cannot be told apart.
The measurement table in BUILT.md is retaken back to back across all four
shapes, because this machine drifts by more than the thing being measured. The
internal socket was ~41us of a 21ms redefinition; --two-process is unchanged
and is not slower than the merged build in any column.
flan dev is one process, so the unix socket between the compiler and the
program was a connect, a write and a read that never left the machine's own
memory. A delivery is now a call into the agent's verb table.
flan_agent.c's handlers write into a sink that is either an fd or a growing
buffer, so serve() and the new flan_agent_request() share one verb table rather
than two that would drift -- every answer in there is assembled from several
pieces, so the seam had to be the writing. Dev.deliver, Dev.result and Dev.ask
were three copies of the same socket dance and are now three lines over one
Dev.request.
Which path is taken is the linker's answer, not a flag: flan_agent_request is
weak in dynload_stubs.c, null in every binary that links no flan_agent.o, and
Agent.request is None there. So --two-process, flan reload and every test go on
using the socket with nothing to configure.
Three things the direct path must not quietly change. The ring's room check is
separate from its store, which was safe only while the accept loop was the sole
producer, so handle_line runs under a mutex. A delivery is still only published
-- the install is one store per function on the game thread at a frame
boundary, and doing it on the spot would be a frame running half in the old
code and half in the new. And the OCaml runtime system is released across the
call, because a delivery is a dlopen.
Measured, and the number is the point: the transport was ~50us of a 23ms
redefinition, so end to end did not move. Code generation is 19 of the 22
milliseconds. The merge's prize was never latency.
The test is a deletion, because a reply cannot say which way it came:
test_dev.ml unlinks the agent's socket file once the merged program has bound
it, and every evaluation after that still installs.
The refusal list was rewritten and the prologues that pointed at it were not,
so prelude.ml claimed in three places that what it now contains is impossible:
the splitting header said `split` is refused at the foot of the file, forty
lines above `split`; the ASCII-case header said Odin's allocating to_lower is
not available here, next to the one that was written; and the UTF-8 header
said the rest of core/strings is refused rather than ported.
Each keeps its point rather than losing it. The iterator is still the shape
that owns nothing and still the right call when there is no result to own;
lower-ascii and bytes-ci=? are still the right calls when a copy is not
wanted, since folding a comparison over two inputs beats lowering both. What
changed is the reason, which used to be the absence of an allocator and is now
a choice between two shapes that both exist.
And strings.flan told the reader the opposite of what it did -- "not freed",
on the line above the free. vec.flan already had the right framing: the free
is written, it keeps the block because an arena cannot release one, and that
is the difference the capability set exists to state.
The list at the foot of prelude.ml was one sentence -- every entry needed to
produce bytes that did not exist in its input, and there was no allocator --
and that sentence has been false since Vec landed. Seven entries move up into
the code, and string-from-bytes turns out to have been the `string` builtin
all along: (string (as-slice v)) is the round trip, free precisely because
the layouts are identical.
What is left is refused for four different reasons and is written that way
now: pad and center for nothing at all except that no caller has asked;
format and sprintf for variadics of mixed type; map, filter, reduce and
sort-by for function values; map-keys and map-values for a map iterator that
does not exist in the runtime.
NEXT.md's queued section is struck and carries the four findings, each with
the change it wants named -- flan_map_next plus one builtin for the iterator;
milestone 5's function values for the higher-order three; vec_new_elem taking
a type expression rather than a bare name, which is what forces slices-new to
exist; and an array literal with no way to say it is [f32], which is what
forces every float in algorithms.flan to be cast. BUILT.md gets the section.
A dotimes whose every iteration continues still counts to its trip count. The
existing case fails by hanging if the latch is wrong; this one fails by
counting wrong, which is the off-by-one the four-block layout could have.
A restart-case clause was made a barrier on reasoning alone and nothing
observed it. Now something does.
And say what a labelled continue means, which is the half that is not obvious:
it advances the named loop's counter and skips the rest of its body, not just
the rest of the innermost one.
return is refused inside handler-bind and restart-case blanketly, and rightly:
a return always crosses the frames they pushed. A break does not. A loop
written wholly inside a restart-case body has a perfectly good local break, so
the rule is a barrier on the loop stack rather than a flag — a jump is refused
exactly when a barrier stands between it and the loop it names, and the message
says which construct. handler-bind and restart-case bodies are barriers, so is
a restart clause, so are a defer's forms; a handler clause is lifted into its
own function and needs no rule at all. in_frames is untouched: a return is the
special case where the target is always outside every barrier.
continue wanted the other blocker. check_dotimes folded its step onto the end
of the body, which a continue would jump past, so the counter would never
advance and the loop would hang. Tast.While carries a latch now — condition,
body, latch — the step goes there, and emit_while emits four blocks. A while's
latch is empty and folds away.
Labels are Odin's, in the head position: (while :outer c ...) and (break
:outer). A keyword there is unambiguous because a loop condition is never one,
so one label function serves while, until, dotimes, break and continue. It is
not a goto — the checker resolves a label against the loops the form is
lexically inside, so control can only leave a loop it is already in.
Break and Continue carry a relative depth rather than a name, because that is
what a backend already has: emit keeps one entry per While the way it keeps
one pad per frame, and indexes it.
Nothing in the prelude wants either. Every early exit there is a return from
the function, which break cannot replace; the sentinel-flag loop break exists
to remove does not appear in it. The two the compiler emits are that shape and
are the one place it cannot help — their sentinel is set inside a restart-case.
reach.ml and render.ml take the While arity change and nothing else.
