From a mutation-testing pass: about sixty small, plausible changes to the
compiler and runtime, each applied, run and restored. Nineteen of them left the
whole suite green. The compiler was right in every case - what was missing was
anything that looked.
The two programs here close the severe cluster. cleanup.flan covers six claims:
an early return runs the defers registered above it, and runs them innermost
first; a defer that calls something, which is what puts a guard inside a defer
on the transfer path; a transfer out of a handler-bind pops its frames; a
two-clause handler-bind pops both; and a signal stops once a handler has
answered it by transferring. The numbers differ per failure, so a wrong answer
names its own cause rather than just being wrong.
signedness.flan covers the ashr/lshr and slt/ult choices. Either could have been
hardcoded to one arm and nothing would have noticed, because no program in the
corpus shifted a negative integer right or compared an unsigned value above
2^31 - where a signed compare answers the other way on every operator.
Each was verified able to fail, with the numbers the report predicted: hardcode
lshr and -4 becomes 9223372036854775804; drop the defers from the return path
and 21 becomes 0; reverse them and it becomes 12; let the signal walk continue
past a handler that transferred and the outer handler runs too.
The ones left open are recorded for the next pass: Reach's walk of index
expressions, addr places and restart clause bodies; the dev registry's
size-change guard; a local shadowing an imported name; and the 4K result cap,
which has no coverage at all rather than a missing assertion.
Found by a read-only audit of emit.ml's failwith sites, each of which is a claim
that the checker guarantees something. Three of those claims were false, and
every one failed in the shape NEXT.md calls the worst available: type checks,
then dies with no source location.
An enum comparison is lowered now rather than refused. Types.is_comparable
already admits an enum, so the checker was stating an intent the backend never
honoured - (= k :a) is the first thing anyone writes with an enum, and it raised
Failure("comparison on K"). An enum is an i32 at run time, so all six
operators are an icmp. Signed, because (defenum K [a -1]) is accepted and an
unsigned compare would call -1 the largest member.
A union in a type position is refused instead. Constructing a union value and
reading a field of one were already refused, so nothing could ever be done with
such a value - only the declaration got through, and it reached clang as a
reference to an undefined %"U", which is a link error naming an emitted symbol
with the source location long gone.
A function type annotation is refused too. The function *value* was refused
where it is written; the annotation was refused nowhere, so (defn f [g (Fn []
i32)]) died with "no layout for". It now sits beside the Map line directly
above it, which is the same shape of not-yet.
The audit also found the sentence that covered the last two: NEXT.md and
check.ml's header both claim unions and function values are rejected by name.
That is true of values and false of types, which is exactly the gap the two
findings lived in.
The simulation was in a package of its own for one reason: importing raylib
linked libraylib on every target, so the headless run could not name the
package the interactive one needs. That reason is gone, and the split was
never anything else — the physics is the same code either way.
So sim.flan is back inside sand.flan, and test/programs/sand-headless.flan
imports sand.flan itself: window, raylib bindings, dev agent and all. It builds
for wasm32 anyway. Nothing it calls reaches raylib, so no shim is compiled, no
-lraylib is passed, and the front-end's functions are never emitted; sand.flan's
main is not exported, so the only main is the headless one. The hash is
unchanged on both targets at both optimisation levels, which is the point —
a refactor that moved the number would have moved the simulation.
The new cases cover what made it possible rather than only the result: a
package nothing calls into, native and wasm32; raylib reached both directly and
through sand.flan and read once; and the three refusals — sand/main, one
directory under two aliases, and two mains.
test_session's package-qualification case moves to vendor/agent, which is now
the package in the tree with a defn in it.
vendor/edn rather than the prelude: the prelude is prepended to every program
and everything in it is emitted, so a reader nobody imports would be a cost
every build pays.
The tokenizer only. A type-directed reader - the compiler emitting a parser
from a walk over a struct's fields, the dual of the printer C-x C-e already has
- lands in check.ml and emit.ml and is not this. What a caller writes today is
a struct reader by hand against the cursor, and the acceptance program carries
one, because that is what proves the API is usable rather than present.
Every token is a slice into the source, so nothing allocates and the buffer has
to outlive the tokens. That contract is stated at the top of the package,
because it is the kind of thing found the hard way.
Escaped strings are refused rather than half-supported: unescaping needs a copy
and there is nowhere to put one, and handing back the raw bytes would return a
three-byte string as four with a backslash in it. Each other refusal carries its
own sentence - #inst and #uuid separately from tagged literals, because a file
is most likely to contain those two and being told tagged literals are refused
would not say that the timestamp is the thing to delete.
Errors live on the cursor, a code and a byte offset, not in the return type: an
Option loses the position, which is the whole point for an editor. A failed
cursor is poisoned so a caller's loop terminates on a malformed file rather than
spinning.
# Conflicts:
# test/test_acceptance.ml
err-too-deep was the one error code nothing observed. The message is the least
of it: the plausible wrong version is `>` where the guard wants `>=`, which
writes one element past a [32 i32] and traps at exit 134 rather than answering
anything. 33 opening brackets is the input that separates them, and it is the
whole justification for a fixed array instead of a growable stack — the place
this lane pushes hardest against having no allocator.
`.5` reads as a float here and does not in EDN, where a number must start with
a digit and `.` is a legal symbol-start byte. That makes it a reinterpretation
of a token that is already legal as something else, which is exactly what the
house rule says to name rather than leave to be discovered, so it is written
beside the refusals.
Also: every symbol in the table was lowercase, so the A-Z half of alpha? was
unexercised and a version missing it passed. Enemy/Goblin in an existing dump
rather than a new case. And a line under "Internal helpers" saying the heading
is intent and not enforcement — a package has no visibility, so edn/scan-atom
is as callable as edn/next, the same way rl/get-color-raw is.
Both new cases verified by mutation: the depth guard traps, and alpha? without
its uppercase range fails Enemy/Goblin.
A package handed over its .c files and its `link` arguments the moment it was
imported, whatever the importing program did with it. That is what made sand's
two halves two files: anything naming vendor:raylib linked libraylib on every
target, and on wasm32 that link cannot succeed, so the headless run could not
so much as mention the package the interactive one needs.
Reach.link answers it from the checked program instead. Start at main and at
the globals that run before it, follow every call — including the Handled
frames, where a lifted handler clause is reached by address and by nothing
else — and keep what is reached. A package none of whose externs survive
contributes no C and no linker argument.
Dropping the flags alone would only move the failure: the bodies that called
into raylib would still be emitted, and wasm-ld would fail on the symbols
rather than on the argument. So the same walk prunes the functions and externs
too. Only those — globals, structs and unions stay, because an unreferenced
global is bytes in BSS and a dropped one is a silently different program.
Dev builds keep everything. What a REPL may redefine next is not a function of
what has been called so far.
read-enemy in test/programs/edn.flan is the worked example the API is for: the
map opened, the keys looped over, each known one dispatched onto its field and
the rest skipped, written by hand because the compiler cannot emit it yet. It
is there rather than in a doc comment because an API only a compiler could
call would be present without being usable, and writing one out is the only
way to find out which it is. Two things came back from writing it — that
float-of has to accept an integer token, since a config file writing `:speed 2`
for an f32 field is not making a mistake, and that a caller needs `fail` on the
cursor, because a reader's own "expected an integer here" has nowhere else to
get a position from.
The expected output is a raw literal. The dump is brackets and quotes end to
end, and escaping it into an ordinary OCaml string would put a second reader
between the test and what the program printed.
Every case was checked by breaking the tokenizer and watching it go red;
sixteen of them, each restored afterwards. The ones worth naming, because they
are the ones that could have been quietly unobservable: dropping the escape
refusal, accepting `#{`, and collapsing every refusal onto one message — that
last is the shape where a table asserting only "it failed" stays green while
observing nothing. Also: a semicolon no longer ending an atom, a comment scan
that does not test for end of input (which traps rather than differing, on the
comment with no trailing newline), the ratio rule widened to any atom
containing a slash (which takes foo/bar with it), text slices left including
the quote and the colon, a closer counted but not matched, any byte accepted as
a symbol start, a comma not counted as whitespace, and skip-value consuming one
token instead of a whole collection.
Image first and deliberately: it is CPU-side, so it is the only large piece of
raylib that can be asserted headlessly rather than looked at. gen-image-color,
the pixel reads, both flips, a PNG round trip through export and load, and the
resize and crop dimensions and contents are all in the table at -O2 and -O0.
The shapes, text and timing calls are observed only, by running sand under Xvfb
and looking, and the program and NEXT.md both say which is which.
Five permutations were run red and restored: Image's width against height and
mipmaps against format, GetImageColor's two indices, the two flip wrappers
bound to each other, and the crop rectangle's width against height. The third
of those also broke the export and load lines, which is what makes the PNG
round trip verified rather than merely plausible.
Two corrections to the brief it was given. MeasureText is not headless material
- it measures with the default font, which only InitWindow loads, and a C probe
returns 0 - and the same is true of the frame-time and screen-size calls. And
the raylib.h on this machine is 5.1-dev while the linked library is 5.5, so
every signature was checked against nm -D instead: IsImageValid rather than
IsImageReady, and DrawRectangleRoundedLines takes no thickness.
Font loading is refused by name. A Font carries a Texture2D, a Rectangle* and a
GlyphInfo*, and a GlyphInfo carries an Image - two more aggregates and two owned
arrays, for something with no headless test.
f32 only, and each refusal by name: clamp and abs stay compositions of the
min/max builtins, split-at wants a pair type there is no way to spell, and the
f64 and other-element-type copies wait for a program that wants them.
-lm goes on every link, after the objects. The default --as-needed drops a
library named before the object that wants it, and at -O2 LLVM folds most sqrtf
calls into the hardware instruction so nothing has to resolve - which makes the
flag look unnecessary until the -O0 build emits the call and fails to link. That
is how it was found, on the -O0 acceptance run.
sqrt is libm's rather than Newton's, because there is no bit cast between f32
and u32 to seed a guess from, and IEEE-754 makes sqrt correctly rounded so
libm is bit-identical across targets anyway. llvm.sqrt.f32 as a builtin would be
better still - one instruction, no symbol, no link flag - and belongs to
whoever next touches check.ml.
The finding worth keeping is a test that came back green when it should have
been red: nothing in the table could observe floor's zero guard, because
(ceil-f32 0.0) is +0.0 either way. (floor-f32 -0.0) is the only case where it
shows, and the prelude comment had claimed the wrong justification for it.
floor, ceil and round over f32, which is what a position and a tile coordinate
are here. The only rounding mode available is the cast's truncation toward
zero, so each of these is that cast plus the correction the mode does not
make, and the content is which inputs make the cast itself undefined. NaN
fails every comparison, so it needs its own (not (= x x)) and nothing else
finds it; the infinities fall out of the magnitude test; and above 2^23 an f32
has no fractional bits left, which makes returning the input there the exact
answer and also the guard that keeps the cast inside i32.
round is half away from zero, written as floor of the magnitude and mirrored.
The obvious (floor-f32 (+ x 0.5)) is wrong twice: half-up rather than
half-away, so -2.5 comes out -2, and at the largest f32 below 0.5 the addition
alone rounds to 1.0 and answers 1 for a number under a half. Both are in the
table, which is why every case there is a negative or a half.
sqrt is the decision in this commit and it goes out to libm, which is a change
to the release link and so is said out loud. Every other number in the prelude
is reachable from the four operations and a cast; a square root is not.
Newton's method needs a starting guess, the good guess comes from
reinterpreting the exponent bits, and the language has only value-preserving
casts - no bit-cast between f32 and u32. Without one the iteration needs a
scaling loop to normalise and still produces a result that is merely close,
which is the one thing a standard library must not hand back. IEEE-754 makes
sqrt correctly rounded, so libm's answer is the same bit pattern on native and
on wasm32; for this function the byte-identical argument points at C rather
than away from it.
The cost is -lm on every link, and its placement matters. It goes after the
objects, not in the leading flags, because --as-needed drops a library named
before the object that wants it. Worse, at -O2 LLVM folds most sqrtf calls
into the hardware instruction and the symbol never has to resolve - so this
looked linked before the flag existed and failed only at -O0, which is exactly
why the table runs both. Untested against --target=wasm32: wasi-libc ships
libm.a as a stub because the symbols live in libc, so it should be inert
there, but nothing here exercises it.
The better fix is not in this lane. llvm.sqrt.f32 as a builtin in check.ml and
emit.ml is one instruction, no symbol and no flag, and it belongs to whoever
owns the compiler.
Finishing the text family the previous lane started. All three are over [u8]
and none of them allocates, which is what decides their shapes.
trim answers a slice of its input. That is the only shape available without an
allocator, and it is also the better one: there is no new storage, only a
narrower view of the caller's, so the result dies with its owner and trimming
modifies nothing. Both loops test (< lo hi), because an all-whitespace input
otherwise walks lo past hi and (slice s lo hi) traps on a reversed range - the
same trap the bounds table already asserts on. That input is in the case list.
index-of-bytes is naive and stays naive. Boyer-Moore wants a skip table sized
by the needle, which is an array, which is an allocation. The empty needle
answers Some 0 so that index-of-bytes and starts-with? agree on every needle,
and the length test returns before the loop so a needle longer than the
haystack cannot build a window off the end.
parse-f64 splits the work where the two halves actually differ: the grammar is
Flan's and the rounding is libc's. parse-i64 is entirely Flan because strtoll's
answers are wrong for a caller - 0 for "", 0 for "abc", 12 for "12x" - and not
because decimal-to-binary conversion is suspect. Reimplementing correctly
rounded conversion is a different and much larger problem than rejecting junk,
and IEEE-754 already guarantees strtod gives the same bits everywhere. So this
validates the whole slice and only a slice that is entirely a number reaches
bytes->f64. Every refusal in the table - "", "abc", "1x", ".", "1e", " 1",
"1 ", "0x10", "nan" - is a plausible number out of strtod.
Two caveats, both written into the source rather than discovered later. The
locale worry that keeps parse-i64 in Flan does apply to strtod's decimal point,
and is moot only because nothing in the runtime calls setlocale; if that stops
being true this is what breaks. And the length is capped at 511 because
flan_bytes_to_f64 truncates there - a validator that approved 600 digits would
be approving a different number than the one strtod reads.
digit? and space? exist because parse-f64 and trim need them, and calc-me loses
its own byte-identical digit?. One top-level namespace makes the second
definition an error rather than a shadow, which is the rule doing its job: two
copies that later drift apart is exactly what it prevents.
The break loop was reachable from a raw socket. This is the half that makes
it reachable from an editor, and it all follows from one fact: a program
stops at a moment nobody asked about.
So the state is learned twice, on purpose. It rides on every reply, beside
the program's output and for the same reason -- the likeliest instant for a
program to stop is the one just after an evaluation, which is a reply the
client is already reading, and learning it a second later from a poll would
mean learning it after the echo area had said the evaluation was fine. And a
timer asks anyway, once a second with `describe', because a program that
stops in a frame of its own game loop produces no reply at all and folding
state into replies that never come says nothing. The timer never reconnects
-- that would quietly erase the `lost' state that exists to be seen -- and
skips while a request is in flight, since accept-process-output runs timers
and a poll firing inside a read would eat that read's reply.
Three ops: `break' for the restart names, `restart' and `abort'. The
annotation owns :stopped and :condition rather than the ops, so one place in
the daemon decides whether the program is stopped and the poll and the prompt
cannot disagree. "ok" from `restart' means accepted, not resumed: the choice
is validated against the stopped stack and taken when that thread next comes
round, so it says so and the client clears its own flag rather than polling
once, finding it stopped, and re-opening the prompt it just answered.
The agent grew one verb, `status', answered in both states. Everything else
the break loop offers is refused while running, rightly; but the question an
editor asks without already knowing had to have an answer either way or there
would be nothing to poll.
And flan_agent_poll had to become re-entrant, which was a bug rather than an
addition. A C-x C-e thunk may itself error, and the break loop that catches
it polls again from inside that call. The old loop cached both indices and
stored tail at the end, rewinding over everything the nested poll consumed --
re-running the thunk that had just stopped the program, which is an unbounded
recursion of breaks. Each job is now claimed before it is run. test_dev.ml
evaluates an expression that errors and resumes it, which fails against the
old shape.
Every other struct in the package is handed to raylib and handed back, and
that proves nothing: store-and-return is symmetric, so C writes and reads the
same wrong slots for any field order. An Image is different. raylib computes
with it, and two computations answer differently per axis.
gen-image-color takes two scalars and returns a struct reading 4, 2, 1, 7 —
four distinct values in four adjacent i32 slots, with no input struct for a
permutation to cancel against. Texture2D never got that: nothing without a GPU
reads its width, height or mipmaps at all.
And get-image-color indexes y*width + x, so on a 4-wide, 2-tall image (3,0)
exists and its transpose does not. That is the axis discriminator the
collision family could not be — exchange x and y in the wrapper and the read
goes out of bounds. The two flips say it twice more: on two rows, one moves a
mark the other leaves alone.
The PNG round trip is not the symmetric trap either. stb's encoder and decoder
are external ground truth; they agree with each other, not with whatever field
order Flan believes in.
Verified to fail, each restored after: width against height, mipmaps against
format, x against y in the shim, the two flips bound to each other, and the
crop rectangle's width against its height.
Six of them were bound, linked, and had never been called by anything. That is
the state a wrong argument order survives indefinitely: the link succeeds, the
program runs, and the answer is nonsense that nobody has looked at. An audit for
wrappers with no caller is worth doing after any binding lane.
All six turned out to be correct, which is worth recording either way - the
point of the audit is not that it finds bugs but that it converts "probably
fine" into "called, and the answer checked".
Each has a case that must come out the other way, because a predicate that
always said yes would pass a single one.
The one that earns the most is the polygon, the only binding here that crosses a
slice, so the only place ptr+len has to arrive as raylib's pointer and count.
Everything else about it would pass with a hardcoded count or with the pointer
alone; the same point against the same array with three corners instead of four
is what pins the length. Verified by hardcoding the count in the shim and
watching it go the wrong way.
Finishing the 2D lane's unfinished work: the collision family was written and
had no tests when the session ended. It is the best material a headless table
gets, since every one of these is pure and needs no GL context.
Two plausible tests in a row turned out to check nothing, and that is the part
worth keeping. A struct round trip is symmetric and passes for any field order -
the texture lane found that one. The second is subtler: no axis-aligned geometry
can pin Vector2's fields, because exchanging x and y is a reflection that is
applied on the way in and undone on the way out. Swapping the shim's own typedef
leaves every collision case passing. Distances never even see it.
What does pin Vector2 is the rotated camera, because a rotation is not
axis-aligned and does not commute with the reflection. That case is load-bearing
and the comment now says so, because the collision cases look like they cover
the same ground and do not.
What the new cases do pin is Rectangle, completely: swapping width and height
turns three of the four predicates the wrong way. Verified by doing it.
collision-lines answers (Option Vector2) rather than a bool and an
out-parameter, because raylib leaves the out-parameter untouched when the
segments do not meet and a caller who forgets reads whatever was there.
GetScreenToWorld2D computes from every field of a Camera2D - offset, target,
rotation and zoom - so a wrong field order produces a wrong coordinate rather
than the same numbers back. That is the standard the texture lane arrived at
the hard way: a struct round trip is symmetric and passes for any layout.
The standard library's first real content beyond printers. Everything is
in-place over a slice, because there is no allocator and nowhere to put a copy
- and a slice aliases its owner's storage, so sorting one sorts the original.
No generics means no single sort. Each is spelled per concrete type, which is a
cost paid deliberately rather than worked around.
parse-i64 is written in Flan rather than bound to strtoll, which answers 0 for
an empty string, for a string of letters, and for a genuine zero, and reports
overflow through errno.
bytes->i64 is strtoll behind a primitive, and strtoll returns 0 for "", for
"abc", for a lone "-", and for the "12" in "12x". None of those is
distinguishable from a real 0, so any program that parses input it did not
write is already wrong and has no way to find out. parse-i64 takes the whole
slice or refuses it and says so with None. It is also the version that answers
the same on wasm32: strtoll is libc's and locale-sensitive, which is the same
argument that put the PRNG in the prelude rather than leaving it to rand().
The byte predicates are over [u8] rather than over string on purpose. (bytes s)
is one call at the call site, and in exchange one copy of each function serves
strings and byte slices both — which is as near a generic as this gets. Each
tests its length before it slices, and `and` short-circuits, so a prefix longer
than the subject answers false instead of tripping the slice bounds check.
sign-f32 and lerp are the only two numeric helpers here, because they are the
only two that decide something. clamp is (min hi (max lo x)) and abs is
(max x (- 0 x)) over builtins that already exist — a prelude wrapper is a
function emitted into every program to save a caller nothing. sign-f32 answers
0.0 for NaN, which is a choice and is written down. lerp is the weighted sum
and not a + t*(b - a): the latter does not land on b exactly at t = 1.0, and a
position that never quite arrives is what interpolation gets bug reports for.
floor, ceil and round are deliberately absent. (f32 (i32 x)) is fptosi, which
is poison out of range, and shipping that as a documented limitation is the
same class of bug NEXT.md already records twice under Sharp edges. Correct
lowering is llvm.floor.f32 in emit.ml, which is not this lane. sqrt is absent
for a different reason: it is an extern to libm, and what libm means on wasm32
is a decision the FFI owns, not the prelude.
rand-i32-range answers lo for an empty or reversed range rather than dividing
by zero, which is immediate undefined behaviour and not merely a wrong number.
Both range functions draw exactly one rand-u32 and neither changes it, so the
sand hash still pins the generator; the new test pins the derivations off a
fixed seed, which nothing else would have caught.
GetScreenToWorld2D and GetWorldToScreen2D are pure arithmetic over every
field of a Camera2D, so they run with no window at all — the best headless
material the package has had. Both directions are asserted as absolute
answers rather than as a round trip, because an inverse cancels a permuted
layout exactly the way store-and-return does.
The rotated case earns its awkwardness: exchanging x and y in Vector2
mirrors every component-wise formula and the answer comes back mirrored
too, so nothing until now could tell the two floats apart. A rotation mixes
them. It reports ok/bad against a tolerance because 90 degrees goes through
sinf and the answer is 27.9999981, and the table compares stdout byte for
byte at -O0 and -O2.
A sequence library normally returns new sequences. There is no allocator, so
every one of these mutates the storage it was handed and a slice is the handle
that makes that useful: (slice grid 4 9) is ptr+len into grid, so sorting it
sorts those five elements and leaves the rest of grid alone. The test asserts
exactly that — it sorts a subslice and prints the whole owning array — because
it is the property that would die silently the day a slice parameter started
being copied rather than passed by value, and -O2's mem2reg would hide it.
Insertion sort rather than anything faster. Quicksort wants a stack and
mergesort wants a buffer, and neither exists; insertion sort needs a swap and
two indices. It is also the only one of the three whose inner loop is short
enough to read, which matters more than the asymptotics on the slice sizes a
frame loop actually sorts. The `and` guarding it short-circuits, and that is
load-bearing: at j = 0 the left test fails and (at s -1) is never evaluated,
so the bounds check never fires.
Over [i32] and nothing else. There are no generics, so a second element type
is a second copy of all seven functions emitted into every program that links
the prelude, and i32 is the type indices, ids and tile values already have.
An f32 set waits for a program that wants one.
min-i32 and max-i32 return (Option i32) rather than a sentinel because there
is no i32 that means "the slice was empty" and is not also a possible element.
sum-i32 accumulates in i64 and widens each element explicitly — there is no
implicit widening anywhere, and an i32 total over a screenful of i32 is how a
sum wraps without anyone noticing.
spec-conditions.md §2, and the reason the transfer was worth building. An
unhandled error runs a hook instead of rt_die(), on the frame that erred with
nothing unwound, lists the restarts between there and the top, and waits.
A hook rather than a direct call because the loop lives in vendor/agent, which
is an optional package, and flan_rt.c is the release runtime - a program with no
agent leaves it null and dies the way it always did. The hook resumes by writing
a restart into the transfer channel, which is the channel an invoke-restart
writes and reaches the same guard, so choosing from the break loop and choosing
from a handler are one act lowered once. §6 needed no change.
The break loop is the poll loop, run from the error rather than from the frame
boundary. That is load-bearing: an expression evaluated while stopped is a
module the listener queues and the game thread runs, so a loop that did not
drain that queue would hang C-x C-e exactly when it is wanted most. Installing
while stopped is allowed, which contradicts the rule that a redefined function
must not be swapped while it is on the stack - that rule is about mid-frame
consistency and there is no frame in progress here. The old body keeps running
and a retry reaches the new one through the cell, which is the whole point.
A restart frame carries its name now, beside the hash. Matching never needs it;
showing someone their choices does, and nothing at run time can turn a hash back
into a name.
A choice is checked on the listener thread against a stack the stopped game
thread is holding still. Answering ok and finding out on the game thread that
nothing offers that name would report success for something that cannot happen.
The test errors twice and takes a different restart each time, so a loop that
always resumed the same way fails it.
Texture2D and Rectangle are the two structs the texture calls need, and they
are the ones whose layout can be silently wrong: five 4-byte fields in a row,
and four floats in a row, so a permutation still reads as plausible numbers
everywhere.
The obvious test — hand raylib a struct, read it back, compare — is worthless
here, and I only found that out by trying it. Storing and returning is
symmetric: swap two fields in the Flan defstruct and the round trip still
agrees with itself, because C writes and reads the same wrong slots. That test
passes whatever the layout is, which is the kind of test this project would
rather not have at all.
So the headless case uses the two things raylib computes from the fields
without a GPU. GetCollisionRec turns (0,0,10,4) and (6,1,10,10) into
(6,1,4,3), four different numbers each derived from a different pair of
fields, and no permutation of Rectangle survives it. SetShapesTexture keeps a
Texture2D without touching GL and substitutes 1 1 1 1 7 when the id is zero,
so a zero id pins the first field, the 7 pins the last, and a zero width
stored rather than substituted is what stops that pair from passing with id
and width swapped. Each of those was checked by permuting the defstruct and
watching the case fail.
What is left unpinned is width, height and mipmaps against each other; nothing
raylib does without a GL context reads them. That is stated in the program
rather than papered over, because the alternative is a case that looks like it
covers them.
set-shapes-texture, get-shapes-texture, get-shapes-texture-rectangle and
get-collision-rec are real bindings, not test scaffolding — they are bound
here because they are also the only pure consumers of these two structs.
spec-conditions.md §2. The same lookup as signal, and the difference is
entirely what happens when the walk ends: signal returns Unit and the
signalling function carries on, error has type Never and the program stops.
Only a transfer gets past it, so emit puts a guard after the call and then
unreachable - and flan_error cannot be marked noreturn for the same reason, it
does return, on exactly one path.
Being Never is what lets it stand where a value was expected, which is the
fall-through shape §1's load-texture example needs and the reason it is worth
having before the break loop rather than after. An unhandled one names the
condition on stderr and dies the way every other trap does; flan_error is where
the dev-build break loop will go.
The two spellings share one AST and IR node with a kind beside them, the same
shape Ast.unwrap already uses for some and try, because they differ in one
decision and nothing else. test/programs/error.flan is the unhandled case,
asserted on the exit code and the reason rather than through the outputs table,
which only has room for a program that exits 0.
The reload path had never seen a restart-case or a handler-bind: the
acceptance table's dev build proves whole-program codegen with cells, but not
Emit.redefinition, where the callees are declares or cell loads and the restart
frame is an alloca in a module the process was not built with. Driving it found
a hole step 1 left - a lifted clause was numbered by its position in the whole
program's lifted list, so the name was neither stable against an unrelated
handler-bind being added nor attributable to the function it came out of, and
redefining a function that established a handler died in llc with an undefined
value.
A clause is now named after its parent - handler/step/0/Missing - and carries
Tast.fn.fparent, which is what lets a redefinition module emit the clauses
belonging to the bodies it is replacing and nothing else. They are hidden for
the same reason a redefined body is: taking the address of an interposable
symbol would resolve to the host's copy, so the module would install the very
handler it was replacing. A clause is reached by address from its parent and
from nowhere else, so it is kept out of the cell and registry machinery
entirely rather than given a slot nobody uses.
test_dev.ml now sends a third evaluation: step redefined to a restart-case
whose frame is an alloca in the new module, whose guarded call goes through the
host's cell, and whose transfer starts in a handler and crosses probe, which
the host was compiled with. The transcript's fourth line is the clause's value.
spec-conditions.md §3 to §6. A handler runs where the signal was, decides, and
control resumes at a restart-case further out - so unlike step 1 this one does
alter control flow, and it is lowered explicitly rather than through platform
unwinding, because wasm32 cannot unwind and because a cmp/jne after a call
reads like ordinary code.
The channel is the out-parameter §6 settled on: one ptr appended to every Flan
signature, written by an invoke-restart and checked after every call. The
return type stays what the source says, one pointer threads down the whole
chain, and a frame that sees the channel set just returns early - which reuses
the existing return path and with it §5's defers for free. Emit.signature was
already the one place a signature is spelled, which is what made that part
small.
Every function is transfer-transparent, release included. §6's escape analysis
is an optimisation; in a dev build a cell can hold anything, so the honest
answer to what a call can reach is anything, and uniform means redefinition
acquires no new refusal class.
The transfer target is the restart frame's own address and not a static clause
id, which corrects what the handoff note had settled. An id has to be unique
against every module a running program may later load, and a hash is only
probably unique - two restart-cases colliding means the inner one silently
catches a transfer aimed at the outer. The frame is an alloca in the function
that offers it, so the address is exact and it also says which clause, which is
how clause ids disappeared. Re-entering a restart-case then needs nothing
extra, since each activation allocates its own frames.
Cleanup is landing blocks, one per region rather than one per function: a
restart-case's pops its frames and either dispatches or forwards, a
handler-bind's pops the handler frames on the way past, and the function's own
runs its defers and returns. One function-wide block would have jumped straight
past the very restart-case that was meant to catch the transfer. The channel is
cleared before any cleanup runs and put back after, or a defer's first call
would branch straight back into the block it came from.
flan_signal takes the channel and passes it to each handler, stopping once one
writes to it. That makes the one C frame every handler is reached through
transparent to a transfer, which it has to be; it is also the only one, since
extern is Flan-to-C only and there are no function values yet.
Refused by name with the reason, each with a test on the reason: restarts with
parameters, return inside a restart-case body, one restart-case offering a name
twice, and invoke-restart inside a defer - a defer is the cleanup a transfer
already runs, so starting one there leaves the defers half run with two targets
and no way to choose. The lexical case is the checker's and the one that
reaches a function through a call is trapped at run time. No restart of that
name is a located runtime error at the invoke site, because there is nowhere to
resume.
Two things found on the way. `{ ctx with in_handler = true }` was a latent bug:
ctx.slots is mutable, so a copy allocated the body's slots into a record the
function never saw again - harmless only because no handler-bind body in the
tests had a let in it. And test/reload_host.c calls flan.outer through an asm
label, which does not fail at link time when the prototype is a parameter
short; it reads garbage as the channel and dies somewhere else.
test/programs/restarts.flan runs at -O2, at -O0 and as a dev build. -O0 is not
redundant: the guard after every call is control flow the optimiser would
otherwise launder, and the dev build is where each of those calls goes through
a cell.
spec-conditions.md §1 and §2 and nothing else, because those two are worth
having alone: signal returns Unit whatever it finds, a handler that returns
normally leaves the signalling function to carry on, and with nothing matching
it is a no-op. So none of §6's transfer machinery exists yet and no signature
changed - which is the whole reason to do this step first.
The runtime is a linked list. Establishing a handler is two stores and a push
onto a frame on the establishing function's own stack, and signal with an empty
stack is a null check, which is what §2 asks for. Popping is by frame rather
than by count, so restoring what this one displaced is right even if something
below it left the stack out of step.
A condition's type is a hash of its name and not an index: an index would shift
the moment a struct were added, and every handler a running program had already
pushed would match the wrong type. The condition crosses as a pointer, since a
handler runs while the signalling frame is alive and there is nothing to copy -
but what the clause binds is the condition itself, the pointer being a hidden
parameter and the name a slot loaded from it, so a handler passing c to
something expecting the struct is not handed an address.
A clause is lifted into a function of its own, because a handler runs from
wherever the signal was and cannot be a branch in the function that wrote it.
That gives two refusals, both by the house rule. A handler cannot see the
establishing function's locals - that is a closure with an explicit
environment, so a reference to one is refused for that reason rather than
reported as an unknown name. And return inside a handler-bind body is refused,
since the frames are popped on the way out and an early exit would leave them
pointing into a function that has gone.
Settled in advance for the next step: in a dev build every function is
transfer-transparent, because a cell can hold anything and the honest answer to
what it can call is anything. Same bargain as the indirect call, and it means
redefinition acquires no new refusal class. Still open is whether the
discriminated result is returned by value or through an out-parameter.
C-x C-e rendered the scalars and refused the rest, which made it a calculator
rather than a REPL. The renderer is now a compile-time walk over the type,
emitting a piece at a time: structs, nested structs, fixed arrays, slices,
options, enums by name, and pointers as their shape. A raylib Color comes back
through the FFI as (rl/Color {:r 17 :g 34 :b 51 :a 68}).
Piecewise emission is what makes composites possible at all - a struct is its
fields with punctuation between them, and concatenating that in generated IR
would need an allocator the language does not have.
u64 now renders, in C, with %llu. It used to refuse because i64->bytes is
signed and it would otherwise come back as -1, but refusing a whole struct
because one field is a u64 is much worse than adding a runtime entry point.
Strings are quoted and escaped in C for the same reason: unescaped content does
not round-trip and reads as a framing bug rather than as the value it is.
An enum renders as :name, recovered from the checker's table as a chain of
comparisons, since members are erased to i32 before the backend sees them; a
value outside the declared members falls through to its number, which is what
you would want to see. A pointer is rendered and never followed - it is the
only thing that could make the walk cycle, and dereferencing one a REPL was
handed is not a safe thing to do on someone's behalf.
Three bounds, easy to conflate. depth and span bound the walk, so sand's
[100 [100 u32]] grid does not unroll into ten thousand render sites. The output
is bounded once in the runtime, since a slice renders through a loop the
compiler cannot bound, and one place enforcing it means no renderer carries a
budget.
emit.ml's cast now treats an enum as the i32 it is. Nothing in the surface
language produces that - a keyword resolves against its enum and never widens -
but the renderer needs an enum's number when it falls outside the members.
Its stdout is a pipe into the daemon now, and whatever it printed since the
last reply rides along with the next one into *flan-output*. Arriving with a
reply rather than by a separate request is the point: the output an evaluation
itself caused is the output anyone wants to see.
Draining that pipe is a liveness requirement, not a nicety. A pipe nobody reads
fills at 64K and the next write blocks the program forever, so it is read from
the accept loop's select whether or not an editor is asking, and the buffer is
capped - a program printing every frame must not grow the daemon without limit,
and the newest text is the useful end.
test_dev read the program's transcript off the daemon's stdout, which is no
longer where it goes; it collects :output from replies instead, which is also
what the editor does. The emacs test moved to a fixture that keeps running,
since it now evaluates more times than the old one had reloads to give.
Refusing every defconst was right about the class and wrong about most of the
instances. A constant the checker consumed - (defconst rows (/ h c)), which
decides grid's type before anything else resolves - is in the shape of the
program and no store can reach it. A constant that is only ever read at run
time is just bytes in memory. sand's colors is the second kind, and tuning a
colour table live is exactly the thing you would want a dev loop for.
So a dev build emits every defconst as a mutable global rather than a constant.
LLVM can then no longer fold a read of it and a module can store into it, and a
changed one is published at the frame boundary the same way a new function body
is. Release builds emit constant and get all the folding back.
Tast.global.gfolded records which kind it is, because nothing downstream of the
checker can tell: env.consts holds exactly the constants the folding pass
consumed, and membership is the question "is this value in the program's
shape?". The session keys its refusal on that, with a message that says what
the constant is used for rather than just that it changed.
Verified against a running sand: sim/colors is accepted, sim/rows is refused
and says why.
A different primitive from redefining a name. There is no name to install a
body into, so the expression is wrapped in a function with nowhere to be called
from; the module exports flan_reload_call to say "run this once", and the agent
calls it after the install - on the game thread, at a frame boundary, so an
expression that reads the program's state sees a point the program agrees is
consistent.
Nothing is marshalled back because nothing could be. A Flan value carries no
header, so no code at run time can say what it is; the compiler knows the type
and renders it there, in the thunk. That is the layout decision's bill, and it
is why the printer set is the scalars rather than everything.
The rendering does not go through stdout. Stdout belongs to the program, it is
in the hot path for anything that prints, and a dev-only feature must not put a
branch in it - so flan_rt.c is untouched and the value goes to flan_dev_result,
read back over the agent's socket. Safe without a handshake because the
generation counter is bumped last: the daemon waits for it to move rather than
assuming the program has reached a frame boundary.
u64 refuses by name, because i64->bytes is signed and anything past 2^63 would
come back negative. Everything without a derived printer refuses the same way.
A number that is quietly wrong is the failure this whole thing exists to
prevent.
An evaluation is not a declaration: the thunk is built against the program and
never spliced into it, so describe does not fill up with an eval/N for every
expression ever typed.
The test that matters is the same expression twice. The fixture increments
ticks every frame, so two evaluations must disagree - a value computed in the
compiler, or read from a copy of the program's state, would not.
Asked whether a defconst could be redefined, probed it, and got ":status ok"
for a change that did nothing at all - the module was built, delivered,
installed, and the program went on using the old value. That is the
silent-wrongness class the house rule exists to prevent, so it is now four
refusals and a fix.
A defconst's value is folded into its call sites - into an array length at
worst, which is decided before any type resolves - so it lives in the running
program's code and not only in its storage. Refused. A defenum member is the
same thing: :space is erased to an i32 literal in the caller. Refused, and
compared over declarations rather than over Tast.program, which carries no
enums at all for exactly that reason.
A defvar's initial value is deliberately not refused. Its storage holds live
state the program moved past long ago, and refusing to change the initialiser
would be refusing "edit the code, keep the sand". Same Tast.global record as a
defconst, opposite answers, told apart by gconst.
The value comparison is structural and conservative - anything it does not
recognise counts as changed. Comparing emitted text would be wrong, since
Emit.const on a string allocates a name off a per-module counter and two
different strings in two throwaway modules both come out as @".str.0".
Third: a new global's declared initial value was being dropped. flan_dev_global
callocs, so (defvar n i64 42) added at run time was silently zero. It now takes
the initial value as a blob, copies it on the allocation and ignores it
afterwards - the second half being where "a reload must not reset the program's
state" lives. In the allocation path rather than a branch at the call site, so
it cannot be got wrong at one of them.
Fourth: a change with no body to publish and no storage to allocate now answers
"nothing to install" instead of shipping an empty module. That is what the
defconst probe actually did, and it cost the program a frame's worth of reload
it did not need.
The piece between an editor and everything else. One long-lived Session, the
program it belongs to launched and owned by the same process, and a socket that
takes forms and installs them. What it adds over flan reload is that the
session persists - a defvar added by one evaluation is part of what the next is
checked against - and that it owns the build, which is what makes its layout
rules describe the process actually running rather than a guess about it.
The protocol is s-expressions rather than bencode, and I changed my mind about
that. The case for nREPL was reusing a designed op set and not re-litigating
session identity, but with the client ours too there is no CIDER to be
compatible with, its eval is string-in/string-out with no slot for which form
from which file, and Emacs already has read and prin1. So: one sexp per
message, length framed because the payload contains newlines. No parsing code
on the editor side, and on this side the parser is the language's own reader,
where :op is already a keyword and Flan source is already a string literal. An
nREPL front end can sit on the same Session later; it should not gate the
editor.
Two silent failures the daemon refuses to have. The agent socket is chosen by
the daemon and forced through FLAN_AGENT_SOCKET before spawning, because a
program's source has to name some path and a daemon that guessed would compile,
build and deliver a module to nobody. And delivery is checked: agent/start
returning 0 means a socket was bound, not that anyone connected, so a failed
connect or a reply that is not ok becomes an error the editor sees.
It waits for the program to bind before accepting an evaluation, since one
arriving first fails for a reason that reads like a compiler bug, and it
accepts with a timeout so a program that has exited takes the daemon with it
instead of leaving an editor waiting on a socket nobody serves.
Everything so far installed one module. The daemon's job is N of them against
one long-lived session, and that is where a registry that hands out fresh
storage per module would show up. So the agent test now takes two: the first
introduces a global the process was never built with, the second only reads it.
1007 rather than 7 is the whole assertion.
Getting there needed stdout to be line buffered, set in flan_rt_init. The C
default when stdout is a file or a pipe is a 4K block, so a program running for
minutes with a REPL attached shows nothing until it exits, and a test driving
one cannot see its progress at all - which is how this was found. One write per
line instead of per 4K.
Also written down: flan reload builds a fresh session from source each time, so
if the program file was edited since the process launched, its idea of the
host's names and memory describes a binary that is not running. That is a limit
of the command, not of sessions. And Session.eval's origin defaults to <eval>,
so the daemon has to pass the editor's real buffer path or errors point at a
file that does not exist.
lib/session.ml holds the declarations a running process was built from plus
every change accepted since, which is what an editor needs and what a one-shot
compiler cannot have.
Transactionality came for free. Check.program builds a fresh environment from a
declaration list on every call, so a form that fails to check mutates nothing
and the accumulated list is simply not replaced - no scratch-environment
machinery, which is what I was about to build. Re-checking the whole program
each evaluation costs the frontend, under 10ms, less than the llc after it.
There is a test for the case that matters: a typo, then a good form, in the
same session.
Which names the process was built with comes from the checked program, not from
any accumulated AST, because Check.program prepends the prelude and no AST
contains it. Derive it from declarations and print-line reads as new, gets a
registry cell nobody publishes, and the first call jumps to null.
Three changes are refused with a reason rather than loaded. A function's
signature, because a cell is a bare ptr and every call site compiled before the
change still passes the old arguments through it. A global's type, because the
storage exists and has a shape - reusing it reads at the wrong offsets, and
replacing it discards the state the reload exists to preserve. A struct's
fields, because the values the process is holding have the old layout. Note
what the checker already catches on its own: change a parameter type and the
caller fails to type check first, loudly. These rules only get a turn on a
change the checker accepts, which is a name nothing else in the program uses -
exactly where the silent version lives. Hence an unused defvar and a C-called
defn in the fixtures.
The accumulated list is the post-Load one, so an evaluated import is spliced as
its expansion. Otherwise re-evaluating a file that imports something appends a
second import, Load expands it again, and the duplicate-name pass rejects it.
C-c C-k on sand.flan's own text is the test.
flan reload now takes a program and a file of changed forms rather than a list
of function names and a --new list: the session works out which names are new,
which is the thing a bare CLI could not.
Also fixed, found by running the agent test under load: the agent took SIGPIPE
when a sender read part of a reply and closed. Replies go out with
MSG_NOSIGNAL, per call rather than by installing a handler, because the signal
disposition belongs to the program the agent is embedded in.
vendor/agent/ is a package like any other - agent.flan declares three calls,
flan_agent.c implements them, link asks for -lpthread. start listens on a unix
socket, poll installs whatever arrived and says how many, wait does the same
after waiting for something.
The split between poll and the listener is the whole design. dlopen relocates a
module and takes the loader lock, which is milliseconds and unbounded, so it
happens on the listener thread. flan_reload_install is one store per function
and must not land while a redefined function is on the stack, so it happens on
the game thread at the top of the frame, when the program asks. A ring and two
atomics connect them; the game thread never blocks on the loader.
wait exists for tests. A test that races the frame rate fails on a loaded
machine, so test/programs/agent.flan waits for the reload rather than sleeping
past it. It also sends a junk path first: the daemon is a separate process and
can send anything, and a bad path must be refused rather than take down the
program it was sent to.
Two things came out of running it. The reply goes out before the module is
queued, because the other way round the game thread can install and the program
can exit between the two, and the answer reaches the sender as a connection
reset instead of as ok. And ok means queued, not installed - the sender does
not get to know when the swap happened, since only the program knows when it is
between frames.
sand.flan now polls at the top of its loop, which is what this step was for.
Under Xvfb, one line on the socket and 455 consecutive frames drew from a
game-draw that did not exist when the process started. Building without --dev
still works: there are no cells, so a module is refused on the listener thread
and the loop never notices.
flan reload builds one module the way the daemon will. --new names what the
host was not built with, which is the one thing the command cannot work out for
itself and exactly what the session will track.
Editing a defvar or a defn is a symbol the host exports. Adding one is not:
there is nothing to bind to and ELF cannot grow a symbol. runtime/flan_dev.c is
the two lookups that cover it - flan_dev_cell for a new function's cell,
flan_dev_global for a new global's storage - both idempotent, so the second
module to mention a name gets what the first one got. That is the whole point:
two modules with their own copy of a new function would each call their own,
and redefining it would update one of them.
The compiler picks per name. A name the host has is a symbol and costs one load
at a call site; a name it lacks is a registry lookup cached at install time in
a module-local slot, and costs two. The common case pays nothing for the
general one.
The redefinition unit is now a list of top-level forms rather than one
function. It has to be: v3 of the fixture adds a var and uses it from a
redefined bump, and splitting that into two loads leaves a module referring to
storage that does not exist yet. C-c C-c passes one name, C-c C-k passes a
file's worth, one path either way.
Four rules, each silent if broken. Every lookup resolves before any body is
published, or a caller reaches a function whose slots are still null - asserted
on the emitted flan_reload_install, since it cannot be race-tested.
flan_dev_global refuses a size change, which is the layout-drift rule's first
enforcement point rather than another exception to it. Nothing is ever
dlclosed, because a cell holds an address inside a module's text. And the table
is fixed capacity, because a module holds a cell's address for as long as it is
loaded and a realloc would strand it.
The test that separates this from a plausible wrong version is v4, which
redefines a name v3 introduced at run time. v3's bump is already installed and
is not rebuilt, so it picks v4 up only if its call goes through a cell both
modules found by the same name. Had v3 cached the function's address instead,
every other assertion would still pass and the transcript would read 246
instead of 432.
Sizes are spelled LLVM's way, ptrtoint getelementptr null 1, rather than by a
layout calculator in OCaml that would have to agree with LLVM's on every
target.
Two things, and either alone is useless, so they are one commit.
Emit.redefinition compiles one function into its own module against a host
that is already running. What it does *not* define is the design: a global is
external, so state survives a reload and sand's grid is not reset by editing
the code; every other function is a declare, so a redefined settle calls the
host's move-grain rather than a frozen copy; there is no main. Build.shared
puts that text through llc + ld -shared. ld, not clang, because a shared object
is allowed undefined symbols and that is the whole mechanism - and because the
driver is 50ms of a 20ms job. Measured here: llc 16ms, ld 3ms, dlopen 0.04ms.
Loading a body is not installing it, though. A call bound at link time cannot
notice a new one, so a dev build routes every Flan-to-Flan call through a cell
- a mutable global holding the address of the function that is current - and a
module publishes itself with one store. The cell load is emitted after the
arguments, so a redefinition between two calls cannot land inside one.
Three details that are not free choices. flan_reload_install is a named
function rather than an ELF constructor, because the agent has to choose when
the store happens and a constructor would do it during dlopen, mid-frame, on
whatever thread called it. A redefinition's own body is hidden, because default
visibility in a shared object is interposable and that applies to taking the
address too: plain @"flan.bump" inside the module resolves to the host's copy,
so the installer would publish the function it was replacing and the reload
would silently do nothing. And -rdynamic is what exports the cells at all, so
it and cells are one flag: Build.opts.dev, flan build --dev, the first time
opts means something semantic rather than an optimisation level.
The test is one process, because two runs would prove nothing about a swap,
and two .so paths, because dlopen caches by path and would hand back the first
handle. Every call in it goes through outer, compiled once into the host and
never rebuilt, so a changed answer can only mean its call site followed. v2
recurses through its own cell, which is the interposition case; it would print
the old body's text if it did not. helper differs between the fixtures purely
as a tripwire for a module that grew its own copy.
LLVM cannot fold the indirection - the cell is an external mutable global - and
a --dev calc-me keeps 46 indirect calls at -O2. values, machine and
sand-headless now run as dev builds in the acceptance table too; the sand hash
is the one result that would notice a call reaching the wrong function.
