The checker half. {K V} and (Map K V) resolve, and map-new, put, get,
has-key?, len, reserve, clone and free are named calls over the
type-erased runtime, with the two sizes and the key's hash and equality
pair produced at the site because the site is where the concrete types
are known. len, reserve, clone and free were extended rather than given
map-shaped names of their own, which is what at and len already did for
Vec: one question, one word.
The key's pair is resolved per key type and mostly is not emitted at
all. Every integer, enum, bool and fixed array of those is compared
bytewise and served by one runtime pair over (pointer, size). A string
is not, because its bytes are elsewhere and two equal strings at
different addresses must hash alike. A struct is not, because its
padding bytes are indeterminate — two structs equal field by field can
differ bytewise — and because it may hold a string. So a struct gets a
pair emitted for it, walking its fields in declaration order and
addressing nothing but fields, and that is the only case that does. Two
maps with the same key type share one pair, and a struct reached twice
through two fields emits one.
get returns (Option V) and builds it here rather than in the runtime,
which has no idea what an Option's layout is — keeping it that way is
what lets one entry point serve every value type. put is upsert
returning Unit. Both bind their arguments to slots before the guard, so
a retry re-attempts the allocation and not the expressions that produced
the key and the value.
Refusals, each by name: a float key has no usable equality at all, which
is not a milestone question; a Ptr, slice, Vec or Map key would hash an
address rather than what it points at; a move-only value would have its
header duplicated by clone, which is the refusal (Vec (Vec T)) already
carries; Unit as a value has no bytes to store, and it is the natural
spelling of a set, so it is refused by name rather than by dividing a
cache line by zero.
Work in progress: it builds and the runtime is exercised and green, but
no Flan program can reach it yet — the checker half is not written, so
(Map K V) is still refused where it is resolved.
runtime/flan_rt.c is Odin's map, followed deliberately: open-addressed
Robin Hood hashing at a 75% load factor, cache-line cell packing so no
key or value straddles a line, and the probe loop kept to pointer-width
integers. One type-erased runtime over (key size, value size) plus a
hash and equality pair, the same arrangement the Vec runtime has over
(size, align).
Two departures from Odin, both deliberate and both commented where they
are made. There are no tombstones, because removal is deferred by
spec-memory.md, and that deletes the backward-shift loop entirely — it is
the single largest reason this is shorter than the original. And the
header does not stuff log2cap into the low bits of the data pointer:
Odin does that because Raw_Map must be three words, whereas this header
already carries an allocator, a generation and an epoch, so the tagging
would buy nothing, cost a mask on every access, and make correctness
depend on the block being 64-byte aligned rather than merely faster
when it is.
The scaffolding around it: a Map is 48 bytes and six words like a Vec,
it crosses to the runtime by address because it is move-only and must be
mutated in place, and it has a DWARF type showing all six fields.
Tast.FnAddr is new — the address of a function, either one this compiler
emitted or a runtime C symbol. It is not a function value: nothing in
the surface language can produce one, name its type or call through it.
Odin's Map_Info reaches its hash and equality pair exactly this way.
reach.ml learns that edge, because a function reached only by address is
invisible to the reachability walk otherwise, which is the same hazard
handler-bind clauses already had.
The hash and equality pair carries the transfer channel as its last
parameter, because a pair emitted for a struct key is an ordinary Flan
function and every Flan function's signature ends with one.
The refusal for a frame whose body has been redefined underneath it did not
fire because four of its five hand-offs were never written. `Emit.fninfo` has
been storing `slot_fingerprint` in the last `i32` of every `%fninfo` all along;
`flan_dev.c` called that field `spare`, there was no accessor for it, the agent
never snapshotted it, the backtrace line never carried it, and `Dev.locals`
compared slot counts and nothing else. The handoff note's "every piece is
written and the refusal does not happen" was a guess, and the first step it
suggested — printing both sides of the comparison — could not have found it,
because there was no comparison.
So: `spare` becomes `slotsig` and gets `flan_dev_frame_slotsig`; the agent
snapshots it beside the slot count and puts it on the backtrace line *before*
the location, since the name is the one field that can contain a space and has
to stay last; `Dev.backtrace` parses it; `Dev.locals` compares it against
`Emit.slot_fingerprint` of the body this session holds and refuses by name when
they differ. No change to `emit.ml` — the value was already there.
The mechanism itself is right and stays. `slot_fingerprint` hashes every slot's
name together with the spelling of its type, so a rename that keeps the count
and the types — exactly the case this exists for — changes it. The count check
stays in front of it because its message is the more specific one.
The fingerprint stays off the wire. A hash is not something an editor can act
on, and the refusal says the fact in words: this frame's body was redefined
since it was entered, so its names no longer describe its values.
`test_dev.ml` gains the inverse and the control. A body that drops a `let` is
refused on the count, and `main` — untouched by the redefinition of `look` —
must still answer, which is the assertion that would catch a fingerprint that
never matched anything and made the verb useless while turning the suite green.
The half the shadow stack was built for. A slot's entry in the frame is its
address, null until the binding that fills it has run, so "not bound yet at
this point" is a null and needs no liveness analysis. The daemon compiles a
thunk that renders the types it already knows -- Tast.fn.slots, with snames
beside them -- at the addresses the stopped program supplies, and reads the
text back the way C-x C-e does. Nothing is copied out, because a value with
no header is bytes with no meaning anywhere but in the program that holds
it.
That is render.ml's walk with its root changed, which is the pointer-rooted
thunk NEXT.md said this needed, and one new arm in the backend: a cast from
one pointer type to another, which emits nothing.
Only named slots are recorded. A recorded slot escapes and stops being
promotable, and the slots that would cost most are the ones with nothing to
show -- dotimes' bound, the temporaries min and max use, the walk's own
scratch. They are refused by name rather than shown under an invented one.
Recording every slot was built and timed and is inside the noise, so the
rule stands on what it shows.
Four refusals, each by name and with its reason: a slot nobody named, a
slot the program has not reached, a type the printer has no arm for, and
two whole frames -- an evaluation's thunk, and a frame running a body that
has been redefined since, where every slot index would be a guess.
Measured, minimum of nine runs: +61% on call-heavy code over globals
against +33% for the frames alone, 0.06% of a frame at 60fps.
plan.org has specified a shadow stack in the dev column since the beginning
and nothing had ever built it. A frame is four words on the calling
function's own stack: the one it displaced, a pointer to a static
description of the function, and two words reserved for its locals. The
name and the location travel on the frame, so a backtrace needs no debug
information, no symbol table, and nothing from the platform unwinder that
plan.org deliberately does not use.
The pop is at every ret, the landing block a transfer leaves through
included. That is the half that is easy to get wrong: a pop written only on
the normal path leaves a dead frame behind every handled error, and the
test takes five breaks and resumes all of them by transfer before asking
for two frames.
(:op "backtrace") answers from a snapshot the stopped thread takes, beside
the restarts and for the same reason, and marks which frames belong to the
program and which to the evaluation the break is inside. It is refused
while the program runs.
Measured, interleaved, three pairs of binaries: 29% on 600 frames of sand,
7.6% on a benchmark that is nothing but calls -- 32us per frame of sand, a
fifth of a percent of a frame at 60fps. An array with a stack pointer was
built and timed as the alternative and is worse on both.
Decision 1. Odin's #load and #load_directory are the model, spelled as
ordinary named calls — an s-expression language already has a head
position and does not need Odin's `#`. (embed "p") is a [u8], (embed "p"
string) is a string, and (embed-dir "d") is a [n EmbedFile] sorted by
name.
Two spellings rather than one that changes type with its context. Odin
threads a type_hint everywhere and can afford it; with structural
equality and no implicit widening, the same text meaning two types here
would be a wart. The path is a literal and resolves relative to the file
the form is written in, both of which are Odin's rules and for Odin's
reasons: the bytes must be in hand before any value exists, and a
package's assets must not depend on where flan was invoked from.
The bytes reach the program as a [Str] node typed [u8], not as a [Bytes]
prim over a string. [Bytes] is identity — emit.ml lowers String and
Slice _ to the same %slice — and wrapping the literal in a prim would
make the node non-constant, so an (embed-dir) bound with defconst could
not be an LLVM constant. Both string emitters take the bytes and ignore
the node's type, so it is the same constant either way and one a global
can hold. emit.ml's escape is byte-exact, so a PNG survives the .ll.
The directory lookup is a linear scan in the prelude over a slice of
EmbedFile. A directory embed is tens of entries out of cache-warm
.rodata, and a compile-time perfect hash would be a build-time map with
its own failure modes that nothing has asked for. Sorted because readdir
order is filesystem-dependent and an unsorted embed would make two
builds of identical sources emit different .ll.
The slice points into .rodata, so a store through it segfaults at -O0
and is deleted at -O2 — the same measured trap the prelude's ASCII-case
note describes for (bytes "Hi"). Inherited, not widened; clone into a
Vec for a mutable copy.
The debug-info arm and the structural printer are each a separate path from
everything the suite was exercising: `outputs ~dev:true` goes through the cells,
not through DWARF, and no program printed a Vec or an allocator. That is
NEXT.md's landed item 2 exactly — field_addr took only Types.Named, so the
printer's Option arm had never run and would have died on the first (Option T)
pointed at it. Both arms work; both are now reached, and the DWARF row asserts
the composite's size as well as its name, because an element count that
disagreed with `lay` would print plausible values for the wrong fields.
Printing a Vec did not work: `println` checked its argument as an ordinary read,
so it moved, and every printing of a Vec would have been its last. Printing is a
borrow — the walk goes over the value and keeps nothing.
And `vec-new` with an explicitly named null allocator no longer substitutes the
heap for it. Adopting the context for a *zeroed* Vec is the documented rule;
quietly substituting for an allocator the program named is the same "released
the region / never made one" collapse free-all already traps for, except silent
and found later as a leak. The no-allocator-named case never arrives as null —
the checker passes flan_context_allocator(), which always answers one.
Two element types, one runtime, and the element type appears nowhere below
the call site: size_of and align_of are produced where the concrete type is
known, which without generics is simply the concrete call site. That is
Odin's arrangement and it is what spec-memory.md specifies. `at` and `len`
were already the names for a fixed array and a slice, so a Vec extends them
rather than adding a parallel pair — the asymmetry `nth` was removed for —
and the value form and the place form go through one helper so they cannot
drift apart.
StorageExhausted lands with step 2 rather than after it, because the
signatures depend on it: `push` and `reserve` are Unit, `clone` is the
container, and nothing grows a Result. It is built out of nodes that already
existed — a while, a restart-case and an error — so the backend learned
nothing about allocation. The restart is established at the failing
allocation, which spec-memory.md names as the exception to "restarts go at
the resync point, once", and the element a push was given is bound to a slot
before the loop so a retry re-attempts the allocation and not the expression.
Move-only is a dead set on the checker context, and it is flow-sensitive at
an `if`: both arms start from the same set and the union survives the join,
so `(if c (free v) (free v))` is legal and a one-armed free still kills the
binding. The case a dead set cannot answer is a move inside a loop — merged
once at the end of the body it counts one move, not two — so that is a rule,
refused with its reason.
Four decisions the spec did not settle:
The Vec header is six words in every build, not four in release. A layout
that changes with a build flag can disagree across the reload boundary
silently: a redefinition module is built by llc and ld against a host built
separately, and nothing makes the two agree on a struct size. The 32-byte
release layout is deferred on that.
A zeroed Vec has a null allocator, and the first operation needing storage
adopts the context allocator. Odin's behaviour. The alternative was refusing a
Vec-typed struct field until drop lands; shipping the null was a null deref on
the first push.
A Vec's length and index are i32, like every other length here. Widening
indices is one change across all the containers, not a Vec question.
`let` has no type annotation, so a local Vec has nowhere to say what it holds
and the element type is written at the call: `(vec-new i32)`. This is not the
explicit instantiation syntax the generics section rules out — nothing here is
generic and the name resolves as an ordinary type. Where the context says, it
may be left out.
The allocator grew a budget: a ceiling on live bytes, 0 for none. The retry
restart is only answerable by a handler that can make the *same* request
succeed, and for a fixed backing store the handler that works is the one that
raises the ceiling — releasing the region a container lives in invalidates
the container, which is what the epoch check catches. The spec's "grows the
arena and then invokes retry" needed something to grow.
The generation word is bumped on every reallocation and read by nothing. The
stale-slice trap it is for needs a slice that can carry the Vec's identity,
and a slice is ptr+len. Said plainly rather than implied by the word's
presence.
spec-memory.md defines an allocator as a procedure plus an opaque data
pointer, which reads as a function value, which check.ml refuses four ways.
None of the four is anywhere near this: `Allocator` is a `Types.t` case with
no user-writable constructor, the way `string` is a builtin ptr+len, its
procedure is a C symbol the emitter names, and every operation is an ordinary
named call that `check_call` already routes through `named_call`. The one
thing that really does need milestone 5 is a *user-written* allocator — it
wants a defn's name in value position — and that is refused by name with that
reason rather than left to come back as an unknown function.
An `Allocator` value is a pointer to the runtime's struct and never a copy of
one. That is forced, not chosen: the capability set has to be readable from
wherever a container landed, and `free-all` bumps an epoch every container
made from the allocator has to observe. A copy would give each its own epoch
and the dev trap would never fire.
Two decisions the spec left to be made here, both announced in BUILT.md:
`free-all` is retain-capacity — offset = 0, the pages stay — and handing the
pages back is `arena-destroy`, a separate operation. Zig's reset takes a mode;
Odin's arena_free_all is already retain-capacity in effect. Taking the mode
would have grown the operation table the spec froze at four. The epoch is
bumped either way, because the pages being the same does not make a container
made before the reset valid.
`context/allocator` and `context/temp` are dynamic variables with save and
restore, not extra parameters. The spec calls the allocator part of the
calling convention; the literal reading touches every signature, the FFI shim,
the dev trampolines and the reload ABI for the same observable behaviour.
`with-allocator` is its own IR node rather than a let and two calls, because
the restore has to happen on the transfer path too. A body that errors leaves
through the landing pad, and a context allocator left pointing into a region
nobody outside the body has heard of would be wrong in the break loop, which
is exactly where something is about to allocate to render a condition. The
acceptance program asserts that path by taking a restart out of a body.
The backend grew one prim, `Rt of string`: a call into the runtime's C named
by symbol, with argument and result types read off the expression nodes. The
container runtime is type-erased and therefore *is* a list of C entry points,
so one arm covers all of them rather than one arm each.
The thunk calls flan_dev_result_begin before it evaluates anything, so an
expression that signals is stopped inside the seqlock's window — and a restart
taken from that break transfers past the thunk, so the matching end never runs.
An unpaired begin cost nothing while the counter only moved at the end. It
costs everything now: incrementing would leave the count odd for the life of
the process, every later read reporting a write in progress, and C-x C-e dead
until the program restarts.
So begin sets the low bit rather than incrementing, and end clears it by
setting rather than adding. The ordinary sequence is unchanged — 2k, 2k+1,
2k+2 — and an abandoned write is over as soon as the next evaluation starts.
What that does not fix, because one buffer cannot: an evaluation running while
another is stopped mid-render shares the buffer, so the inner value is the one
that survives. That was true before the counter was a seqlock and is not a
regression.
Also noted in NEXT.md: rt_die in flan_rt.c has the same exit-with-the-loader-
lock-held shape the break loop just lost. Not fixed with it, because rt_die is
the non-dev path too, where there is no listener to deadlock against — whether
it should be _exit always or only under --dev is a decision.
And the 4K-cap assertions clamp their own String.sub, so a short body prints a
failure instead of raising out of the test.
The 4K result cap and condition_name[128] are on the agent's socket path, which
is why the sanitizer corpus cannot reach them: a program in the sweep has no
socket and nobody on the other end of it. test_agent has both.
A 5000-byte string literal evaluated into the running program comes back as
exactly 4096 bytes ending in the ellipsis result_end puts there to say it
clamped — and it comes back through the seqlock's copy, so the cap and the new
reader are pinned by the same case. The header also shows the generation as 1,
which is the count of complete values rather than the raw counter.
A condition class of 198 characters comes back from `status` as 127 and a
terminator. Aborting out of that break is what pins the exit status at 134 now
that the loop leaves with _exit rather than exit.
SNAP_MAX, SNAP_NAMES and the dev registry's overflow guard are still read
rather than tested. Sixty-five nested restart-cases and four thousand interned
names are a lot of program to write for a clamp each, and neither is on a path
this session changed.
flan_dev_result_cap() exists so the size is asked for rather than written down
in two files: "the copy is never truncated" is only true while the agent's
buffer and the runtime's bound agree, and the agent checks that where the copy
happens.
The pipe the queue program blocks on is close-on-exec, or the child inherits
the write end and its own stdin never reaches end of file — it sat in its last
read waiting for a byte only it could send.
spec-conditions.md §3's remaining half: a clause binds parameters, an
invoke-restart supplies them, and what a restart takes is compared at run
time because a restart is found by name on a dynamic stack — neither end
of the transfer can see the other.
The parameters live in a buffer the restart-case owns, not the invoker's
frame. A clause runs after every frame between the two has returned (§5),
so anything on the invoking side is gone by then; the invoker stores into
the target frame while both are still alive, which is the one moment they
are.
The frame carries the parameter count and a hash of how the types are
spelled, and every frame carries them whether it takes parameters or not:
a clause taking none has to refuse arguments as loudly as one taking two
of the wrong type. The count is not redundant with the hash — it is what
makes a 32-bit collision between two different signatures harmless — and
the spelling itself rides along so that a mismatch can say what was
wanted and what was given, which neither end alone knows.
The arguments are evaluated into slots before the invoke node rather than
hanging off it. An argument that transfers on its own is then guarded
before anything aims the channel, and a call written in an argument is on
the ordinary walk Reach and Load already do — a node they treat as a leaf
would have dropped the function and failed to link.
The other way a transfer starts is the break loop, which chooses by
position and has nothing to fill parameters in with. It reaches a clause
through the same channel, so nothing downstream could tell the two apart:
the frame is pushed with the buffer marked unfilled and a clause with
parameters checks that mark before reading it. Refused with the reason
rather than run on values no one supplied.
runtime/flan_rt.c gains two message functions and nothing else; the
restart frame's first four fields, which are the ones C declares, do not
move.
It read the generation, then a non-atomic length, then returned the buffer
itself — and the agent sent those bytes down a socket some time later, while
the game thread was free to be a hundred bytes into the next value. A seqlock
cannot validate a read that finishes after it returns, so the pointer was the
bug and not the ordering.
Made into a real one rather than documented down to what it guaranteed,
because what it guaranteed was nothing: the generation told the daemon a new
value had arrived and said nothing about whether the bytes it then read were
that value. Writing the honest comment would have left the daemon's only way of
reading a result unsound with a note beside it.
The counter is odd for exactly as long as a value is being written.
flan_dev_result_read copies into the caller's buffer and checks the counter
either side of the copy, retrying if it moved; a reader that loses the race
reports the last complete generation and no bytes, so a daemon polling for a
new value keeps polling rather than being shown half of one. The count handed
out is the number of complete values, so "has it moved" still means what
lib/dev.ml takes it to mean. The agent's buffer is RESULT_MAX, so the copy is
never truncated.
The race itself has no regression test. Arranging it means landing a socket
read inside a render thunk from outside the process, which is the same hook the
snapshot generation wants. What is tested is that eval still reads back the
value it rendered, through test_dev's existing cases.
(slice s 2 1) has length 2 - 1 - 2 = -1. flan_bytes_to_i64 and
flan_bytes_to_f64 both wrote their clamp as (size_t)n < sizeof buf - 1,
and (size_t)(-1) is 18446744073709551615, which is not less than 511 --
so k took the cap and the memcpy copied 63 or 511 bytes out of a
five-byte string constant. ASan calls it a global-buffer-overflow in
flan_bytes_to_i64; the regression case is in test_sanitize.
Every other (ptr, len) entry point in the runtime already guarded the
negative case -- flan_write_stdout tests n > 0, flan_escape_bytes and
flan_dev_emit both fold a negative length to zero -- so this was two
exceptions rather than a missing convention. A checked build traps on
the reversed slice before reaching either, which is why it took an
--no-bounds-checks run to show.
Also clamps the three snprintf shims that publish scratch as a slice.
snprintf returns what it would have written, not what it did, so a
format that overran the 64-byte buffer would hand out a length past its
end. No format here can: %g is 13 characters and %lld is 20. Found by
reading, and the sweep could not have found it -- nothing in forty
programs prints a number that long.
A restart the innermost frame shadows could be seen and not taken;
it is taken by position now, off a snapshot that stopped moving under
the break loop. The editor half this was briefed as building already
existed — the stale line that said otherwise is fixed.
The guard reserves 9 bytes but the comment explained 5, which is the
longest escape alone -- it did not account for the three writes after the
loop (the ellipsis and the closing quote), so the next person to touch
the escape table would have preserved the wrong invariant.
Swept every length to 1300 against \x01, a quote, a backslash and 'a'
under ASan with a red zone past the buffer: no write past 1024, worst
output 1021. Correct, but by three bytes, which is exactly why the
reserve is now written down as the four things it is spent on.
Nothing exercised truncation -- the longest nested string in the fixture
was 18 bytes -- so println.flan now prints a struct with an 1100-byte
string field, and the expected output spells the surviving count out as
a number so a change to the buffer shows up as one.
Two frames offering `retry` put both on the break loop's list and only the
inner one within reach: §4's walk takes the first frame offering a name, by
definition, so the outer clause was drawn, offered, and unreachable. The old
prompt showed `retry` twice and sent the string either way. An index is the
only thing that can say which one, which is why SBCL identifies them
positionally too.
An index is worthless against a stack that moves, though, and this one moves:
the break loop is the poll loop, so every restart-case an evaluation enters
pushes and pops the same global list between the listing and the choice. So
the list is read once on entry and copied — names into the agent's own buffer,
frames as the addresses a transfer carries — and every answer comes from that.
The name still travels with the index as a receipt, checked against the
snapshot and refused if the two have drifted, so a bare integer can be wrong
out loud.
And the third state. A restart below the thunk a break is inside was accepted,
announced, and silently not taken: `flan_reload_call` holds its own transfer
channel and drops it on return, so the unwind stops at the thunk. The boundary
is now recorded where it is made, at the call — frames a restart-case inside
the thunk pushes are above it and still work — and such a restart is listed,
marked, and refused with the reason.
`break.flan` grew the shadowed pair, and 900 is a value no by-name lookup in
that file can produce.
session.ml already had this: a compile-time walk over a Tast type that
emits the calls to print a value of it, handling every concrete type the
language has. It was dev-build-only and went to flan_dev_emit, and
prelude.ml justified the per-type print-* functions by saying a real
println had to wait for milestone 5 and generics. It did not. plan.org
specifies println as compiler-provided and per concrete type, which is
not overloading: there is nothing to dispatch on at run time and no
user-supplied printer to choose between, so no type variables appear.
The walk moves to render.ml, parameterised on an emitter and a slot
allocator. The emitter is five functions rather than five extern names
because the two sides are not both extern calls -- the REPL's are, and
stdout's compose a conversion with a write. The slot allocator differs
too: the REPL builds a thunk's frame, println takes slots from the
enclosing function being checked, once per call site.
Two runtime shims, both only reachable from the walk. flan_u64_to_bytes,
because routing u64 through the signed printer makes 0xFFFF...F read as
-1, which is the one way println could disagree with the REPL about a
value both can hold. flan_escape_bytes, so a string nested in a printed
structure is quoted and escaped -- same table as flan_dev_emit_str, noted
in both, because the REPL and println must not disagree about what a
struct looks like.
A string at top level prints raw and nested prints quoted. Not a conflict:
(println "hello") has to print hello, and a struct's string field has to
be distinguishable from the punctuation around it. The split is top-level
vs nested, so it lives in check.ml and not in the walk.
Found on the way: a field of an Option had no gep in emit.ml, so the
walk's Option arm had never run -- the REPL would have failed on one too.
Option is { i8, T } with no declared name, so its layout is now spelled
out. Nothing in the surface language reaches a field of an Option; the
printer does, to read the tag without unwrapping a None.
The print-* functions stay. They print without a newline, which println
cannot express -- slices.flan's show prints elements separated by spaces
-- and they are raw where print is structural.
println.flan covers every arm at -O0 and -O2: the u64, the raw/quoted
split, both Option arms, the depth and span caps, and the slice arm's
loop twice over plus once inside a dotimes, which is where per-call-site
slot allocation would show if it were per-iteration.
spec-conditions.md §2, and the reason the transfer was worth building. An
unhandled error runs a hook instead of rt_die(), on the frame that erred with
nothing unwound, lists the restarts between there and the top, and waits.
A hook rather than a direct call because the loop lives in vendor/agent, which
is an optional package, and flan_rt.c is the release runtime - a program with no
agent leaves it null and dies the way it always did. The hook resumes by writing
a restart into the transfer channel, which is the channel an invoke-restart
writes and reaches the same guard, so choosing from the break loop and choosing
from a handler are one act lowered once. §6 needed no change.
The break loop is the poll loop, run from the error rather than from the frame
boundary. That is load-bearing: an expression evaluated while stopped is a
module the listener queues and the game thread runs, so a loop that did not
drain that queue would hang C-x C-e exactly when it is wanted most. Installing
while stopped is allowed, which contradicts the rule that a redefined function
must not be swapped while it is on the stack - that rule is about mid-frame
consistency and there is no frame in progress here. The old body keeps running
and a retry reaches the new one through the cell, which is the whole point.
A restart frame carries its name now, beside the hash. Matching never needs it;
showing someone their choices does, and nothing at run time can turn a hash back
into a name.
A choice is checked on the listener thread against a stack the stopped game
thread is holding still. Answering ok and finding out on the game thread that
nothing offers that name would report success for something that cannot happen.
The test errors twice and takes a different restart each time, so a loop that
always resumed the same way fails it.
spec-conditions.md §2. The same lookup as signal, and the difference is
entirely what happens when the walk ends: signal returns Unit and the
signalling function carries on, error has type Never and the program stops.
Only a transfer gets past it, so emit puts a guard after the call and then
unreachable - and flan_error cannot be marked noreturn for the same reason, it
does return, on exactly one path.
Being Never is what lets it stand where a value was expected, which is the
fall-through shape §1's load-texture example needs and the reason it is worth
having before the break loop rather than after. An unhandled one names the
condition on stderr and dies the way every other trap does; flan_error is where
the dev-build break loop will go.
The two spellings share one AST and IR node with a kind beside them, the same
shape Ast.unwrap already uses for some and try, because they differ in one
decision and nothing else. test/programs/error.flan is the unhandled case,
asserted on the exit code and the reason rather than through the outputs table,
which only has room for a program that exits 0.
spec-conditions.md §3 to §6. A handler runs where the signal was, decides, and
control resumes at a restart-case further out - so unlike step 1 this one does
alter control flow, and it is lowered explicitly rather than through platform
unwinding, because wasm32 cannot unwind and because a cmp/jne after a call
reads like ordinary code.
The channel is the out-parameter §6 settled on: one ptr appended to every Flan
signature, written by an invoke-restart and checked after every call. The
return type stays what the source says, one pointer threads down the whole
chain, and a frame that sees the channel set just returns early - which reuses
the existing return path and with it §5's defers for free. Emit.signature was
already the one place a signature is spelled, which is what made that part
small.
Every function is transfer-transparent, release included. §6's escape analysis
is an optimisation; in a dev build a cell can hold anything, so the honest
answer to what a call can reach is anything, and uniform means redefinition
acquires no new refusal class.
The transfer target is the restart frame's own address and not a static clause
id, which corrects what the handoff note had settled. An id has to be unique
against every module a running program may later load, and a hash is only
probably unique - two restart-cases colliding means the inner one silently
catches a transfer aimed at the outer. The frame is an alloca in the function
that offers it, so the address is exact and it also says which clause, which is
how clause ids disappeared. Re-entering a restart-case then needs nothing
extra, since each activation allocates its own frames.
Cleanup is landing blocks, one per region rather than one per function: a
restart-case's pops its frames and either dispatches or forwards, a
handler-bind's pops the handler frames on the way past, and the function's own
runs its defers and returns. One function-wide block would have jumped straight
past the very restart-case that was meant to catch the transfer. The channel is
cleared before any cleanup runs and put back after, or a defer's first call
would branch straight back into the block it came from.
flan_signal takes the channel and passes it to each handler, stopping once one
writes to it. That makes the one C frame every handler is reached through
transparent to a transfer, which it has to be; it is also the only one, since
extern is Flan-to-C only and there are no function values yet.
Refused by name with the reason, each with a test on the reason: restarts with
parameters, return inside a restart-case body, one restart-case offering a name
twice, and invoke-restart inside a defer - a defer is the cleanup a transfer
already runs, so starting one there leaves the defers half run with two targets
and no way to choose. The lexical case is the checker's and the one that
reaches a function through a call is trapped at run time. No restart of that
name is a located runtime error at the invoke site, because there is nowhere to
resume.
Two things found on the way. `{ ctx with in_handler = true }` was a latent bug:
ctx.slots is mutable, so a copy allocated the body's slots into a record the
function never saw again - harmless only because no handler-bind body in the
tests had a let in it. And test/reload_host.c calls flan.outer through an asm
label, which does not fail at link time when the prototype is a parameter
short; it reads garbage as the channel and dies somewhere else.
test/programs/restarts.flan runs at -O2, at -O0 and as a dev build. -O0 is not
redundant: the guard after every call is control flow the optimiser would
otherwise launder, and the dev build is where each of those calls goes through
a cell.
spec-conditions.md §1 and §2 and nothing else, because those two are worth
having alone: signal returns Unit whatever it finds, a handler that returns
normally leaves the signalling function to carry on, and with nothing matching
it is a no-op. So none of §6's transfer machinery exists yet and no signature
changed - which is the whole reason to do this step first.
The runtime is a linked list. Establishing a handler is two stores and a push
onto a frame on the establishing function's own stack, and signal with an empty
stack is a null check, which is what §2 asks for. Popping is by frame rather
than by count, so restoring what this one displaced is right even if something
below it left the stack out of step.
A condition's type is a hash of its name and not an index: an index would shift
the moment a struct were added, and every handler a running program had already
pushed would match the wrong type. The condition crosses as a pointer, since a
handler runs while the signalling frame is alive and there is nothing to copy -
but what the clause binds is the condition itself, the pointer being a hidden
parameter and the name a slot loaded from it, so a handler passing c to
something expecting the struct is not handed an address.
A clause is lifted into a function of its own, because a handler runs from
wherever the signal was and cannot be a branch in the function that wrote it.
That gives two refusals, both by the house rule. A handler cannot see the
establishing function's locals - that is a closure with an explicit
environment, so a reference to one is refused for that reason rather than
reported as an unknown name. And return inside a handler-bind body is refused,
since the frames are popped on the way out and an early exit would leave them
pointing into a function that has gone.
Settled in advance for the next step: in a dev build every function is
transfer-transparent, because a cell can hold anything and the honest answer to
what it can call is anything. Same bargain as the indirect call, and it means
redefinition acquires no new refusal class. Still open is whether the
discriminated result is returned by value or through an out-parameter.
C-x C-e rendered the scalars and refused the rest, which made it a calculator
rather than a REPL. The renderer is now a compile-time walk over the type,
emitting a piece at a time: structs, nested structs, fixed arrays, slices,
options, enums by name, and pointers as their shape. A raylib Color comes back
through the FFI as (rl/Color {:r 17 :g 34 :b 51 :a 68}).
Piecewise emission is what makes composites possible at all - a struct is its
fields with punctuation between them, and concatenating that in generated IR
would need an allocator the language does not have.
u64 now renders, in C, with %llu. It used to refuse because i64->bytes is
signed and it would otherwise come back as -1, but refusing a whole struct
because one field is a u64 is much worse than adding a runtime entry point.
Strings are quoted and escaped in C for the same reason: unescaped content does
not round-trip and reads as a framing bug rather than as the value it is.
An enum renders as :name, recovered from the checker's table as a chain of
comparisons, since members are erased to i32 before the backend sees them; a
value outside the declared members falls through to its number, which is what
you would want to see. A pointer is rendered and never followed - it is the
only thing that could make the walk cycle, and dereferencing one a REPL was
handed is not a safe thing to do on someone's behalf.
Three bounds, easy to conflate. depth and span bound the walk, so sand's
[100 [100 u32]] grid does not unroll into ten thousand render sites. The output
is bounded once in the runtime, since a slice renders through a loop the
compiler cannot bound, and one place enforcing it means no renderer carries a
budget.
emit.ml's cast now treats an enum as the i32 it is. Nothing in the surface
language produces that - a keyword resolves against its enum and never widens -
but the renderer needs an enum's number when it falls outside the members.
A different primitive from redefining a name. There is no name to install a
body into, so the expression is wrapped in a function with nowhere to be called
from; the module exports flan_reload_call to say "run this once", and the agent
calls it after the install - on the game thread, at a frame boundary, so an
expression that reads the program's state sees a point the program agrees is
consistent.
Nothing is marshalled back because nothing could be. A Flan value carries no
header, so no code at run time can say what it is; the compiler knows the type
and renders it there, in the thunk. That is the layout decision's bill, and it
is why the printer set is the scalars rather than everything.
The rendering does not go through stdout. Stdout belongs to the program, it is
in the hot path for anything that prints, and a dev-only feature must not put a
branch in it - so flan_rt.c is untouched and the value goes to flan_dev_result,
read back over the agent's socket. Safe without a handshake because the
generation counter is bumped last: the daemon waits for it to move rather than
assuming the program has reached a frame boundary.
u64 refuses by name, because i64->bytes is signed and anything past 2^63 would
come back negative. Everything without a derived printer refuses the same way.
A number that is quietly wrong is the failure this whole thing exists to
prevent.
An evaluation is not a declaration: the thunk is built against the program and
never spliced into it, so describe does not fill up with an eval/N for every
expression ever typed.
The test that matters is the same expression twice. The fixture increments
ticks every frame, so two evaluations must disagree - a value computed in the
compiler, or read from a copy of the program's state, would not.
Asked whether a defconst could be redefined, probed it, and got ":status ok"
for a change that did nothing at all - the module was built, delivered,
installed, and the program went on using the old value. That is the
silent-wrongness class the house rule exists to prevent, so it is now four
refusals and a fix.
A defconst's value is folded into its call sites - into an array length at
worst, which is decided before any type resolves - so it lives in the running
program's code and not only in its storage. Refused. A defenum member is the
same thing: :space is erased to an i32 literal in the caller. Refused, and
compared over declarations rather than over Tast.program, which carries no
enums at all for exactly that reason.
A defvar's initial value is deliberately not refused. Its storage holds live
state the program moved past long ago, and refusing to change the initialiser
would be refusing "edit the code, keep the sand". Same Tast.global record as a
defconst, opposite answers, told apart by gconst.
The value comparison is structural and conservative - anything it does not
recognise counts as changed. Comparing emitted text would be wrong, since
Emit.const on a string allocates a name off a per-module counter and two
different strings in two throwaway modules both come out as @".str.0".
Third: a new global's declared initial value was being dropped. flan_dev_global
callocs, so (defvar n i64 42) added at run time was silently zero. It now takes
the initial value as a blob, copies it on the allocation and ignores it
afterwards - the second half being where "a reload must not reset the program's
state" lives. In the allocation path rather than a branch at the call site, so
it cannot be got wrong at one of them.
Fourth: a change with no body to publish and no storage to allocate now answers
"nothing to install" instead of shipping an empty module. That is what the
defconst probe actually did, and it cost the program a frame's worth of reload
it did not need.
Everything so far installed one module. The daemon's job is N of them against
one long-lived session, and that is where a registry that hands out fresh
storage per module would show up. So the agent test now takes two: the first
introduces a global the process was never built with, the second only reads it.
1007 rather than 7 is the whole assertion.
Getting there needed stdout to be line buffered, set in flan_rt_init. The C
default when stdout is a file or a pipe is a 4K block, so a program running for
minutes with a REPL attached shows nothing until it exits, and a test driving
one cannot see its progress at all - which is how this was found. One write per
line instead of per 4K.
Also written down: flan reload builds a fresh session from source each time, so
if the program file was edited since the process launched, its idea of the
host's names and memory describes a binary that is not running. That is a limit
of the command, not of sessions. And Session.eval's origin defaults to <eval>,
so the daemon has to pass the editor's real buffer path or errors point at a
file that does not exist.
Editing a defvar or a defn is a symbol the host exports. Adding one is not:
there is nothing to bind to and ELF cannot grow a symbol. runtime/flan_dev.c is
the two lookups that cover it - flan_dev_cell for a new function's cell,
flan_dev_global for a new global's storage - both idempotent, so the second
module to mention a name gets what the first one got. That is the whole point:
two modules with their own copy of a new function would each call their own,
and redefining it would update one of them.
The compiler picks per name. A name the host has is a symbol and costs one load
at a call site; a name it lacks is a registry lookup cached at install time in
a module-local slot, and costs two. The common case pays nothing for the
general one.
The redefinition unit is now a list of top-level forms rather than one
function. It has to be: v3 of the fixture adds a var and uses it from a
redefined bump, and splitting that into two loads leaves a module referring to
storage that does not exist yet. C-c C-c passes one name, C-c C-k passes a
file's worth, one path either way.
Four rules, each silent if broken. Every lookup resolves before any body is
published, or a caller reaches a function whose slots are still null - asserted
on the emitted flan_reload_install, since it cannot be race-tested.
flan_dev_global refuses a size change, which is the layout-drift rule's first
enforcement point rather than another exception to it. Nothing is ever
dlclosed, because a cell holds an address inside a module's text. And the table
is fixed capacity, because a module holds a cell's address for as long as it is
loaded and a realloc would strand it.
The test that separates this from a plausible wrong version is v4, which
redefines a name v3 introduced at run time. v3's bump is already installed and
is not rebuilt, so it picks v4 up only if its call goes through a cell both
modules found by the same name. Had v3 cached the function's address instead,
every other assertion would still pass and the transcript would read 246
instead of 432.
Sizes are spelled LLVM's way, ptrtoint getelementptr null 1, rather than by a
layout calculator in OCaml that would have to agree with LLVM's on every
target.
