x86 is what flan dev takes by default and dyn is the iteration feature, so
a backend that refused dyn meant the two halves of the dev loop could not
be in the same program. The refusal was one arm of is_agg, and it said the
true thing: it was never the representation that was missing. A dyn is
uint64_t, a scalar in both calling conventions, classified by every rule
this file already had; every operation on one is a Tast.Rt primitive and
call_rt has always known how to make one of those. What the lane actually
cost was the collector's root discipline.
Which is emit.ml's, reused rather than rewritten: Emit.dyn_roots counts the
roots for both backends now, so the pushes and the pops balance because one
counter decides both ends, and the two backends root the same nodes because
there is one counter and not two. A zeroed frame slot per dyn slot and per
dyn-producing call, minted beside the channel and outside every scoped --
the bump allocator reclaims at the end of a statement and a slot minted in
the body would be handed out again while the collector still held its
address. Pushed from the body buffer, not the prologue's, because a call
clobbers the registers the prologue is still spilling from. And one pop in
the epilogue, which is the whole of why this backend needed no landing-pad
work for it: there is exactly one epilogue, and the return, the fall-through
and the transfer exit all arrive at it. emit.ml needs the same pop at five
separate rets.
The ABI point the dyn handoff left open for the integrator is settled by
reading the other side rather than by agreeing: flan_dyn.c's mark follows a
value only when the quiet-NaN prefix is set, and the zero word does not have
it, so a zeroed root decodes as the double 0.0 and is never an address
anything dereferences. Zero is safe for a reason. The header says so now.
And one line in dev.ml that was never x86's: the merged dev host resets the
condition stacks and the frame chain between runs, because main is
re-entered by longjmp and pops no frame -- and it never reset the root
stack, so every root a finished run pushed still named stack the next run
was about to write over. That gap was an LLVM dev build's too.
Verification, and one of the numbers is new. @x86: MATCH 129 -> 135, DIFFER
0, REFUSED 0 -- the five dyn programs off survey.sh's llvmonly list, which
is gone rather than empty, plus p13. dune test --force green, with --x86
acceptance rows beside the LLVM ones for all five dyn programs, dyn-boundary
asserted on the same exit 134 and the same sentence on both.
p13-dyn-collect.flan is the one that is not a formality. Nothing else in
this repository allocates past flan_dyn.c's one-megabyte floor, so nothing
else collects even once, so a program whose roots are entirely wrong passes
every output test there is -- the handoff wrote that about the stub and it
outlived the stub. p13 allocates several megabytes of garbage while holding
live values across it: at forty times the corpus size it peaks at 4MB of
RSS, which is the collector running many times over, and both backends
still print the same four lines.
Two decisions in this lane went against the brief and both are written down
where the next reader will meet them: the return slot stayed mandatory, so the
third state ret = None was to grow does not exist and neither does the fallout
listed for load.ml, shim.ml and cimport.ml; and the parameter rule is resolved in
Check rather than in parse.ml, because cimport passes C type names through
verbatim and POSIX's lowercase stat and timespec are writable in parameter
position, which is what makes a syntactic rule unsound rather than merely
awkward.
The open ABI point is in flan_dyn.h beside the root functions rather than only
in the handoff, because the header is what the two sides diff. A rooted slot
holding 0 is not a value: the compiler zeroes every root at entry because the
push happens before the code that fills it and possibly for a branch that never
runs, and 0 is the only pattern it can write without knowing the encoding. If
the real runtime NaN-boxes and integer zero is the zero word then this is wrong
and both sides change together.
Session.compatible needed nothing: it compares with Types.equal over the
parameters and the return, and dyn is equal to itself and to nothing else. Both
directions are pinned anyway, because this is the one place "changes signature"
covers a change the source does not spell out -- a parameter can become dyn, or
stop being dyn, by a type being declared elsewhere in the program.
@x86 128 match 0 differ 0 refused, @sanitize clean, dune test green.
Two claims corrected against the thing they claimed about.
A named object is linked whole -- symbol-driven selection is an archive rule,
and dropping unreached code inside an included object needs -ffunction-sections
and --gc-sections, which the link line does not pass. nm on any corpus program
finds flan_dyn_add and flan_gc_collect in it. So the file said something the
build does not do. What makes --no-gc possible is the other half of the same
argument and was already written beside it: nothing refers to flan_dyn.c, so
not compiling it is a change at three sites and nowhere else.
And the boundary. "Typed Flan has no implicit widening" is true of a value and
not of a literal: (g 1) against (defn g [x f64] ...) compiles, because the
checker gives the literal the type the parameter asks for, while (defn h [y
i64] f64 (g y)) is refused. A dyn value written as 1 has been through
flan_dyn_from_i64 and cannot remember, so the same source read as dyn traps
where read as typed it does not. Stated where the compiler lane will find it,
with the two ways to close it, both of them the compiler's.
The boundary and the operators, which are the two halves of dyn being a type
rather than a word the checker tolerates.
Typed to dyn is implicit and dyn to typed is not, and the asymmetry is the
design: boxing loses nothing and can happen wherever a dyn is wanted, while
unboxing can fail at run time on a value the compiler cannot inspect, so it
happens only where somebody wrote a type. Both go through expect, because
expect is already the one place a wanted type meets a produced one, and every
annotating site already calls it.
Literals take their width from the dyn, not from the default. (defvar x dyn 5)
holds an i64 five: the ABI carries one integer width, so the defaulting question
never arises, and the literal is built at i64 rather than boxed after defaulting
to i32 -- which also means 3000000000 is a dyn integer.
An operator with one dyn operand is the runtime's. binary has already checked
the second operand against the first, so a mixed pair arrives with the typed
side boxed and the fold only has to call flan_dyn_add instead of adding. The
comparisons answer bool and not a dyn holding one, because a comparison is
almost always the test of an if; a program that wants it as a value boxes it
again for free at that boundary. = and != never trap -- two values of unrelated
types are unequal, not an error -- and the orderings do.
Types.equal had no Dyn case, so dyn was equal to nothing including itself.
print hands the whole value to the runtime rather than walking it: every other
arm of the structural printer exists because a Flan value carries no header and
only the compiler knows what it is, and a dyn is the exact reverse.
The compiler carries the dyn runtime the way it already carries flan_rt.c, with
the header pasted in front of the stub so there is one self-contained
translation unit and one contract.
Milestone 1 of dynamic-by-default, the runtime half: NaN-boxed values in one
machine word, a mark-sweep heap, and the operations over them.
A double is itself, which is what a language with a physics loop and a float
calculator in its corpus wants; everything else hides in the quiet-NaN space,
three tag bits and a 48-bit payload that is exactly an x86-64 user pointer.
The negative-NaN collision is answered by canonicalising every NaN on the way
in, which flan_rt.c had already decided was the right thing to print. An i64
past the payload goes on the heap rather than becoming a 48-bit integer with a
64-bit name.
The collector is mark-sweep and nothing else -- no generation, no barrier, no
free list -- because the answer to wanting it faster is to type the program.
Roots are pushed, not scanned: NaN-boxing makes a conservative guess wrong in
both directions, and flan_dev.c's frame chain is the precedent. A fixed ring
of the last sixty-four allocations is marked unconditionally, which closes the
window where an expression with two constructors in it can collect its own
first result before the compiler has rooted either.
A type mismatch traps rather than aborting, through a flan_trap exported from
flan_rt.c so it takes the same path the six existing traps take: parked for
inspection in a dev session, dead where it stands otherwise. The sentence
names the operation, both tags as words, and both values.
flan_dyn.c is its own translation unit and nothing in the release runtime
names a symbol in it, so a program with no dyn operation links no collector
and --no-gc can be file-level selection rather than an argument with the
linker.
docs/SPIKE-DYNAMIC.md carries the argument. test/dyn_ops.c drives every
operation and all twenty-four refusals from C, the way dev_limits.c does,
including a million allocations against a hundred live and the control that
says an unrooted object really is reclaimed.
The type itself, the ABI its operations call into, and the one decision the
feature could not avoid: (defn f [x y]) is one parameter or two, and which one
depends on whether y names a type.
Parse does not decide it. That lookup is the one its defn comment records being
removed for being wrong twice in one day -- the set of type names is incomplete
at parse time by construction, and macros generating definitions is what
widened the failure. So the vector is carried undecided, as Ast.pitems, and
paired in Check, after every file is loaded, every macro expanded and every
header imported. The set is complete there. It is not complete across time, and
the comment says so: a defstruct written later changes a signature with no edit
to the function.
The return slot stays mandatory and dyn is written out in it. The ambiguity
there has no syntactic resolution at all -- a capitalised head in a list is both
a type application and a struct literal -- so the third state the parameters
needed does not exist for the return type, and ret = None goes on meaning Unit.
What the feature costs, and what is taken back: a slot with no type used to be a
syntax error, so a mistyped type now reads as an extra parameter with no
diagnostic. A name within one edit of a type's gets the resolver's own
did-you-mean, and an unknown capitalised name is reported as the unknown type it
is -- not one parameter in the corpus is capitalised. A lowercase name
resembling no type is the feature working, and is the residual.
The x86 backend refuses dyn by name; both callers already name --llvm, and the
daemon takes that backend by default, so this is the first thing a user of dyn
sees. The JS dialect refuses it too, for the opposite reason -- every value
there is already dynamic and what is missing is only the lowering.
runtime/flan_dyn.h is the fixed ABI. flan_dyn_stub.c stands in until the real
collector lands and says in its header that it verifies nothing about roots.
The !-means-mutates convention distinguished nothing — there is no
immutable counterpart to contrast with — so every mutating name drops
the mark: sort, sort-by, sort-bytes, swap, reverse, append, append-i64,
append-f64, encode-rune, split-next, map-remove, map-next, and the test
helpers beside them. Two could not simply shed it: map! is map-in-place,
because map is the into transform's word and means the non-mutating
thing; put! is put-at, because put is the Map builtin. The ?-means-asks
convention stays. Dated records keep the old spellings; watch.clj's
reset-spies! and the other Clojure names are not ours to rename.
Six refusals in the runtime called _exit(134) where every other error had
learned to park: no restart by that name, a restart taken with the wrong
arguments or with none, a defer that invoked one, a null allocator, and
free-all on something with no region. Under a merged flan dev the compiler is
in that process, so a program that named a restart nobody established took the
session down with it, which is the one thing the break loop exists to prevent.
They park now. Not through flan_break_hook, which is what bounds and
arithmetic use: that hook may answer by aiming a transfer channel, and these
six are called by emitted code that falls off the end with no channel anywhere
in the call, so a restart chosen against one would be accepted and dropped.
flan_trap_hook says the other thing instead — stop here, let everything be
read, and refuse the resume with a reason.
All six park, for two reasons rather than one. Four are guards that fire
before the operation they guard, so nothing is half done and the frame reads
like any other. The other two fire mid-transfer, with the frame's defers
possibly half run, and they park only to be looked at: stopping on a torn
unwind is strictly more than exiting before anyone can ask what tore it.
The break loop grew a per-snapshot resumable flag for it. Restarts are still
listed and still numbered, the terminal marks them untakeable and the socket
reports the same positions as unreachable, and the listener refuses a choice
with the trap's own sentence rather than the thunk-boundary one.
Standalone builds die exactly as they did: nothing installs the hook in a
program that did not import the agent, and the acceptance case for free-all
still wants exit 134 and the same message.
The review entry that asked for this named flan_exit_hook, which is normal
termination and not this at all; it is struck out with the correction.
dev-chatty.flan outlives the surveys' twenty seconds by design, and unlike
dev-repl it prints while it does -- so the two backends stop at different
lines and the diff reports on scheduling rather than on lowering. It joins
dev-loop and dev-watch in the excluded-by-name list in both sweeps, with the
distinction written down.
rt_flush_out is guarded on __wasm__: the pipe it is careful about belongs to a
merged flan dev, which is only ever a native host, and wasm32 need not answer
for a descriptor mode its runtime may model differently.
And three comments that went false with the _exit: the atexit registration in
the merged entry point is no longer there for rt_die, which unlinks the socket
for itself now, so both places that said so say what it is actually left
covering.
Retrying immediately looked like eight chances and was one: a walk that bails
at the epoch check costs almost nothing, so all eight fit inside the single
compaction they were all losing to, and the listing refused having waited for
nothing. A quarter of a millisecond between attempts -- the agent's break-loop
idiom, legal here because the waiter is the listener thread and never the game
loop -- bounds the whole refusal at two milliseconds. Measured with a writer
noting and freeing on top of three thousand live blocks: 8 right answers in 200
without the pause, 200 in 200 with it. It is not magic, and the comment says
so: a writer that spends most of its time rearranging the table still gets
refused, which is the honest answer and used to be a zero-row lie.
The two cases the last commit left unwired are wired now, and a third joins
them: a listing taken while the table really is being compacted, which nothing
covered -- the full-of-live case never compacts and the churn case is
single-threaded, so the retry itself was exercised by nothing. It asserts only
what a slower machine cannot change: never zero rows, never a count that is
neither right nor a refusal. How the rest divides is printed, not pinned.
(/ 0.0 0.0) printed nan through LLVM, which folds it at compile time to
the positive quiet NaN, and -nan through x86, where divsd computes the
negative one. Put the operands in globals so nothing folds and both say
-nan, so the divergence is the folding path and not the arithmetic.
The sign bit of a NaN is not a property of the number and IEEE 754 does
not specify it, so the print site is where this is answered.
flan_f64_to_bytes renders any NaN as nan, and the two dev emitters do
the same. That is not a new rule: format-f64 in the prelude has always
answered nan for this value, so a build where (print x) said -nan and
(show x 2) said nan was contradicting itself inside one backend. An
infinity still prints signed.
format.flan prints the three non-finite values through print as well as
through show. It is in the survey corpus, so the one program pins the
printed form under dune test and the agreement between backends under
the survey.
The registry compacted whenever the table was three quarters full, and a
compaction reclaims dead entries and nothing else. A program holding more than
three quarters of the table in live blocks therefore compacted on every
allocation for the rest of its life, reclaiming nothing each time and holding
the table-wide epoch odd while it did. A listing racing that loop lost all
eight of its attempts and answered with zero rows -- "nothing is held", about a
program holding three thousand blocks, from the verb that exists to find a
leak. Measured at 199 wrong answers in 200.
The trigger now also asks whether there is an eighth of a table's worth of dead
to reclaim, which is a count four places maintain: a death, an arena's
free-all, a note written over a dead slot, and the sweep itself. That bounds
the cost from the other side too, since a sweep that runs reclaims at least 512
slots and so cannot run twice in 512 allocations.
Separately, flan_dev_reg_by_type answered a walk it could not take with zero
rows, which is the same number a program that had freed everything gets, and
stepped past slots flan_reg_snap could not copy while still calling the walk
whole. It now counts those slots and returns -1 with the count, the agent
refuses in a sentence the daemon already renders, and the snap contract says
which caller keeps it and why reg_at is allowed not to.
A note that finds no slot is still dropped -- dying because a diagnostic ran out
of room would be the diagnostic shooting the patient -- and now says so on
stderr once, quoting how many entries were dead rather than claiming the table
is all live.
test/dev_limits.c gains three modes, driven from test_reload: 3100 live blocks
read under a writer thread (1 right in 200 before, 200 after), 3000 live with
600 churned on top of them to prove the sweep still runs, and a genuinely full
table that must say so exactly once.
The lo <= hi test in check_slice and slice-from-ptr's n >= 0 sat behind
--no-bounds-checks in both backends, while the comment beside each said
they could not be dropped. They are not bounds checks: hi <= len asks
whether a range fits inside a length, and lo <= hi asks whether the word
about to be written into a %slice's length field is a count at all. The
first stays behind the flag, the second is now emitted everywhere, the
way flan_vec_as_slice has always validated its own l > h in plain C.
emit.ml emits two signal blocks rather than one and i1, so an unchecked
build carries one compare. x86.ml keeps all three frame temporaries
stored outside the flag and gates only the second compare, because the
third is the length the message prints.
The IR assertion in test_acceptance now says the two slice calls are
present under --no-bounds-checks rather than absent, and the same build
is run: case 2 and case -2 of bounds.flan must still die.
rt_die began with fflush(stdout) and ended with exit(134), and under a merged
flan dev both could hang. The flush waits on a 64K pipe whose reader is busy;
exit runs the atexit chain and the ELF destructors, which want the loader lock
a dlopening listener thread may be holding, and in that build the chain also
holds OCaml's shutdown. A bounds failure that hangs is worse than one that
kills the process, because nothing is printed and nothing exits.
So the flush every trap does first goes through rt_flush_out, which puts fd 1
into non-blocking mode and accepts a truncated tail, and rt_die takes _exit
and unlinks FLAN_DEV_SOCK by hand -- the same shape as die_now in
flan_agent.c, which argued all of this first and is quoted rather than
diverged from. On a terminal, a file, or a pipe with room, O_NONBLOCK changes
nothing, and the acceptance corpus diffs that output.
One consequence worth knowing: _exit skips ASan's exit-time leak report, so a
trapping program under @sanitize no longer reports leaks.
Closing the window of a printing program left the pipe full, and the flushes
on the way out blocked on a compiler thread that was busy answering the very
request asking to run it again. program_state still said RUNNING, so rerun
refused a finished program as "the program is already running".
Two flushes moved. flan_exit no longer flushes before the hook -- exit(3)
flushes for itself, so that call only ever ran on the merged path, and there
it was the first thing to block. flan_merged_park flips the state under the
lock first and flushes after, because a state flip is two stores and cannot
wait on anything.
What widens is the window in which the program is parked and not yet on the
condvar. Nothing is lost there: program_asked is the fact and a signal
delivered to nobody is discarded. Two re-runs in that window are both
answered ok for one run, which is the trade against a refusal that was false.
Every number-to-text conversion wrote into one file-static in the runtime and
answered a slice over it, and nothing copied. Two of them in one expression
printed the second number twice — no crash, no diagnostic, and nothing a
sanitizer could find, because every byte read was inside an object that was
alive. The wrong object.
The buffer is now the caller's, one frame slot per call site. The slot is
allocated in the checker rather than in either backend: a slot is a
function-lifetime location in both of them, where an x86 backend temporary is
bump-allocated and reclaimed at the end of the expression that made it — which
is the one lifetime a returned slice must outlive. Each backend gains one
pointer argument and no reasoning of its own, which is what keeps them
symmetric.
The static is gone rather than left unused, since a buffer with nothing but a
comment beside it is a loaded gun. What remains is the ordinary lifetime a
pointer into a frame has: storing one of these slices in a container that
outlives the frame, or returning it, is still a copy the caller has to make.
NEXT.md's sharp edge now says that instead of what it used to say.
The writer is the game thread, in every allocation and every free; the reader
is the agent's listener, on a program that is running. Nothing stood between
them. The consequence is not a slightly wrong count: a row's type pointer and
its length mean nothing apart, and a reader that takes the new pointer with the
old length reads off the end of a string literal.
Each entry now carries the watch table's seqlock, odd while it is written, and
a reader copies the entry and re-reads the counter before believing it. The
compaction bumps a table-wide counter around itself, because it moves entries
between slots and no per-slot counter can describe that; a scan that sees that
counter move walks again. It clears the table slot by slot rather than with one
memset, since the memset would zero the counters a reader was holding.
The breakdown and the leak report stay answerable while the program runs, which
is the moment they are for. reg at does not: whether one address is still live
is exactly what a running program is changing, so it is refused the way every
break verb is refused, which is what the daemon already did on its own side.
(map-remove! m k) answers the value that was there, or None, which is the
answer get already gives and for the same reason: a key that is not in the map
is an answer, not a failure. Handing the value back rather than dropping it
makes "take this out and use it" one call instead of two that hash the key
twice.
The removal shifts the probe run back over the hole. A Robin Hood lookup stops
at the first empty slot, so a hole left in the middle of a run hides every
entry after it — and the hidden ones are precisely what a test that only asks
after what it removed never looks at, which is why the program removes a
thousand of two thousand keys and then asks for the other thousand.
Odin was read rather than recalled here, and it does the opposite: its erase
marks a tombstone and its insert carries the repair loop. Staying tombstone-
free keeps the shape the rest of the file already assumed, and the lookups —
which outnumber the removals — pay nothing for it. The note in the runtime and
the two in BUILT.md that said Odin deletes by backward shift were describing
Odin's insert, and now say which is which.
It allocates nothing and releases nothing, so there is no guard around it and
it means the same thing on a map in an arena as on one in the heap: a key and a
value live inside the one block the map allocated, and there was never anything
per entry to hand back.
The argument vector's malloc was unchecked, and a failure there would have
published a null pointer with a length beside it. It now dies naming what it
was building, because argv has no allocation site for a condition to hang on.
flan_slurp_into read a capacity of elements as a capacity of bytes and skipped
the epoch check every other container operation runs. The element size is now
a parameter and the length it publishes counts whole elements, so the day slurp
answers something other than (Vec u8) it does not answer with bytes nobody
wrote.
A string with a NUL in it is refused at the C boundary, which is the policy
flan_path_cstr has always had for a path: C reads to the first NUL, so what
crosses is a prefix of what was passed, and a window title is no different from
a filename in that respect. The refusal names the declare-c, which is the name
the program's author wrote.
The runtime's two translation units are compiled with -Wall -Wextra. They were
already clean under both; the flag is there so the next one is caught rather
than read.
The generation word keeps its place and loses its "yet": a reader for it is a
third word on every slice in the language, which is a spec amendment rather
than a runtime patch, and the comment now says so where someone deciding to
trust the word would read it.
Five more: file-exists?, file-size, delete-file, rename-file and
make-directory. The interesting thing is not the list, it is the line drawn
through it.
file-exists? and file-size answer a value -- a bool and an (Option i64) -- and
are prelude functions over one declare that the compiler knows nothing about.
Absence is the reply to those two questions and not a fault, so a condition
would make the ordinary case pay for a handler search, and there is no restart
a handler could take that would turn "it is not there" into a different
answer.
delete-file, rename-file and make-directory answer () and signal FileError,
and they are check.ml builtins for the one thing a declare cannot do: they go
through file_guard, so each failure arrives under retry and use-value. Those
are restarts a handler really can take -- make the parent directory and retry,
or supply another path -- which is exactly the case a bool return throws away.
op continues the prelude's numbering as 2, 3 and 4.
One C function behind the two questions rather than two, because they are one
question: stat answers whether the path resolves and how big it is in the same
breath. It is stat and not flan_file_size's fopen-plus-ftell, which is shaped
by slurp being about to read the file and is wrong as a general size -- fopen
on a directory succeeds on Linux and ftell then answers a number that is not a
file size. The two coexist and answer different questions.
rename holds the source in the guard's path slot, so a use-value renames a
different file to the same destination. Both readings are plausible until
somebody says which, so check.ml says which.
The errno mapping is not extended. Its three buckets are what a handler can
act on; EEXIST and ENOTEMPTY land in io with everything else, and that is
honest until conditions have a hierarchy to hang a fourth reason off.
All three carry barf's decision 2 unchanged: they change the filesystem, so on
the web they signal rather than succeeding quietly into a filesystem the page
throws away.
Not here, and not half-parsed either: a directory listing, which needs an
allocating builtin and a Vec of owned strings, and streaming IO. Neither has
a name to trip over.
programs/files.flan makes and removes its own tree and takes both restarts on
operations that write. The runtime additions continue the block at the end of
flan_rt.c.
Nothing in it could. A game got a clock from raylib and a program without a
window had none at all, so "how long did that take" was unanswerable in the
half of daily use that is a tool rather than a game.
Two clocks, because the mistake a single one invites is using it for the other
job. monotonic-ns measures: it never goes backwards, nothing adjusts it, and
its zero is arbitrary, so it is meaningless alone and correct as a difference.
unix-ns dates: nanoseconds since 1970, which is what goes in a save file, and
which jumps in either direction when somebody sets the system clock. The names
are picked so that reaching for the wrong one reads wrong.
This is Odin's shape, from core/time/time.odin and core/time/time_linux.odin:
Tick against Time, both an i64 of nanoseconds, over MONOTONIC and REALTIME,
with the seconds-valued face derived rather than a second syscall. Three C
functions here and six Flan names over them, which is the rule flan_rt.c's own
header states -- a primitive is the only thing implemented twice.
The monotonic origin is the first read of the clock in the process, not boot,
and that is the one decision worth arguing. CLOCK_MONOTONIC counts from boot,
so on a machine up a hundred days the raw value is past 2^53 nanoseconds and
monotonic-seconds would lose sub-microsecond resolution depending on the
machine's uptime rather than on anything the program did. Latched to first
read it stays integer-exact for a hundred days of process life, and it also
matches what a game already has: raylib's GetTime is seconds since
InitWindow, so the two numbers now mix without a conversion at every site.
sleep-ns loops on EINTR, because otherwise a signal cuts the wait short and a
frame loop wobbles for reasons nothing in the program explains. It is
documented as at-least and not as a frame limiter; the shape that actually
paces a loop is a deadline recomputed from monotonic-ns each turn, and the
comment says so where somebody will read it.
getenv answers an (Option [u8]) viewing the process environment, which needs
no allocator and no free and is safe precisely because nothing in this
language can call setenv or spawn a process. The absent case rides in the
length rather than in the pointer: there is no null test to write, since a
(Ptr T) here always addresses something, so flan_getenv answers -1 and a
pointer at a valid empty string and the Flan side tests arithmetic.
The runtime additions are a single block at the end of flan_rt.c, with
<time.h> inside it for the reason <errno.h> sits beside the file section.
programs/time.flan asserts invariants and never a reading -- t2 >= t1, a sleep
that did not return early, a date after 2020 and before 2100 -- because the
same file is in the corpus @x86 builds twice and diffs, so a timestamp would
fail a correct compiler on its second run.
Every size the containers compute is a product of a capacity the program chose
and an element size the checker did, and a product that wraps leaves a block
that fits beside a capacity that does not. The next write goes past the end of
an allocation a sanitizer was told to expect, which is the one corruption
nothing in the suite could have found.
The Vec's growth, the Pool's two blocks and their sum, the map's five runs and
the budget check now go through checked arithmetic. A size with no
representation reports along the path an out-of-memory already takes, with the
largest number the condition's field can hold, since the true one has none.
The test pins the case the guard exists for: an element of 2^33 + 1 bytes at a
capacity of 2^31 wraps to 2 GiB, which a heap allocator answers.
A (Vec Value) where a Value may itself hold a (Vec Value) — the recursive
dynamic value an EDN reader has to answer with when nobody hands it a target
struct type — was refused five different ways, and every one of the five gave
the same reason: the container runtime is type-erased, so it copies and
releases slots bytewise and cannot reach inside a slot. A free would release
the slots and leave every block they point at stranded.
That reason is about teardown, and it does not hold for a region. free-all
never releases an individual slot; it takes the whole arena, and every block
the elements own is in it, because they came out of it. The refusals were
over-broad, and what they were guarding was never ownership — ownership
tracking is untouched here, moves are still moves, and Types.is_move_only is
the same function it was.
So the question moved rather than disappeared. It could not stay at the type,
because can-free is a capability on an allocator value and with-allocator
rebinds a dynamic variable: which tier a (vec-new) will meet is not a property
of the place its type is written. What is decided at compile time is only
whether to ask, which is a property of the element type; the answer is a
run-time branch on the allocator, one per container and never per element,
because the alternative is a walk at release and a walk at release is the
registry of destructors the frame tier's reset exists to not have. It is
emitted at every growth and not only at the construction, because ZII means a
container can exist without ever passing through (vec-new) — a case field left
out of a literal, a global that starts zeroed — and those adopt the context on
their first push.
free on such a container is refused rather than made quietly shallow. It cannot
recurse, which is the whole premise, and releasing the outer block alone would
be "I freed it" written over a program that stranded everything inside; this
runtime refuses that collapse everywhere else. The message names free-all,
which is reachable by construction. clone stays refused for a reason the region
does not dissolve, and the old message had bundled the two failures under one
sentence: what disqualifies clone is not that it copies a header — so do at and
get, and they are fine, because they promise nothing — it is that clone
allocates a new block and promises independence, and a bytewise copy hands back
elements still pointing into the original's region.
A struct or union field is admitted only where the field's container holds
owning elements, because that container can only have been built against a
region. A field holding a plain (Vec u8) stays refused: nothing would force
that one into a region, and two copies of the aggregate would be two headers
over one heap block. vec-in-struct.flan still pins that.
The epoch already covered use after free-all, including the case this makes
reachable — an inner header copied out of an arena-held element into a local
still traps, because an Allocator is a pointer and a copied-by-value one would
carry its own epoch.
arena-value.flan builds the value by hand; arena-edn.flan reads a real document
through the tokenizer, and its reader takes no allocator and names none,
because spec-memory.md already puts the allocator in the calling convention.
arena-region.flan is the branch itself: run 0 is the (Vec (Vec i32)) control
that must not trap, and runs 1 and 2 are the two ways this dies.
You run a program under flan dev, it opens a raylib window, you close the
window, main returns — and there is no way to get another window short of
flan-dev-restart-program, which throws away the build, the session and every
global with it. In Common Lisp or Clojure the image outlives main, so you call
it again. The process here already outlived main: the exit hook flushed, closed
stdout and sat in for (;;) pause(). Nothing could wake it.
So main() is a loop. The hook records the status and longjmps back into a
setjmp in main() — there is no return available, since flan_exit is reached
from wherever the program happened to be — and the thread waits on a condition
variable until the new rerun op signals it. The main thread is the one that
runs main again: a window belongs to the thread that opened it, and on macOS to
the first thread of the process. A longjmp pops no frame, so the park first
empties the handler stack, the restart stack and the shadow frame chain, each
of which was a chain of allocas in stack the next run is about to write over.
Nothing else is reset; the second run reads whatever the first left in the
globals, which is the semantics that was asked for.
Closing stdout had to go with it. That was how the compiler learned the program
was done, but a pipe delivers EOF once, so the signal and the program's output
were the same resource and spending it left the second run with nowhere to
print. The descriptor hazard the old code reopened /dev/null for goes away with
the close that caused it. Liveness is asked for instead, through a weak symbol
in the same style as the agent's, and is now three states rather than two: Live,
Parked and Gone. Every guard branches on that before consulting the break
state, because the agent's listener answers "running" while the program is
parked and telling somebody whose program has finished that it is running is
worse than saying nothing. Only eval accepts a parked program — it queues and
waits for nothing, and the queued module installs at the first frame boundary
of the next run, so a body can be fixed while parked and the re-run executes
it. Everything else needs a frame boundary or a stopped stack, has neither, and
says which, naming the command that gets the program back.
A re-run while the program is running is refused rather than queued: the test
and the signal happen under one mutex, so two mains writing the same globals at
once never starts.
:parked rides on every reply beside :stopped, for the reason :stopped does —
finishing is as unannounced as stopping, more so when the way it happens is a
mouse click on a title bar. Emacs shows flan:parked in the modeline and binds
flan-rerun to C-c C-M-x.
Three arithmetic situations had no defined behaviour and the two backends
disagreed about all three: a divide or remainder by zero, which was a raw
SIGFPE with no message and no location; (/ min -1), whose quotient is one past
the top of the type; and a float to integer cast whose value does not fit,
which LLVM called undefined and would fold to anything.
They now signal ArithError with `error`, exactly as a bad index signals
BoundsError, and die with a sentence naming the file, the line and the operands
only if nothing answered. The guards ride the same --checks flag as the bounds
check and are elided with it.
No restart is established at the failing operation. The sketch this started
from asked for use-value, and the implementation ruled it out: a restart frame
is allocated by the restart-case that offers it, on its own stack, so the
runtime cannot hold one on a program's behalf and use-value here would mean an
alloca and a restart frame at every division in every checked build. That is
the cost already refused for indexing, buying a silently different answer.
The x86 backend is unchanged and is the next commit.
Two loose ends from NEXT.md.
slice-from-ptr's run-time refusal borrowed @flan_slice_error and reported a
range and a length the caller never wrote. It has flan_slice_promise_error
now: signals BoundsError, walks the handlers, offers the break loop, falls
through to a message and a status like the two beside it. The sentence names
what was promised and what was passed, and a second line says what is not
checked. The condition fields stay (0, n, 0) — the violated condition as a
range, and not (0, n, n), which reads as in bounds.
And a session now holds the buffer's own defmacros: seeded in Session.create
from the same read that produced decls, and added by Session.eval so a
defmacro typed at the editor joins the set the way a defn does. Not a re-read
of the file, which would put unsaved-versus-saved skew inside expansion. The
commit stays below the checker. Macro.program dedupes the ambient set against
the forms being parsed, left-wins, because unqualified names can now collide.
Two loose ends.
The arena was invisible to memcheck. free-all is retain-capacity, so from
malloc's point of view nothing died and round two of a reset arena could read
a byte it never wrote, print round one's value, and draw no report.
flan_arena_proc now issues memcheck's MAKE_MEM_UNDEFINED over the whole
capacity beside its registry call. Measured on the same machine: the control
produced ERROR SUMMARY 0 before and 6 errors from 4 contexts after, with
--track-origins naming the client request. It is a control in
test_valgrind.ml now rather than a printed note.
The macro is vendored, not included, and the argument is measurement: the
machine that runs the sweep has valgrind and not valgrind-devel, so a guarded
#include would compile to nothing exactly where it matters and the control
would go quiet with no diagnostic. There is also nowhere to put an -I --
flan_rt.c is cat'd into an OCaml string literal and handed to clang in a
scratch directory. The __x86_64__ guard is load-bearing: the same runtime is
built for wasm32-wasi and emscripten.
Cost outside valgrind: 23 instructions on the free-all path only, about 1ns
per reset over fifty million of them, against a run-to-run spread wider than
the effect. Nothing on alloc, resize or free. valgrind.supp still holds no
suppressions; the corpus stayed clean across the change, which is its own
finding.
merged_serve's warning path deserved a test and has one. The discriminating
fact is not the log line but the policy: two_process kills its child and
fails where merged_serve warns and serves anyway, and nothing held that
second answer in place. dev-noagent.flan plus the last block of test_dev.ml
assert the session still answers describe after the wait runs out. Verified
by reverting the policy: the block reports rather than passing. It costs the
full ten seconds and there is no way to spend less. HANDOFF-f1.md is deleted.
The allocation registry had a recording side and half a reader. This is the
rest of the reader: point at any heap address, a breakdown by type, what is
still held, and the test that stops dev-ptr.flan's header from being read by
hand.
The recorded name, back to a type. The table records a string and has to —
the note is built where the concrete type exists and what crosses into the
runtime is bytes. What closes it is that the string is Types.to_string, which
is the source spelling, so the round trip is the language's own reader,
Parse.texpr and Check.resolve. No table of spellings is written down, so
nothing can fall behind Types.to_string, and a name that is not a type —
"pool slots" — is refused with the name quoted rather than defaulted.
The address root renders a (Ptr T) and not the pointee, which puts it through
render.ml's pointer arm: permission is asked in one place in the compiler, and
an address root and a slot root reach the same two answers by the same code.
Flan has no integer-to-pointer cast, so flan_dev_reg_addr is an extern beside
flan_agent_frame_slot, for the same reason.
One walk and two questions: a leak report is a breakdown with the dead left
out, so flan_dev_reg_by_type is one function and the agent formats it.
"At exit" is not a hook. A program killed by a signal runs no handler, which
is how a game under the editor ends, so (:op "leaks") is the authoritative
reader and can be asked at any moment including the one before the kill. The
atexit hook is for the program that returns from main, is registered from
inside flan_dev_reg_enable rather than by a file-scope destructor so that a
release build does not grow a third not-free place, and is off unless
FLAN_DEV_LEAKS is set because the acceptance table reads stderr.
The memcheck half of item 6 is deliberately not here.
The watch accumulator clears a slot lazily, on its next sample, and the reader
never compares epochs. A previous lane read that as a defect and left
watch_render_num's n=0 arm behind as dead code against the day the reader
became epoch-aware.
It should not. A stopped program takes no samples, so an epoch-aware reader
would report every slot as empty for as long as the program sat in a break
loop — and reading the numbers from the moment you stopped is the whole point
of stopping. The lazy clear is the right answer there.
What was wrong was narrower and lives in the editor: flan-watch--tick sent
`:reset t` five times a second at a program that could not answer it. The read
still goes out every tick; only the reset field drops, guarded on
flan-dev--stopped, which flan-dev.el's background poll already keeps current.
The n=0 arm is deleted rather than commented, since the only way to reach it is
the epoch check just rejected and dead code is an invitation to add one. n is
at least 1 whenever watch_render_num runs, so sum/n cannot divide by zero.
test-flan-watch.el asserts both halves with no daemon: a running tick carries
:reset, a stopped one does not, both still read the table and both leave a
reply in flight. Verified by mutation. What it cannot reach is the daemon
honouring the absent field; test_dev.ml drives a real program for that.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
flan_dev_reg_dead was reaching for the containment scan, and it is on the free
path: a dev build would have paid a 4096-entry sweep per free. A free hands back
the base address the allocator gave out, which is what the slot is keyed on, so
the question there is equality and never containment. Only free-all needs the
scan, and that runs once a frame.
The table is allocated when it is armed, not declared. A fixed array was a
quarter of a megabyte of BSS in a shipped game for a table that build never
writes; now a release build carries a null pointer and the not-taken branch.
The pointer arm binds its subject to a slot before naming it three times — the
slice arm's rule, and its reason: an inspect with a path reaches a leaf through
a bounds check, and three of those to render one pointer is the walk paying for
its own shape.
dev-ptr.flan shows both halves on a stopped stack. It was read by hand; the
test_dev.ml case that would drive it is another lane's file, and NEXT.md says so.
(Ptr Enemy) already says Enemy, at compile time, in the walk. What the renderer
lacked was any way to know whether the storage at the far end is still there —
and an allocation registry is exactly a record of which addresses it is still
true to read. So the inspector follows a live one and renders the pointee by the
same walk as anything else, and names what died at a dead one.
println does not, and the split is not squeamishness: spec-memory.md fixes what
a printed Ptr prints, a printed line belongs to the program and has to read the
same in a release build, and a release build has no registry to ask. The two
callers already differ in an emitter record; they differ in one more.
No address appears in the text. An address is not stable across two runs, so
printing one would make a rendering depend on where the heap landed — the rule
Render already follows for an allocator. What a reader wants from a dangling
pointer is what died.
registry.flan is one program read twice: a dev build answers for an address at
the heap, arena and pool tiers, and a release build answers 0 to all of it. The
arena row is the free-all Valgrind cannot see — this does not make memcheck
report it, it makes the same read answerable.
The table, and the half of the wiring that needs no type name. A struct is its
C layout with no header and no tag word, so nothing at run time can say what is
at an address — and adding a tag would break the FFI. The registry sidesteps it:
the compiler knows the type at the moment memory is asked for, so the insert is
emitted, and the dead-marking is not, because an address needs no type.
Entries are blocks rather than values and lookup is containment, which is not an
optimisation: every heap pointer a program can hold is interior. (at v i) is
v->ptr + i*size and (resolve p h) is an item in the middle of a pool. Exact hits
would answer nothing anyone can ask.
Dead entries stay until the allocator hands the address out again, which is when
the old answer stops being true. An arena's free-all marks its whole range dead
— the release memcheck is never told about. That does not make memcheck report
it; it makes the inspector able to.
(Handle T) and (Pool T) land as types and as a runtime. A handle is one
int64_t — slot index low, generation high — so it copies, zeroes and
compares like the integer it is and owns nothing. A live slot's generation
is odd, which makes a zeroed handle resolve to nothing rather than to slot
zero, and makes iteration free. Wrapping retires the slot rather than
reissuing it: 2^31 reuses is rare, and rare is not an answer when the
failure is the silent wrong one the type exists to prevent.
No surface yet — the checker still has no names for any of it.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
RESULT_MAX was written down twice, once in flan_dev.c and once in the agent,
with a run-time check that the two had not drifted. That second copy was a
buffer sized to be sent through a socket, and it is gone: the agent asks
flan_dev_result_cap() and allocates, which it can do because it is never the
game thread.
The bound itself stays, and the reason is the rule everything else here is
built around. result is what the game thread writes into, from a render thunk
at a frame boundary; a growable one is the frame thread calling realloc, which
is an allocation in the one place this design exists to keep allocation out of.
It would also break the seqlock, which is a protocol about torn contents and
assumes the address it copies from neither moves nor goes away underneath the
reader. Growing on the writer's side is a use-after-free the counter cannot
see.
So it is a render budget and not a wire size, and it only looked like one
because the agent had a copy of it. Removing the bound is a redesign of the
read -- probe, allocate, re-read, validate, retry -- and belongs with moving
the read to a frame boundary.
Also: flan_agent_request answers "err path too long" where the socket does,
instead of nothing. One verb table is only worth having if the two callers
cannot be told apart.
The measurement table in BUILT.md is retaken back to back across all four
shapes, because this machine drifts by more than the thing being measured. The
internal socket was ~41us of a 21ms redefinition; --two-process is unchanged
and is not slower than the merged build in any column.
`flan dev` now builds one binary that is the compiled Flan program and holds
the whole OCaml compiler, and execs it. The program keeps main() — macOS needs
the window there — and caml_startup happens on a pthread beside it, next to the
listener flan_agent.c already starts. The editor's socket and the wire protocol
are untouched: Emacs cannot tell the difference.
Two rules are written into lib/dev.ml rather than discovered later. The game
thread must never call into OCaml, because a native thread has no safe points
and so can never be stopped by the collector — which is exactly why a frame is
never paused, and exactly what one convenient direct call would undo. And no
OCaml value may be stored in Flan memory without caml_register_global_root,
which is the way the spike's "the GC does not touch the arenas" measurement
stops being true.
The link is spelled in dev.ml out of Build's existing public pieces rather than
as a mode of Build.executable: lib/build.ml belongs to another lane this week.
It should collapse into Build once that lands.
A Flan main does not return — Emit ends it with flan_exit and an unreachable —
so in one process that call would take the compiler down with a program that
merely finished. flan_rt.c grows a hook, null in every other build, that the
merged entry point uses to flush, close stdout and park. The compiler then
learns the program is done the same way the daemon did: the pipe reads EOF.
--two-process keeps the old shape for a machine that cannot build the compiler
object, and nothing has been deleted.
The globals section attributed a frame by its slot fingerprint, which is the
wrong cut for it: a redefined body can name entirely different globals while
binding identical locals, so the check saw no change and the new body's
reference set went into the union under the old body's frame, with the frame
numbers beside an entry saying so.
So a second fingerprint. Reach.ref_fingerprint hashes the set of globals a body
names — sorted and deduplicated, because a reference set is not ordered, where
slot indices make the slot fingerprint order-sensitive on purpose — and it
travels the path the first one already cut: %fninfo, flan_dev_frame_refsig, the
agent's snapshot, the backtrace line, Dev.globals_op. Different means the frame
is skipped by name with its reason, and the rest of the stack still contributes.
Two numbers rather than one, because they are two facts. A frame whose slots
match and whose globals do not has locals that are perfectly readable and
attribution that is not, and a combined hash would make locals refuse a frame
with nothing wrong with it. locals still checks the slot fingerprint alone.
It lives in reach.ml because expr_refs is already the walk that answers what a
body refers to, and is the walk the union itself is built from. One consequence:
emit now reaches reach, which closes a cycle through Load if cimport calls
Build.cachedir, so the header cache spells the object cache directory itself.
test_dev.ml drives the exact case — a body that binds identical locals and names
untouched where the stopped frame names pressure. With the check disabled it
fails twice: the missing refusal, and untouched appearing under frame 0.
Measured rather than guessed, and the guesses were wrong twice: the
per-slot cell division and the block-size divisions were each replaced
first, and neither moved the number. A profile named the four that did.
The hash was FNV one byte at a time, a serial multiply chain per byte
and a quarter of the operation. It is eight bytes at a time now, and a
key that is one machine word — every integer, every enum, every bool,
so very nearly every key — is one load and one mix with no loop at all.
This is where "the hash is compiled concretely per key type" stops
describing the arrangement and starts being the reason it is quick.
Equality on eight bytes was a call into libc's vectorised memcmp, an
eighth of the operation, and copying a value out was a call into
memmove. Both are a load and a compare now for the sizes that are one
word.
The block geometry was recomputed five times over inside one function,
and that function ran twice per lookup — once in the probe and once
again in get. It is one struct built once and handed back. The seed was
a five-multiply avalanche on the critical path of every probe, for
mixing the hasher does again immediately afterwards; one multiply is
all it has to do. And 64/size is a table, which is Odin's Map_Cell_Info
by another route — Odin precomputes it per type because the probe loop
must not divide, and the sizes reach this runtime as plain arguments.
Numbers, on this machine, i64 to i64, against CPython 3.13's dict on
the same workload. Cache-resident, 10k entries, 10M lookups: 21ns
against 132ns, so about six times quicker. That is the answer to "is
this another Python dict", and it is the one the design predicted.
At a million entries it loses, 1.41s to 1.16s, and that is worth
writing down rather than leaving out. Both are waiting on memory there,
and this layout waits longer: keys, values and hashes are three
separate runs, so a lookup that misses everything takes three cache
misses where a compact dict takes two, and the hash run is a full eight
bytes a slot. The layout buys probe locality, which is a win while the
hash run is resident and a loss once nothing is.
