check_constants makes two kinds of finding and they were treated alike.
A value that does not match, or a C name the header does not have, is
the library contradicting the package and stops a build the way a
permuted defstruct does. An enum nobody mapped and a rule that reaches
nothing are about the package's own bindings file -- real, and worth
fixing, but telling a lane that added a defenum to go and edit a config
in a message shaped like "your layout is wrong" is the wrong thing to
fail a build with. Those gate generate-c, where that file is edited.
Also: a const prefix now counts as reaching a name before an explicit
constant line is consulted, so a rule whose every match is also spelled
out by hand is not reported as matching nothing.
Two gaps the raylib examples hit.
The layout check compared a Flan enum against the header's `int` and
called it a disagreement. It is not one: Shim.cty lowers a defenum to
int32_t in a struct field exactly as it does in a parameter, which is
what the signature check already knew and the layout check did not. One
predicate now serves both, symmetric, and tolerant of a 32-bit integer
and nothing else -- f64 against the library's float still fails, in the
very struct whose other field is an enum. Camera3D.projection is a
CameraProjection again and rl/camera-projection is gone with it, so
`.projection :perspective` resolves at the construction site.
And generate-c's claim said nothing about a defconst or a defenum
member, so a wrong flag bit was completely silent. `bindings` gained
`enum`, `const` and `constant` lines saying what a Flan constant is
called in C -- the prefix is nowhere in the Flan name, so it is declared
rather than guessed. Nothing goes quiet in either direction: a name the
rule builds and the header lacks is reported, a rule that reaches
nothing is reported, and a defenum with no line is itself a finding,
because otherwise the silence just moves up one level.
clang's dump gives anonymous EnumDecls for every raylib enum and no
value at all for an enumerator written without `= n`, so the constants
are one flat table and the values are counted the way C counts them.
cache_format bumped with the dump type.
Build.cachedir sat under TMPDIR, which dune makes private per run, so no
test run ever reused an object and every build in the suite was cold. It
moves to $XDG_CACHE_HOME/flan/objcache (FLAN_CACHE_DIR overrides), which
is safe because the keys are total: compile_c digests the source text,
the compiler's stamp and every flag; wasm_resource_dir digests the
builtins archive; compiler_object digests flan.cmxa and flan.a. Writes
were already .tmp-then-rename, so concurrent dune jobs are fine.
Macro.key was the one key that was not total -- prelude text plus the
call's forms, and nothing about the compiler whose codegen produced the
.so it names, which is dlopened straight back into this binary. Under a
per-run TMPDIR that never showed; under a durable cache it is a stale
expander that crashes rather than a compile error. It carries the
compiler's stamp now, handed across start_merged's exec in
FLAN_COMPILER_STAMP because a merged dev binary lives at a per-session
path and keying on that rebuilt a macro module every dev start.
Measured on dev-repl.flan, launch to bound socket: 2.0s cold against
0.48s warm. Whole-program flan build: 1.44s against 0.06s. Full dune
test 25.7s/30.1s before, 24.0s after, user CPU ~50s down to ~34s.
And the await: one timer covered two waits, a build then a bind, so
'the daemon never listened' was a wrong diagnosis of a build that had
not finished. listening now polls the process alongside the socket and
says which -- exited with a status, or still running and therefore still
building. A daemon that dies fails in milliseconds instead of costing
the whole timeout. Thirty seconds, down from a minute, because the build
it waits on is warm now.
Two follow-ups to the marking commit.
`Dev.eval_expr`'s new wait matched `Stopped _`, which fires on the first
iteration when the program is already parked on something else — the
break loop allows evaluating, so that is reachable — and answers for a
thunk that has not run yet, on a reply whose own `:condition` names the
other condition. It now waits for `Stopped "Pause"`, which the agent
reports under a nested break because `condition_name` is overwritten on
the way in and restored on the way out. `dev-pause.flan` grows a
`Missing` and a `boom` so the test can park the program on something
else first and tell the two apart.
And the flake NEXT.md had as "seen once and unexplained": `the daemon
never listened` is not a race, it is an llc-and-link of the whole
program before `flan dev` binds — ~600ms idle, measured at 6.6s and 6.8s
with the rest of the suite beside it, against a 5s and 8s await. All
three test binaries now wait a minute; the watchdog is what bounds the
run. Two consecutive full runs green.
Finishes DISCUSS.md §9's `pause` marking: the daemon half was already
built, this is the editor half plus the one daemon path it was missing.
`C-u C-c C-c` marks the form point is inside, `C-u C-u C-c C-c` the
top-level form (stop on entry), `C-u C-x C-e` the expression before
point. The buffer is never edited — the position rides beside the code
and the `(pause)` call goes into the tree after parsing, so no source
location moves.
`C-x C-e`'s path needed the daemon: its 5s `wait` answered "the program
did not reach a frame boundary", which is exactly what a thunk parked at
a breakpoint looks like from out here. `wait` is now three-way and asks
`state t = Stopped` only when a pause was requested, so the no-pause
shape `test_dev.ml` pins is unchanged.
The overlay is an annotation and not feedback, so unlike an error marker
it survives `pre-command-hook`; what takes it down is an accepted
evaluation with no `:pause` on it, which is the same thing that takes
the mark itself down.
Tests: a `test_dev.ml` block over the new `dev-pause.flan` that marks,
stops, re-evaluates plainly and then polls half a second confirming it
does not stop again — one sample after `continue` proves nothing, the
resumed frame is still in the old body — and an `emacs/test-flan-dev.el`
block for which form a prefix picks, the byte column, the overlay's
lifetime, and one live round trip.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
C-u before an eval marks a form so the program stops when it runs
(DISCUSS.md 9). The mark arrives as a position in a separate :pause field
and is applied to the Ast after parsing: splicing text into the source
would move every line and column after it, and the error overlays, the
layout, the break loop's frame locations and DWARF all read those.
Ast.mark_pause puts a (pause) call at whatever starts at that position --
wrapping a sub-expression in a do, or going to the front of a defn's body,
since a declaration cannot be wrapped. A position that matches nothing is
refused rather than installed unmarked, which would report a breakpoint
that is not there.
It sticks with no extra state: the marked declaration is what goes into
the session, so an ordinary C-c C-c over the same form clears it.
The daemon half only; the Emacs command and its overlay are not built.
HANDOFF-f2.md has the rest, in order.
INCOMPLETE AND NOT WIRED IN. lib/x86.ml is not in lib/dune, so nothing
compiles it and nothing calls it; `dune test --root . -j 1` was green at
the tip this branched from and is unaffected, because no file the build
reads was changed. The module itself has never been type-checked.
What is here: the instruction encoder (integer and SSE, loads and stores
at every width, division, shifts, setcc, rip-relative addressing, rep
movsb), the layout bridge to Emit.lay, the frame allocator, and the
.rodata constant emitters. What is not here: the expression lowering,
the call sequence, the function prologue and epilogue, the assembly file
assembly, the build.ml flag and the differential harness. The header
comment is the design; the second half of the file is missing.
THE INTERNAL CONVENTION, which is the decision hardest to recover from
the code, and which is chosen rather than inherited:
- Scalars -- integers, bool, ptr, enum, handle, allocator, Fn -- in
SysV's integer registers rdi rsi rdx rcx r8 r9, then right to left
on the stack. bool is one byte, zero-extended on load.
- Floats in xmm0-xmm7, then on the stack.
- EVERY aggregate by pointer. An argument is a pointer to a copy the
caller made; a return is a hidden sret pointer in the FIRST integer
register with every other argument shifted along, and that same
pointer comes back in rax. Nothing is classified, nothing is split
across register classes, there is no eightbyte rule.
- The transfer channel is the last argument of all, a pointer, in the
integer sequence -- emit.ml's `signature` rule, unchanged. It is a
pointer to a pointer: main allocates one cell, stores null, and
threads its address down; a callee that transfers stores non-null
into it and every caller loads, tests and branches to its pad.
- Frame: every intermediate value is a frame temporary, bump-allocated
below rbp with a high-water mark, and the outgoing-argument area is
reserved once in the prologue. rsp is written exactly twice, by the
prologue's sub and by leave. So rsp % 16 == 0 at every call site is
a property of one rounded sub, and the spike's depth counter is not
needed -- its bug class is removed rather than guarded against.
WHY THE CONVENTION IS OURS TO PICK, confirmed rather than assumed: a dev
build compiled by this backend never emits a .ll at all, and a release
build never runs this backend, so no process holds code from both. The
only boundary that must match SysV exactly is C, and check.ml rejects an
aggregate in a `declare` while the generated shim flattens every struct,
so no Flan-emitted call ever hands C an aggregate. I found no path that
mixes the two backends in one process. I did NOT get far enough to test
that claim by running anything, so it stands on reading build.ml's
`executable` and emit.ml's `signature`, not on an experiment.
WHAT THE MEASUREMENT SAYS, and it is the one new fact this branch has.
spike/backend/hist.ml histograms Tast nodes over a program after Reach
prunes it. Item 15's four buckets undercount what a whole-program build
must do on day one:
- enum-compare.flan needs Str, Make, Field and Call before it prints
anything, because the prelude builds a slice to print one. Aggregates
are not a later row; they are in the first program.
- loops.flan carries Handled, RestartCase and Signal one each. The
"no plan" row is in the reachable set of a program that only loops,
so conditions cannot be deferred behind a whole-program flag.
- The text primitives (Bytes, I64ToBytes, WriteStdout) are C calls,
not instruction work, so they are cheap.
WHAT THE NEXT PERSON SHOULD DO FIRST, in order:
1. Finish the lowering as destination-driven: `eval f e ~dst` writes
e's value into [rbp+dst] and nothing is ever live in a register
across a statement. That is what makes aggregates and scalars one
code path and what keeps the frame model's promise.
2. Emit an assembly file -- .byte blobs with `call sym` and
`.long lbl - . - 4` for the few relocated fields -- and add the
flag to build.ml as FLAN_X86 plus an `opts` field, off by default.
Do not write an ELF writer; it produces no Flan progress and a bug
in it looks exactly like an encoding bug.
3. Copy test/test_sanitize.ml's shape for the differential harness.
There is no differential run yet, so nothing about correctness has
been demonstrated on this branch.
4. Bounds checks are implementable and should not be skipped:
flan_bounds_error(ptr, i64, i64, i64, ptr) and flan_slice_error
take the transfer channel, so they are an ordinary guarded call.
THE TWO LANGUAGE PREREQUISITES, unchanged and still not decided here.
Uninit is the one that bites: this backend gives whatever the stack slot
held, LLVM may reason from poison, and that is the one construct where
the two backends are supposed to differ. Division by zero, INT64_MIN/-1
and the float-to-int cast are the other three that x86 answers
differently from LLVM's "undefined" -- idiv raises SIGFPE where LLVM
says nothing, and cvttsd2si answers the integer indefinite value. The
Fn-value question -- body pointer or cell pointer -- is untouched: the
lowering here would have emitted direct calls, which means no
redefinition, and that is a gap to close before this backend is the dev
backend rather than an experiment.
A type-changing (map f) is the case into's single shadowed element name
would break if the shadowing were a trick rather than the language's
rule; it is not, because each stage is a fresh slot at its own type, and
into.flan now runs an i32 source into a (Vec f32) to say so.
A move-only accumulator carried round by recur is the shape BUILT.md
pitches the form on and was untested. It works, and recur.flan now
carries a Vec three times round and answers with it.
block's empty-body arm returned before the loop that distributes the
tail, so (do) in a tail position left ctx.tail set for whatever was
checked next. Latent rather than live — every consumer sets it
immediately before use, and the leaking form is always Unit-typed — but
it is one line to close and the invariant is easier to state closed.
Also the PORTING.md line listing loop/recur among the things with no
customer: it was built, and the half of that finding that still stands is
tail calls, which were not.
(into xs (vec-new i32) (map double) (filter even?)). The function name is
syntax and never a value, so (map double) is (double x) written into the
loop body: no intermediate collection, no closure, no generics, nothing
to inline. Transducers would compose at run time and Rust's iterators
need monomorphisation; a macro needs neither. into.flan counts the pulls,
which is the assertion a unit test cannot make.
The destination is in the form because the destination is the allocation,
and that is what makes spec-memory.md's explicit-allocator rule true by
construction rather than by convention. Which also settles the open
question: reductions do not share the form. A seed is not an allocation,
so (into xs 0 (map cost) (sum)) would be a second form wearing the same
spelling, and the destination would stop being honest about what it is.
A source that is already a name is used as it is, not bound. A (Vec T) is
move-only, so binding it would take the caller's ownership for something
that only reads; a fixed array would be copied once per into. A source
that is anything else is still bound once, which is what a call needs.
The wart is that an owning temporary there leaks, because the binding has
a name the caller cannot reach — a call in that position should borrow,
and drop is what would close it.
All four of the prelude's macro limits bit and none blocked anything. The
three refusals are names nothing defines, which is the only error
facility a macro has. into-wrap is a defn using only special forms, so
Macro.reduce does not drop it, and it is the first thing in the prelude
written as a loop/recur.
flan_dev_reg_dead was reaching for the containment scan, and it is on the free
path: a dev build would have paid a 4096-entry sweep per free. A free hands back
the base address the allocator gave out, which is what the slot is keyed on, so
the question there is equality and never containment. Only free-all needs the
scan, and that runs once a frame.
The table is allocated when it is armed, not declared. A fixed array was a
quarter of a megabyte of BSS in a shipped game for a table that build never
writes; now a release build carries a null pointer and the not-taken branch.
The pointer arm binds its subject to a slot before naming it three times — the
slice arm's rule, and its reason: an inspect with a path reaches a leaf through
a bounds check, and three of those to render one pointer is the walk paying for
its own shape.
dev-ptr.flan shows both halves on a stopped stack. It was read by hand; the
test_dev.ml case that would drive it is another lane's file, and NEXT.md says so.
There is no TCO here and recur is not a cheaper substitute for one: the
compiler verifies the call is in the loop body's tail position, so the
mistake is a compile error where it was written rather than a stack
overflow somewhere else. A loop is a let, a While whose condition is
true, and two jumps — emit.ml is untouched, and the barrier question
recur asks is the one labelled break already answered.
Tail position is a permission that is withdrawn at the top of check, the
same read-and-withdraw defer_ok does, handed back only by a block's last
form, both arms of an if and a match arm. So nothing enumerates the forms
that are not tails, which a pre-pass over the Ast would have had to, and
would have had to keep doing.
loop is also a barrier for break and continue, which is added rather than
inherited: a loop answers with the value of its body and a jump out has
no value to give. That is also why it takes no label. A while inside a
loop keeps its own break.
Two things the shape forced. A loop binding is a plain name, because
destructuring would make recur's argument count unreadable off the
binding vector. And in_loop's "moves a value bound outside the loop"
rule had to be told about the loop's own names, or (loop [v (vec-new
i32)] ...) would have been refused for doing the ordinary thing.
(Ptr Enemy) already says Enemy, at compile time, in the walk. What the renderer
lacked was any way to know whether the storage at the far end is still there —
and an allocation registry is exactly a record of which addresses it is still
true to read. So the inspector follows a live one and renders the pointee by the
same walk as anything else, and names what died at a dead one.
println does not, and the split is not squeamishness: spec-memory.md fixes what
a printed Ptr prints, a printed line belongs to the program and has to read the
same in a release build, and a release build has no registry to ask. The two
callers already differ in an emitter record; they differ in one more.
No address appears in the text. An address is not stable across two runs, so
printing one would make a rendering depend on where the heap landed — the rule
Render already follows for an allocator. What a reader wants from a dangling
pointer is what died.
registry.flan is one program read twice: a dev build answers for an address at
the heap, arena and pool tiers, and a release build answers 0 to all of it. The
arena row is the free-all Valgrind cannot see — this does not make memcheck
report it, it makes the same read answerable.
The checker builds one note after every operation that may have allocated,
because the checker is the only place the concrete element type exists — and it
builds them in every build, because a tree that differed by build flag would
make every pass between here and the backend ask which one it was looking at.
The backend drops them when [dev] is off, before walking the arguments: a note
takes the container's address, and emitting that only to discard the call would
leave an escaped alloca that mem2reg will not promote.
Armed by a global constructor rather than a line in main. A defvar initialiser
can allocate before main runs, and a note that arrived before the flag was set
would be a block the table never heard of.
A dev build reports the live block, answers 1 for a pointer into it, and 0 for
the same pointer after the free. A release build answers 0 to all of it.
The imported-bindings program was skipped without FLAN_RAYLIB_H, which was
right when the bindings only existed if a header was read and is now the one
gate hiding the change. It runs on the same terms as every other raylib case:
libraylib linkable, no raylib-devel. A generated.flan regenerated empty or
stale is now caught on an ordinary machine rather than only on one with a
header exported.
Reach.link already answers the shim worry BUILT.md's cold-build attribution
would otherwise raise: sand.flan links 110 wrappers, not 425, because the
bindings nothing reachable calls are dropped. What is left is 65ms of frontend
on a cold build, against a header read that was 60-90ms of a fresh session and
15.5ms of every redefinition. Both numbers are in BUILT.md now.
Also a swallowed line continuation in the exclusion message.
generated.flan carries the 253 declarations the importer reads out of raylib's
header, so a build needs libraylib linkable and no header at all. The opt-in
no longer decides how many bindings a package has — every build now gets all
425, they are greppable, and they diff when raylib moves.
What that gives up is the build-time check, so `flan generate-c` is the only
thing that writes the file and it compares first: every defstruct against the
header's record, every hand-written declare-c against the header's signature,
and it writes nothing when they disagree. Against the 5.1-dev header on this
machine that is ten real differences and no write.
The 172 hand-written lines stay, and not out of caution. Everything the
generator emits agrees with the header by construction, so diffing generated
output against its own source is a tautology; the hand-written lines were
transcribed by a person, so they are the only thing here a header can
contradict. All ten of those differences came from them.
`bindings` beside `headers` is what survives regeneration, because a hand-edit
to a committed generated file does not. Two directives: `exclude` drops
raylib's three allocator entry points, and `name` gives the 19 generated
predicates the `?` spelling the hand-written ones already use.
Loc.Errors is a second exception, and the handlers in the session and the
daemon name only Loc.Error — so a list reaching them is an unhandled
exception and a dead session, which is the one thing the dev loop exists to
prevent. A flag on the function the session already calls left that one
label away from happening. Parse.program_all and Check.program_all are
separate names, so the session's call site has to be edited by a person for
its behaviour to change, and the guarantee stops being a default argument.
Placeless diagnostics now sort last rather than first. A wrong main signature
is raised against unknown, which is line 0, and sorting on the number alone
put it above every error that can actually be clicked. It is a real error and
it is not anywhere, so it goes after the ones that are.
A kind is a stable id per error, so a test can assert which error this is
without matching on prose and a message can be reworded without breaking
anything. The reader's fourteen refusals all have one; in the checker they
go on the errors a test names and the handful that are common enough to be
worth classifying. Not a hundred of them, because jank has a hundred from
being mature and the number is not the feature.
The notes are the part that could not be said before. A duplicate definition
now points at the second and notes the first; a duplicate parameter and a
duplicate field do the same; an unknown field, an unknown struct and a
non-exhaustive match all note the declaration and list what is actually
there, so the reader's next move arrives with the question instead of after
it. The reader's unclosed bracket is the clearest case — the error sits on
the bracket, because that is where the fix goes, and the note sits where the
file ran out, because that is the surprise.
No message text changed, so every existing needle still means what it meant.
The new assertions are on kinds and on note positions, which is the house
rule about asserting the reason, made stable.
The provenance rides on the location, not on the form, because the location
is the thing that already travels: Expand.unmarshal stamps the call site onto
every node a macro answers with, and that stamp goes on through the AST and
the typed IR untouched. Tagging it there means an error raised anywhere
downstream can name the macro with no field added to Form, to Ast or to Tast.
Outermost wins. The macro the author wrote is the one worth naming, not
whatever it expanded into on the way down.
The honest limit, since it would otherwise read as a claim: a macro's
expansion has no source of its own to point at, so the note lands on the call
site along with the error. What it buys is the reader knowing the code being
refused is not the code they wrote.
A sink collects what a pass found so the pass can go on to the next thing.
It is switched on by the caller, not by the code that raises, which is what
leaves the interactive path untouched: the daemon checks one form, asks for
a sink that is off, and still gets one exception.
Two resync points, and both are places the work already had a boundary. In
the parser it is a top-level form — the reader found where each declaration
ends, so skipping a bad one cannot lose its place, while inside a
declaration there is no such landmark and one bad defn stays one error. In
the checker it is the two passes: pass one, which builds every name and
signature, still stops at the first refusal, because a signature it could
not make sense of leaves a hole that pass two would report once per mention.
Thirty unknown-name lines under one wrong signature are not thirty errors.
Pass two is where the volume is and where collecting pays, and by then every
signature is sound, so a body that fails cannot make the next body fail.
That is what makes a declaration a resync point needing no resynchronising.
The surface: (pool-new T), (insert p x) answering a handle, (resolve p h)
answering (Option (Ptr T)), (release p h) answering whether this call was
the one that released it, (len p) and (live p), and (pool-handle p i) for
enumeration. free extends to the pool and refuses a handle by name, because
a handle owns nothing and consuming one copy would say nothing about the
others.
resolve answers a pointer rather than a value because spec-memory.md's own
worked example does, and says why a line above it: a pattern binding binds
a value, and a copy cannot be written back.
test/programs/handles.flan prints <handle 1:1> and <handle 1:3> for the same
slot before and after a death, and the projectile still holding the first
gets -1 rather than the newcomer's 99.
The first line of an entry is still exactly file:line:col: message, because
that is the GNU format compilation-mode already parses and the whole of the
editor story. Everything under it is indented, which compilation-mode
ignores, so the underline is free. A note gets an entry of its own rather
than being folded into the error's block — that is what makes the second
place somewhere next-error can go, and is the reason notes carry locations.
Every part of it degrades to the bare first line: a location the checker
invented has line 0, the prelude and the REPL have names that are not paths,
and a file can change under us between being read and being blamed. An error
printer that can raise is worse than one that prints less.
(Handle T) and (Pool T) land as types and as a runtime. A handle is one
int64_t — slot index low, generation high — so it copies, zeroes and
compares like the integer it is and owns nothing. A live slot's generation
is odd, which makes a zeroed handle resolve to nothing rather than to slot
zero, and makes iteration free. Wrapping retires the slot rather than
reissuing it: 2^31 reuses is rare, and rare is not an answer when the
failure is the silent wrong one the type exists to prevent.
No surface yet — the checker still has no names for any of it.
Loc.Error now carries a diagnostic: a stable kind, a span, notes that each
have their own span and severity, and the macro expansion it came from. The
notes are the part that was actually missing — "this is wrong here" plus
"because of that, over there" is two places and two explanations, and a
single string can state only one of them.
The compatibility story for the daemon, which was the open question: the
single-diagnostic exception stays the single-diagnostic exception. Session
and dev evaluate one form and have one failure to report, so they take a
location and a message out of it with Loc.summary and are otherwise
unchanged. A second exception carries a list, and only a driver that
compiles a whole file raises it, so nothing interactive has to know it is
there.
No message text changed.
Loc.t grows an exclusive end, defaulting to the start, so a location nobody
widened is a zero-width span at a point and every existing call site keeps
its old meaning. Only the reader knows where a form ends, so only the reader
fills them in — one helper in the one place that holds both ends, which is
why nothing above Reader had to learn a span exists.
The width assertion is the point of the tests: the field could exist, nothing
could fill it, and every underline would be one character long while the
feature looked finished.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
The slot after a defn's parameters is about to become mandatory, and a void
function has to have something to write there. () is ML's spelling and it
cannot collide: an empty call is not a valid expression, so () has no reading
in value position for a body form to be confused with.
Additive on its own. Internally it stays Tname "Unit" -- the resolver, the
shim and the emitter all speak that name and none of them change -- so this is
two arms in parse.ml: texpr reads () as the unit type, and is_type_form says
that a leading () is a return type rather than the first form of a body.
flan dev is one process, so the unix socket between the compiler and the
program was a connect, a write and a read that never left the machine's own
memory. A delivery is now a call into the agent's verb table.
flan_agent.c's handlers write into a sink that is either an fd or a growing
buffer, so serve() and the new flan_agent_request() share one verb table rather
than two that would drift -- every answer in there is assembled from several
pieces, so the seam had to be the writing. Dev.deliver, Dev.result and Dev.ask
were three copies of the same socket dance and are now three lines over one
Dev.request.
Which path is taken is the linker's answer, not a flag: flan_agent_request is
weak in dynload_stubs.c, null in every binary that links no flan_agent.o, and
Agent.request is None there. So --two-process, flan reload and every test go on
using the socket with nothing to configure.
Three things the direct path must not quietly change. The ring's room check is
separate from its store, which was safe only while the accept loop was the sole
producer, so handle_line runs under a mutex. A delivery is still only published
-- the install is one store per function on the game thread at a frame
boundary, and doing it on the spot would be a frame running half in the old
code and half in the new. And the OCaml runtime system is released across the
call, because a delivery is a dlopen.
Measured, and the number is the point: the transport was ~50us of a 23ms
redefinition, so end to end did not move. Code generation is 19 of the 22
milliseconds. The merge's prize was never latency.
The test is a deletion, because a reply cannot say which way it came:
test_dev.ml unlinks the agent's socket file once the merged program has bound
it, and every evaluation after that still installs.
The refusal list was rewritten and the prologues that pointed at it were not,
so prelude.ml claimed in three places that what it now contains is impossible:
the splitting header said `split` is refused at the foot of the file, forty
lines above `split`; the ASCII-case header said Odin's allocating to_lower is
not available here, next to the one that was written; and the UTF-8 header
said the rest of core/strings is refused rather than ported.
Each keeps its point rather than losing it. The iterator is still the shape
that owns nothing and still the right call when there is no result to own;
lower-ascii and bytes-ci=? are still the right calls when a copy is not
wanted, since folding a comparison over two inputs beats lowering both. What
changed is the reason, which used to be the absence of an allocator and is now
a choice between two shapes that both exist.
And strings.flan told the reader the opposite of what it did -- "not freed",
on the line above the free. vec.flan already had the right framing: the free
is written, it keeps the block because an arena cannot release one, and that
is the difference the capability set exists to state.
The list at the foot of prelude.ml was one sentence -- every entry needed to
produce bytes that did not exist in its input, and there was no allocator --
and that sentence has been false since Vec landed. Seven entries move up into
the code, and string-from-bytes turns out to have been the `string` builtin
all along: (string (as-slice v)) is the round trip, free precisely because
the layouts are identical.
What is left is refused for four different reasons and is written that way
now: pad and center for nothing at all except that no caller has asked;
format and sprintf for variadics of mixed type; map, filter, reduce and
sort-by for function values; map-keys and map-values for a map iterator that
does not exist in the runtime.
NEXT.md's queued section is struck and carries the four findings, each with
the change it wants named -- flan_map_next plus one builtin for the iterator;
milestone 5's function values for the higher-order three; vec_new_elem taking
a type expression rather than a bare name, which is what forces slices-new to
exist; and an array literal with no way to say it is [f32], which is what
forces every float in algorithms.flan to be cast. BUILT.md gets the section.
return is refused inside handler-bind and restart-case blanketly, and rightly:
a return always crosses the frames they pushed. A break does not. A loop
written wholly inside a restart-case body has a perfectly good local break, so
the rule is a barrier on the loop stack rather than a flag — a jump is refused
exactly when a barrier stands between it and the loop it names, and the message
says which construct. handler-bind and restart-case bodies are barriers, so is
a restart clause, so are a defer's forms; a handler clause is lifted into its
own function and needs no rule at all. in_frames is untouched: a return is the
special case where the target is always outside every barrier.
continue wanted the other blocker. check_dotimes folded its step onto the end
of the body, which a continue would jump past, so the counter would never
advance and the loop would hang. Tast.While carries a latch now — condition,
body, latch — the step goes there, and emit_while emits four blocks. A while's
latch is empty and folds away.
Labels are Odin's, in the head position: (while :outer c ...) and (break
:outer). A keyword there is unambiguous because a loop condition is never one,
so one label function serves while, until, dotimes, break and continue. It is
not a goto — the checker resolves a label against the loops the form is
lexically inside, so control can only leave a loop it is already in.
Break and Continue carry a relative depth rather than a name, because that is
what a backend already has: emit keeps one entry per While the way it keeps
one pad per frame, and indexes it.
Nothing in the prelude wants either. Every early exit there is a return from
the function, which break cannot replace; the sentinel-flag loop break exists
to remove does not appear in it. The two the compiler emits are that shape and
are the one place it cannot help — their sentinel is set inside a restart-case.
reach.ml and render.ml take the While arity change and nothing else.
