Finishes DISCUSS.md §9's `pause` marking: the daemon half was already
built, this is the editor half plus the one daemon path it was missing.
`C-u C-c C-c` marks the form point is inside, `C-u C-u C-c C-c` the
top-level form (stop on entry), `C-u C-x C-e` the expression before
point. The buffer is never edited — the position rides beside the code
and the `(pause)` call goes into the tree after parsing, so no source
location moves.
`C-x C-e`'s path needed the daemon: its 5s `wait` answered "the program
did not reach a frame boundary", which is exactly what a thunk parked at
a breakpoint looks like from out here. `wait` is now three-way and asks
`state t = Stopped` only when a pause was requested, so the no-pause
shape `test_dev.ml` pins is unchanged.
The overlay is an annotation and not feedback, so unlike an error marker
it survives `pre-command-hook`; what takes it down is an accepted
evaluation with no `:pause` on it, which is the same thing that takes
the mark itself down.
Tests: a `test_dev.ml` block over the new `dev-pause.flan` that marks,
stops, re-evaluates plainly and then polls half a second confirming it
does not stop again — one sample after `continue` proves nothing, the
resumed frame is still in the old body — and an `emacs/test-flan-dev.el`
block for which form a prefix picks, the byte column, the overlay's
lifetime, and one live round trip.
Three handoffs at the root carry the detail; NEXT.md carries the queue.
F2's Emacs half is seven items with two traps in it, F1 leaves a second
unexplained flake and the sweep that would let its section be deleted,
F3 leaves the two tests its own change did not get. The raylib examples
were never started, and Tier 1 item 6 still has not moved.
Also the worktree problem, because it nearly cost two lanes their work:
at least two were handed out about 485 commits stale, on a tree without
lib/dev.ml. Both noticed. One that did not would look like it worked.
765 lines to 201, at parity with lisp/sand.lisp, clojure/src/fnm/sand.clj
and src/fnm/sand.jank. Gone: audio and tone synthesis, the brush textures
and the embedded PNG, the render-texture scene, the HUD font, the camera,
the world cursor, the HUD and the input-state read-out. None of it is in
any reference version, and none of it was the language being exercised.
Inlined the single-use helpers -- empty-at?, move-grain, draw-grid, paint.
settle also carried an unused (rl/Vector2 ...) binding, which put raylib
inside the section whose banner says there is none, and a stray (pause).
The physics is untouched on purpose: the references disagree there, so
parity does not name a target, and settle is what the pinned hash covers.
It still prints 15595743031174623232 at -O2, -O0 and as a dev build.
test_web.ml asserted brush.png's bytes reached the wasm module, proving an
embed survives the web build. web-files.flan is web-built and *run* under
node and asserts the embedded bytes print, which is the same property
checked harder, so the sand assertion goes rather than the embed staying.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
flan-watch-ghost-mode paints each watched value inline, after the line holding
the call that wrote it. An addition to the watch buffer and not a replacement:
both can be on at once, and turning either off leaves the other running.
The earlier note said ghost text was gated on a (watch ...) form in check.ml,
because nothing in the table carries a source location. That is true of the
table and the conclusion did not follow. The call site is in the buffer, and
the name in the table is the string literal in it, so the anchor is searched
for rather than reported. Nothing new is asked of the daemon. The head of the
call is a defcustom regexp, because watch-i64 is a name the program's author
chose in their own declare-c and only the C symbol behind it is fixed.
Both pictures are painted from one reply in flan-watch--absorb, so they cannot
disagree and there is no second watch request in flight. That meant the watch
buffer could no longer be the subscription: arming and the timer now hang off
flan-watch--consumers, and only the last consumer out disarms the table.
Overlays are replaced wholesale on every repaint rather than followed through
edits, which is the whole answer to invalidating one whose line moved. Only
buffers shown in a window are scanned.
Settled and written down: two sites of one name both show it and say so,
because the table has one slot and the last writer wins; a watch in a loop
shows the last value written, as the buffer does, because every better answer
is the query UI this design exists to avoid; a stopped program's values say
"last frame" and change face, since inline they sit in code that looks live;
a site with no row is annotated only when the table reports overflow.
syntax-ppss moves point and clobbers the match data, so calling it inside a
re-search-forward loop and then reading match-string restarts the scan and the
loop never ends. Everything is read out before the check now.
emacs/test-flan-watch.el covers it, loaded from test-flan-cider.el the way
test-flan-mode.el is, so no build change is needed. 203 checks, 0 failures.
A type-changing (map f) is the case into's single shadowed element name
would break if the shadowing were a trick rather than the language's
rule; it is not, because each stage is a fresh slot at its own type, and
into.flan now runs an i32 source into a (Vec f32) to say so.
A move-only accumulator carried round by recur is the shape BUILT.md
pitches the form on and was untested. It works, and recur.flan now
carries a Vec three times round and answers with it.
block's empty-body arm returned before the loop that distributes the
tail, so (do) in a tail position left ctx.tail set for whatever was
checked next. Latent rather than live — every consumer sets it
immediately before use, and the leaking form is always Unit-typed — but
it is one line to close and the invariant is easier to state closed.
Also the PORTING.md line listing loop/recur among the things with no
customer: it was built, and the half of that finding that still stands is
tail calls, which were not.
(into xs (vec-new i32) (map double) (filter even?)). The function name is
syntax and never a value, so (map double) is (double x) written into the
loop body: no intermediate collection, no closure, no generics, nothing
to inline. Transducers would compose at run time and Rust's iterators
need monomorphisation; a macro needs neither. into.flan counts the pulls,
which is the assertion a unit test cannot make.
The destination is in the form because the destination is the allocation,
and that is what makes spec-memory.md's explicit-allocator rule true by
construction rather than by convention. Which also settles the open
question: reductions do not share the form. A seed is not an allocation,
so (into xs 0 (map cost) (sum)) would be a second form wearing the same
spelling, and the destination would stop being honest about what it is.
A source that is already a name is used as it is, not bound. A (Vec T) is
move-only, so binding it would take the caller's ownership for something
that only reads; a fixed array would be copied once per into. A source
that is anything else is still bound once, which is what a call needs.
The wart is that an owning temporary there leaks, because the binding has
a name the caller cannot reach — a call in that position should borrow,
and drop is what would close it.
All four of the prelude's macro limits bit and none blocked anything. The
three refusals are names nothing defines, which is the only error
facility a macro has. into-wrap is a defn using only special forms, so
Macro.reduce does not drop it, and it is the first thing in the prelude
written as a loop/recur.
BUILT.md on the split that is the design — the type is emitted because only the
checker knows it, the death is not because an address needs no type — and on the
part that reads backwards: (Ptr Enemy) already said Enemy, so the registry
supplies permission rather than identification.
The honesty is in the same section rather than a footnote. A release build pays
a load and a not-taken branch per free, because flan_dev.c is in every build and
a second allocator selected by a build flag is worse than a branch. The table is
calloc'd when armed rather than declared, so nothing else is carried. The arena
free-all is answerable now and still invisible to memcheck, and those are two
different claims.
NEXT.md keeps the entry open for what was not built: an op that points at an
arbitrary address, the breakdown by type, leak attribution, the memcheck half,
and the test_dev.ml case that would drive dev-ptr.flan.
There is no TCO here and recur is not a cheaper substitute for one: the
compiler verifies the call is in the loop body's tail position, so the
mistake is a compile error where it was written rather than a stack
overflow somewhere else. A loop is a let, a While whose condition is
true, and two jumps — emit.ml is untouched, and the barrier question
recur asks is the one labelled break already answered.
Tail position is a permission that is withdrawn at the top of check, the
same read-and-withdraw defer_ok does, handed back only by a block's last
form, both arms of an if and a match arm. So nothing enumerates the forms
that are not tails, which a pre-pass over the Ast would have had to, and
would have had to keep doing.
loop is also a barrier for break and continue, which is added rather than
inherited: a loop answers with the value of its body and a jump out has
no value to give. That is also why it takes no label. A while inside a
loop keeps its own break.
Two things the shape forced. A loop binding is a plain name, because
destructuring would make recur's argument count unreadable off the
binding vector. And in_loop's "moves a value bound outside the loop"
rule had to be told about the loop's own names, or (loop [v (vec-new
i32)] ...) would have been refused for doing the ordinary thing.
If a frame mutates a global and then signals, taking a retry re-runs the
mutation. Control resumes at the restart-case and runs forward; nothing is
undone. Common Lisp has the same property and offers no help either, so this
is written down rather than fixed.
The discipline is that the author chooses where the retry boundary is: a
restart-case above the mutations re-runs them, one below re-runs only what
follows. Put the restart before anything mutates, make the retried section
idempotent, or snapshot what will be re-applied.
It matters more here than in most Lisps because the intended use is a game
loop, and a bad index signalling BoundsError rather than ending the process
made abandoning and retrying a frame an ordinary thing to do.
conditions.org and web/index.html already carried the mechanical half as a
one-line gotcha; those are rewritten in place rather than gaining a second
bullet beside them. spec-conditions.md takes it in section 5, which already
enumerates what a transfer does and does not do. No numbered case changed
meaning.
web/index.html had no section on the FFI's generated half at all; it has one
now, with the command, the config, and the reason committing the output is what
makes the no-header property honest rather than a caveat.
BUILT.md gets why the 172 stay, which is the part that is easy to get wrong:
136 of them are exactly what the rule produces and the rest are expressible as
overrides, so the superset argument is sound and still leads somewhere bad —
deleting them reduces the signature check to a tautology.
DISCUSS.md 6a and 6b are answered rather than left open, and 6b's own point
about enums turns out to be live in the tree: key-down? keeps its Key
parameter because it is hand-written, and the generated key-up? beside it
takes an i32.
compile.el puts note in the same capture group as info — group 7, level 0 —
while warning is group 6, level 1, and compilation-skip-threshold defaults to
1. So next-error walks the errors with no configuration, which is the claim
M-x compile rests on and it holds; it steps over the notes until the
threshold is 0. They are still parsed, coloured and clickable.
Labelling notes warning: would make them navigable at the default and is
refused. A note is not a warning, and a compile whose only complaint is an
error would start reporting warnings that are not warnings.
The macro expansion field gets the test it was missing, through a real
expansion rather than a unit test on either half: the tag is put on by Macro
and defaulted into the diagnostic by Loc, and either half alone would pass
with the other broken. clamp misused expands into a call to a name that does
not exist, so the checker refuses something the author never wrote, which is
the case the field is for.
BUILT.md gets the section: the 32/32 split, live-is-odd and the two things
that fall out of it, wrapping retiring the slot, why resolve answers
(Option (Ptr T)) and where the spec already said so, why len is the slot
high-water and not the live count, and why a slot is released through the
pool rather than through free.
Two amendments to a frozen spec, both deferrals: .field and at do not
auto-deref a handle, and deref is not overloaded on one. Neither can answer
"gone", which is the whole job, and the spec's own worked example resolves
first and matches.
The pointer hole is written down rather than implied: a (Ptr T) from resolve
dies on any insert that grows, which is the slice contract one level down.
NEXT.md swept, not just struck — five places beyond item 6 were still
asserting that Handle did not exist.
BUILT.md gets the design: why the span went into Loc.t rather than beside it,
why macro provenance went the same way, why the first line of a report is
still the GNU format, and what the daemon sees.
NEXT.md item 8 is struck through, with the parts that were not built stated
plainly so they do not read as oversights — the reader does not collect,
because a paren stream cannot be resynchronised; pass one of the checker does
not collect, because thirty unknown-name lines under one wrong signature are
the same error thirty times; and there are not a hundred kinds, because the
count was never the feature.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
The prose still said --two-process measured faster in every column, which was
true of the one number taken before item 1 and not of the four retaken back to
back. In the after pair the merged build is 0.9ms ahead; in the before pair
two-process was 1.3ms ahead. A difference that changes sign between runs is a
difference made of noise, and that is a better argument than a direction would
have been.
NEXT.md's file table gains lib/agent.ml and stops describing dev.ml and the
agent as though the compiler reached the program over a socket. It is the index
a next lane reads first.
Third on the list to delete, and nothing goes. "The compiler can read the
stopped frame's memory directly, so copying it is ceremony" is the right
instinct and the wrong diagnosis: the snapshot was never about two address
spaces.
A stopped program is not holding still. The break loop polls, flan_agent_poll
runs whatever was delivered, and a C-x C-e thunk is arbitrary Flan that pushes
and pops the one global restart list and the shadow stack while it runs. The
compiler is a thread beside it either way. Restart names are copied because
serving them off the live list hands the reader a pointer into a frame the
loop's own poll may already have popped; a pointer is meaningful to the
compiler now, and the frame it points into is no more alive for that. The
fingerprints have a sharper reason still, already written down: the module a
frame's description lives in can be unloaded before the comparison happens.
The generation stamp is about nested breaks. A thunk this loop runs can error,
push a break of its own, and reach chosen_ready first, claiming an index someone
chose from the outer list. Depth cannot tell those apart because a resumed
outer break and a new one reuse the number. One process changes none of that.
So of the three things the merge was expected to make deletable, one was. The
socket was transport. The result cap and the snapshot are concurrency, and were
only mistaken for transport because the socket was in front of them.
What the merge does unlock here is item 4, deliberately not touched:
flan_agent_frame_slot already hands back an address, and in one process the
compiler could read the value there instead of compiling a thunk to print it.
RESULT_MAX was written down twice, once in flan_dev.c and once in the agent,
with a run-time check that the two had not drifted. That second copy was a
buffer sized to be sent through a socket, and it is gone: the agent asks
flan_dev_result_cap() and allocates, which it can do because it is never the
game thread.
The bound itself stays, and the reason is the rule everything else here is
built around. result is what the game thread writes into, from a render thunk
at a frame boundary; a growable one is the frame thread calling realloc, which
is an allocation in the one place this design exists to keep allocation out of.
It would also break the seqlock, which is a protocol about torn contents and
assumes the address it copies from neither moves nor goes away underneath the
reader. Growing on the writer's side is a use-after-free the counter cannot
see.
So it is a render budget and not a wire size, and it only looked like one
because the agent had a copy of it. Removing the bound is a redesign of the
read -- probe, allocate, re-read, validate, retry -- and belongs with moving
the read to a frame boundary.
Also: flan_agent_request answers "err path too long" where the socket does,
instead of nothing. One verb table is only worth having if the two callers
cannot be told apart.
The measurement table in BUILT.md is retaken back to back across all four
shapes, because this machine drifts by more than the thing being measured. The
internal socket was ~41us of a 21ms redefinition; --two-process is unchanged
and is not slower than the merged build in any column.
flan dev is one process, so the unix socket between the compiler and the
program was a connect, a write and a read that never left the machine's own
memory. A delivery is now a call into the agent's verb table.
flan_agent.c's handlers write into a sink that is either an fd or a growing
buffer, so serve() and the new flan_agent_request() share one verb table rather
than two that would drift -- every answer in there is assembled from several
pieces, so the seam had to be the writing. Dev.deliver, Dev.result and Dev.ask
were three copies of the same socket dance and are now three lines over one
Dev.request.
Which path is taken is the linker's answer, not a flag: flan_agent_request is
weak in dynload_stubs.c, null in every binary that links no flan_agent.o, and
Agent.request is None there. So --two-process, flan reload and every test go on
using the socket with nothing to configure.
Three things the direct path must not quietly change. The ring's room check is
separate from its store, which was safe only while the accept loop was the sole
producer, so handle_line runs under a mutex. A delivery is still only published
-- the install is one store per function on the game thread at a frame
boundary, and doing it on the spot would be a frame running half in the old
code and half in the new. And the OCaml runtime system is released across the
call, because a delivery is a dlopen.
Measured, and the number is the point: the transport was ~50us of a 23ms
redefinition, so end to end did not move. Code generation is 19 of the 22
milliseconds. The merge's prize was never latency.
The test is a deletion, because a reply cannot say which way it came:
test_dev.ml unlinks the agent's socket file once the merged program has bound
it, and every evaluation after that still installs.
The list at the foot of prelude.ml was one sentence -- every entry needed to
produce bytes that did not exist in its input, and there was no allocator --
and that sentence has been false since Vec landed. Seven entries move up into
the code, and string-from-bytes turns out to have been the `string` builtin
all along: (string (as-slice v)) is the round trip, free precisely because
the layouts are identical.
What is left is refused for four different reasons and is written that way
now: pad and center for nothing at all except that no caller has asked;
format and sprintf for variadics of mixed type; map, filter, reduce and
sort-by for function values; map-keys and map-values for a map iterator that
does not exist in the runtime.
NEXT.md's queued section is struck and carries the four findings, each with
the change it wants named -- flan_map_next plus one builtin for the iterator;
milestone 5's function values for the higher-order three; vec_new_elem taking
a type expression rather than a bare name, which is what forces slices-new to
exist; and an array literal with no way to say it is [f32], which is what
forces every float in algorithms.flan to be cast. BUILT.md gets the section.
return is refused inside handler-bind and restart-case blanketly, and rightly:
a return always crosses the frames they pushed. A break does not. A loop
written wholly inside a restart-case body has a perfectly good local break, so
the rule is a barrier on the loop stack rather than a flag — a jump is refused
exactly when a barrier stands between it and the loop it names, and the message
says which construct. handler-bind and restart-case bodies are barriers, so is
a restart clause, so are a defer's forms; a handler clause is lifted into its
own function and needs no rule at all. in_frames is untouched: a return is the
special case where the target is always outside every barrier.
continue wanted the other blocker. check_dotimes folded its step onto the end
of the body, which a continue would jump past, so the counter would never
advance and the loop would hang. Tast.While carries a latch now — condition,
body, latch — the step goes there, and emit_while emits four blocks. A while's
latch is empty and folds away.
Labels are Odin's, in the head position: (while :outer c ...) and (break
:outer). A keyword there is unambiguous because a loop condition is never one,
so one label function serves while, until, dotimes, break and continue. It is
not a goto — the checker resolves a label against the loops the form is
lexically inside, so control can only leave a loop it is already in.
Break and Continue carry a relative depth rather than a name, because that is
what a backend already has: emit keeps one entry per While the way it keeps
one pad per frame, and indexes it.
Nothing in the prelude wants either. Every early exit there is a return from
the function, which break cannot replace; the sentinel-flag loop break exists
to remove does not appear in it. The two the compiler emits are that shape and
are the one place it cannot help — their sentinel is set inside a restart-case.
reach.ml and render.ml take the While arity change and nothing else.
