That commit is about C-M-x and touches three files; the third is a
snapshot of an editing session caught mid-thought:
(defvar frame Allocator (arena-new 262144))
(defvar game-data (embed (with-allocator frame )))
The second does not check — embed takes a path, and there is no path
there — so the acceptance corpus fails to build and `dune test` has been
red at the tip on its own account. Removed rather than repaired, because
what it was going to say is the author's to finish.
`arena-new` is a builtin, so it is in no program's symbol table and `defs`
never mentioned it — which made the editor answer "the running program
defines no arena-new" about a name that works. The fix is not a better
refusal: it is that the seventy-eight names the checker answers without
being told are now in the reply, under a kind of their own.
check.ml carries the table, beside the arms it describes, because a table
in another file drifts from them with nothing said. test_flan reads both
the arms and the table and fails on either having a name the other does
not, in both directions — there is no reflecting over a match, so it reads
the source.
Each entry is a signature in `signature_of_fn`'s shape and one line. The
arms that do not have one shape say what is true instead of pretending:
`?` for an argument that may be left out, `|` for the types an arm really
takes, and the checker's own predicate names for the type-directed ones.
The three user-allocator names carry a bare name and no bracket list,
because they are refused wherever they are written.
`defs` grows a fifth string for the prose, and builtins are appended last
so a completion table does not bury the names being worked on. A defn has
no docstring to put there and does not get one here: the Tast keeps none,
and that is a different piece of work.
The editor reads the kind, not a special case. `flan-doc--where` and the
xref backend both answer before their empty-location branch, because a
global's missing location and a builtin's absent one are different facts
and only one of them is about the daemon.
Emacs binds it in emacs-lisp-mode-map and nowhere else, so a mode derived
from lisp-mode inherits nothing and the key answers "undefined" — which
reads as the client being broken rather than as the key being free.
comint hands the input sender its buffer text with font-lock's properties
still on it, and prin1 writes such a string as #(...) — which the daemon
reads as a bare symbol followed by a stray list, so the field stops being
a string without anything saying so. The wire layer strips rather than
asking every caller to remember: it is the one place that knows the text
is about to become bytes.
`-dev-` was in every Emacs symbol this client owns and meant nothing to anyone
typing one: the daemon is `flan dev` at a shell, but from inside Emacs there is
no other kind of connection to distinguish it from. `M-x flan-dev` is now
`M-x flan`, `flan-dev-quit` is `flan-quit`, the private prefix `flan-dev--` is
`flan--`, and every defcustom follows — ninety-odd symbols, with the two files
renamed to emacs/flan.el and emacs/test-flan.el so the file names say the same
thing as the symbols in them.
No aliases. Renaming a defcustom breaks a config that names it and there is no
way around that; the repo has no precedent for softening one, and an alias left
behind is what keeps a rename from finishing. MANUAL.md says the old names are
gone and how to fix a config, which is the whole of the migration path.
Three strings are not symbols and keep their spelling: `.flan-dev.sock`, which
bin/main.ml writes and which a renamed variable searching for a renamed file
would simply never find; and the two buffer names `*flan-dev*` and ` *flan-dev*`,
which name the `flan dev` subcommand's own output rather than anything in elisp.
`flan dev` with a space is the CLI and is untouched everywhere.
The entry point also stops asking a question it already has the answer to. From
a buffer visiting a .flan file it starts that file; from anywhere else it reads
one from the minibuffer as before; `C-u` reads one either way, which is how you
start a second program without leaving the first. The current buffer is still
the only source of the default — the bug where a previous project won over the
buffer you were in was fixed by removing `flan--file` from that position, and
nothing here puts it back.
Four checks on the `interactive' form, evaluated on its own rather than by
calling the command, because calling it would build and launch a program and
the question is only which file the form arrives at and whether it had to ask.
A fifth asserts that nothing answers to the old names. test/test_emacs.ml loads
the test file by path and test/test_session.ml names the client file in a
comment, so the rename reaches those two lines; nothing else outside emacs/ and
the docs moved. Verified by byte-compiling every file
clean and by `dune test` and `@page`.
Three gaps in the first pass. The park flushes after the poll, which is the
only thing that gets a printing expression's line out of a fully buffered pipe
before the next run — now pinned by its position in the transcript. The globals
section under a parked break answers with everything skipped, which is a
different reply from the timeout a job nobody polled would give, so it is asked
for rather than assumed. And the render thunk's five-second sentence no longer
asks a parked program whether it is calling agent/poll.
The park waited on one flag and could do one thing, so C-x C-e on (+ 1 1) was
refused for want of a frame boundary — an expression that needs nothing from
the program, in a process holding every global the run left.
It waits on two now. A re-run leaves the park; a wake drains the agent's ring
and waits again, with the state still PARKED, which is what makes running the
thunk there exactly as safe as running it at a frame boundary: while parked
there is no concurrency to be unsafe against. The break loop is the precedent
and CL's spawned worker is deliberately not copied.
A thunk that stops now stops on a parked thread, so the restart ops refuse on
whether a break is engaged rather than on the state, and a paused expression
against the park is resumable.
#{{:a 1} {:a 1} {:a 2}} answers 2 whether tables=? compares anything or
compares nothing, so it was proving the count and not the compare. The
pair beside it isolates both halves: one map twice must collapse to 1,
and two maps of one entry each with different keys must stay 2, which is
what a size-only compare would get wrong.
And read's comment stops implying a property it does not have: empty
input answers (Some Value.Nil), indistinguishable from the document that
is nil. Empty is not malformed and the reader is not the thing that gets
to decide it is.
vendor/json is vendor/edn's shape with one decision reversed. edn never
allocates, so its tokens are views into the source buffer and escaped
strings are refused for want of anywhere to put the unescaped copy. This
one has an allocator, so it unescapes, and to unescape it copies —
string-of is the only function in the package that allocates, and it
copies even when there was no escape to resolve, because a Value whose
lifetime depended on which bytes happened to be in it is not a contract
anyone can hold. Odin answered the same question the same way:
tokenizer.odin allocates nothing, parser.odin's unquote_string does the
copy, and it clones in the no-escape branch too.
What that buys is at the bottom of test/programs/json.flan, which is
programs/edn.flan and programs/arena-edn.flan in one file because for
JSON they are one claim. The source buffer is overwritten with `?` bytes
while the document is live and the strings read back afterwards are
still the strings. arena-edn's header has a section admitting it cannot
do that.
Strict JSON and not Odin's JSON5 default, and the difference is where
most of the refusals come from: comments, single quotes, +1, .5, 1.,
0x1f, 01, NaN, Infinity and unquoted keys each get a sentence naming the
dialect they belong to, rather than one shared unexpected-byte. A lone
surrogate is refused too, and that one is forced rather than chosen —
rune-size answers None for the whole D800-DFFF block, so encode-rune!
would write nothing and the character would vanish.
The tokenizer refused #{} because "it needs a hash set to even
represent" — which is a claim about a reader, and a tokenizer represents
nothing. #{ now pushes } on the same balance stack { does, there is one
new token kind and no new closer, and err-set is gone rather than kept
with a message it no longer earns. skip-value needed nothing: it is
written against the depth and not against the kinds.
The dynamic reader moves out of test/programs/arena-edn.flan and into
vendor/edn/read.flan as (edn/read bytes), answering an (Option Value)
against whichever allocator the caller bound. Two decisions are written
down where they are made:
* a set is a Value.Set holding a deduplicated (Vec Value), because
(Map Value bool) does not typecheck — keyable refuses a key holding
a Vec or a Map — and restricting elements to keyable Values would
refuse #{[0 0] [1 0]}, which is the file this was built for. Insert
is O(n) against a structural value=?, so building the tileset's 54
pairs is 1458 comparisons, once.
* a Value copies every string into the allocator where a Token stays
a view. A view handed back out of the function that owns the buffer
is a dangling pointer, and free-all would not even take it. Odin's
json parser clones for the same reason.
An imported defdata was a refusal in load.ml — "not implemented yet
(milestone 4)" — and it had to go first. It is the type's name plus the
Type. half of a constructor symbol, which arrives as a Var node when the
case has no fields and a Struct node when it has; a match pattern needed
nothing, because a case resolves against the scrutinee's type and was
never a top-level name. programs/pkg-data.flan is that on its own.
programs/edn-read.flan reads assets/edn/tileset.edn, which is the
editor's real output: :texture-path and a :selected-cells of 54 integer
pairs, with no type declared for any of it. It also overwrites the
source buffer in place after reading and prints the document back, which
is the copy contract asserted rather than described.
[close] was the only way out of the accept loop that anybody ever took, and
an editor that is killed rather than quit never sends it. Four of the eight
orphans found on the author's machine were that.
The signal is the socket, not a new op: Emacs opens one connection for the
whole session and drops it only on paths that send [close] first, so an
editor left open overnight is an editor still attached and the grace cannot
accumulate under it. Armed only once a client has been there, so a daemon
still waiting for its first one is untouched. Parked sessions get the short
clock and live ones six times it, because a parked program is invisible and
a running one is a window somebody may be looking at.
Two things were missing from the REPL's history rather than broken in it. The
ring worked; nothing reached it by the key almost everyone presses, and nothing
survived quitting Emacs.
`<up>' and `<down>' are history on the line being typed and line motion
everywhere else. The conditional is the whole point: the text above the prompt
is a transcript people scroll back through, and a blunt binding would take that
away to buy something `M-p' already does. Inside a form typed over several
lines the keys still move between them, and reach history from the first. The
search is prefix-filtered, which is comint's `...-from-input' pair and what a
shell does. Each wrapper hands `this-command' over to the comint command it
delegates to, because comint tells a continued search from a fresh one by
looking for its own name in `last-command' -- without it the first press looks
perfect and the second never moves.
History is kept in `flan-repl-history' under `user-emacs-directory', through
`locate-user-emacs-file' so that no-littering and a moved state directory take
it along; the file and the size are both defcustoms. It is read when the mode
starts and written on kill and on exit, but not when the ring is empty and not
when the ring is shorter than the file -- a write replaces the file whole, so
either would throw away a session somebody else's Emacs saved. The exit sweep
is not installed in batch, where every prompt belongs to a test and the file it
would land in belongs to whoever ran it.
Entries are separated by a form feed on its own line rather than by comint's
newline, which is also what is inside a multi-line form: with the default the
file shreds one such form into fragments. The separator is let-bound around
each call instead of set in the buffer, because `comint-write-input-ring' reads
it inside a temporary buffer where a buffer-local value is invisible.
And the multi-line case the recall made reachable: `comint-send-input' sends
the process mark to *point*, and `comint-eol-on-send' carries it no further
than one line, while `flan-repl-return' judged completeness over the whole
input. Recall parks point where the typing stopped -- for an empty prompt, at
the start of what was just recalled -- so RET on a recalled form sent nothing
at all. It now sends what it measured.
emacs/test-flan-repl.el covers both halves from a stub prompt, loaded from
test-flan-cider.el for the reason test-flan-mode.el gives: `emacs/*.el' is
already a dependency of that stanza, so a file here needs no build change. No
daemon: history is a function from a ring and a position to what the buffer
says. Seventeen checks, which cost that suite nothing measurable -- the whole
of it still runs in 0.15s.
Under --two-process the compiled program is a child of the daemon, and the
daemon is the only thing that ever ends it: the program is a loop with no
reason of its own to stop, and lib/dev.ml kills it from a Fun.protect finally
block on the way out. That block does not run when the daemon is SIGKILLed —
by a harness tearing down a daemon it gave up on, by a watchdog, by anybody —
and what was left behind was a program with ppid 1, sleeping, holding a socket
nothing will ever connect to. Eight of those were on the machine this was
written on, the oldest six days old, and one had been minted by this suite.
The child now arms PR_SET_PDEATHSIG when it starts its agent, gated on a
FLAN_DEV_PARENT the daemon sets and nothing else does, and closes the race
where the daemon died before the arming by comparing getppid against that pid
rather than against 1 — under a subreaper an orphan is not reparented to init.
The signal is SIGPWR and not SIGTERM, because the daemon's ordinary teardown
already sends the child SIGTERM and the child's stderr is the daemon's own; a
handler there would print "the daemon is gone" into the daemon's output on
every clean close. The handler writes one sentence, unlinks the agent socket
it bound, and exits 0, and it does nothing else because it can run on the game
thread inside stdio's lock.
A merged build arms none of this and must not. There the daemon is the
process, so a dead daemon is a dead program; its parent is whoever typed
flan dev, which is a shell or an emacs and not the session's owner. Merged
daemons do leak on this machine — four of the eight — but for a different
reason, which is that accept_loop has no idea whether its editor still exists.
That wants a mechanism that tracks the client rather than the parent, and it
is not this change.
The daemon's read end of the program's stdout pipe is also cloexec now. The
child used to inherit it, which is both a leak and the reason waiting for EOF
on that pipe could never have been the mechanism: the pipe always had a
reader. A printing program now takes SIGPIPE once its daemon is gone, which is
a second net under chatty programs and not a substitute — the test uses
dev-watch, which is silent, so that only the thing under test can end it.
The test spawns the two-process shape, sends one request so that what it kills
is demonstrably serving, reads the program's pid out of the daemon's
/proc children before killing it — it is a grandchild, so waitpid is ECHILD —
and SIGKILLs the daemon. SIGKILL and not SIGTERM: SIGTERM lets the finally
block run and the assertion would be green with this change reverted. It fails
that way, and passes with it, in well under a second.
Six refusals in the runtime called _exit(134) where every other error had
learned to park: no restart by that name, a restart taken with the wrong
arguments or with none, a defer that invoked one, a null allocator, and
free-all on something with no region. Under a merged flan dev the compiler is
in that process, so a program that named a restart nobody established took the
session down with it, which is the one thing the break loop exists to prevent.
They park now. Not through flan_break_hook, which is what bounds and
arithmetic use: that hook may answer by aiming a transfer channel, and these
six are called by emitted code that falls off the end with no channel anywhere
in the call, so a restart chosen against one would be accepted and dropped.
flan_trap_hook says the other thing instead — stop here, let everything be
read, and refuse the resume with a reason.
All six park, for two reasons rather than one. Four are guards that fire
before the operation they guard, so nothing is half done and the frame reads
like any other. The other two fire mid-transfer, with the frame's defers
possibly half run, and they park only to be looked at: stopping on a torn
unwind is strictly more than exiting before anyone can ask what tore it.
The break loop grew a per-snapshot resumable flag for it. Restarts are still
listed and still numbered, the terminal marks them untakeable and the socket
reports the same positions as unreachable, and the listener refuses a choice
with the trap's own sentence rather than the thunk-boundary one.
Standalone builds die exactly as they did: nothing installs the hook in a
program that did not import the agent, and the acceptance case for free-all
still wants exit 134 and the same message.
The review entry that asked for this named flan_exit_hook, which is normal
termination and not this at all; it is struck out with the correction.
dev-chatty.flan outlives the surveys' twenty seconds by design, and unlike
dev-repl it prints while it does -- so the two backends stop at different
lines and the diff reports on scheduling rather than on lowering. It joins
dev-loop and dev-watch in the excluded-by-name list in both sweeps, with the
distinction written down.
rt_flush_out is guarded on __wasm__: the pipe it is careful about belongs to a
merged flan dev, which is only ever a native host, and wasm32 need not answer
for a descriptor mode its runtime may model differently.
And three comments that went false with the _exit: the atexit registration in
the merged entry point is no longer there for rt_die, which unlinks the socket
for itself now, so both places that said so say what it is actually left
covering.
bounds.flan covers this already and the x86 survey cannot see it: the
program picks its case out of (at args 1) and survey.sh runs everything
with no arguments, so re-gating x86's lo <= hi would have failed
nothing. This probe reaches the reversed slice on its own, through
(len args) so that neither optimiser can fold the branch and the checker
has no literal to object to. It matches under the default sweep and
under SURVEY_FLAGS=--no-bounds-checks, which is the claim.
Also records what the same reading turned up and did not fix: x86 still
reports a negative slice-from-ptr promise through flan_slice_error, so
the two backends print different sentences for it, and no corpus program
reaches that case without arguments.
Retrying immediately looked like eight chances and was one: a walk that bails
at the epoch check costs almost nothing, so all eight fit inside the single
compaction they were all losing to, and the listing refused having waited for
nothing. A quarter of a millisecond between attempts -- the agent's break-loop
idiom, legal here because the waiter is the listener thread and never the game
loop -- bounds the whole refusal at two milliseconds. Measured with a writer
noting and freeing on top of three thousand live blocks: 8 right answers in 200
without the pause, 200 in 200 with it. It is not magic, and the comment says
so: a writer that spends most of its time rearranging the table still gets
refused, which is the honest answer and used to be a zero-row lie.
The two cases the last commit left unwired are wired now, and a third joins
them: a listing taken while the table really is being compacted, which nothing
covered -- the full-of-live case never compacts and the churn case is
single-threaded, so the retry itself was exercised by nothing. It asserts only
what a slower machine cannot change: never zero rows, never a count that is
neither right nor a refusal. How the rest divides is printed, not pinned.
It ended "send it again once the program is running", which reads as a
withdrawal of what eval promises a parked program on every other evaluation:
that a body redefined while parked installs when the program is run again.
What is true is narrower — this module was not taken, and the run that drains
the sixty-four already queued is what makes room for it.
The three named arms are not every way a step after the check can fail.
write_file makes the daemon's copy of the module's text before llc is called at
all, and a Sys_error from it went out through serve's guard with the session
already holding the declaration — the same stranded declaration under a
different exception. Dev.eval now restores on the way past anything that
escapes, and carries an accepted flag so that a module the agent has already
taken is never rolled back out from under the process.
That is also the seam the crash can be reproduced through. test_dev takes the
daemon's working directory away from it — its own, not the macro cache the two
failures above use, which cannot fail a build with no macro in it — evaluates a
defn, and then calls it. Without the restore the daemon lists probe-two in
describe and the call closes the socket: the null cell, the game thread, and
address 0.
Two seconds was measured on one warm machine and has no margin against the
6.8s cold build listening already records. The status and the absence of the
frame-boundary sentence are what discriminate; this bound exists only so a
timeout cannot be read as an answer, so it moves to 4.5 and says so.
The unreadable-frame case wrote both frames as one string, where `(:a14' reads
as a typo rather than as the end of one frame and the start of the next; they
are two inserts now, with a line saying which is which. And the check that a
request after a daemon restart does not wait out a reply the old connection
owed was asserting against the same number it had bound the timeout to -- the
one arrangement that cannot tell "fast" from "timed out a moment ago". The
timeout is ten seconds and the assertion is three, which is the gap the claim
is actually about.
