84 hand-written C wrappers is the shape of a job the compiler should be
doing. The reason the shim exists is unchanged and is not negotiable: a
small aggregate's calling convention is a per-target classification, not
part of its layout, and reproducing x86-64, arm64 and wasm32 inside
emit.ml is three classifiers to keep correct forever, where a mistake
reads as a field full of garbage rather than as a link error. clang does
it, per target, for free. So the C stays; the typing of it stops.
declare-c names the library's own function in the library's own
signature, and Shim emits the typedefs, the extern prototype, the
flattening wrapper and the flattened declaration the Flan side calls.
It is a second form rather than a change to declare because no
structural rule can separate them: (declare start-raw [path string] i32
"flan_agent_start") means the symbol takes ptr+len, and (declare-c
init-window [w i32 h i32 title string] "InitWindow") means it takes a
NUL-terminated char *. Same shape, opposite claims. declare is
untouched, so sqrtf and vendor/agent keep working unedited.
The generated C rides on Tast.program rather than beside it, so the CLI,
the REPL and the acceptance table all carry it without being told about
it. `flan shim` prints it, because a wrong binding is wrong in a wrapper
that is otherwise on no disk anywhere.
The wasi-sdk candidate had an LLVM version in it, which moves release to
release — so the path advertised as the proper article would have matched only
by coincidence, while the emscripten one beside it was derived. Both are
derived now.
calc-me on wasm32 covers what the other three cases cannot: flan_argv hands
Flan an array of flan_slice built in C, so what it pins is the element stride
of a ptr+len pair — 16 bytes native, 12 on wasm32 — rather than a field
offset. It is also the claim in this file's own header, that the table runs on
the second target, honoured for the first time.
flan emit refuses --target rather than stripping it. The IR really is
target-free, so ignoring it is correct and silence about it is not.
--target= carries a value, so the flag test becomes a prefix match and the
residual-argument filter uses the same test — otherwise -o out --target=X fell
into the usage error. A wasm build defaults to a .wasm name, since the
extension is what tells a runtime, and a reader, what the file is.
flan run refuses the flag by name. It builds and execs, a cross-built module is
not something this host execs, and choosing a runtime for it is not a decision
this command should be making quietly.
The piece between an editor and everything else. One long-lived Session, the
program it belongs to launched and owned by the same process, and a socket that
takes forms and installs them. What it adds over flan reload is that the
session persists - a defvar added by one evaluation is part of what the next is
checked against - and that it owns the build, which is what makes its layout
rules describe the process actually running rather than a guess about it.
The protocol is s-expressions rather than bencode, and I changed my mind about
that. The case for nREPL was reusing a designed op set and not re-litigating
session identity, but with the client ours too there is no CIDER to be
compatible with, its eval is string-in/string-out with no slot for which form
from which file, and Emacs already has read and prin1. So: one sexp per
message, length framed because the payload contains newlines. No parsing code
on the editor side, and on this side the parser is the language's own reader,
where :op is already a keyword and Flan source is already a string literal. An
nREPL front end can sit on the same Session later; it should not gate the
editor.
Two silent failures the daemon refuses to have. The agent socket is chosen by
the daemon and forced through FLAN_AGENT_SOCKET before spawning, because a
program's source has to name some path and a daemon that guessed would compile,
build and deliver a module to nobody. And delivery is checked: agent/start
returning 0 means a socket was bound, not that anyone connected, so a failed
connect or a reply that is not ok becomes an error the editor sees.
It waits for the program to bind before accepting an evaluation, since one
arriving first fails for a reason that reads like a compiler bug, and it
accepts with a timeout so a program that has exited takes the daemon with it
instead of leaving an editor waiting on a socket nobody serves.
lib/session.ml holds the declarations a running process was built from plus
every change accepted since, which is what an editor needs and what a one-shot
compiler cannot have.
Transactionality came for free. Check.program builds a fresh environment from a
declaration list on every call, so a form that fails to check mutates nothing
and the accumulated list is simply not replaced - no scratch-environment
machinery, which is what I was about to build. Re-checking the whole program
each evaluation costs the frontend, under 10ms, less than the llc after it.
There is a test for the case that matters: a typo, then a good form, in the
same session.
Which names the process was built with comes from the checked program, not from
any accumulated AST, because Check.program prepends the prelude and no AST
contains it. Derive it from declarations and print-line reads as new, gets a
registry cell nobody publishes, and the first call jumps to null.
Three changes are refused with a reason rather than loaded. A function's
signature, because a cell is a bare ptr and every call site compiled before the
change still passes the old arguments through it. A global's type, because the
storage exists and has a shape - reusing it reads at the wrong offsets, and
replacing it discards the state the reload exists to preserve. A struct's
fields, because the values the process is holding have the old layout. Note
what the checker already catches on its own: change a parameter type and the
caller fails to type check first, loudly. These rules only get a turn on a
change the checker accepts, which is a name nothing else in the program uses -
exactly where the silent version lives. Hence an unused defvar and a C-called
defn in the fixtures.
The accumulated list is the post-Load one, so an evaluated import is spliced as
its expansion. Otherwise re-evaluating a file that imports something appends a
second import, Load expands it again, and the duplicate-name pass rejects it.
C-c C-k on sand.flan's own text is the test.
flan reload now takes a program and a file of changed forms rather than a list
of function names and a --new list: the session works out which names are new,
which is the thing a bare CLI could not.
Also fixed, found by running the agent test under load: the agent took SIGPIPE
when a sender read part of a reply and closed. Replies go out with
MSG_NOSIGNAL, per call rather than by installing a handler, because the signal
disposition belongs to the program the agent is embedded in.
vendor/agent/ is a package like any other - agent.flan declares three calls,
flan_agent.c implements them, link asks for -lpthread. start listens on a unix
socket, poll installs whatever arrived and says how many, wait does the same
after waiting for something.
The split between poll and the listener is the whole design. dlopen relocates a
module and takes the loader lock, which is milliseconds and unbounded, so it
happens on the listener thread. flan_reload_install is one store per function
and must not land while a redefined function is on the stack, so it happens on
the game thread at the top of the frame, when the program asks. A ring and two
atomics connect them; the game thread never blocks on the loader.
wait exists for tests. A test that races the frame rate fails on a loaded
machine, so test/programs/agent.flan waits for the reload rather than sleeping
past it. It also sends a junk path first: the daemon is a separate process and
can send anything, and a bad path must be refused rather than take down the
program it was sent to.
Two things came out of running it. The reply goes out before the module is
queued, because the other way round the game thread can install and the program
can exit between the two, and the answer reaches the sender as a connection
reset instead of as ok. And ok means queued, not installed - the sender does
not get to know when the swap happened, since only the program knows when it is
between frames.
sand.flan now polls at the top of its loop, which is what this step was for.
Under Xvfb, one line on the socket and 455 consecutive frames drew from a
game-draw that did not exist when the process started. Building without --dev
still works: there are no cells, so a module is refused on the listener thread
and the loop never notices.
flan reload builds one module the way the daemon will. --new names what the
host was not built with, which is the one thing the command cannot work out for
itself and exactly what the session will track.
Two things, and either alone is useless, so they are one commit.
Emit.redefinition compiles one function into its own module against a host
that is already running. What it does *not* define is the design: a global is
external, so state survives a reload and sand's grid is not reset by editing
the code; every other function is a declare, so a redefined settle calls the
host's move-grain rather than a frozen copy; there is no main. Build.shared
puts that text through llc + ld -shared. ld, not clang, because a shared object
is allowed undefined symbols and that is the whole mechanism - and because the
driver is 50ms of a 20ms job. Measured here: llc 16ms, ld 3ms, dlopen 0.04ms.
Loading a body is not installing it, though. A call bound at link time cannot
notice a new one, so a dev build routes every Flan-to-Flan call through a cell
- a mutable global holding the address of the function that is current - and a
module publishes itself with one store. The cell load is emitted after the
arguments, so a redefinition between two calls cannot land inside one.
Three details that are not free choices. flan_reload_install is a named
function rather than an ELF constructor, because the agent has to choose when
the store happens and a constructor would do it during dlopen, mid-frame, on
whatever thread called it. A redefinition's own body is hidden, because default
visibility in a shared object is interposable and that applies to taking the
address too: plain @"flan.bump" inside the module resolves to the host's copy,
so the installer would publish the function it was replacing and the reload
would silently do nothing. And -rdynamic is what exports the cells at all, so
it and cells are one flag: Build.opts.dev, flan build --dev, the first time
opts means something semantic rather than an optimisation level.
The test is one process, because two runs would prove nothing about a swap,
and two .so paths, because dlopen caches by path and would hand back the first
handle. Every call in it goes through outer, compiled once into the host and
never rebuilt, so a changed answer can only mean its call site followed. v2
recurses through its own cell, which is the interposition case; it would print
the old body's text if it did not. helper differs between the fixtures purely
as a tripwire for a module that grew its own copy.
LLVM cannot fold the indirection - the cell is an external mutable global - and
a --dev calc-me keeps 46 indirect calls at -O2. values, machine and
sand-headless now run as dev builds in the acceptance table too; the sand hash
is the one result that would notice a call reaching the wrong function.
Second stage of the milestone-2 frontend. calc-me.flan (12 decls) and
sand.flan (20 decls) both parse end to end, and both are test deps so a
regression fails `dune test` rather than surfacing at the CLI.
Three silent-misparse bugs fixed along the way -- all cases that read
cleanly and meant something else:
- dotimes/defer/some/try/fn fell through to Call, discarding their
binding and control-flow meaning. Now special forms. Forms from later
milestones (handler-bind, restart-case, loop/recur, defmacro, signal,
with-allocator, errdefer, await) are rejected outright rather than
parsed as calls.
- (Some 1) in first body position was read as a return type, because
(Option f64) and (Some 1) are identical s-expressions and the
heuristic was capitalisation. Now decided by the set of names actually
declared as types, collected in a pre-pass -- exact, and
order-independent so a type declared below its user still resolves.
- Array literals in value position were rejected outright.
Also adds NEXT.md with the handoff for the checker.
