with_errors had no Sys_error arm, so flan check nosuch.flan ended in OCaml's
default handler; the daemon has had that arm since before the CLI did. A
Not_found backstop joins it — nothing reaches it today, and the day something
does the failure should name the file rather than say nothing at all.
flan run handed every flag it did not understand to the compiled program:
flan run game.flan --debug built at -O2 and gave the game a --debug. Build
flags are now the build's, -- ends them, and an unknown dash argument before
-- is refused by name with -- named as the way to mean it for the program.
-O0 through -O3 get a spelling on build and run, which they did not have at
all: Build.default pinned -O2 and --debug was the only route to anything
else. Four levels and not five, because -Os is clang's and llc rejects it,
and the same string reaches both. --debug with a higher level is refused
rather than quietly overruled by Build's own -O0.
The name freed up by the rename now means what C means by it: the members
overlay one storage, the size is the largest of them, the alignment the
strictest, and nothing anywhere records which one was written. It serves
two things that wanted it. Binding a C header means holding the union the
library holds and reading whichever member the library's own tag says is
live -- a tag Flan cannot see, because the rule relating them is prose in
a manual. Overlaying an f32 on a u32 to look at its bits is the other,
and it is the same read.
So that read is defined rather than refused. This is the one place in the
checker where bytes win over safety on purpose, and the alternative was
not a safer language, it was no feature: type punning *is* reading the
member that was not written. The promise is the one C's implementations
make and C's standard does not -- the layout is the target's, the bytes
are the bytes, a read is a reinterpretation of them -- and what is not
promised is anything about bytes nobody wrote, where a member wider than
the one last stored reads a tail that is indeterminate exactly as a
struct's padding is. ZII narrows that to almost nothing: a union starts
all-bytes-zero unless uninit says otherwise.
uninit on one is allowed, unlike on a defdata. The refusal there was
never about garbage; it is that a tag steers, and a tag no case names
falls past every comparison in a match into a block LLVM may treat as
unreachable. An untagged union steers nothing.
Which is also why three things are refused, each for a reason that does
not expire with a milestone. No move-only member: nothing knows which
member is live, so nothing can tear one down, and unlike the struct and
defdata refusals this is not waiting on recursive teardown -- there is no
fact for teardown to read. No bool at any depth: an i1 loaded from a byte
that is neither 0 nor 1 is a value the optimiser may assume cannot exist,
and a union is the only type that can produce one. No defdata at any
depth, for the reason uninit gives, arriving the other way round. An
Option member is fine and the walk says why: its match is a tag test and
a branch, not a chain with an unreachable tail.
Two members in one literal, a match on a union, a union map key and a
member written into a global initialiser are each refused by name.
A union is a field list whose every offset is zero, so it travels as a
Tast.structure and the checker, the emitter and the x86 backend each grow
one table rather than one shape. A value is a zeroed temporary and a
store -- Set over Pfield, which every backend already has -- so there is
no new IR node and no layout rule spelled out a second time per backend.
The LLVM type is the blob clang gives a union, the DWARF is
DW_TAG_union_type with every member at zero, and the printer names the
type and does not walk it: it cannot know which member is live, and one
of them may be a pointer.
cimport can now check what it could not. A C record holding a union
member was not recorded at all, so the defstruct beside it went unchecked
rather than checked wrongly; a named union member resolves to a defunion
now and the whole record is compared field by field. The defunion itself
is compared against the header's union as a set and not in order --
every member is at offset zero, so a permuted one is the same type and
reporting it would be a finding that is not one -- while a member the
header has and Flan lacks is reported, because that is what changes the
size. A defunion against a C struct, or a defstruct against a C union,
is reported in both directions. An anonymous union member is still
skipped, and the comment now says that the gap is on the Flan side:
there is nothing to declare.
Flan's tagged sum has been spelled defunion since it landed, which was
accurate right up until the language wanted C's untagged union as well.
Both cannot be called the same thing, and the tagged one is the one with
an alternative name that says what it is: a case, its fields, and a tag
that steers which case is live is a data type, not a union.
So the form is defdata everywhere -- the parser, the AST, the checker,
both backends, the prelude's Form, the editor's font-locking and imenu,
the docs and every .flan file in the tree. The internal vocabulary moves
with it: Tast.union is Tast.data, uname is dname, the tables the checker
and the emitter keep are datas. Leaving them would have inverted the
words permanently, with surface defunion meaning one thing and
env.unions meaning the other, which is exactly the kind of drift the
comments in those files exist to prevent. What did not move is case,
variant and vfields: a tagged sum still has cases, and it still has one
live at a time.
defunion is not kept as an alias. An alias would compile the day the
untagged form lands and mean the opposite of what it used to -- the same
silent misparse that made defn's return type mandatory, and worse,
because the reader would have no reason to look. The old spelling is a
named refusal instead, parse/defunion-renamed, which says what it is now
called and that the name is reserved for something else. It fires on the
head alone, so (defunion U [A B]) -- which would otherwise have parsed
cleanly as one field A of type B -- is refused with the rest.
`flan emit --x86` printed a three-line header and then nothing but .byte
blobs. The information was all there and none of it was written down.
Each run of bytes is now headed by the Flan form that produced it, with the
position it was written at, indented by how deeply the form nests. The
headings are queued rather than written, so a form that emits nothing does
not leave its heading on the next form's bytes; atoms queue none at all,
because a literal operand would otherwise steal the heading standing above
the imul that consumes it.
Above each function is a frame map, which is the half no disassembly
recovers: every value in this backend lives in a frame temporary, so
-0x20(%rbp) is the whole vocabulary of the listing and nothing says what it
means. It is read out of what emit_fn already keeps, so it cannot drift.
Beside it, where the arguments arrived and whether there is a hidden sret.
And the bookkeeping is named where it appears -- the transfer guard, the
bounds triple, the arithmetic guards, rep movsb, the dev indirection cell --
with each explained once in a legend at the top rather than at every site.
Always on for `emit --x86`, which exists to be read, and never for a build,
whose .s is a temp file handed to clang. spike/x86/annot.sh is the check that
this costs no byte: emit both ways, assemble both, compare every section.
342 SAME / 0 DIFFER over the corpus in default, --dev and --debug. dump.sh
now shows the annotated listing beside objdump's disassembly -- why beside
what, which is the pairing that answers the mnemonics question.
survey.sh has not been run on this; see the handoff.
Item 3, and the reason the backend was written. Until now --x86 was read only
by flan build's argument list; the daemon built both halves through LLVM, so
none of this reached the dev loop at all.
The choice is a session setting, not a per-command flag, and it is spelled
exactly as [debug] already is -- one field on Session.t, set once in Dev.start,
carried on every change the session emits. session.ml's comment on [debug]
already gives the reason and it is the same one: the modules have to match the
process they are loaded into. Session.redefinition is the single place that
picks a backend, so the six call sites cannot disagree and the refusal has one
home. Session.change carries the answer beside the text, so the builder and the
text can never come from two different decisions.
There is no fallback and there must not be one. X86.redefinition refusing a form
is reported to the editor; quietly building an LLVM module instead is precisely
the crossed pair flan.abi.x86 exists to refuse at dlopen. A refusal reaches the
editor as a diagnostic like any other -- X86.Unsupported is re-raised as a
Loc.Error at the form it is about, because every caller already handles that and
none handled the other, and a session that died on the first unsupported form
would be worse than one that says so and stays up.
flan reload got the same flag at the same time. A command that could build a
module for a host the other backend compiled is how the crossed pair was
reachable from the CLI at all; the aggregate handoff's two-line reproduction no
longer has a second half.
And the finding: flan dev --x86 refuses the merged daemon. A merged build is the
program and the compiler in one process, and the compiler expands macros by
dlopening a module Build.macro_module made through Emit.program, cached on disk
by the macro source rather than by the backend. The merged host is linked
-rdynamic so a redefinition module can reach its cells, which also exports every
flan.* body it has -- so the macro module's own copy of a prelude function is
interposed by the host's. With an LLVM host nobody notices. With an --x86 host
the caller is LLVM and the body it lands in is this backend's, and the process
dies inside flan.[clamp] during the first macro expansion, before the program
has started. flan.abi.x86 does not catch it and was never meant to: a macro
module deliberately neither defines nor requires a marker. The honest fix is
hidden visibility on a macro module's Flan bodies, which changes the cached
object for both backends and wants a lane of its own. Until then the refusal
names the mechanism and the remedy, and --two-process has no such meeting.
start_merged keeps its --x86 plumbing, unreachable for now, because it is the
half that is right and will be wanted the day the macro module is fixed.
test_dev.ml drives an --x86 daemon through C-c C-c, C-x C-e, a literal, a new
defvar with a value of its own and a new defn, and asserts (twice fresh) is 82 --
which only holds if both registry lookups resolved. The merged refusal is
asserted there too. bin/main.ml learned to print a bare Failure as a sentence
rather than an uncaught exception and its backtrace.
The transfer channel was the only thing between 41 programs and the
corpus. It is there now: a guard after every Flan call, a landing pad
per restart-case, handler-bind and with-allocator, a transfer exit per
function that runs its fdefers, and check_at and check_slice, which
could not exist until the guard did.
Measured by what the programs print and what they exit with, never by
reading bytes. spike/x86/survey.sh builds every program in
test/programs both ways and diffs stdout and the exit status; it did
not exist, so it is here too, and it is the progress meter.
before 41 MATCH 1 DIFFER 41 refused by name
after 83 MATCH 0 DIFFER 0 refused by name
The one DIFFER was bounds.flan, and it was the honest answer to
"--x86 is silently a --no-bounds-checks build". It is not one any
more: check_at and check_slice signal through the channel exactly as
emit.ml's do, so a bounds violation signals, a restart-case catches
it, and an unhandled one exits 134 on both backends. The transitional
refusal that would have said so retired before it was written.
check_no_transfer is not removed, it is narrowed to the one place the
argument still holds: a global's initialiser runs from
flan..init-globals, before main and before anything can handle
anything, so a transfer out of it has nowhere to go.
Four bugs, and three of them are the shape item 16 predicted -- code
that reads correctly and answers wrong, found by output and not by
objdump:
- The body fell through into the transfer exit, so every fdefer ran
twice on a normal return. emit.ml cannot have this bug: its ret
terminates the block.
- A Vec crossed to the runtime as the address of a *copy*, so pushes
grew the copy and an in-bounds (at v 1) signalled against a length
of zero.
- ucomis sets CF, ZF and PF together for a NaN, so sete answered true
for (= x x) and the prelude's NaN test never fired: (/ 0.0 0.0)
formatted as -9223372036854775808. Flan's comparisons are LLVM's
ordered ones, so < and <= swap and =, != take a setnp beside them.
- A union read field 0 through the struct table and was refused by
name rather than laid out as a tag and a payload.
And one that could not have been found later: emit_globals_init stored
a null *into* the channel slot rather than a cell address into it,
which is a null pointer for every callee to write through. Harmless
while nothing could transfer; a fault the first time a guard loaded
through it.
Load.program takes forms: it reads the import forms, resolves them with the
one resolver it always had, and parses the file with the packages' macros in
front of it. The refusal said this needed a second import resolver at the Form
level. It did not notice that the file being compiled is parsed before Load
runs too, so no shape of the feature could have left import resolution where
it was.
Names arrive qualified, as a defn's do. (mac/twice 4) is a call and (twice 4)
is an unknown name.
Stopped mid-task: dune test was never run and the acceptance wiring is
unfinished. HANDOFF-macros.md has what is left.
x86.ml was an encoder and a frame model with nothing calling it. It now
lowers a whole Tast.program to an assembly file, and `flan build --x86`
hands that file to the same clang invocation the LLVM path uses, against
the same runtime objects. The flag is off by default; LLVM stays the
release backend and the default one.
Three programs, built both ways and compared by what they print and what
they exit with rather than by reading bytes: exit 0; a dotimes that
prints; and a fizz over a call, an if, a remainder and two string
literals. All three agree with the LLVM build.
The measurement decided the target. hist.ml over the fizz program shows
no Signal, no Handled, no RestartCase — a loop that prints does not drag
conditions in. What does is the bounds check and the allocator, and
neither is in the reachable set of a program that prints a number.
That is why there is no transfer guard here, and check_no_transfer is
what makes the omission sound rather than hopeful: if nothing reachable
can write the channel, no call can return with it set. It is a
whole-program property, so it is checked once per build and the build
stops with the node's name when it fails.
check_constants makes two kinds of finding and they were treated alike.
A value that does not match, or a C name the header does not have, is
the library contradicting the package and stops a build the way a
permuted defstruct does. An enum nobody mapped and a rule that reaches
nothing are about the package's own bindings file -- real, and worth
fixing, but telling a lane that added a defenum to go and edit a config
in a message shaped like "your layout is wrong" is the wrong thing to
fail a build with. Those gate generate-c, where that file is edited.
Also: a const prefix now counts as reaching a name before an explicit
constant line is consulted, so a rule whose every match is also spelled
out by hand is not reported as matching nothing.
Two gaps the raylib examples hit.
The layout check compared a Flan enum against the header's `int` and
called it a disagreement. It is not one: Shim.cty lowers a defenum to
int32_t in a struct field exactly as it does in a parameter, which is
what the signature check already knew and the layout check did not. One
predicate now serves both, symmetric, and tolerant of a 32-bit integer
and nothing else -- f64 against the library's float still fails, in the
very struct whose other field is an enum. Camera3D.projection is a
CameraProjection again and rl/camera-projection is gone with it, so
`.projection :perspective` resolves at the construction site.
And generate-c's claim said nothing about a defconst or a defenum
member, so a wrong flag bit was completely silent. `bindings` gained
`enum`, `const` and `constant` lines saying what a Flan constant is
called in C -- the prefix is nowhere in the Flan name, so it is declared
rather than guessed. Nothing goes quiet in either direction: a name the
rule builds and the header lacks is reported, a rule that reaches
nothing is reported, and a defenum with no line is itself a finding,
because otherwise the silence just moves up one level.
clang's dump gives anonymous EnumDecls for every raylib enum and no
value at all for an enumerator written without `= n`, so the constants
are one flat table and the values are counted the way C counts them.
cache_format bumped with the dump type.
generated.flan carries the 253 declarations the importer reads out of raylib's
header, so a build needs libraylib linkable and no header at all. The opt-in
no longer decides how many bindings a package has — every build now gets all
425, they are greppable, and they diff when raylib moves.
What that gives up is the build-time check, so `flan generate-c` is the only
thing that writes the file and it compares first: every defstruct against the
header's record, every hand-written declare-c against the header's signature,
and it writes nothing when they disagree. Against the 5.1-dev header on this
machine that is ten real differences and no write.
The 172 hand-written lines stay, and not out of caution. Everything the
generator emits agrees with the header by construction, so diffing generated
output against its own source is a tautology; the hand-written lines were
transcribed by a person, so they are the only thing here a header can
contradict. All ten of those differences came from them.
`bindings` beside `headers` is what survives regeneration, because a hand-edit
to a committed generated file does not. Two directives: `exclude` drops
raylib's three allocator entry points, and `name` gives the 19 generated
predicates the `?` spelling the hand-written ones already use.
Loc.Errors is a second exception, and the handlers in the session and the
daemon name only Loc.Error — so a list reaching them is an unhandled
exception and a dead session, which is the one thing the dev loop exists to
prevent. A flag on the function the session already calls left that one
label away from happening. Parse.program_all and Check.program_all are
separate names, so the session's call site has to be edited by a person for
its behaviour to change, and the guarantee stops being a default argument.
Placeless diagnostics now sort last rather than first. A wrong main signature
is raised against unknown, which is line 0, and sorting on the number alone
put it above every error that can actually be clicked. It is a real error and
it is not anywhere, so it goes after the ones that are.
A sink collects what a pass found so the pass can go on to the next thing.
It is switched on by the caller, not by the code that raises, which is what
leaves the interactive path untouched: the daemon checks one form, asks for
a sink that is off, and still gets one exception.
Two resync points, and both are places the work already had a boundary. In
the parser it is a top-level form — the reader found where each declaration
ends, so skipping a bad one cannot lose its place, while inside a
declaration there is no such landmark and one bad defn stays one error. In
the checker it is the two passes: pass one, which builds every name and
signature, still stops at the first refusal, because a signature it could
not make sense of leaves a hole that pass two would report once per mention.
Thirty unknown-name lines under one wrong signature are not thirty errors.
Pass two is where the volume is and where collecting pays, and by then every
signature is sound, so a body that fails cannot make the next body fail.
That is what makes a declaration a resync point needing no resynchronising.
The first line of an entry is still exactly file:line:col: message, because
that is the GNU format compilation-mode already parses and the whole of the
editor story. Everything under it is indented, which compilation-mode
ignores, so the underline is free. A note gets an entry of its own rather
than being folded into the error's block — that is what makes the second
place somewhere next-error can go, and is the reason notes carry locations.
Every part of it degrades to the bare first line: a location the checker
invented has line 0, the prelude and the REPL have names that are not paths,
and a file can change under us between being read and being blamed. An error
printer that can raise is worse than one that prints less.
Loc.Error now carries a diagnostic: a stable kind, a span, notes that each
have their own span and severity, and the macro expansion it came from. The
notes are the part that was actually missing — "this is wrong here" plus
"because of that, over there" is two places and two explanations, and a
single string can state only one of them.
The compatibility story for the daemon, which was the open question: the
single-diagnostic exception stays the single-diagnostic exception. Session
and dev evaluate one form and have one failure to report, so they take a
location and a message out of it with Loc.summary and are otherwise
unchanged. A second exception carries a list, and only a driver that
compiles a whole file raises it, so nothing interactive has to know it is
there.
No message text changed.
`flan dev` now builds one binary that is the compiled Flan program and holds
the whole OCaml compiler, and execs it. The program keeps main() — macOS needs
the window there — and caml_startup happens on a pthread beside it, next to the
listener flan_agent.c already starts. The editor's socket and the wire protocol
are untouched: Emacs cannot tell the difference.
Two rules are written into lib/dev.ml rather than discovered later. The game
thread must never call into OCaml, because a native thread has no safe points
and so can never be stopped by the collector — which is exactly why a frame is
never paused, and exactly what one convenient direct call would undo. And no
OCaml value may be stored in Flan memory without caml_register_global_root,
which is the way the spike's "the GC does not touch the arenas" measurement
stops being true.
The link is spelled in dev.ml out of Build's existing public pieces rather than
as a mode of Build.executable: lib/build.ml belongs to another lane this week.
It should collapse into Build once that lands.
A Flan main does not return — Emit ends it with flan_exit and an unreachable —
so in one process that call would take the compiler down with a program that
merely finished. flan_rt.c grows a hook, null in every other build, that the
merged entry point uses to flush, close stdout and park. The compiler then
learns the program is done the same way the daemon did: the pipe reads EOF.
--two-process keeps the old shape for a machine that cannot build the compiler
object, and nothing has been deleted.
declare-c generates the wrapper, the typedefs and the prototype from one
declaration, so they cannot disagree with each other. What nothing checked was
whether the declaration matched the library — BUILT.md records that as trusted
rather than guaranteed, because no header was ever read.
This reads one. clang is asked for a JSON AST dump of the header and shelled
out to, not linked: -Xclang -ast-dump=json is the same binary on PATH that
every build already runs, which is plan.org's "Why LLVM IR as text" applied a
second time. Zig's old @cImport linked clang as a library and that is precisely
the dependency plan.org rejected.
cjson.ml is enough JSON to read the dump and no more, so this adds no opam
package to parse it.
What comes out of the header is signatures and nothing else — not structs, not
enums, not macros. The bound on how much is imported is the package's own
defstructs: a function whose signature mentions a struct the package has not
described is refused with that reason, so vendor/raylib describing thirteen
structs is what makes the import thirteen structs wide. Keeping the layouts
hand-written is also what makes checking them against the header's records
worth doing — a _Static_assert was rejected in BUILT.md as circular, and this
is not, because the two sides have different authors.
Refusals are demotions, taken from Zig's translator: it never drops a
declaration it cannot handle, it binds the name to a @compileError carrying the
reason so the failure lands at the use site. Load.refuse_hidden is already that
mechanism. So a returned char * does not kill the header — it makes one name
unavailable, with the reason attached.
flan import-c prints what it would produce, what it refused, how the package's
defstructs compare with the header's records, and how the hand-written
declare-c lines compare with the header's signatures.
Against raylib 5.5, the version whose .so vendor/raylib/link names: all 16
defstructs and all 172 hand-written declare-c agree exactly. Against the 5.1-dev
header installed in /usr/local it reports ten differences, nine functions that
version does not have and one that gained a parameter — so the check has teeth
and the clean run is not a vacuous one.
flan build --target=web produces a page, its JS and a .wasm. The two wasm
targets share the word and almost nothing else, so is_wasi and is_web are
separate predicates and is_wasm is their union — the union is exactly the
facts about the machine, 32-bit pointers and no dlopen, which is what the
refusals are about.
Everything the wasi target has to find by hand is what emcc already is: no
sysroot, no builtins archive, no shadow resource directory, and no
__main_argc_argv shim, because emscripten's start code calls main under that
name. target_flags for web is empty and the only thing checked is that emcc
exists. The one fact this rests on is that emcc takes a .ll on its command
line, so Emit's output needs no change.
The main loop is -sASYNCIFY rather than emscripten_set_main_loop, which
BUILT.md predicted. The prediction had the browser right and the cost wrong:
set_main_loop wants the loop body as a callback, so every example that writes
(until (rl/window-should-close?) ...) would be split by hand into an init and
a tick and would stop being the native program. raylib's web platform is built
for asyncify instead — WindowShouldClose on PLATFORM_WEB is an
emscripten_sleep(16) that returns false — so the loop yields at a call it
already makes and no example changed a character. Asyncify goes on every web
link, because whether a program blocks is not a question Build can answer and
a per-program flag set is a per-program cache key.
A link line may now be addressed to one target — @native, @wasi, @web — and
${NAME} expands from the environment. The selection is here and not in Load,
which reads the file, because Load resolves imports before a target is chosen.
The object cache now keys on whichever compiler the target uses, so an emcc
object and a clang one of the same source cannot collide. The refusals name
the target that was asked for; --sanitize on web says the weaker truth, that
emscripten ships an ASan and nothing here has ever run it.
ASan is an LLVM pass but instruments only functions carrying
sanitize_address, which clang's C frontend adds and nothing adds to IR
written by hand. Passing -fsanitize=address to the clang run over the
.ll therefore instruments flan_rt.c and not one instruction of Flan: an
out-of-bounds read of a defvar array, built --no-bounds-checks, printed
its garbage and exited 0. With Emit naming an attribute group on every
define, the same program reports global-buffer-overflow in flan.main.
UBSan has no such lever. Its checks are branches the C frontend emits to
__ubsan_handle_*, not a pass, so -fsanitize=undefined covers the runtime
and nothing else; (<< 1 32) still goes unremarked. Recorded where it
will be read rather than discovered again.
The flag does not force -O0 the way --debug does -- the UB worth finding
is what the optimiser does with it -- and it does pull in -g, since a
report with no line costs more than the build. compile_c's cache key now
digests the same cflags list the command line uses, because an
unsanitized flan_rt.o served out of the cache links fine and reports
nothing.
Emit.redefinition has taken ~debug since it was written and was tested
with it; Session.eval never passed it, so every body installed by C-c C-c
lost its debug info in the running process.
Passing it alone would have been half a fix. Build.shared is what forces
-O0, and dev.ml built modules at -O2, so the llvm.dbg.declares would have
been emitted and then deleted by mem2reg: a line table, and no locals.
And a module with DWARF loaded into a host without it lines up against
nothing. So it is one flag — flan dev --debug and flan reload --debug —
and it sets the host build, the module builds and the emitted metadata
together. Off by default: a debug build is an -O0 build, and quietly
making every reloaded body -O0 changes the frame time of the one function
you are iterating on, in the loop whose point is watching that number.
What a dlopen'd module does to a breakpoint, measured against the reload
fixture rather than reasoned about:
- lldb reads the new module's DWARF on the dlopen and says so: "1
location added to breakpoint 3".
- A breakpoint set by NAME gains a second location either way, so
dlopen was never the difficulty. What the line table buys is that it
stops with source instead of disassembly.
- A FILE AND LINE breakpoint on the new body resolves only with it;
without, it sits at locations = 0 (pending) forever.
- A FILE AND LINE breakpoint on the HOST's copy stays pinned at
locations = 1. That is correct, not stale: the old body is still
mapped and every call site that has not gone through its cell again
still reaches it.
- The stack crosses intact — a frame in the reloaded .so and the one
below it in the host each name their own .flan file.
(lldb) frame variable
(long) step = 10
(long) prior = 11
The transcripts are in flan-dape.el, replacing the note that said the
module carries no DWARF yet.
flan-cnr.el's stack pane was refusing for the wrong reason. DWARF was
never its gap; nothing is attached to the stopped program, and a socket
cannot read another process's frames. Reworded to say that.
Source interleaving in the disassembly buffer is unblocked and not done:
objdump -dS interleaves a --debug module's Flan source correctly, so
Dev.asm_of needs the -S and a parse_listing that tolerates source lines.
--debug is a third flag beside --dev and the optimisation level because it
answers a third question. --dev is "can I redefine this while it runs";
--debug is "can I stop it and read it". Either is useful without the other,
and a REPL session that is not being stepped should not pay for DWARF.
Not implied by -O0 in particular, for a reason already written down in this
file: the acceptance table runs the same programs at -O0 and -O2 to compare
the emitted IR against what mem2reg makes of it. If -O0 pulled in debug info,
every one of those comparisons would be against a different module.
It does imply -O0 downwards, and sets it. The whole mechanism is an
llvm.dbg.declare hanging off an alloca, and mem2reg deletes the alloca.
Refused for wasm32 by name. The member offsets in the DWARF are computed for
the host — ptr is 8 bytes — and wasm32's pointer is 4, so a slice's len sits
at byte 8 there and byte 16 here. Emitting the host numbers would hand a
debugger a confident wrong answer for every slice and every struct holding
one, which is the exact failure this project keeps meeting at the FFI
boundary. Silence would be worse than the refusal.
-g reaches the C compiles too, and joins compile_c's digest key with it, or
an object built without it would be served to a build that asked for it.
vendor/raylib has no C in it any more: shim.c is deleted and its 84 wrappers
are emitted from declare-c, which names the library's function in the library's
own signature. The reason the shim exists is unchanged - a small struct's
calling convention is a per-target classification and clang reproduces it for
free - but writing it by hand has stopped.
declare-c is a second form rather than a change to declare, because the two make
opposite claims about the same shape: (declare start-raw [path string] ...) says
the symbol takes ptr+len, and (declare-c init-window [... title string] ...)
says it takes a NUL-terminated char*. No structural rule separates them, so the
author says which.
The merge needed two fixes that neither lane could have found alone.
Load's uses-walker matches decl_kind exhaustively and did not know DeclareC, so
the reachability work and the generator did not compile together.
And the generated C is now emitted in parts keyed by the wrapper's own C symbol,
not as one translation unit. Reach.link drops the bindings nothing reachable
calls; a single TU holding every wrapper referenced every raylib symbol, so
sand-headless - which deliberately links no libraylib, and is the reason Reach
exists - failed at the link with undefined references to GetTime and its
neighbours. The first attempt keyed the parts by Flan name and broke the other
way, dropping a wrapper that was called: the flattened declaration is named
foo-c when a Flan wrapper is generated over it and foo when none is needed, so
the Flan name is not one thing. The wrapper's C symbol is what the declaration
binds in both branches.
Worth recording how close that came to passing: the acceptance suite died with
an exception rather than printing FAIL, so a grep for failures counted zero and
the suite looked green. Only the count of reporting suites - ten where there had
been eleven - showed it.
84 hand-written C wrappers is the shape of a job the compiler should be
doing. The reason the shim exists is unchanged and is not negotiable: a
small aggregate's calling convention is a per-target classification, not
part of its layout, and reproducing x86-64, arm64 and wasm32 inside
emit.ml is three classifiers to keep correct forever, where a mistake
reads as a field full of garbage rather than as a link error. clang does
it, per target, for free. So the C stays; the typing of it stops.
declare-c names the library's own function in the library's own
signature, and Shim emits the typedefs, the extern prototype, the
flattening wrapper and the flattened declaration the Flan side calls.
It is a second form rather than a change to declare because no
structural rule can separate them: (declare start-raw [path string] i32
"flan_agent_start") means the symbol takes ptr+len, and (declare-c
init-window [w i32 h i32 title string] "InitWindow") means it takes a
NUL-terminated char *. Same shape, opposite claims. declare is
untouched, so sqrtf and vendor/agent keep working unedited.
The generated C rides on Tast.program rather than beside it, so the CLI,
the REPL and the acceptance table all carry it without being told about
it. `flan shim` prints it, because a wrong binding is wrong in a wrapper
that is otherwise on no disk anywhere.
A package handed over its .c files and its `link` arguments the moment it was
imported, whatever the importing program did with it. That is what made sand's
two halves two files: anything naming vendor:raylib linked libraylib on every
target, and on wasm32 that link cannot succeed, so the headless run could not
so much as mention the package the interactive one needs.
Reach.link answers it from the checked program instead. Start at main and at
the globals that run before it, follow every call — including the Handled
frames, where a lifted handler clause is reached by address and by nothing
else — and keep what is reached. A package none of whose externs survive
contributes no C and no linker argument.
Dropping the flags alone would only move the failure: the bodies that called
into raylib would still be emitted, and wasm-ld would fail on the symbols
rather than on the argument. So the same walk prunes the functions and externs
too. Only those — globals, structs and unions stay, because an unreferenced
global is bytes in BSS and a dropped one is a silently different program.
Dev builds keep everything. What a REPL may redefine next is not a function of
what has been called so far.
The wasi-sdk candidate had an LLVM version in it, which moves release to
release — so the path advertised as the proper article would have matched only
by coincidence, while the emscripten one beside it was derived. Both are
derived now.
calc-me on wasm32 covers what the other three cases cannot: flan_argv hands
Flan an array of flan_slice built in C, so what it pins is the element stride
of a ptr+len pair — 16 bytes native, 12 on wasm32 — rather than a field
offset. It is also the claim in this file's own header, that the table runs on
the second target, honoured for the first time.
flan emit refuses --target rather than stripping it. The IR really is
target-free, so ignoring it is correct and silence about it is not.
--target= carries a value, so the flag test becomes a prefix match and the
residual-argument filter uses the same test — otherwise -o out --target=X fell
into the usage error. A wasm build defaults to a .wasm name, since the
extension is what tells a runtime, and a reader, what the file is.
flan run refuses the flag by name. It builds and execs, a cross-built module is
not something this host execs, and choosing a runtime for it is not a decision
this command should be making quietly.
The piece between an editor and everything else. One long-lived Session, the
program it belongs to launched and owned by the same process, and a socket that
takes forms and installs them. What it adds over flan reload is that the
session persists - a defvar added by one evaluation is part of what the next is
checked against - and that it owns the build, which is what makes its layout
rules describe the process actually running rather than a guess about it.
The protocol is s-expressions rather than bencode, and I changed my mind about
that. The case for nREPL was reusing a designed op set and not re-litigating
session identity, but with the client ours too there is no CIDER to be
compatible with, its eval is string-in/string-out with no slot for which form
from which file, and Emacs already has read and prin1. So: one sexp per
message, length framed because the payload contains newlines. No parsing code
on the editor side, and on this side the parser is the language's own reader,
where :op is already a keyword and Flan source is already a string literal. An
nREPL front end can sit on the same Session later; it should not gate the
editor.
Two silent failures the daemon refuses to have. The agent socket is chosen by
the daemon and forced through FLAN_AGENT_SOCKET before spawning, because a
program's source has to name some path and a daemon that guessed would compile,
build and deliver a module to nobody. And delivery is checked: agent/start
returning 0 means a socket was bound, not that anyone connected, so a failed
connect or a reply that is not ok becomes an error the editor sees.
It waits for the program to bind before accepting an evaluation, since one
arriving first fails for a reason that reads like a compiler bug, and it
accepts with a timeout so a program that has exited takes the daemon with it
instead of leaving an editor waiting on a socket nobody serves.
lib/session.ml holds the declarations a running process was built from plus
every change accepted since, which is what an editor needs and what a one-shot
compiler cannot have.
Transactionality came for free. Check.program builds a fresh environment from a
declaration list on every call, so a form that fails to check mutates nothing
and the accumulated list is simply not replaced - no scratch-environment
machinery, which is what I was about to build. Re-checking the whole program
each evaluation costs the frontend, under 10ms, less than the llc after it.
There is a test for the case that matters: a typo, then a good form, in the
same session.
Which names the process was built with comes from the checked program, not from
any accumulated AST, because Check.program prepends the prelude and no AST
contains it. Derive it from declarations and print-line reads as new, gets a
registry cell nobody publishes, and the first call jumps to null.
Three changes are refused with a reason rather than loaded. A function's
signature, because a cell is a bare ptr and every call site compiled before the
change still passes the old arguments through it. A global's type, because the
storage exists and has a shape - reusing it reads at the wrong offsets, and
replacing it discards the state the reload exists to preserve. A struct's
fields, because the values the process is holding have the old layout. Note
what the checker already catches on its own: change a parameter type and the
caller fails to type check first, loudly. These rules only get a turn on a
change the checker accepts, which is a name nothing else in the program uses -
exactly where the silent version lives. Hence an unused defvar and a C-called
defn in the fixtures.
The accumulated list is the post-Load one, so an evaluated import is spliced as
its expansion. Otherwise re-evaluating a file that imports something appends a
second import, Load expands it again, and the duplicate-name pass rejects it.
C-c C-k on sand.flan's own text is the test.
flan reload now takes a program and a file of changed forms rather than a list
of function names and a --new list: the session works out which names are new,
which is the thing a bare CLI could not.
Also fixed, found by running the agent test under load: the agent took SIGPIPE
when a sender read part of a reply and closed. Replies go out with
MSG_NOSIGNAL, per call rather than by installing a handler, because the signal
disposition belongs to the program the agent is embedded in.
vendor/agent/ is a package like any other - agent.flan declares three calls,
flan_agent.c implements them, link asks for -lpthread. start listens on a unix
socket, poll installs whatever arrived and says how many, wait does the same
after waiting for something.
The split between poll and the listener is the whole design. dlopen relocates a
module and takes the loader lock, which is milliseconds and unbounded, so it
happens on the listener thread. flan_reload_install is one store per function
and must not land while a redefined function is on the stack, so it happens on
the game thread at the top of the frame, when the program asks. A ring and two
atomics connect them; the game thread never blocks on the loader.
wait exists for tests. A test that races the frame rate fails on a loaded
machine, so test/programs/agent.flan waits for the reload rather than sleeping
past it. It also sends a junk path first: the daemon is a separate process and
can send anything, and a bad path must be refused rather than take down the
program it was sent to.
Two things came out of running it. The reply goes out before the module is
queued, because the other way round the game thread can install and the program
can exit between the two, and the answer reaches the sender as a connection
reset instead of as ok. And ok means queued, not installed - the sender does
not get to know when the swap happened, since only the program knows when it is
between frames.
sand.flan now polls at the top of its loop, which is what this step was for.
Under Xvfb, one line on the socket and 455 consecutive frames drew from a
game-draw that did not exist when the process started. Building without --dev
still works: there are no cells, so a module is refused on the listener thread
and the loop never notices.
flan reload builds one module the way the daemon will. --new names what the
host was not built with, which is the one thing the command cannot work out for
itself and exactly what the session will track.
Two things, and either alone is useless, so they are one commit.
Emit.redefinition compiles one function into its own module against a host
that is already running. What it does *not* define is the design: a global is
external, so state survives a reload and sand's grid is not reset by editing
the code; every other function is a declare, so a redefined settle calls the
host's move-grain rather than a frozen copy; there is no main. Build.shared
puts that text through llc + ld -shared. ld, not clang, because a shared object
is allowed undefined symbols and that is the whole mechanism - and because the
driver is 50ms of a 20ms job. Measured here: llc 16ms, ld 3ms, dlopen 0.04ms.
Loading a body is not installing it, though. A call bound at link time cannot
notice a new one, so a dev build routes every Flan-to-Flan call through a cell
- a mutable global holding the address of the function that is current - and a
module publishes itself with one store. The cell load is emitted after the
arguments, so a redefinition between two calls cannot land inside one.
Three details that are not free choices. flan_reload_install is a named
function rather than an ELF constructor, because the agent has to choose when
the store happens and a constructor would do it during dlopen, mid-frame, on
whatever thread called it. A redefinition's own body is hidden, because default
visibility in a shared object is interposable and that applies to taking the
address too: plain @"flan.bump" inside the module resolves to the host's copy,
so the installer would publish the function it was replacing and the reload
would silently do nothing. And -rdynamic is what exports the cells at all, so
it and cells are one flag: Build.opts.dev, flan build --dev, the first time
opts means something semantic rather than an optimisation level.
The test is one process, because two runs would prove nothing about a swap,
and two .so paths, because dlopen caches by path and would hand back the first
handle. Every call in it goes through outer, compiled once into the host and
never rebuilt, so a changed answer can only mean its call site followed. v2
recurses through its own cell, which is the interposition case; it would print
the old body's text if it did not. helper differs between the fixtures purely
as a tripwire for a module that grew its own copy.
LLVM cannot fold the indirection - the cell is an external mutable global - and
a --dev calc-me keeps 46 indirect calls at -O2. values, machine and
sand-headless now run as dev builds in the acceptance table too; the sand hash
is the one result that would notice a call reaching the wrong function.
Second stage of the milestone-2 frontend. calc-me.flan (12 decls) and
sand.flan (20 decls) both parse end to end, and both are test deps so a
regression fails `dune test` rather than surfacing at the CLI.
Three silent-misparse bugs fixed along the way -- all cases that read
cleanly and meant something else:
- dotimes/defer/some/try/fn fell through to Call, discarding their
binding and control-flow meaning. Now special forms. Forms from later
milestones (handler-bind, restart-case, loop/recur, defmacro, signal,
with-allocator, errdefer, await) are rejected outright rather than
parsed as calls.
- (Some 1) in first body position was read as a return type, because
(Option f64) and (Some 1) are identical s-expressions and the
heuristic was capitalisation. Now decided by the set of names actually
declared as types, collected in a pre-pass -- exact, and
order-independent so a type declared below its user still resolves.
- Array literals in value position were rejected outright.
Also adds NEXT.md with the handoff for the checker.
