Build.cachedir sat under TMPDIR, which dune makes private per run, so no
test run ever reused an object and every build in the suite was cold. It
moves to $XDG_CACHE_HOME/flan/objcache (FLAN_CACHE_DIR overrides), which
is safe because the keys are total: compile_c digests the source text,
the compiler's stamp and every flag; wasm_resource_dir digests the
builtins archive; compiler_object digests flan.cmxa and flan.a. Writes
were already .tmp-then-rename, so concurrent dune jobs are fine.
Macro.key was the one key that was not total -- prelude text plus the
call's forms, and nothing about the compiler whose codegen produced the
.so it names, which is dlopened straight back into this binary. Under a
per-run TMPDIR that never showed; under a durable cache it is a stale
expander that crashes rather than a compile error. It carries the
compiler's stamp now, handed across start_merged's exec in
FLAN_COMPILER_STAMP because a merged dev binary lives at a per-session
path and keying on that rebuilt a macro module every dev start.
Measured on dev-repl.flan, launch to bound socket: 2.0s cold against
0.48s warm. Whole-program flan build: 1.44s against 0.06s. Full dune
test 25.7s/30.1s before, 24.0s after, user CPU ~50s down to ~34s.
And the await: one timer covered two waits, a build then a bind, so
'the daemon never listened' was a wrong diagnosis of a build that had
not finished. listening now polls the process alongside the socket and
says which -- exited with a status, or still running and therefore still
building. A daemon that dies fails in milliseconds instead of costing
the whole timeout. Thirty seconds, down from a minute, because the build
it waits on is warm now.
The provenance rides on the location, not on the form, because the location
is the thing that already travels: Expand.unmarshal stamps the call site onto
every node a macro answers with, and that stamp goes on through the AST and
the typed IR untouched. Tagging it there means an error raised anywhere
downstream can name the macro with no field added to Form, to Ast or to Tast.
Outermost wins. The macro the author wrote is the one worth naming, not
whatever it expanded into on the way down.
The honest limit, since it would otherwise read as a claim: a macro's
expansion has no source of its own to point at, so the note lands on the call
site along with the error. What it buys is the reader knowing the code being
refused is not the code they wrote.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
plan.org milestone 5 says when, unless, until, cond and dotimes are special
forms only until macros land. This is the first one to stop being one, and
running test/programs/macro-unless.flan means the compiler built a shared
object, dlopened it into itself and called a Flan function to find out what
(unless c a b) means.
unless is the one that moved because it is the one nothing else needs: zero
uses in the prelude, so moving it cannot make the prelude depend on the
expander that compiles it. Its coverage is sand.flan, seven calls, compiled
through Session in test_session -- which is the in-process path and the reason
lib/dune now passes -linkall. Say plainly what that coverage is not: nothing
in test/programs used unless before today, so macro-unless.flan is a test
written after the feature. The corpus that was written before it is sand.flan
and web/examples/control.flan, and both compile unchanged.
lib/macro.ml is the half of expansion that has to compile something. Expand is
the image format and the quasiquote desugaring and depends on nothing above
Form; this needs Check, Build and Emit, so it sits above the parser it feeds
and arrives through Parse.expander.
What it does, in order:
- Collects every defmacro from the prelude and from the file. Not from an
imported package: Load learns a package's imports by parsing it, so
collecting from one means a second import resolver over Forms, and that is a
bigger thing than this.
- Builds them in rounds, because a macro's body may call a macro and a body
with an unexpanded call in it will not compile at all -- the call is a name
nothing defines. Round 0 takes every macro that names no macro still
waiting; round 1 expands the rest against round 0's module. A round that
takes nothing while macros remain is a ring and is named. macros.flan has
the round-1 case and macro-cycle.flan has the ring, and the distinction
between them is the one thing here that is easy to get wrong: a call inside
a quasiquote is *not* a compile-order dependency. It is part of what the
macro answers, and the answer is expanded again after it returns. The first
macro-cycle.flan written for this commit quasiquoted, and it was not a cycle
at all -- it hit the fuel instead, correctly.
- Walks bottom up, so a macro never sees a call to another macro in what it is
handed, and re-expands what comes back, so a macro that expands into a call
to itself keeps going. That loop is bounded at 200 and says which macro ran
out: macro-spin.flan.
- Skips all of it when the file names no macro, which is nearly every file.
Otherwise every build in the suite would pay a clang driver to answer a
question nobody asked. When it does build, the module is cached under the
object cache and keyed by the prelude's source plus the file's defmacros, so
a second process pays a dlopen.
lib/dune passes -linkall, which is the one line in another lane's file. The
module installs itself into Parse.expander at initialisation and nothing
references it, so without -linkall the linker drops it from every executable
that does not name the module -- bin/main.exe among them -- and a program
calling a macro fails with an unknown name. The alternative was an install
call at every entry point, including ones in files this lane must not touch.
The one thing a macro cannot do that parse.ml could is give a reason. A macro
runs inside the compiler and anything it signals aborts the compile with no
location, so a malformed (unless) answers a name nothing defines and the
report is "unknown name unless-takes-a-test-and-a-body" at the call site --
right place, wrong sentence. NEXT.md says so.
test_flan.ml's "unless -> if(not)" assertion is gone, because it asserted a
desugaring in a file that no longer does one. Nothing else in the suite
changed.
