JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'dev-loop' into lane-implicit-widening

Browse Source 
# Conflicts:
#	FIX.org
This commit is contained in:
Joseph Ferano 2026-09-20 20:01:54 +07:00
commit f8dfdaa9a0
19 changed files with 1933 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

461
FIX.org
View File

@ -1540,7 +1540,10 @@ inspector reads.
~migrate-instances~. A heterogeneous map has no layout to be stale, so ~migrate-instances~. A heterogeneous map has no layout to be stale, so
nothing breaks today when a class gains a slot: old instances simply lack nothing breaks today when a class gains a slot: old instances simply lack
it. Enumerating live instances is the part that is missing, and it is the it. Enumerating live instances is the part that is missing, and it is the
pool's question rather than this lane's. pool's question rather than this lane's. **Built after all, and without
the enumeration**: see "Lazy instance migration" below, where the answer
turned out to be CLHS 4.3.6's — do not walk the heap, stamp the instances
and migrate each one when it is next touched.
- *The JS backend.* It refuses dyn wholesale, so none of this compiles there. - *The JS backend.* It refuses dyn wholesale, so none of this compiles there.
Same parking as the string-equality hole above. Same parking as the string-equality hole above.
@ -2042,6 +2045,11 @@ wins every call and the definition is still unreachable; what changed is that
the arity message says so and notes the definition. Refusing the shadowing is the arity message says so and notes the definition. Refusing the shadowing is
a language decision and was left to the author. a language decision and was left to the author.
[Superseded the same day by the author's decision — see "Shadowing a builtin"
below. The builtin no longer wins, the definition is no longer unreachable,
and the arity note this paragraph describes has been removed along with the
world it described.]
* ~int~ and ~float~ as builtin aliases, 2026-09-20 * ~int~ and ~float~ as builtin aliases, 2026-09-20
The author, on the foreign-spelling list the diagnostics pass had just The author, on the foreign-spelling list the diagnostics pass had just
@ -2098,8 +2106,12 @@ was a builtin. The rule is decided by the target, at registration:
- Anything else — refused: "int is a builtin alias for i32 and cannot be - Anything else — refused: "int is a builtin alias for i32 and cannot be
redefined as i64 — delete this defalias, or give the type another name". redefined as i64 — delete this defalias, or give the type another name".
The alternative was the ~arity~ precedent, where the builtin wins and a note The alternative was the ~arity~ precedent, where the builtin won and a note
surfaces at the error the shadowing caused. It does not transfer: a surfaced at the error the shadowing caused — a precedent deleted later the
same day, when shadowing a builtin became legal and the user's definition
started winning instead (see "Shadowing a builtin" below); the reasoning
below stands either way, because neither world has anywhere to put the note.
It does not transfer: a
~(defalias int i64)~ has no later error site to hang a note on. ~resolve_name~ ~(defalias int i64)~ has no later error site to hang a note on. ~resolve_name~
reaches ~ikind_of_name~ before the alias table, so the declaration would be reaches ~ikind_of_name~ before the alias table, so the declaration would be
read as ~i32~ at every use and nothing would ever say so. Silence was the one read as ~i32~ at every use and nothing would ever say so. Silence was the one
@ -2442,6 +2454,449 @@ failures were that row and the sixth was ~dev-trap-free-all~, so what is racy
is ~trap_park~ itself and every row that calls it — which is exactly what the is ~trap_park~ itself and every row that calls it — which is exactly what the
mechanism described there predicts. Per the sweep policy the ~@x86~ and mechanism described there predicts. Per the sweep policy the ~@x86~ and
~@sanitize~ sweeps were not run here. ~@sanitize~ sweeps were not run here.
* Shadowing a builtin, 2026-09-20
The author's decision, in the author's words:
#+begin_quote
"allow shadowing but warn" — a user ~(defn get ...)~ colliding with a builtin
is legal, the USER'S definition wins at call sites (real shadowing, Clojure's
model: the def takes over, a warning says so), and the compiler warns once at
the definition site.
#+end_quote
** Where builtin-wins actually lived
Not in a table and not in a precedence list. ~named_call~ is one
~match name with~ whose arms are the builtin names written out as string
literals, and the three arms that look anything up — a local of ~Fn~ type,
~gsigs~, then ~env.fns~ — are the last three in that match. So a builtin won
because OCaml tried its arm first, and for no other reason. ~env.fns~ never
outranked anything; it was simply never reached for a name spelled like a
builtin. The old comment above ~arity~ said this outright ("the dispatch
above reaches every builtin arm before it ever looks in [fns]") and is the
only place it was written down.
** The resolution change
One guard, first arm of ~named_call~:
: | _ when shadows_builtin ctx loc name -> ordinary_call ctx ~want loc name args
and the three trailing arms factored into ~ordinary_call~ so that both routes
— falling past every builtin, and being sent straight there by the guard —
resolve a name by exactly the same rules. Order is now total and reads the
way a reader would guess: local of function type, then generic signature,
then the function table, then the builtins, then the struct and the
did-you-mean refusals.
~shadows_builtin~ asks two questions, in this order. Is the name a
builtin's: one lookup in ~builtin_set~, false for every call to an ordinary
function, and asking it first is also what keeps the arms that are not calls
— an enum cast, a cast to a type variable, a machine-type cast — exactly
where they were. Then, and only then, is there a definition that reaches
this call: a local of function type, or a defn written in this same file.
~builtin_set~ is a ~Hashtbl~ and is new. The guard is the first arm of the
dispatch, so it runs at every named call, and the list ~builtin_names~ that
already existed is walked linearly — about a third of check time on a
program of twenty thousand calls, measured in review. The list stays for the
did-you-mean, whose order is its order; the set answers the membership.
** The warning, verbatim
: shadow-builtin.flan:20:7: warning: get shadows the builtin get — every call in this program now reaches your definition
: 20 | (defn get [p P] i32 (.x p))
: | ~~~
Rendered by ~Loc.entry ~mark:'~' ~label:"warning: "~, which is the
~--warn-memory~ precedent, so flycheck parses it exactly as it parses an
error. Nothing raises and the exit status does not move. Unlike
~--warn-memory~ it is behind no flag: there is nothing to tune, and the line
is one line and rare.
It is printed from ~Check.build_program~ rather than from ~bin/main.ml~
beside ~print_memory_warnings~, because every route into the compiler passes
through that function — build, check, run, and the dev daemon's reload, which
is where a defn is most likely to be written. The list itself is
~Check.shadowed_builtins~, a pure function over the declarations, which is
what the tests ask.
** Scope, settled from the code
*Package-wide or program-wide: neither, and the mechanism already decided
it.* ~Load~ qualifies every name an imported package declares to ~alias/name~,
including its own uses of them, so a package's ~get~ is ~rl/get~ and cannot
collide with a builtin at all. What is left is the other direction: a program
that defines ~get~ and imports a package whose body calls the builtin ~get~.
That call must keep meaning the builtin, and it does: the shadow reaches
exactly the file the definition was written in, which is the same visibility
a defn has everywhere else. The prelude falls out of the same rule rather
than needing one of its own — it is a file, and not the one the program is
in.
The file and not the enclosing function's name, which is what this first
shipped with and was wrong. A package's functions are qualified at the
import, so "does the owner's name carry a slash" answers correctly wherever
a call sits inside a function — and wrongly in the one place a call does
not. Review demonstrated it: a program defining ~(defn len ...)~ reached
inside an imported package's ~(defvar sz i32 (len "abcd"))~, which is
checked with no owner at all, and made it 999. A global initialiser has no
enclosing name; it does have a file.
~programs/shadow-builtin.flan~ is every half in one program: 7 is the
program's own one-argument ~(get p)~, 4 is the builtin ~get~ called inside
the package it imports, 99 is a shadowed ~+~, 999 is the program's own
~len~, and the last 4 is that same ~len~ inside the package's global
initialiser, where the builtin still means the builtin.
*Prelude macros.* No rule was needed: the namespace is already one.
~(defn comment [x i32] i32 ...)~ against the prelude's ~(defmacro comment
...)~ is refused today as "comment is defined twice", with a note at the
prelude's definition, and the same for ~inc~ and ~dec~. Shadowing a builtin
is a different question precisely because a builtin is not a declaration —
it is an arm in the compiler, with nothing for a redefinition check to point
at. Macros expand before checking and key on the head name unconditionally,
so if the redefinition check were ever relaxed the macro would win and the
defn would be unreachable; that is not a state this compiler can reach, and
nothing was written to handle it.
*What the file rule costs.* A bare REPL expression — ~C-x C-e~ on a form,
evaluated with origin ~<eval>~ and no file behind it — is not the file the
defn was written in, so it reaches the builtin. ~C-c C-c~ sends the buffer's
own path and is unaffected, which is the case the dev loop is actually made
of. It is the conservative direction: a REPL line meaning the builtin is a
surprise, a REPL line silently meaning a definition somewhere else is a
worse one. If it ever bites, the fix is for the session to evaluate with the
buffer's path as origin, which it already knows.
*A macro named after a builtin warns too, and that is right.* ~(defmacro get
[args] ...)~ is an ~Ast.Defn~ like any other by the time the declaration
list is collected — a macro is a function the compiler runs — so
~shadowed_builtins~ names it and the warning reads the same. The macro also
wins, and by a different mechanism: expansion runs before checking and keys
on the head name, so the call never becomes a call at all. The one wrinkle
is that a file carrying macros is checked twice, the macro module first, so
its warning is printed twice. Disclosed rather than suppressed: dropping a
duplicate means keeping state across the two checks, and the second line is
the same line.
*The dead end: a shadowed builtin has no remaining spelling.* Nothing in
this language qualifies a name — there is no ~core/get~, no ~(builtin get)~
— so a file that defines ~get~ has given up the builtin ~get~ for the whole
file, and a definition that wants to *wrap* the builtin cannot. ~(defn len
[s string] i32 (+ 1 (len s)))~ is not a wrapper, it is unbounded recursion:
the inner call reaches the definition being written, and the program
stack-overflows at run time with no diagnostic from the compiler, which has
nothing to object to. The warning says the name is taken over; it does not
say this. An escape hatch is a language decision and is with the author.
** Pins
- ~test_flan.ml~: the warning's kind, line and column; its message, matched
whole and not by needle; that it carries no notes; that the source which
used to be refused now checks; and that a program shadowing nothing warns
not at all.
- ~test_flan.ml~, from review: a shadowed operator warns with the same
sentence and lowers to a ~Call~ to the definition rather than the ~Add~
prim; and a call read with another file's name, against the same
declaration list, reaches the builtin and is refused at the builtin's
arity — the global-initialiser case at its smallest.
- ~test_acceptance.ml~: ~programs/shadow-builtin.flan~ outputs
~7\n4\n99\n999\n4\n~, and the ~@x86~ sweep compares both backends over
the same file.
- Removed: the ~check/builtin-arity~ kind, its message ("this is the builtin
get, which a defn of the same name does not replace"), its note ("is also
defined here, and this call is not reaching it — rename it to call it"),
and the three checks that pinned them. The situation cannot arise: the call
reaches the user's defn, whose arity is whatever it declared.
- Changed: the builtin-arm/~Check.builtins~ cross-check reads ~named_call~'s
source down to ~ | _ ->~ rather than ~ | _~, because the new first arm is
guarded and stopping at it read the whole region as empty.
** One thing the new package cost
A package under ~test/programs/pkgs/~ needs a ~glob_files~ line of its own in
four places in ~test/dune~ — the test stanza and the ~@valgrind~, ~@x86~ and
~@js~ sweeps — because dune's glob does not descend and the sweeps walk
~programs/*.flan~ whole. Without it the corpus row fails with "no package
at ..." and prints no FAIL line, only "1 failure(s)" at the end of the log:
worth knowing, because a grep for FAIL says green over it.
** What was run
~dune test --root .~ in the lane's worktree, forced: exit 0. Rebased onto
dev-loop before the review follow-ups, so the ~arity~ signature this lane
cuts down is the one the byte-fill lane had just given a ~ctx~ argument, and
the ~int~/~float~ section's paragraph about "the ~arity~ precedent, where
the builtin wins" is revised in place — that precedent is what this lane
deleted.
The heavy sweeps (~@x86~, ~@sanitize~, ~@valgrind~) were left to the batch.
* Lazy instance migration for a redefined defclass, 2026-09-20
CLHS 4.3.6 — the ~update-instance-for-redefined-class~ protocol — adapted to
the dyn side's classes, minus the user hook. Redefining a ~defclass~ in the
dev session used to be *silent*: a class is compile-time sugar for a
constructor ~defn~, so the edit replaced a function body, the instances
already in the program kept their old keys for ever, and nothing anywhere
said so. Now the instances follow the class.
The research is ~docs/SBCL-REDEFINITION-NOTES.md~, candidate C. Its central
finding is why this was cheap and why the same thing is not available for a
typed ~defstruct~: every SBCL mechanism of this kind rests on an instance
carrying a pointer to its shape, and a dyn instance *has a header* where a
flat struct does not.
** What it does
#+begin_src lisp
(defclass point [x y])
;; ... a program runs, builds instances, holds them in globals ...
(defclass point [x z]) ; C-c C-c, with the file's callers if any
;; every live instance, at its next touch:
;; :x keeps the value it had (matched by name)
;; :z appears as nil (gained)
;; :y is gone (dropped)
;; the object is the same object (identity preserved)
;; (class-of p) is still :point (so every method still reaches it)
#+end_src
Nothing is enumerated and no heap is walked, which is the part the old
deferral thought was missing. The redefinition is O(1) — one registry entry
updated — and the work is paid per instance, once, by whoever touches it.
** The three pieces
*** A registry, in the runtime
~runtime/flan_dyn.c~, under "Classes": one entry per class name, holding the
current slot list and a generation counter. ~flan_dyn_class_def(name, slots,
n)~ registers or re-registers one; ~slots~ is the names packed into a single
string with newlines between.
*Nothing in it is a collector object, and that is the whole GC argument.* A
class's name and its slots are interned ~kw_entry~ pointers — immortal, not
on the collected heap, never traced — which is the same argument the ~klass~
header field already makes. The table itself is ~malloc~ed, append-only and
never freed. So no root is pushed for the registry, the marker has nothing to
reach in it, and a collection triggered from inside a migration cannot see a
half-built slot list. A registry of dyn vectors would have needed all three
of those worried about.
*** A generation, in the instance's header
A ~uint32_t~ in ~flan_obj~, *in the padding between ~mark~ and ~len~*.
~sizeof(flan_obj)~ is 48 with it and was 48 without it — the union is exactly
24 bytes (~items~, ~cap~, ~klass~), so there is no spare word inside the arm
and a field placed after the union would have cost eight bytes on every dyn
value in the heap for a word only class instances read. The obvious guess
before reading the struct is that ~view.is_vec~ leaves four spare bytes at
offset 44; it does not. That word is the *view* arm's and is aliased with
~klass~ — the arms overlap, so nothing inside the union is free. The free
bytes are the ones alignment already wastes, in front of it.
The number is asserted rather than commented: ~flan_dyn_obj_size()~ is a new
entry point and ~dyn_ops.c~'s ~classes~ mode checks it against 48, so a later
field that pushes it out fails a test instead of costing that silently.
Zero means "built before any definition was registered", which is every
instance of every program that was built and never reloaded. The first
registration of a name lands on 1, so those instances migrate exactly once,
the first time the class is redefined under them — which is what makes a
program that predates this correct rather than merely unbroken.
*** A registration thunk, per reload
~lib/session.ml~'s ~change~ emits one nullary function per evaluation that
declared any class, calling ~flan_dyn_class_def~ once per class, and hands it
to ~Emit.redefinition~/~X86.redefinition~ as ~?call~ — the mechanism ~C-x
C-e~ already uses, where the agent finds ~flan_reload_call~ by ~dlsym~ and
runs it after the module's bodies are published and on the game thread. Both
backends, unchanged: the thunk is an ordinary Tast function and the backends
lower ~Rt~ calls generically.
*It has to be a thunk and not something in the constructor.* The case this
exists for is a class redefined and *not* constructed — old instances touched
after the edit — and a registration that only ran at construction would never
fire for it. That is the same reasoning that rules out registering from
~main~: reload modules re-execute their definitions, not their program.
*Every* class in the form is registered, not only the ones whose slots
changed, because a class the registry has never seen has to arrive somehow.
The bump is what is conditional: re-registering an identical list changes
nothing, so a ~C-c C-k~ costs one comparison per class and migrates nothing.
Without that rule every save would migrate every instance in the program.
** Where a migration happens
~want_map~ (so ~get~, ~put~ and ~has-key?~), ~flan_dyn_len~'s map arm, and
~dyn_equal~'s. CLHS asks for "no later than the next time a slot of that
instance is read or written"; those are the three places that read or write
the slot *set*.
*Neither printer is one of them*, and that has a consequence somebody will
meet. ~render~ — which ~print~ goes through, and which the editor renders
every dyn value with — and ~say_render~ — the 96-byte sentence a trap
prints — both walk the entries raw and neither syncs. ~say_render~ runs
inside trap reporting, where the heap is whatever the trap left, and a
printer that frees an object's entry block and installs another is not
something to have on that path; ~render~ is its sibling and is reached from
it for nested values, so splitting them would put the mutation one recursion
below a trap anyway.
So: *a stale instance shows its old slots to the editor until something
touches it.* A watch expression, the value ~C-x C-e~ answers and the
inspector's render of a dyn all arrive through ~render~, so in the moment
after a ~defclass~ is redefined the inspector can show a slot the class no
longer has and omit one it has gained — while ~(get p :z)~ typed at the same
instant answers the new definition, migrates the instance, and makes the
inspector agree from then on. CLHS's "implementation-dependent time" permits
it; it is the price of the printer staying a printer; and it is disclosed
here rather than discovered.
The migration rebuilds the entry block rather than compacting it in place,
and writes the slots in the *class's* order. One ~malloc~ per instance per
redefinition, and the property bought is that a migrated instance is
indistinguishable from a freshly constructed one — ~dyn_equal~ compares maps
by lookup and would not have cared, but ~len~ and ~render~ work in insertion
order and would have.
** Equality across generations: migrate first
Two instances of one class built either side of a redefinition, holding equal
values for the slots the class still has, *are equal*. ~dyn_equal~ migrates
both operands before comparing the tag or the length. The decision recorded:
equality is over the class as it is now, not over the shapes the two values
happened to be born with. The alternative — comparing key sets literally —
would answer "not equal" about a difference the class no longer has, and
would make the answer depend on which of the two had been touched since.
** The registry is advisory, and this is the honest cost
A class instance is an open map. ~put~ takes any key — FIX.org already defers
refusing ~(get p :z)~ — so a program can write a key the class never
declared, and the next migration *drops it*, because the migration's rule is
that an instance's keys are the class's slots.
That is data loss, and there is no enforcement behind it to make the loss
impossible. Enforcing would mean refusing an unknown key at ~put~, which is
the static slot discipline the dyn side deliberately does not have, and the
research names this exact risk: "if ~put~ of an arbitrary key stays legal,
the registry describes an intention rather than a constraint". It describes
an intention. ~test_dev.ml~ pins the loss as behaviour rather than leaving it
to be discovered.
** What the session had to give up to allow it, and what it kept
A slot added or removed is a *constructor signature change*, which
~session.ml~'s ~compatible~ refuses by default — a call site compiled to pass
two dyn words into a three-parameter body leaves the third holding a
register, and a dyn word that is not a value is a wild pointer rather than a
wrong answer. Item 6's own line above — the constructor "is an ordinary
~defn~ — so its arity refusal, its cell in a dev build and its behaviour
under redefinition are the ones every function already has" — is true and
was read one step too far: what every function already has *is* the signature
refusal, so a class could not change its slots at all. That was the first
thing this lane had to fix, before any of the runtime work could be reached.
The refusal is now lifted for a ~defclass~ constructor *and nothing else*,
and only when no compiled caller is left behind. In practice the checker gets
there first: the whole declaration list is re-checked against the new
constructor before ~compatible~ is consulted, so a declaration still calling
it with the old count is refused at the call site with a line number — which
is the sentence a reader sees, and is what ~test_session.ml~ pins. What
~change~ adds is the *reason* held locally rather than inherited: a caller
that type-checks under the new arity is one whose source changed, so it is in
this form and is republished with the class. The walk over ~t.program~'s
bodies asserts that instead of assuming it, and if it ever fires the answer
is a refusal naming the callers rather than a wild pointer.
Not touched: typed ~defstruct~ layout changes and typed global type changes
keep their refusals. ~SBCL-REDEFINITION-NOTES.md~ §5 is why — a flat unboxed
struct has no header to stamp and cannot change size in place, so none of
this is available there at any price.
** Deferred, with the reason
- *~update-instance-for-redefined-class~ itself*, the user hook. CLOS hands
the discarded slots' values to a method so a coordinate change can be
written by hand; the obvious Flan spelling is a generic,
~(defmethod update-for-redefined point [p added discarded] ...)~, riding
the dispatch that already exists. Left out of v1 because the automatic
half — name matching — is the half that makes redefinition usable, and the
hook is what makes it *expressive*. Nothing about the design blocks it:
the migration already computes both lists.
- *Initargs validation.* CLOS's default method signals on an initarg the
class does not declare. There are no initargs here; construction is
positional.
- *Refusing an unknown ~put~*, which is what would turn the registry from
advisory into enforcing. Same gate as the deferred ~(get p :z)~ check.
- *Rolling a failed migration back.* SBCL wraps the user hook in
~nlx-protect~ so a signalling method leaves the instance on its old
wrapper. With no user hook the migration cannot signal, so there is nothing
to roll back yet; it becomes a real question the day the hook lands.
- *The whole-program build registers nothing.* A program that is built and
never reloaded has no registry at all, its instances carry generation zero,
and everything behaves exactly as it did before this existed. Registering
at startup would need an initialiser in both backends' executable paths and
buys only introspection — there is no *stale* instance in a program whose
classes never changed.
** Pinned
- ~test/dyn_ops.c~'s ~classes~ mode, run by ~test_dyn.ml~: the object size,
an unregistered class behaving as before, a slot gained, a slot lost, both
at once, three definitions an instance slept through, a re-registration of
the same list migrating nothing, two generations compared, a plain map
untouched by any of it, and two thousand instances migrated while the
collector runs. Driven from C because the event has no Flan spelling: a
class definition changes between two *modules*, so no single program can
see one change.
- ~test_sanitize.ml~'s ~dyn_sweep~ runs that mode under ASan and UBSan. It is
the one mode that frees an object's entry block while the object stays live
and reachable, which is the shape a wrong marker would show as a
use-after-free and as nothing at all in the checked build.
- ~test_dev.ml~, "a class redefined under its own instances": a real daemon
over ~test/programs/dev-classes.flan~, instances pushed into a dyn global
by ~C-x C-e~ thunks, then five ~C-c C-c~ evaluations of the class — four
of which change the slot list — with the program's own heap answering
between them: gained slot nil, kept slot kept, count right, *a generic
still dispatching after the migration*, an untouched instance migrating on
its own first touch, a lost slot gone, the third generation, the tag
surviving, the one unchanged re-evaluation migrating nothing, and a
raw-~put~ key dropped by the next real redefinition.
- And the same protocol once more against a ~flan dev --llvm~ daemon. The
block above runs on x86, which is what ~flan dev~ takes unasked; the subset
under LLVM is the part that is backend-specific — whether the registration
thunk reaches the runtime at all — and everything past that point is
flan_dyn.c's, which does not know who called it. Written because
~x86.ml~'s header had claimed for some time that it did *not* emit
~flan_reload_call~, which is exactly the kind of sentence not to trust
twice.
- ~test_session.ml~: a slot added and a slot removed both accepted, the
module carrying a ~call~ to ~flan_dyn_class_def~, a definition of
~flan_reload_call~ and the packed slot-list constant, an unchanged class
registering anyway with its own list, the refusal when a
compiled caller is in the way, and the same edit accepted when the caller
comes with it.
** Found on the way
~lib/x86.ml~'s ~redefinition~ header said "the transient ~flan_reload_call~
thunk is not built here, and is refused by name". It has been built there for
some time — the code is at the bottom of the same function — and the comment
had simply not moved with it. Corrected rather than worked around; this
lane's thunk goes through that path on every ~C-c C-c~ of a class, which is
the default backend for ~flan dev~.
A second one, found by the review rather than by the lane: two of the
~test_session.ml~ pins above asserted the string ~flan_dyn_class_def~ against
the module's IR text, and ~emit.ml~ writes a ~declare~ for every runtime
entry point into every module it emits — so both passed against a module that
registered nothing. They assert ~call void @flan_dyn_class_def~ and the packed
slot-list constant now. Confirmed by mutation: with the thunk suppressed the
old needles pass and the new ones fail. Worth carrying as a habit rather than
as a fix — a needle that names a runtime symbol is matching the declare block
unless it says ~call~.
** What was run
~dune test --root .~ green (exit 0, no FAIL lines) before and after the
rebase onto dev-loop, and ~test_dev.exe~ run directly afterwards because its
label can be swallowed by a cached run. ~dune build --root . @sanitize~ clean
on the committed source, which is where the two-thousand-instance migration
under collection actually gets looked at.
The rebase is worth a line of its own. Three conflicts were additive —
FIX.org, ~want_map~ (the diagnostics lane gave ~trap2~ a location pair, this
one put a ~class_sync~ beside it, both wanted), and ~test_dev.ml~'s
agent-socket block beside this one's. The fourth was not a conflict at all
and is the one to remember: ~flan_dyn_class_def~'s argument check was written
against the four-argument ~trap1~ and merged clean into a tree where ~trap1~
takes a location first, so the class name would have been read as a length.
*~dune build~ does not compile ~flan_dyn.c~* — it is a string the compiler
carries and hands to clang at ~flan run~ — so a green build is not evidence
about that file at all. ~dune test~ is, and so is running any program.
* Implicit widening, 2026-09-20 — "go with C" * Implicit widening, 2026-09-20 — "go with C"
Answers DISCUSS.org's *implicit numeric conversions with a warning flag, Answers DISCUSS.org's *implicit numeric conversions with a warning flag,
instead of hard errors*. The ask there was a warn-instead-of-refuse mode; the instead of hard errors*. The ask there was a warn-instead-of-refuse mode; the

2
docs/DIAGNOSTICS-AUDIT.md
View File

@ -49,7 +49,7 @@ runtime has only preformatted loc strings, no access to source text).
| 4 | `unknown function prinltn` / `unknown name n` | check.ml:6259, 2732 | `(prinltn "hi")` | B | C | D | B | No did-you-mean, although `near_miss` (check.ml:670) is written, tested and wired — to **types only**. Point it at `env.fns`, `env.globals` and the local scope. Cheapest structural win on the list. | | 4 | `unknown function prinltn` / `unknown name n` | check.ml:6259, 2732 | `(prinltn "hi")` | B | C | D | B | No did-you-mean, although `near_miss` (check.ml:670) is written, tested and wired — to **types only**. Point it at `env.fns`, `env.globals` and the local scope. Cheapest structural win on the list. |
| 5 | `expected bool, found i32` | check.ml:1866 via `check_truthy` (3596) | `(let [x 1] (if x …))` | A | C | D | A | Caret is exactly right (the `check_truthy` loc work paid off). But the message never states Flan's truthiness rule — bool or dyn, nothing else — and never names the fix (`(not= x 0)`). Special-case the condition position. | | 5 | `expected bool, found i32` | check.ml:1866 via `check_truthy` (3596) | `(let [x 1] (if x …))` | A | C | D | A | Caret is exactly right (the `check_truthy` loc work paid off). But the message never states Flan's truthiness rule — bool or dyn, nothing else — and never names the fix (`(not= x 0)`). Special-case the condition position. |
| 6 | `binding 5 has no value — let takes name/value pairs` | parse.ml:670 | `(let [x i32 5] …)` | C | D | D | B | A type annotation in `let` is the single most natural thing for someone arriving from a typed language, and `let` has none. The message reads as if the user miscounted. Detect "middle form names a type" and say so: "`let` bindings take no type annotation — write `[x 5]`". | | 6 | `binding 5 has no value — let takes name/value pairs` | parse.ml:670 | `(let [x i32 5] …)` | C | D | D | B | A type annotation in `let` is the single most natural thing for someone arriving from a typed language, and `let` has none. The message reads as if the user miscounted. Detect "middle form names a type" and say so: "`let` bindings take no type annotation — write `[x 5]`". |
| 7 | `get takes 2 arguments, given 1` against the **user's own** `(defn get [p P] …)` | check.ml:5296 (builtin dispatch) + 6233 | `(defn get [p P] i32 …)` + `(get p)` | D | D | D | B | A user defn whose name collides with a builtin is silently shadowed, and then the arity refusal is measured against the *builtin*, pointing at the user's call. Either refuse the shadowing definition at its `dloc` with a note, or report the arity against the definition the user can see. | | 7 | `get takes 2 arguments, given 1` against the **user's own** `(defn get [p P] …)` | check.ml:5296 (builtin dispatch) + 6233 | `(defn get [p P] i32 …)` + `(get p)` | D | D | D | B | A user defn whose name collides with a builtin is silently shadowed, and then the arity refusal is measured against the *builtin*, pointing at the user's call. Either refuse the shadowing definition at its `dloc` with a note, or report the arity against the definition the user can see. **Settled 2026-09-20, neither way: the author chose "allow shadowing but warn" — the defn wins at every call site in its own file and the compiler warns once at the definition. See FIX.org, "Shadowing a builtin".** |
| 8 | `unhandled Boom` | flan_rt.c:646 | `(defstruct Boom [why i32])` + `(error (Boom {.why 7}))`, `flan run` | D | C | D | B | Three words. No location (not even the `error` site, which the emitter knows), no field values, no list of the handlers that were in scope. The condition system is a headline feature and this is its failure mode. | | 8 | `unhandled Boom` | flan_rt.c:646 | `(defstruct Boom [why i32])` + `(error (Boom {.why 7}))`, `flan run` | D | C | D | B | Three words. No location (not even the `error` site, which the emitter knows), no field values, no list of the handlers that were in scope. The condition system is a headline feature and this is its failure mode. |
| 9 | `the collection nosuch: is a directory named nosuch somewhere above /…/. , and there is none` | load.ml:120 | `(import zz "nosuch:thing")` | B | C | C | D | Reads as an assertion immediately contradicted. Also emits a bare `/.` on the path. Rewrite as a plain statement of the search ("no directory named `nosuch` between here and the root") and list what collections *were* found. | | 9 | `the collection nosuch: is a directory named nosuch somewhere above /…/. , and there is none` | load.ml:120 | `(import zz "nosuch:thing")` | B | C | C | D | Reads as an assertion immediately contradicted. Also emits a bare `/.` on the path. Rewrite as a plain statement of the search ("no directory named `nosuch` between here and the root") and list what collections *were* found. |
| 10 | `and`'s last operand gets the previous operand's caret | parse.ml `shortcircuit`, via check.ml:3632 | `(println (and true true (vec-new i32)))` | D | B | C | A | Already diagnosed in FIX.org:1036 with three rejected fixes; the accepted one — `check_if` preferring the arm that is not a compiler temp when choosing which to blame — is a check.ml change nobody owned. This pass owns check.ml. | | 10 | `and`'s last operand gets the previous operand's caret | parse.ml `shortcircuit`, via check.ml:3632 | `(println (and true true (vec-new i32)))` | D | B | C | A | Already diagnosed in FIX.org:1036 with three rejected fixes; the accepted one — `check_if` preferring the arm that is not a compiler temp when choosing which to blame — is a check.ml change nobody owned. This pass owns check.ml. |

189
lib/check.ml
View File

@ -310,6 +310,15 @@ let foreign_spelling = function
and letting the two drift. Filled once, immediately after that table. *) and letting the two drift. Filled once, immediately after that table. *)
let builtin_names : string list ref = ref [] let builtin_names : string list ref = ref []
(* The same names as a set, and the two are not one because they are asked
two different questions. The list above is read once, at a refusal, and
its order is the order the did-you-mean walks. This is asked at *every*
named call — [shadows_builtin] is the first arm of the dispatch — and a
linear walk of eighty-odd strings per call is a cost a whole-program check
pays in full: measured at about a third of check time on a program of
twenty thousand calls. Filled beside the list. *)
let builtin_set : (string, unit) Hashtbl.t = Hashtbl.create 128
(* What a [break] or a [continue] may be talking about, innermost first. (* What a [break] or a [continue] may be talking about, innermost first.
[Lloop] is a loop it is lexically inside, carrying its label if it was given [Lloop] is a loop it is lexically inside, carrying its label if it was given
@ -5001,35 +5010,18 @@ and call_value ctx ~want loc (callee : Tast.expr) args =
fail loc "this is a %s and not a function, so it cannot be called" fail loc "this is a %s and not a function, so it cannot be called"
(Types.to_string other) (Types.to_string other)
(* A builtin's arity, and the one thing the caret cannot show: whether the (* A builtin's arity. The count is the builtin's and can only be the
count being measured against is the builtin's or a defn of the same name. builtin's: a defn of the same name written in the program now takes the
A defn does not shadow a builtin — the dispatch above reaches every builtin call over before any builtin arm is reached ([shadows_builtin] at the top
arm before it ever looks in [fns] — so a user function called [get] is of [named_call]), so a call measured here is a call to the builtin and
silently unreachable, and the refusal that followed measured the call there is no second signature for the reader to have meant. The note that
against the builtin while pointing at a call the reader had written for used to say otherwise — "this is the builtin get, which a defn of the same
their own. Said outright, with the definition alongside. *) name does not replace" — described a resolution order this compiler no
and arity ctx loc name n args = longer has. *)
if List.length args <> n then begin and arity _ctx loc name n args =
let notes = if List.length args <> n then
if Hashtbl.mem ctx.env.fns name then fail loc "%s takes %d argument%s, given %d" name n
match Hashtbl.find_opt ctx.env.fn_locs name with (if n = 1 then "" else "s") (List.length args)
| Some at ->
[ Loc.note at
(name ^ " is also defined here, and this call is not reaching \
it — rename it to call it") ]
| None -> []
else []
in
let shadowed = notes <> [] in
if shadowed then
Loc.failk "check/builtin-arity" loc ~notes
"%s takes %d argument%s, given %d — this is the builtin %s, which a \
defn of the same name does not replace"
name n (if n = 1 then "" else "s") (List.length args) name
else
fail loc "%s takes %d argument%s, given %d" name n
(if n = 1 then "" else "s") (List.length args)
end
(* The operators that fold: [+ - * /], [min]/[max] and the three bitwise (* The operators that fold: [+ - * /], [min]/[max] and the three bitwise
combining operators all take two operands or more, and mean the same thing combining operators all take two operands or more, and mean the same thing
@ -5419,6 +5411,19 @@ and vec_at ctx loc (target : Tast.expr) (idx : Ast.expr list) =
and named_call ctx ~want loc name args = and named_call ctx ~want loc name args =
let prim p ty args = expect ctx loc ~want (mk loc ty (Tast.Prim (p, args))) in let prim p ty args = expect ctx loc ~want (mk loc ty (Tast.Prim (p, args))) in
match name with match name with
(* The user's own definition, first — before every builtin arm below.
Clojure's rule: a [(defn get ...)] takes the name over, and a call
written in the program that defines it reaches that definition rather
than the builtin it is named after. The defn site is warned about once
(see [shadowed_builtins]); the call sites say nothing, because at a call
site there is nothing surprising left — the name means what the file
says it means.
What this arm does NOT do is let one file's definition reach into
another's: [shadows_builtin] answers false for a call in the prelude and
for a call in imported package code, which is the same visibility rule a
defn has everywhere else. *)
| _ when shadows_builtin ctx loc name -> ordinary_call ctx ~want loc name args
(* ── arithmetic and comparison ─────────────────────────────────── *) (* ── arithmetic and comparison ─────────────────────────────────── *)
| "+" | "-" | "*" | "/" -> | "+" | "-" | "*" | "/" ->
let p = match name with let p = match name with
@ -7245,13 +7250,22 @@ and named_call ctx ~want loc name args =
| _ -> prim (Tast.Cast target) target [ a ]) | _ -> prim (Tast.Cast target) target [ a ])
(* ── ordinary calls ────────────────────────────────────────────── *) (* ── ordinary calls ────────────────────────────────────────────── *)
| _ -> ordinary_call ctx ~want loc name args
(* Everything that is not a builtin arm: a local of function type, a generic
signature, the function table, and the refusals for a name that is none of
them. Reached two ways — by falling past every arm above, and by the
shadowing guard at the very top of [named_call], which sends a call whose
name the program has defined straight here. One function so that both
routes resolve a name by exactly the same rules. *)
and ordinary_call ctx ~want loc name args =
match () with
(* A local or a parameter holding a function value, called by the name it is (* A local or a parameter holding a function value, called by the name it is
bound to — which is what the body of [map] looks like. It is checked bound to — which is what the body of [map] looks like. It is checked
before the global function table and after every builtin: a binding before the global function table: a binding shadows a defn of the same
shadows a defn of the same name (one namespace, ordinary lexical name (one namespace, ordinary lexical scoping). A local of any *other*
scoping), and nothing shadows [+]. A local of any *other* type falls type falls through to the table, so a program that shadows a function
through to the table, so a program that shadows a function name with an name with an i32 and then calls the function still means the function. *)
i32 and then calls the function still means the function. *)
| _ when (match lookup ctx name with | _ when (match lookup ctx name with
| Some b -> (match b.bty with Types.Fn _ -> true | _ -> false) | Some b -> (match b.bty with Types.Fn _ -> true | _ -> false)
| None -> false) -> | None -> false) ->
@ -7333,6 +7347,64 @@ and named_call ctx ~want loc name args =
name name name name
else Loc.failk "check/unknown-function" loc "unknown function %s" name else Loc.failk "check/unknown-function" loc "unknown function %s" name
(* Does the program's own definition of this name take this call over?
Two questions, in this order, and the order is what makes the guard cheap
enough to be the first arm of the dispatch.
Is the name a builtin's at all. One lookup in [builtin_set], false for
every call to an ordinary function — which is most calls in most programs
— and the question that stops the second from being asked at all. Asking
it first also keeps the arms that are not calls — an enum cast, a cast to
a type variable, a machine-type cast — exactly where they were, since a
name that reaches one of those is not a builtin's either.
Is there a definition of it that reaches this call: a local of function
type, or a defn — ordinary or generic — written in this same file.
And is the definition visible here, which is asked of the two files: the
one the definition was written in and the one this call is written in. A
definition shadows the builtin through its own file and no further, which
is the same visibility a defn has everywhere else — the prelude is the
language's own source and means the builtin wherever it writes one, and an
imported package keeps the builtin it was written against no matter what
the program importing it decides to call [get].
The file and not the enclosing function's name. A package's functions are
qualified at the import ([rl/get]), so asking whether the owner's name
carries a slash answers correctly everywhere a call sits inside a
function — and wrongly in the one place a call does not: a package's
global initialiser, which is checked with no owner at all. An importer
defining [len] reached inside an imported [(defvar sz i32 (len "abcd"))]
and changed what it computed. The files were never wrong about it.
What it costs is the REPL: an expression evaluated with no file behind it
is not the file the defn was written in, so it reaches the builtin. That
is the conservative direction, and C-c C-c — which sends the buffer's own
path — is not affected. *)
and shadows_builtin ctx loc name =
(* Where the definition was written, if this name has one. A generic is in
[generics] and nowhere near [fn_locs], so both tables are asked. *)
let declared_in () =
match Hashtbl.find_opt ctx.env.fn_locs name with
| Some at -> Some at.Loc.file
| None ->
(match Hashtbl.find_opt ctx.env.generics name with
| Some fn -> Some fn.Ast.nloc.Loc.file
| None -> None)
in
(* A local of function type is lexical: it cannot be in scope anywhere but
the file that bound it, so there is no file to compare. *)
let local_fn () =
match lookup ctx name with
| Some b -> (match b.bty with Types.Fn _ -> true | _ -> false)
| None -> false
in
Hashtbl.mem builtin_set name
&& (local_fn ()
|| (match declared_in () with
| Some file -> String.equal file loc.Loc.file
| None -> false))
(* ── A call to a generic function ─────────────────────────────────────── (* ── A call to a generic function ───────────────────────────────────────
The whole of instantiation, and it is at the call site because the call The whole of instantiation, and it is at the call site because the call
site is the only place the concrete types exist. Odin does the same thing site is the only place the concrete types exist. Odin does the same thing
@ -8039,7 +8111,41 @@ let builtins : (string * string * string) list =
(* The forward reference declared beside [nearest], filled the moment the table (* The forward reference declared beside [nearest], filled the moment the table
it names exists. Nothing reads it before a call is checked, and no call is it names exists. Nothing reads it before a call is checked, and no call is
checked before this module is loaded. *) checked before this module is loaded. *)
let () = builtin_names := List.map (fun (n, _, _) -> n) builtins let () =
builtin_names := List.map (fun (n, _, _) -> n) builtins;
List.iter (fun (n, _, _) -> Hashtbl.replace builtin_set n ()) builtins
(* ── The one thing shadowing owes the reader ────────────────────────────
A defn named after a builtin is legal and it wins ([shadows_builtin]), and
that is a large thing to have happened in silence: every [(get m k)] in
the file now means something the reader has to go and look at. So it is
said once, where the decision was made, and never again at the call sites
— a footgun notice, not a lint.
It is a warning and it says so in the one way that matters: nothing raises
and the exit status does not move. Unlike [memory_sites] it is behind no
flag, because there is nothing to tune — a program either renamed a
builtin or it did not, and the line is one line and rare.
The prelude is skipped: its defns are the language's own and a collision
there is a compiler bug rather than news for whoever is compiling. So are
qualified names, for the reason [shadows_builtin]'s [visible] gives —
[rl/get] is not [get] and shadows nothing. *)
let shadowed_builtins (decls : Ast.decl list) : Loc.diag list =
List.filter_map
(fun (d : Ast.decl) ->
match d.Ast.d with
| Ast.Defn fn
when Hashtbl.mem builtin_set fn.Ast.name
&& not (String.contains fn.Ast.name '/')
&& not (String.equal fn.Ast.nloc.Loc.file Prelude.file) ->
Some
(Loc.diag ~kind:"check/shadows-builtin" fn.Ast.nloc
(Printf.sprintf
"%s shadows the builtin %s — every call in this program now \
reaches your definition" fn.Ast.name fn.Ast.name))
| _ -> None)
decls
(* ── Declarations: pass 1, collect ─────────────────────────────────── *) (* ── Declarations: pass 1, collect ─────────────────────────────────── *)
@ -9646,6 +9752,19 @@ let build_program ~keep_going (decls : Ast.decl list) : Tast.program * env =
be written anywhere in it, which is also what makes a reload rebuild be written anywhere in it, which is also what makes a reload rebuild
every dispatch from the session's declarations — see lib/classes.ml. *) every dispatch from the session's declarations — see lib/classes.ml. *)
let decls = Classes.expand decls in let decls = Classes.expand decls in
(* And with the declaration list in its final shape — the classes expanded,
the shims flattened, the imports already qualified by [Load] — the one
warning this compiler prints unasked. Here rather than in [bin/main.ml]
beside [print_memory_warnings] because every route into the compiler
passes through this function: build, check, run, and the dev daemon's
reload, which is where a defn is most likely to be written. Printed in
the shape [Loc] gives an error, so a checker in an editor parses it the
same way. *)
List.iter
(fun (d : Loc.diag) ->
prerr_endline
(Loc.entry ~mark:'~' ~label:"warning: " d.Loc.dloc d.Loc.dmsg))
(shadowed_builtins decls);
(* Pass one, and it stops at the first thing it refuses. That is not (* Pass one, and it stops at the first thing it refuses. That is not
laziness: every name, type and signature in the file comes from here, so a laziness: every name, type and signature in the file comes from here, so a
declaration this pass could not make sense of leaves a hole that pass two declaration this pass could not make sense of leaves a hole that pass two

1
lib/emit.ml
View File

@ -3784,6 +3784,7 @@ declare i64 @flan_dyn_vec_new()
declare i64 @flan_dyn_map_new() declare i64 @flan_dyn_map_new()
declare i64 @flan_dyn_map_new_class(i64) declare i64 @flan_dyn_map_new_class(i64)
declare i64 @flan_dyn_class_of(i64) declare i64 @flan_dyn_class_of(i64)
declare void @flan_dyn_class_def(i64, ptr, i64)
declare i64 @flan_dyn_kw(ptr, i64) declare i64 @flan_dyn_kw(ptr, i64)
declare i64 @flan_dyn_map_get(i64, i64) declare i64 @flan_dyn_map_get(i64, i64)
declare void @flan_dyn_map_set(i64, i64, i64) declare void @flan_dyn_map_set(i64, i64, i64)

189
lib/session.ml
View File

@ -204,8 +204,8 @@ let known t n =
(* Everything here is a change that would load cleanly and then be wrong. The (* Everything here is a change that would load cleanly and then be wrong. The
house rule (NEXT.md, Watch for) says recognise it and refuse with the house rule (NEXT.md, Watch for) says recognise it and refuse with the
reason, so each one names what it would have broken. *) reason, so each one names what it would have broken. *)
let compatible ?(origin = fun _ -> None) ~loc (old_ : Tast.program) let compatible ?(origin = fun _ -> None) ?(relaxed = []) ~loc
(new_ : Tast.program) = (old_ : Tast.program) (new_ : Tast.program) =
let find_fn p n = let find_fn p n =
List.find_opt (fun (f : Tast.fn) -> String.equal f.Tast.name n) p.Tast.fns List.find_opt (fun (f : Tast.fn) -> String.equal f.Tast.name n) p.Tast.fns
in in
@ -230,7 +230,16 @@ let compatible ?(origin = fun _ -> None) ~loc (old_ : Tast.program)
and caller tracking, none of which exist — so this stays a refusal and caller tracking, none of which exist — so this stays a refusal
until they do, rather than becoming a silent mismatch. See until they do, rather than becoming a silent mismatch. See
plan.org, Hot reload, and open decision #6. *) plan.org, Hot reload, and open decision #6. *)
if not same then (* [relaxed] is the caller saying it has already proved, by a
narrower argument than this one can make, that no compiled call
site passes the old arguments. Today that caller is [change],
about a (defclass ...) constructor whose slot list changed, and
only when nothing in the running program still calls it — see the
"A class whose slots changed" block in [eval], which walks the
callers and makes the refusal itself, naming the ones in the way.
Nothing else sets it, and a name that is not in it is refused here
as it always was. *)
if not same && not (List.mem f.Tast.name relaxed) then
(* ── When the name is not one the programmer wrote ────────────── (* ── When the name is not one the programmer wrote ──────────────
A generic's instantiations are named [sort-i32], [sort-f32] A generic's instantiations are named [sort-i32], [sort-f32]
and so on, and the mangling carries only the *type variables* and so on, and the mangling carries only the *type variables*
@ -633,7 +642,114 @@ let eval ?(origin = "<eval>") ?pause t src : change =
(* Nothing above this line has changed the session. A [Loc.Error] from here (* Nothing above this line has changed the session. A [Loc.Error] from here
leaves it exactly as it was. *) leaves it exactly as it was. *)
let program, env = Check.program_with_env decls in let program, env = Check.program_with_env decls in
compatible ~origin:(Check.instantiation_origin env) ~loc t.program program; (* ── A class whose slots changed ──────────────────────────────────────
Every (defclass ...) in what arrived, and for the ones the session
already had, whether the slot list is the one it had. A changed list is
a changed constructor signature, which [compatible] refuses by default
and for a good reason — a call site compiled to pass two dyn words into
a three-parameter body leaves the third holding whatever was in the
register, and a dyn word that is not a value is a wild pointer, not a
wrong answer.
So the refusal stands wherever a compiled caller exists, and is lifted
exactly where one cannot: when nothing in the running program still
names the constructor, or when everything that does is being recompiled
by this same evaluation. That is the C-c C-k case — reload the file and
the class and its callers land together — and it is what makes
redefining a class in the dev loop possible at all without the versioned
functions and tracked call sites plan.org's open decision #6 describes.
**The checker gets there first, and this is still not redundant.** By
the time control reaches here the whole declaration list has been
checked against the new constructor, so a declaration still calling it
with the old argument count has already been refused, at the call site,
with a line number — which is the sentence a reader actually sees, and
is why [test_session.ml] pins that one. What remains is the *reason* the
relaxation is sound, held locally instead of inherited from another
pass: a caller that type-checks under the new arity is a caller whose
source changed, so it is in this form and is republished with the class.
The walk below asserts that rather than assuming it. If it ever fires,
something upstream has stopped being true and the answer is a refusal
and not a wild pointer.
A class whose slot list did not change is not in here at all: its
constructor has the same signature and goes through [compatible]
untouched. *)
let class_slots (ds : Ast.decl list) n =
List.fold_left
(fun acc (d : Ast.decl) ->
match d.Ast.d with
| Ast.Defclass (m, slots) when String.equal m n ->
Some (List.map fst slots)
| _ -> acc)
None ds
in
let incoming_classes =
List.filter_map
(fun (d : Ast.decl) ->
match d.Ast.d with
| Ast.Defclass (n, slots) -> Some (n, List.map fst slots)
| _ -> None)
incoming
in
let relaxed =
List.filter_map
(fun (n, slots) ->
match class_slots t.decls n with
| Some old when old <> slots ->
(* Every function of the running program that calls the
constructor or takes its address, minus the ones this
evaluation is recompiling. [Tast.walk] rather than a match on
the body's head: a constructor call can be anywhere in an
expression, and a missed one is the wild-pointer case above.
**Known to be dead today, and deliberately not tightened.**
The checker refuses every caller before this runs, so [stale]
is empty on every path anyone has found; a global's [ginit] is
not walked here for the same reason it need not be — a global
initialised by calling a constructor is re-checked with
everything else, and a mismatch there is the checker's
refusal too. What this branch is for is the day that stops
being true. It is a tripwire, not a filter: if it ever fires,
the answer is a refusal naming the callers rather than a
module that loads and then reads a register as a pointer. Do
not delete it because it is unreachable — unreachable is the
property being asserted. *)
let stale =
List.filter_map
(fun (f : Tast.fn) ->
if List.exists (String.equal f.Tast.name) names then None
else begin
let hit = ref false in
let see (e : Tast.expr) =
match e.Tast.e with
| Tast.Call (m, _) when String.equal m n -> hit := true
| Tast.FnAddr (Tast.Flanfn m) when String.equal m n ->
hit := true
| _ -> ()
in
List.iter (Tast.walk see) f.Tast.body;
List.iter (Tast.walk see) f.Tast.fdefers;
if !hit then Some f.Tast.name else None
end)
t.program.Tast.fns
in
if stale = [] then Some n
else
fail loc
"%s gains or loses slots, so its constructor takes a different \
number of arguments, and %s still call%s it with the old \
one. Evaluate the whole file (C-c C-k) so the class and its \
callers are compiled together, or restart."
n (String.concat ", " stale)
(if List.length stale = 1 then "s" else "")
(* Unchanged slots: the constructor has the signature it had, and
[compatible] has nothing to say about it. *)
| _ -> None)
incoming_classes
in
compatible ~origin:(Check.instantiation_origin env) ~relaxed ~loc t.program
program;
compatible_enums ~loc t.decls decls; compatible_enums ~loc t.decls decls;
(* ── The bodies to install ──────────────────────────────────────────── (* ── The bodies to install ────────────────────────────────────────────
The names the form declared that have a body in the checked program — The names the form declared that have a body in the checked program —
@ -697,7 +813,63 @@ let eval ?(origin = "<eval>") ?pause t src : change =
program.Tast.globals) program.Tast.globals)
names names
in in
let ir = redefinition t ~consts program ~fns in (* ── Telling the runtime what the classes now are ─────────────────────
A (defclass ...) is compile-time sugar for a constructor [defn], so
nothing about it reaches the running process except a function body —
which is why redefining one used to be silent, and why the instances
already in the program kept their old slots for ever.
What closes that is one call per class into
[runtime/flan_dyn.c]'s registry, carried by a thunk the agent runs after
the module's bodies are published and on the game thread, which is the
mechanism [C-x C-e] already uses. It has to be a thunk and not something
in the constructor: the case this exists for is a class redefined and
*not* constructed — old instances touched after the edit — and a
registration that only ran at construction would never fire.
Every class in the form, not only the ones whose slots changed. The
registry ignores a re-registration of the same list, so a C-c C-k costs
a comparison per class and migrates nothing; and a class whose
definition the registry has never seen has to arrive somehow. *)
let class_thunk =
if incoming_classes = [] then None
else begin
t.thunks <- t.thunks + 1;
let tname = Printf.sprintf "classdef/%d" t.thunks in
let str s : Tast.expr = { Tast.e = Tast.Str s; ty = Types.String; loc } in
let body =
List.map
(fun (n, slots) : Tast.expr ->
let kw : Tast.expr =
{ Tast.e = Tast.Prim (Tast.Rt "flan_dyn_kw", [ str n ]);
ty = Types.Dyn; loc }
in
(* The slot names in one string, newline between: the runtime
splits them. A dyn vector would have been the obvious shape
and is the wrong one — it is a collector object, so the
registry would hold something the marker has to reach, where
a packed string reaches interned keywords that are immortal
already. *)
{ Tast.e =
Tast.Prim (Tast.Rt "flan_dyn_class_def",
[ kw; str (String.concat "\n" slots) ]);
ty = Types.Unit; loc })
incoming_classes
in
Some
{ Tast.name = tname; params = []; ret = Types.Unit; body;
fdefers = []; fparent = None; floc = loc;
slots = [||]; snames = [||] }
end
in
let ir =
match class_thunk with
| None -> redefinition t ~consts program ~fns
| Some th ->
redefinition t ~consts ~call:th.Tast.name
{ program with Tast.fns = program.Tast.fns @ [ th ] }
~fns:(fns @ [ th.Tast.name ])
in
let allocates = let allocates =
List.exists List.exists
(fun (g : Tast.global) -> not (known t g.Tast.gname)) (fun (g : Tast.global) -> not (known t g.Tast.gname))
@ -716,7 +888,12 @@ let eval ?(origin = "<eval>") ?pause t src : change =
t.decls <- decls; t.decls <- decls;
t.program <- program; t.program <- program;
t.env <- env; t.env <- env;
{ ir; x86 = t.x86; names; fns; installs = fns <> [] || allocates || consts <> [] } (* [class_thunk] counts: a form that is only a (defclass ...) already
installs its constructor, but a module carrying nothing but the
registration still has something for the program to run. *)
{ ir; x86 = t.x86; names; fns;
installs =
fns <> [] || allocates || consts <> [] || class_thunk <> None }
(* ── Evaluating an expression ──────────────────────────────────────── *) (* ── Evaluating an expression ──────────────────────────────────────── *)

12
lib/x86.ml
View File

@ -4690,9 +4690,15 @@ let program ~checks ?(dev = false) ?(debug = false) ?(annotate = false)
meeting for the first time has no startup to have missed. [emit.ml] answers meeting for the first time has no startup to have missed. [emit.ml] answers
it the same way. it the same way.
{b The scope, and it is still narrower than [Emit.redefinition]'s.} The {b The scope, and it is still narrower than [Emit.redefinition]'s.} What is
transient [flan_reload_call] thunk is not built here, and is refused by name narrower is the IR this backend covers at all: a form it cannot lower is
-- this file's idiom for a case it has not earned the right to compile. *) refused by name, which is this file's idiom for a case it has not earned the
right to compile. The [flan_reload_call] thunk is {i not} one of them -- it
is emitted at the bottom of this function, the same as in [emit.ml], and the
agent finds it by [dlsym] either way. This note used to say it was refused;
the code moved and the sentence did not. Both a [C-x C-e] expression and the
class-registration thunk a redefined [defclass] carries go through it, and
[flan dev] takes this backend unasked. *)
let redefinition ~checks ?(dev = true) ?(known = fun _ -> true) let redefinition ~checks ?(dev = true) ?(known = fun _ -> true)
?(retains = true) ?(consts = []) ?call (p : Tast.program) ~fns : string = ?(retains = true) ?(consts = []) ?call (p : Tast.program) ~fns : string =
if not dev then if not dev then

287
runtime/flan_dyn.c
View File

@ -229,6 +229,27 @@ typedef struct flan_obj {
struct flan_obj *next; /* every object ever allocated, newest first */ struct flan_obj *next; /* every object ever allocated, newest first */
uint8_t kind; uint8_t kind;
uint8_t mark; uint8_t mark;
/* OBJ_MAP with a [klass] only: the generation of the class definition this
instance was built against. Compared against the registry's current
generation on every access that observes the slot set, and a mismatch is
a lazy migration — see [class_sync] and "Classes" below.
It lives *here*, in the padding that [kind] and [mark] leave in front of
[len]'s alignment, and that placement is the whole reason the field is
free: [sizeof(flan_obj)] is 48 with it and was 48 without it. The union
is exactly 24 bytes — [items], [cap], [klass] fill it — so there is no
spare word inside the arm, and a field after it would have cost every
dyn object in the heap eight bytes for a word only class instances read.
[flan_dyn_obj_size] answers the number and dyn_ops.c's [classes] mode
asserts it, so a later field that pushes it past 48 fails a test rather
than costing that silently.
Zero means "built before any class definition was registered", which is
also the answer for every map that is not an instance. The registry's
first registration of a name lands on 1, so a gen-0 instance of a
registered class migrates once, which is what makes a program built
before this existed correct rather than merely unbroken. */
uint32_t gen;
int64_t len; /* bytes of a text, elements of a vec or entries int64_t len; /* bytes of a text, elements of a vec or entries
of a map */ of a map */
union { union {
@ -897,6 +918,7 @@ static flan_obj *gc_alloc(uint8_t kind, int64_t extra) {
o->next = gc_all; o->next = gc_all;
o->kind = kind; o->kind = kind;
o->mark = 0; o->mark = 0;
o->gen = 0;
o->len = 0; o->len = 0;
memset(&o->u, 0, sizeof o->u); memset(&o->u, 0, sizeof o->u);
gc_all = o; gc_all = o;
@ -1100,6 +1122,238 @@ flan_dyn flan_dyn_map_new(void) {
return dyn_make(BOX_OBJ, (uint64_t)(uintptr_t)o); return dyn_make(BOX_OBJ, (uint64_t)(uintptr_t)o);
} }
/* ── Classes ───────────────────────────────────────────────────────────
*
* The registry a redefined (defclass ...) updates, and the lazy migration
* that makes the instances built against the old definition answer the new
* one. This is CLHS 4.3.6 — [update-instance-for-redefined-class] — with the
* user hook left out; docs/SBCL-REDEFINITION-NOTES.md is where the protocol
* was read off and candidate C is this.
*
* **Why a registry at all, when a class instance is already just a map.**
* Because a map cannot be asked what it is *supposed* to hold. The instance
* knows the keys it has; only the class knows the keys it ought to have, and
* "ought to have" is the whole content of a redefinition. So one entry per
* class name, holding the current slot list and a generation, and an
* instance holds the generation it was built against.
*
* **Why nothing here is a GC object.** A class's name and its slots are
* *names*, and [flan_dyn_kw]'s entries are interned, immortal and not on the
* collector's heap — the same argument the [klass] field makes one screen up.
* The table below is [malloc]ed, append-only and never freed, so the marker
* has nothing to trace here and no root has to be pushed for it. A registry
* of dyn vectors would have needed both, and would have needed them to
* survive a collection triggered from inside a migration.
*
* **What the registry does not do.** It does not constrain [put]. A class
* instance is an open map — FIX.org already defers refusing [(get p :z)] —
* so a key nobody declared can be written to one, and the migration below
* will *drop* it at the next redefinition, because its rule is that an
* instance's keys are the class's slots. That is real data loss and it is
* written down as such in FIX.org rather than dressed up as enforcement.
*
* **Where a migration happens.** [want_map], so every [get], [put] and
* [has-key?]; [flan_dyn_len]'s map arm; and [dyn_equal]'s, so two instances
* of different generations are compared as the class currently defines them
* rather than by the shapes they happen to be carrying. CLHS asks for "no
* later than the next time a slot is read or written" and those are the
* three places that read or write the slot *set*.
*
* **The two printers are deliberately not among them**, and the consequence
* is visible to whoever is sitting in front of the editor, so it is written
* out rather than left as a footnote. [render] — which [print] and every
* value the editor renders go through — and [say_render] — the 96-byte
* sentence a trap prints — both walk [items] raw and neither syncs.
*
* The reason is the same for both: [say_render] runs inside trap reporting,
* where the heap is whatever the trap left, and a printer that frees an
* object's entry block and installs another is not something to have on
* that path; [render] is the same function's sibling and is called from it
* for nested values, so splitting them would put a mutation one recursion
* below a trap anyway.
*
* What that costs: **a stale instance shows its OLD slots to the editor
* until something touches it.** A watch expression, the value [C-x C-e]
* answers, and the inspector's render of a dyn all reach a class instance
* through [render], so immediately after a [defclass] is redefined the
* inspector can show a slot the class no longer has, and not show one it
* has gained — while [(get p :z)] typed at the same instant answers the new
* definition and migrates it, after which the inspector agrees. CLHS's
* "implementation-dependent time" permits it and it is the price of the
* printer staying a printer; it is not a bug report waiting to happen only
* because it is written down here, in FIX.org, and nowhere else. */
/* Interning, which is under "Keywords" further down. This file includes no
* header of its own — every entry point is written out in flan_dyn.h and
* defined here in the order the sections read best — so the one call that
* runs ahead of its definition declares itself. */
flan_dyn flan_dyn_kw(const uint8_t *p, int64_t n);
typedef struct class_entry {
kw_entry *name;
kw_entry **slots; /* interned, immortal, in declaration order */
int64_t nslots;
uint32_t gen;
} class_entry;
static class_entry *classes;
static int64_t classes_n, classes_cap;
static class_entry *class_find(kw_entry *name) {
int64_t i;
for (i = 0; i < classes_n; i++)
if (classes[i].name == name) return &classes[i];
return NULL;
}
/* The generation a new instance of [name] is stamped with. Zero for a class
* no definition has been registered for, which is every class in a program
* that was built and never reloaded: nothing has changed shape, so nothing
* needs to migrate, and the registry earns its keep only once an editor has
* sent a new definition. */
static uint32_t class_gen(kw_entry *name) {
class_entry *e = class_find(name);
return e == NULL ? 0u : e->gen;
}
/* One class's current slot list, as the compiler's per-reload thunk hands it
* over: the class's name as a keyword, and the slot names packed into one
* string, newline between and no leading colons — the shape a string literal
* already crosses in, rather than a dyn vector this would have to root.
*
* The generation is bumped only when the list actually differs. That is what
* makes C-c C-k idempotent: reloading a file re-runs every one of its class
* definitions, and a bump per reload would migrate every instance in the
* program every time anybody saved, for no change. */
void flan_dyn_class_def(flan_dyn name, const uint8_t *slots, int64_t n) {
kw_entry *k;
kw_entry **list = NULL;
int64_t count = 0, i, start;
class_entry *e;
if (flan_dyn_tag(name) != FLAN_DYN_TAG_KEYWORD)
/* No location: the caller is the thunk a reload runs, which has no
source position of its own — the class's own [defclass] is where a
reader would look, and it is not on any stack by the time this runs.
Unreachable from written Flan in any case; only the compiler emits
this call, and it emits a keyword. */
trap1(NULL, 0, TYPE_TRAP, "class definition",
"a class name is a keyword", name);
k = dyn_kw(name);
if (n < 0) n = 0;
/* Count first, then fill: one allocation of the right size, and an empty
* class — (defclass marker []) is in the corpus — allocates nothing. */
for (i = 0, start = 0; i <= n; i++)
if (i == n ? i > start : slots[i] == '\n') {
if (i > start) count++;
start = i + 1;
}
if (count > 0) {
list = (kw_entry **)malloc((size_t)count * sizeof *list);
if (list == NULL) trap_oom(count * (int64_t)sizeof *list);
count = 0;
for (i = 0, start = 0; i <= n; i++)
if (i == n ? i > start : slots[i] == '\n') {
if (i > start)
list[count++] = dyn_kw(flan_dyn_kw(slots + start, i - start));
start = i + 1;
}
}
e = class_find(k);
if (e != NULL) {
int same = e->nslots == count;
if (same)
for (i = 0; i < count; i++)
if (e->slots[i] != list[i]) { same = 0; break; }
if (same) { free(list); return; }
free(e->slots);
e->slots = list;
e->nslots = count;
/* Wrapping is not a correctness question — what matters is that the new
* generation differs from the one the live instances carry — but zero is
* reserved for "no definition registered", so it is stepped over. */
e->gen = e->gen + 1u;
if (e->gen == 0u) e->gen = 1u;
return;
}
if (classes_n == classes_cap) {
int64_t cap = classes_cap ? classes_cap * 2 : 8;
class_entry *t =
(class_entry *)realloc(classes, (size_t)cap * sizeof *t);
if (t == NULL) trap_oom(cap * (int64_t)sizeof *t);
classes = t;
classes_cap = cap;
}
classes[classes_n].name = k;
classes[classes_n].slots = list;
classes[classes_n].nslots = count;
/* One, never zero: an instance built before this registration carries zero
* and has to be seen as stale, because the definition it was built from is
* exactly the one nobody recorded. */
classes[classes_n].gen = 1u;
classes_n++;
}
/* The migration. [o] is left holding exactly the class's current slots, in
* the class's order, with the values it already had for the ones it still
* has and nil for the ones it has just gained — which is precisely the
* property CLHS 4.3.6 guarantees, matched by name, with the instance's
* identity preserved because none of this allocates a new object.
*
* Rebuilt into a fresh block rather than compacted in place, and the order is
* the class's rather than the instance's, so that a migrated instance is
* indistinguishable from one the constructor has just built. [dyn_equal]
* compares maps by lookup and would not have cared; [render] and [len] print
* and count in insertion order and would have. One malloc per instance per
* redefinition is the price, and a migration happens once.
*
* Nothing here allocates on the collector's heap, so no collection can run
* part-way through and see an object whose [len] and [items] disagree.
*
* Nor can it free a block something above it is walking. The block it frees
* is [o]'s, and every caller syncs [o] before it starts walking [o] — so a
* re-entry through a nested [dyn_equal], including a map used as a key of
* itself, finds [o] already current and returns at the generation compare.
* The key scan here uses the interned identity compare and calls
* [dyn_equal] not at all, so it cannot re-enter from inside. */
static void class_sync(flan_obj *o) {
class_entry *e;
flan_dyn *fresh = NULL;
int64_t i, j;
if (o->kind != OBJ_MAP || o->u.v.klass == NULL) return;
e = class_find(o->u.v.klass);
if (e == NULL || e->gen == o->gen) return;
if (e->nslots > 0) {
fresh = (flan_dyn *)malloc((size_t)e->nslots * 2 * sizeof *fresh);
if (fresh == NULL) trap_oom(e->nslots * 2 * (int64_t)sizeof *fresh);
}
for (j = 0; j < e->nslots; j++) {
flan_dyn v = dyn_make(BOX_NIL, 0);
for (i = 0; i < o->len; i++) {
flan_dyn key = o->u.v.items[i * 2];
/* [flan_dyn_tag] and not a bare [dyn_box]: a float is not boxed at
all, so its payload bits can read as any box tag, and reading a
non-keyword's payload as a [kw_entry *] is a wild pointer. A raw
[put] can have left a float — or anything else — in here. */
if (flan_dyn_tag(key) == FLAN_DYN_TAG_KEYWORD
&& dyn_kw(key) == e->slots[j]) {
v = o->u.v.items[i * 2 + 1];
break;
}
}
fresh[j * 2] = dyn_make(BOX_KW, (uint64_t)(uintptr_t)e->slots[j]);
fresh[j * 2 + 1] = v;
}
/* Charged the way [map_set]'s growth is, in both directions: a class that
* lost slots gives the bytes back, or the trigger drifts up by whatever
* every migration in the program ever released. */
gc_bytes += (e->nslots - o->u.v.cap) * 2 * (int64_t)sizeof(flan_dyn);
free(o->u.v.items);
o->u.v.items = fresh;
o->u.v.cap = e->nslots;
o->len = e->nslots;
o->gen = e->gen;
}
/* The same map with a shape tag on it: what a (defclass ...) constructor /* The same map with a shape tag on it: what a (defclass ...) constructor
* calls. [k] is a keyword and anything else traps by name — the compiler * calls. [k] is a keyword and anything else traps by name — the compiler
* hands it the class's own name and nothing else can reach this. */ * hands it the class's own name and nothing else can reach this. */
@ -1112,6 +1366,10 @@ flan_dyn flan_dyn_map_new_class(flan_dyn k) {
o->u.v.items = NULL; o->u.v.items = NULL;
o->u.v.cap = 0; o->u.v.cap = 0;
o->u.v.klass = dyn_kw(k); o->u.v.klass = dyn_kw(k);
/* Stamped at construction against whatever the registry currently says, so
* an instance built by the constructor this reload just installed is
* already current and never migrates. */
o->gen = class_gen(o->u.v.klass);
return dyn_make(BOX_OBJ, (uint64_t)(uintptr_t)o); return dyn_make(BOX_OBJ, (uint64_t)(uintptr_t)o);
} }
@ -1119,6 +1377,8 @@ flan_dyn flan_dyn_map_new_class(flan_dyn k) {
* traps. Absence is an answer here for the reason it is one in [map_get] — * traps. Absence is an answer here for the reason it is one in [map_get] —
* asking what class a value is, is a question every value can be asked, and * asking what class a value is, is a question every value can be asked, and
* an ordinary map, a number and nil all truthfully answer "none". */ * an ordinary map, a number and nil all truthfully answer "none". */
int64_t flan_dyn_obj_size(void) { return (int64_t)sizeof(flan_obj); }
flan_dyn flan_dyn_class_of(flan_dyn v) { flan_dyn flan_dyn_class_of(flan_dyn v) {
flan_obj *o; flan_obj *o;
if (flan_dyn_tag(v) != FLAN_DYN_TAG_MAP) return flan_dyn_nil(); if (flan_dyn_tag(v) != FLAN_DYN_TAG_MAP) return flan_dyn_nil();
@ -1611,6 +1871,14 @@ static int dyn_equal(flan_dyn a, flan_dyn b, int depth) {
int64_t i, j; int64_t i, j;
if (x == y) return 1; if (x == y) return 1;
if (depth >= EQ_DEPTH) return 0; if (depth >= EQ_DEPTH) return 0;
/* Two instances of one class built either side of a redefinition hold
different key sets, and comparing those key sets would answer "not
equal" about a difference the class no longer has. So both are brought
to the current definition first and the comparison is then the ordinary
one. The decision this records: equality is over the class as it is
now, and not over the shapes the two values were born with. */
class_sync(x);
class_sync(y);
/* The shape tag is part of the value. Two instances of one class compare /* The shape tag is part of the value. Two instances of one class compare
* by their entries as any two maps do; an instance and a plain map with * by their entries as any two maps do; an instance and a plain map with
* the same entries do not, which is Clojure's answer for a record beside * the same entries do not, which is Clojure's answer for a record beside
@ -1795,7 +2063,15 @@ flan_dyn flan_dyn_view_flat(void *data, int64_t len, int32_t elem) {
} }
flan_dyn flan_dyn_len(flan_dyn v) { flan_dyn flan_dyn_len(flan_dyn v) {
if (is_text(v) || is_map(v)) return flan_dyn_from_i64(dyn_obj(v)->len); if (is_text(v)) return flan_dyn_from_i64(dyn_obj(v)->len);
/* A map's length is its slot count, so a stale instance would answer the
count of a definition that no longer exists. Migrated first for the same
reason [get] is. */
if (is_map(v)) {
flan_obj *o = dyn_obj(v);
class_sync(o);
return flan_dyn_from_i64(o->len);
}
if (is_vec(v)) { if (is_vec(v)) {
flan_obj *o = dyn_obj(v); flan_obj *o = dyn_obj(v);
if (o->kind == OBJ_VIEW) return flan_dyn_from_i64(view_len("len", o)); if (o->kind == OBJ_VIEW) return flan_dyn_from_i64(view_len("len", o));
@ -1922,8 +2198,15 @@ static int64_t map_find(flan_obj *o, flan_dyn k) {
} }
static flan_obj *want_map(const char *op, flan_dyn m, flan_dyn k) { static flan_obj *want_map(const char *op, flan_dyn m, flan_dyn k) {
flan_obj *o;
if (!is_map(m)) trap2(NULL, 0, TYPE_TRAP, op, "only a map answers it", m, k); if (!is_map(m)) trap2(NULL, 0, TYPE_TRAP, op, "only a map answers it", m, k);
return dyn_obj(m); o = dyn_obj(m);
/* The lazy half of the redefinition protocol: [get], [put] and [has-key?]
all arrive here, and CLHS 4.3.6 asks for the update to happen no later
than the next read or write of a slot. A plain map returns from the
first line of [class_sync] untouched. */
class_sync(o);
return o;
} }
flan_dyn flan_dyn_map_get(flan_dyn m, flan_dyn k) { flan_dyn flan_dyn_map_get(flan_dyn m, flan_dyn k) {

28
runtime/flan_dyn.h
View File

@ -94,6 +94,34 @@ flan_dyn flan_dyn_map_new_class(flan_dyn k);
* — an ordinary map included. Never traps. */ * — an ordinary map included. Never traps. */
flan_dyn flan_dyn_class_of(flan_dyn v); flan_dyn flan_dyn_class_of(flan_dyn v);
/* A class definition, registered or re-registered: [name] is the class's name
* as a keyword and [slots]/[n] is its slot names packed into one string,
* newline between and no leading colons. The compiler emits one call per
* (defclass ...) into the thunk a reload runs, so a definition that changed
* lands here before anything touches an instance.
*
* The generation moves only when the slot list really differs, so re-running
* a file's definitions unchanged — every C-c C-k — costs a comparison and
* migrates nothing. When it does move, every instance built against an
* earlier definition migrates lazily at its next [get], [put], [has-key?],
* [len] or equality comparison: slots the class still has keep their values
* matched by name, slots it has gained appear as nil, and keys it no longer
* declares are dropped. The instance's identity is preserved throughout;
* this is CLHS 4.3.6 without the user hook.
*
* The drop is unconditional, which is the honest cost of a class instance
* being an open map: a key written by a raw [put] that the class never
* declared is dropped by the next migration too. The registry describes the
* class's intention and does not enforce it. */
void flan_dyn_class_def(flan_dyn name, const uint8_t *slots, int64_t n);
/* [sizeof(flan_obj)], for the one test that asserts it. The generation a
* class instance carries was fitted into the padding between [mark] and
* [len] precisely so that this number did not move; a field that pushed it
* up would cost every dyn value in the heap, so the number is asserted
* rather than left to a comment. */
int64_t flan_dyn_obj_size(void);
/* A keyword: :foo as a run-time value. Interned — the runtime keeps one entry /* A keyword: :foo as a run-time value. Interned — the runtime keeps one entry
* per distinct name forever, so two keywords with the same bytes are the same * per distinct name forever, so two keywords with the same bytes are the same
* word and equality is an identity compare, never a memcmp. The entries are * word and equality is an identity compare, never a memcmp. The entries are

16
test/dune
View File

@ -56,6 +56,10 @@
(glob_files programs/pkgs/mac/*) (glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*) (glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*) (glob_files programs/pkgs/macspin/*)
; The package whose body calls the builtin get while the program importing
; it defines a get of its own — shadow-builtin.flan, which is the pin that
; a shadow stops at the file that declared it.
(glob_files programs/pkgs/shadowed/*)
; The synthetic C header the importer's table reads. Committed rather than ; The synthetic C header the importer's table reads. Committed rather than
; reached for on the machine: the raylib case needs raylib installed, at the ; reached for on the machine: the raylib case needs raylib installed, at the
; right version, with a variable set, so it skips everywhere and covers ; right version, with a variable set, so it skips everywhere and covers
@ -215,7 +219,9 @@
; And the macro-declaring packages, for the same reason. ; And the macro-declaring packages, for the same reason.
(glob_files programs/pkgs/mac/*) (glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*) (glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*)) (glob_files programs/pkgs/macspin/*)
; And the package shadow-builtin.flan imports.
(glob_files programs/pkgs/shadowed/*))
(action (run ./test_valgrind.exe))) (action (run ./test_valgrind.exe)))
; The corpus a fourth time, through the hand-written x86-64 backend, compared ; The corpus a fourth time, through the hand-written x86-64 backend, compared
@ -272,7 +278,9 @@
(glob_files programs/pkgs/tree/*) (glob_files programs/pkgs/tree/*)
(glob_files programs/pkgs/mac/*) (glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*) (glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*)) (glob_files programs/pkgs/macspin/*)
; And the package shadow-builtin.flan imports.
(glob_files programs/pkgs/shadowed/*))
(action (action
(setenv SURVEY_STRICT 1 (setenv SURVEY_STRICT 1
(setenv SURVEY_QUIET 1 (setenv SURVEY_QUIET 1
@ -435,7 +443,9 @@
(glob_files programs/pkgs/tree/*) (glob_files programs/pkgs/tree/*)
(glob_files programs/pkgs/mac/*) (glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*) (glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*)) (glob_files programs/pkgs/macspin/*)
; And the package shadow-builtin.flan imports.
(glob_files programs/pkgs/shadowed/*))
(action (action
(setenv SURVEY_STRICT 1 (setenv SURVEY_STRICT 1
(setenv SURVEY_QUIET 1 (setenv SURVEY_QUIET 1

207
test/dyn_ops.c
View File

@ -1017,6 +1017,209 @@ static void refuse(const char *what) {
exit(3); exit(3);
} }
/* ── Redefining a class ────────────────────────────────────────────────
*
* CLHS 4.3.6 as flan_dyn.c implements it: a class definition is registered,
* an instance is stamped with the generation it was built against, and the
* first access after the definition changes migrates the instance — slots
* kept by name, gained slots nil, dropped slots gone, identity preserved.
*
* Driven from C rather than from Flan because the *event* being tested has
* no Flan spelling: [flan_dyn_class_def] is called by a thunk the reload
* agent runs, so a single program can never see its class change. The
* daemon case in test_dev.ml is the same protocol with a real editor at one
* end; this is the protocol itself, at a granularity a daemon cannot show. */
static flan_dyn slot(flan_dyn m, const char *name) {
return flan_dyn_map_get(m, flan_dyn_kw((const uint8_t *)name,
(int64_t)strlen(name)));
}
static void define(const char *name, const char *slots) {
flan_dyn_class_def(flan_dyn_kw((const uint8_t *)name,
(int64_t)strlen(name)),
(const uint8_t *)slots, (int64_t)strlen(slots));
}
/* One instance of :point with the two slots the first definition gives it.
* Built through the same entry point a constructor uses, so it is stamped
* exactly as compiled code would stamp it. */
static flan_dyn a_point(int64_t x, int64_t y) {
flan_dyn p = flan_dyn_map_new_class(flan_dyn_kw((const uint8_t *)"point", 5));
flan_dyn_map_set(p, flan_dyn_kw((const uint8_t *)"x", 1),
flan_dyn_from_i64(x));
flan_dyn_map_set(p, flan_dyn_kw((const uint8_t *)"y", 1),
flan_dyn_from_i64(y));
return p;
}
static void classes(void) {
flan_dyn p = flan_dyn_nil(), q = flan_dyn_nil(), plain = flan_dyn_nil();
flan_dyn keep = flan_dyn_nil();
int64_t i;
flan_gc_init();
flan_dyn_root_push(&p);
flan_dyn_root_push(&q);
flan_dyn_root_push(&plain);
flan_dyn_root_push(&keep);
/* The word this whole design rests on. The generation went into the two
bytes of padding between [mark] and [len] precisely so that a dyn object
stayed the size it was; a later field that pushed it out would cost every
value in the heap and would do it silently. */
check(flan_dyn_obj_size() == 48, "a dyn object is still 48 bytes");
/* ── Nothing registered ──
A program built and never reloaded has no registry at all, and its
instances must behave exactly as they did before any of this existed. */
p = a_point(1, 2);
prints(p, "#point{ :x 1 :y 2}");
check(num(slot(p, "x")) == 1, "an unregistered class reads its slot");
check(num(flan_dyn_len(p)) == 2, "an unregistered class has its length");
/* ── A slot gained ──
The first registration lands on generation one, so the instance built a
moment ago — carrying zero, the generation of a definition nobody
recorded — is stale and migrates at the next touch. [:z] appears as nil
and [:x] keeps the value it was constructed with. */
define("point", "x\ny\nz");
check(flan_dyn_tag(slot(p, "z")) == FLAN_DYN_TAG_NIL,
"a gained slot arrives as nil");
check(num(slot(p, "x")) == 1, "a kept slot keeps its value");
check(num(flan_dyn_len(p)) == 3, "a gained slot is counted");
prints(p, "#point{ :x 1 :y 2 :z nil}");
/* And the tag survived: a migration must not turn an instance into a map. */
check(truth(flan_dyn_eq(flan_dyn_class_of(p),
flan_dyn_kw((const uint8_t *)"point", 5))),
"a migrated instance is still an instance");
/* ── A slot lost ──
Dropped by name and not by position: [:y] goes, [:x] and [:z] stay where
they were. This is also the case that quietly documents the advisory
registry — a key put into an instance that the class does not declare
would be dropped by exactly this pass. */
flan_dyn_map_set(p, flan_dyn_kw((const uint8_t *)"z", 1),
flan_dyn_from_i64(9));
define("point", "x\nz");
check(num(flan_dyn_len(p)) == 2, "a lost slot is gone from the length");
check(flan_dyn_tag(slot(p, "y")) == FLAN_DYN_TAG_NIL,
"a lost slot reads as absent");
check(num(slot(p, "z")) == 9, "a slot either side of a lost one is kept");
prints(p, "#point{ :x 1 :z 9}");
/* ── Gained and lost at once, and the third bump ──
Three definitions have now been registered after the first, so the
generation has moved three times; an instance that has been touched
between each of them has followed every step. */
define("point", "z\nw");
check(num(slot(p, "z")) == 9, "a kept slot survives a third redefinition");
check(flan_dyn_tag(slot(p, "w")) == FLAN_DYN_TAG_NIL,
"the third redefinition's new slot is nil");
check(flan_dyn_tag(slot(p, "x")) == FLAN_DYN_TAG_NIL,
"the third redefinition's dropped slot is gone");
/* The slot order is the class's, not the instance's history: a migrated
instance has to be indistinguishable from a freshly constructed one, or
[len], [render] and insertion order would each tell a different story. */
prints(p, "#point{ :z 9 :w nil}");
/* ── Re-registering the same list changes nothing ──
This is what makes evaluating a whole file idempotent. If a bump
happened per registration, every save would migrate every instance in
the program for no change at all — and here it would reset [:z] to nil,
because the value below is written *after* the re-registration. */
flan_dyn_map_set(p, flan_dyn_kw((const uint8_t *)"w", 1),
flan_dyn_from_i64(4));
define("point", "z\nw");
check(num(slot(p, "w")) == 4, "an unchanged redefinition migrates nothing");
/* ── An instance never touched across three redefinitions ──
The migration is lazy, so an instance can sit through any number of
definitions and meet only the last one. [q] is built at the generation
[p] started from and is not read until the end. */
q = a_point(5, 6);
define("point", "z\nw\nq1");
define("point", "z\nw\nq2");
define("point", "z\nw\nq3");
check(num(flan_dyn_len(q)) == 3, "a long-stale instance migrates once");
check(flan_dyn_tag(slot(q, "x")) == FLAN_DYN_TAG_NIL,
"a long-stale instance loses what the last definition dropped");
check(flan_dyn_tag(slot(q, "q3")) == FLAN_DYN_TAG_NIL,
"a long-stale instance gains the last definition's slots");
check(flan_dyn_tag(slot(q, "q1")) == FLAN_DYN_TAG_NIL,
"and none of the definitions in between");
/* ── Mixed generations compare as the class now is ──
Two instances of one class, one built before a redefinition and one
after, holding the same values for the slots the class still has. The
decision recorded here is that they are equal: equality migrates first,
so it is over the class as it is now and not over the shapes the two
values were born with. */
define("point", "x\ny");
p = a_point(1, 2); /* two slots, the definition of the day */
define("point", "x\ny\nn");
/* Built by hand rather than through [a_point], because it is the instance
the *new* constructor would build: three slots, stamped current. */
q = flan_dyn_map_new_class(flan_dyn_kw((const uint8_t *)"point", 5));
flan_dyn_map_set(q, flan_dyn_kw((const uint8_t *)"x", 1),
flan_dyn_from_i64(1));
flan_dyn_map_set(q, flan_dyn_kw((const uint8_t *)"y", 1),
flan_dyn_from_i64(2));
flan_dyn_map_set(q, flan_dyn_kw((const uint8_t *)"n", 1), flan_dyn_nil());
check(truth(flan_dyn_eq(p, q)),
"two generations of one class with equal slots are equal");
/* ── A plain map is untouched by any of it ──
No tag, so no registry entry is ever looked up: the key a class happens
to have declared is just a key here, and nothing is added, dropped or
reordered however many times :point is redefined. */
plain = flan_dyn_map_new();
flan_dyn_map_set(plain, flan_dyn_kw((const uint8_t *)"y", 1),
flan_dyn_from_i64(7));
flan_dyn_map_set(plain, flan_dyn_kw((const uint8_t *)"x", 1),
flan_dyn_from_i64(8));
define("point", "x\ny\nz\nzz");
check(num(flan_dyn_len(plain)) == 2, "a plain map gains no slot");
prints(plain, "{ :y 7 :x 8}");
check(flan_dyn_tag(flan_dyn_class_of(plain)) == FLAN_DYN_TAG_NIL,
"a plain map has no class");
/* ── Migration under collection ──
The one thing a migration must not do is confuse the marker. It frees
the instance's entry block and installs another, so an object whose
[len] and [items] disagreed for even an instant would be a walk off the
end of a block on the next mark. Two thousand instances, held in a
rooted vec, all stale, all migrated while the heap is collecting around
them — and then read back, which is what says the values survived rather
than merely that nothing crashed. */
flan_gc_set_floor(16 * 1024);
define("point", "x\ny");
keep = flan_dyn_vec_new();
for (i = 0; i < 2000; i++) flan_dyn_push(keep, a_point(i, i + 1));
define("point", "y\nx\ndeep");
for (i = 0; i < 2000; i++) {
flan_dyn e = flan_dyn_at(keep, flan_dyn_from_i64(i));
/* Allocation between each migration, so a collection lands part-way
through the set and has both shapes to mark. */
flan_dyn_map_set(e, flan_dyn_kw((const uint8_t *)"deep", 4),
flan_dyn_vec_new());
}
flan_gc_collect();
{
int ok = 1;
for (i = 0; i < 2000; i++) {
flan_dyn e = flan_dyn_at(keep, flan_dyn_from_i64(i));
if (num(slot(e, "x")) != i || num(slot(e, "y")) != i + 1) ok = 0;
if (flan_dyn_tag(slot(e, "deep")) != FLAN_DYN_TAG_VEC) ok = 0;
if (num(flan_dyn_len(e)) != 3) ok = 0;
}
check(ok, "two thousand instances migrated across a collection");
}
flan_dyn_root_pop(4);
printf(failures == 0 ? "classes ok\n" : "classes failed\n");
}
int main(int argc, char **argv) { int main(int argc, char **argv) {
flan_rt_init(argc, argv); flan_rt_init(argc, argv);
if (argc < 2) { if (argc < 2) {
@ -1034,6 +1237,10 @@ int main(int argc, char **argv) {
if (strcmp(argv[1], "unrooted") == 0) { unrooted(); return 0; } if (strcmp(argv[1], "unrooted") == 0) { unrooted(); return 0; }
if (strcmp(argv[1], "park") == 0) { park(); return 0; } if (strcmp(argv[1], "park") == 0) { park(); return 0; }
if (strcmp(argv[1], "desc") == 0) { desc(); return 0; } if (strcmp(argv[1], "desc") == 0) { desc(); return 0; }
if (strcmp(argv[1], "classes") == 0) {
classes();
return failures == 0 ? 0 : 1;
}
if (strcmp(argv[1], "view") == 0) { if (strcmp(argv[1], "view") == 0) {
view(); view();
return failures == 0 ? 0 : 1; return failures == 0 ? 0 : 1;

46
test/programs/dev-classes.flan Normal file
View File

@ -0,0 +1,46 @@
;;;; A class, instances of it, and a session that changes the class.
;;;;
;;;; dev-class.flan already asks whether a *method* added to a running
;;;; program reaches a call site compiled before it existed. This asks the
;;;; harder half: whether the instances already in the program survive their
;;;; class being redefined. They are ordinary dyn maps held in a dyn global,
;;;; so the reload cannot touch them — a redefinition makes a program's
;;;; globals external and never re-initialises them, which is the whole of
;;;; "edit the code, keep the sand" — and what the editor sends is a new
;;;; constructor plus a registration of the class's new slot list. The
;;;; migration happens lazily, in the runtime, at the first touch after that.
;;;;
;;;; [instances] is a global rather than a local for exactly that reason: an
;;;; expression the editor evaluates is a thunk with a frame of its own, so
;;;; anything it is supposed to still be holding an hour later has to live
;;;; somewhere the thunk is not.
;;;;
;;;; The instances are pushed by the editor rather than by [main], and that
;;;; is not incidental. A (defclass ...) whose slots change is a constructor
;;;; whose signature changes, and the session refuses that wherever a
;;;; compiled caller of the constructor is left standing — a call site that
;;;; passes two dyn words into a three-parameter body leaves the third
;;;; holding a register, and a dyn word that is not a value is a wild
;;;; pointer. A [main] calling [(point 3 4)] would be exactly such a caller,
;;;; so this program has none and the instances arrive from thunks, which
;;;; leave nothing behind. test_session.ml pins the refusal itself.
;;;;
;;;; It keeps running rather than returning, for dev-class.flan's reason: an
;;;; expression typed at the editor is a thunk the agent runs at a frame
;;;; boundary, and a parked program has none.
(import agent "vendor:agent")
(defclass point [x y])
(defgeneric area [self] dyn)
(defmethod area point [p] (* (get p :x) (get p :y)))
(defvar instances dyn)
(defn main [] i32
(agent/start "/tmp/flan-dev-classes-fallback.sock")
(set instances (vec-new dyn))
(dotimes [i 4000]
(agent/wait 5))
0)

18
test/programs/pkgs/shadowed/shadowed.flan Normal file
View File

@ -0,0 +1,18 @@
;;;; A package that calls the builtin get, imported by a program that defines
;;;; a get of its own.
;;;;
;;;; The importer's defn takes the name over in the importer's own file and
;;;; nowhere else: this file's names were qualified at the import (this
;;;; function is shadowed/field to everything downstream), so the get written
;;;; here is the builtin's, was compiled as the builtin's, and answers what a
;;;; dyn map holds under a keyword.
(defn field [m] dyn (get m :b))
;;; The same question asked where there is no enclosing function to be
;;; qualified: a global's initialiser, which runs at startup and is checked
;;; with no owner at all. The importer below defines a len of its own; this
;;; one is the builtin's and this global is 4.
(defvar size i32 (len "abcd"))
(defn stored-size [] i32 size)

45
test/programs/shadow-builtin.flan Normal file
View File

@ -0,0 +1,45 @@
;;;; A defn named after a builtin, and what the name means afterwards.
;;;;
;;;; "Allow shadowing but warn": this file's (defn get ...) is legal, it wins
;;;; at every call site in this file, and the compiler says so once at the
;;;; definition — the warning is on stderr and the exit status does not move.
;;;;
;;;; Five lines printed, and between them the whole rule. In order:
;;;;
;;;; - 7: (get p) is one argument, which the builtin get does not take. It
;;;; compiles, and it prints the field, because the name resolves to the
;;;; definition below and the builtin is not consulted about its arity.
;;;; - 4: (shadowed/field m) reaches into the imported package, whose body
;;;; calls the builtin get on a dyn map. The shadow does not follow it
;;;; there: a package's calls mean what they meant when it was written.
;;;; - 99: an operator is a builtin like any other and shadows like one.
;;;; - 999: this file's own len, which is what len means in this file.
;;;; - 4 again, and it is the one that needed the work: the package's global
;;;; initialiser (defvar size i32 (len "abcd")) is checked with no
;;;; enclosing function at all, so there is no qualified name on it to say
;;;; it belongs to a package. The file it was written in says so instead.
(import shadowed "pkgs/shadowed")
(defstruct P [x i32])
(defn get [p P] i32 (.x p))
;;; An operator is a builtin like any other, and shadows like any other: (+ 1
;;; 2) below is this definition and answers 99. Nothing else in the program
;;; adds anything, and the prelude's own additions are untouched — the
;;; prelude is a different file.
(defn + [a i32 b i32] i32 99)
;;; And a builtin the imported package uses in a *global initialiser*, which
;;; is the one place there is no enclosing function to carry a package's
;;; qualified name. The package's (defvar size i32 (len "abcd")) is 4; this
;;; definition answers 999 and is reached only here.
(defn len [s string] i32 999)
(defn main [] ()
(println (get (P {.x 7})))
(println (shadowed/field {:a 1 :b 4}))
(println (+ 1 2))
(println (len "abcd"))
(println (shadowed/stored-size)))

20
test/test_acceptance.ml
View File

@ -2527,6 +2527,26 @@ let () =
shape/Box and not area/shape/Box. *) shape/Box and not area/shape/Box. *)
outputs "a diamond, with a type crossing it" "programs/pkg-diamond.flan" outputs "a diamond, with a type crossing it" "programs/pkg-diamond.flan"
"3\n6\n20\n"; "3\n6\n20\n";
(* A defn named after a builtin, and the boundary the shadow stops at.
The numbers are the whole claim and none of them could be printed by
the other reading: 7 is the program's own one-argument (get p), which
the builtin get has no arity for at all; 4 is the builtin get called
inside the imported package on a dyn map; 99 is an operator shadowed
like any other name; 999 is this program's len.
The last 4 is the one that was a bug. It is the package's global
initialiser, (defvar size i32 (len "abcd")), which is the one place a
call sits inside no function and so carries no package-qualified name
— the importer's len reached into it and made it 999. The shadow is
decided by the file the definition was written in, and a file is
something a global initialiser has.
The warning the definitions earn is on stderr and is pinned in
test_flan.ml, where the line and column can be asked about directly.
What is asserted here is that it changes nothing else: the program
runs and its status is zero. *)
outputs "a defn shadows a builtin, and the package it imports does not"
"programs/shadow-builtin.flan" "7\n4\n99\n999\n4\n";
(* A data type crossing the same boundary, which was a refusal by name (* A data type crossing the same boundary, which was a refusal by name
until vendor/edn needed one. The rename has two halves and the second until vendor/edn needed one. The rename has two halves and the second
is the one that is easy to do by accident only: the type's name is a is the one that is easy to do by accident only: the type's name is a

240
test/test_dev.ml
View File

@ -5317,6 +5317,246 @@ let () =
List.iter (fun f -> try Sys.remove f with Sys_error _ -> ()) List.iter (fun f -> try Sys.remove f with Sys_error _ -> ())
[ psock; pout ]; [ psock; pout ];
(* ── A class redefined under its own instances ──────────────────
CLHS 4.3.6's update protocol, end to end, with a real editor at one
end and the running program's own heap at the other. "A method added
to a running program", further up, adds a *method* to a live
program; this changes what the class *is*,
which is the case that used to be silent — a (defclass ...) is
compile-time sugar for a constructor, so redefining one replaced a
function body and told the instances nothing.
What closes it: the module the session builds now carries a
registration of the class's new slot list, run by the agent after the
bodies are published, and the runtime migrates each instance lazily at
its next touch. Everything below is asked of the program, on its own
thread, against objects it has been holding since before the edit.
Every answer is compared inside the expression for the reason the
block above gives: a dyn value renders to the program's stdout and a
typed [1] lands in the reply's [:value], which takes the timing out of
the test. *)
let msock = tmp "migrate.sock" and mout = tmp "migrate.out" in
(try Sys.remove msock with Sys_error _ -> ());
let mfd = Unix.openfile mout [ Unix.O_WRONLY; Unix.O_CREAT; Unix.O_TRUNC ] 0o600 in
let mpid =
Unix.create_process flan
[| flan; "dev"; "programs/dev-classes.flan"; "-s"; msock |]
Unix.stdin mfd Unix.stderr
in
Unix.close mfd;
if not (listening ~pid:mpid msock) then begin
fail "the migration daemon %s (%S)" !listen_why
(In_channel.with_open_bin mout In_channel.input_all);
(try Unix.kill mpid Sys.sigkill with Unix.Unix_error _ -> ())
end
else begin
let c = connect msock in
let said r = Option.value ~default:"" (Wire.string_field r "message") in
let value r = Option.value ~default:"" (Wire.string_field r "value") in
let ask code =
request c
(Printf.sprintf
"(:op \"eval-expr\" :code %S :file \"programs/dev-classes.flan\")"
code)
in
let redefine code =
request c
(Printf.sprintf
"(:op \"eval\" :code %S :file \"programs/dev-classes.flan\")"
code)
in
(* [what] is the claim, [code] an expression that answers 1 when it
holds. A miss reports the code as well as the answer, because at
this density the line number is not enough to say which step. *)
let holds what code =
let r = ask code in
if status r <> "ok" then fail "%s: %s" what (said r)
else if value r <> "1" then
fail "%s answered %S (%s)" what (value r) code
in
(* Two instances, built by the constructor the program was compiled
with. The first ask is retried for the class block's reason: the
agent's thread is let go only after the socket is bound. *)
let started () = status (ask "(do (push instances (point 3 4)) 1)") = "ok" in
if not (await started) then
fail "the migration daemon never reached a frame boundary"
else begin
holds "a second instance" "(do (push instances (point 5 6)) 1)";
holds "the class the program was built with"
"(if (= (area (at instances 0)) 12) 1 0)";
(* ── A slot gained ── *)
let r = redefine "(defclass point [x y z])" in
if status r <> "ok" then fail "adding a slot to a class: %s" (said r)
else begin
(* The instance is the one that was pushed before the edit — same
object, same position in the same global vec — and it now
answers the new definition. This is the whole feature in three
lines: the gained slot is nil, the kept slots kept their values,
and the count is the new one. *)
holds "a gained slot is nil on an old instance"
"(if (= (get (at instances 0) :z) nil) 1 0)";
holds "a kept slot keeps its value"
"(if (= (get (at instances 0) :x) 3) 1 0)";
holds "the migrated instance has the new slot count"
"(if (= (len (at instances 0)) 3) 1 0)";
(* Dispatch after migration. The generic reaches its method by the
instance's shape tag, and a migration rebuilds the instance's
entries — so this is the line that says the tag came through it.
[area] reads :x and :y, both of which the new definition still
has, so the answer is the one it always was. *)
holds "a generic still dispatches on a migrated instance"
"(if (= (area (at instances 0)) 12) 1 0)";
(* And an instance that has not been touched since the edit is not
special: it migrates when it is asked, not when the class
changed. *)
holds "an untouched instance migrates on its own first touch"
"(if (= (len (at instances 1)) 3) 1 0)"
end;
(* ── A slot lost, and a second generation ──
[:y] goes. Nothing calls [area] after this: its method reads :y,
which is now nil, and a generic that traps on a slot its class no
longer has is the program being wrong rather than the migration. *)
let r = redefine "(defclass point [x z])" in
if status r <> "ok" then fail "removing a slot from a class: %s" (said r)
else begin
holds "a lost slot reads as absent"
"(if (= (get (at instances 0) :y) nil) 1 0)";
holds "a lost slot is gone from the count"
"(if (= (len (at instances 0)) 2) 1 0)";
holds "the slots either side of it are untouched"
"(if (= (get (at instances 0) :x) 3) 1 0)"
end;
(* ── The third redefinition ──
Three changed definitions have now been registered, so the
generation has moved three times and the instances have followed
each move. A generation that was not bumped, or was bumped to a
value an instance already carried, would leave this one stale. *)
let r = redefine "(defclass point [x z w])" in
if status r <> "ok" then fail "a third redefinition: %s" (said r)
else begin
holds "the third definition's slot count"
"(if (= (len (at instances 1)) 3) 1 0)";
holds "the third definition's new slot is nil"
"(if (= (get (at instances 1) :w) nil) 1 0)";
holds "and the value from before the first edit is still there"
"(if (= (get (at instances 1) :x) 5) 1 0)";
(* The tag is not a slot and no migration touches it: [class-of]
answers what it always did, which is what keeps every method
ever written for this class reachable. *)
holds "the instance is still an instance of its class"
"(if (= (class-of (at instances 1)) :point) 1 0)"
end;
(* ── A definition that did not change ──
Every C-c C-k re-runs a file's class definitions, and a generation
bumped per registration rather than per *change* would migrate
every instance in the program on every save. Here that would be
visible: the value written below is put into a slot the class
declares, and a spurious migration would keep it — so the
discriminating half is the raw key on the line after, which a real
migration drops and an ignored re-registration leaves alone. *)
holds "a key written straight into an instance"
"(do (put (at instances 0) :scratch 7) 1)";
let r = redefine "(defclass point [x z w])" in
if status r <> "ok" then fail "re-evaluating an unchanged class: %s" (said r)
else
holds "an unchanged definition migrates nothing"
"(if (= (get (at instances 0) :scratch) 7) 1 0)";
(* And the same key after a definition that *did* change, which is
the advisory registry stated as a test rather than as a hope: a
class instance is an open map, [put] accepts any key, and the next
migration drops the ones the class does not declare. FIX.org says
so in as many words. *)
let r = redefine "(defclass point [x z w q])" in
if status r <> "ok" then fail "a fourth redefinition: %s" (said r)
else
holds "a migration drops a key the class never declared"
"(if (= (get (at instances 0) :scratch) nil) 1 0)"
end;
(try Unix.close c with Unix.Unix_error _ -> ());
(try Unix.kill mpid Sys.sigkill with Unix.Unix_error _ -> ());
(try ignore (Unix.waitpid [] mpid) with Unix.Unix_error _ -> ())
end;
List.iter (fun f -> try Sys.remove f with Sys_error _ -> ())
[ msock; mout ];
(* ── The same thing through the other backend ───────────────────
The block above runs on x86, because that is what [flan dev] takes
when nobody says. The registration a redefined class carries rides
the [flan_reload_call] thunk, which both backends emit and the agent
finds by [dlsym] either way — and "both backends emit it" is a
sentence [x86.ml]'s own header got wrong for long enough to be worth
not trusting a second time. So the shortest subset that would notice:
one instance, one slot added, and the three answers that say the
migration happened.
Short on purpose. What is backend-specific is the thunk reaching the
runtime at all; everything the block above pins beyond that is
flan_dyn.c's, and flan_dyn.c does not know which backend called
it. *)
let lsock2 = tmp "migrate-llvm.sock" and lout2 = tmp "migrate-llvm.out" in
(try Sys.remove lsock2 with Sys_error _ -> ());
let lfd2 = Unix.openfile lout2 [ Unix.O_WRONLY; Unix.O_CREAT; Unix.O_TRUNC ] 0o600 in
let lpid2 =
Unix.create_process flan
[| flan; "dev"; "programs/dev-classes.flan"; "-s"; lsock2; "--llvm" |]
Unix.stdin lfd2 Unix.stderr
in
Unix.close lfd2;
if not (listening ~pid:lpid2 lsock2) then begin
fail "the LLVM migration daemon %s (%S)" !listen_why
(In_channel.with_open_bin lout2 In_channel.input_all);
(try Unix.kill lpid2 Sys.sigkill with Unix.Unix_error _ -> ())
end
else begin
let c = connect lsock2 in
let said r = Option.value ~default:"" (Wire.string_field r "message") in
let value r = Option.value ~default:"" (Wire.string_field r "value") in
let ask code =
request c
(Printf.sprintf
"(:op \"eval-expr\" :code %S :file \"programs/dev-classes.flan\")"
code)
in
let holds what code =
let r = ask code in
if status r <> "ok" then fail "llvm: %s: %s" what (said r)
else if value r <> "1" then
fail "llvm: %s answered %S (%s)" what (value r) code
in
let started () = status (ask "(do (push instances (point 3 4)) 1)") = "ok" in
if not (await started) then
fail "the LLVM migration daemon never reached a frame boundary"
else begin
let r =
request c
"(:op \"eval\" :code \"(defclass point [x y z])\" \
:file \"programs/dev-classes.flan\")"
in
if status r <> "ok" then
fail "llvm: adding a slot to a class: %s" (said r)
else begin
holds "a gained slot is nil through the LLVM backend"
"(if (= (get (at instances 0) :z) nil) 1 0)";
holds "a kept slot keeps its value through the LLVM backend"
"(if (= (get (at instances 0) :x) 3) 1 0)";
holds "the slot count through the LLVM backend"
"(if (= (len (at instances 0)) 3) 1 0)";
holds "a generic still dispatches through the LLVM backend"
"(if (= (area (at instances 0)) 12) 1 0)"
end
end;
(try Unix.close c with Unix.Unix_error _ -> ());
(try Unix.kill lpid2 Sys.sigkill with Unix.Unix_error _ -> ());
(try ignore (Unix.waitpid [] lpid2) with Unix.Unix_error _ -> ())
end;
List.iter (fun f -> try Sys.remove f with Sys_error _ -> ())
[ lsock2; lout2 ];
List.iter (fun f -> try Sys.remove f with Sys_error _ -> ()) List.iter (fun f -> try Sys.remove f with Sys_error _ -> ())
[ sock; out; bsock; bout ]; [ sock; out; bsock; bout ];
Test_support.report ~label:"dev" () Test_support.report ~label:"dev" ()

21
test/test_dyn.ml
View File

@ -20,6 +20,12 @@
desc an aggregate root: a struct whose dyn fields are named by a desc an aggregate root: a struct whose dyn fields are named by a
descriptor rather than pushed one at a time. descriptor rather than pushed one at a time.
Without it a collector that never freed would pass everything Without it a collector that never freed would pass everything
classes a (defclass ...) redefined: the registry, the generation an
instance carries, and the lazy migration at its next touch —
a slot gained, a slot lost, both at once, three definitions
an instance never woke up for, two generations compared,
a plain map left alone, and two thousand instances migrated
while the collector runs
nested a chain of vecs sixty-four deep, traced through one root nested a chain of vecs sixty-four deep, traced through one root
sharing one object held three times — written through one path and read sharing one object held three times — written through one path and read
through another, and swept once when the last goes through another, and swept once when the last goes
@ -41,7 +47,7 @@
mismatched write on each of the three element kinds, and a mismatched write on each of the three element kinds, and a
push against a flat (slice or array) view push against a flat (slice or array) view
One binary, built once, run thirty-eight times. The build is the expensive One binary, built once, run thirty-nine times. The build is the expensive
part and the runs are milliseconds, which is what keeps this inside part and the runs are milliseconds, which is what keeps this inside
`dune test` rather than behind an alias. *) `dune test` rather than behind an alias. *)
@ -137,6 +143,17 @@ let () =
fail "an aggregate root\n got: %S (exit %d)\n wanted: %S" fail "an aggregate root\n got: %S (exit %d)\n wanted: %S"
out code want_desc; out code want_desc;
(* Redefining a class, which is CLHS 4.3.6's lazy update protocol as
flan_dyn.c implements it: a registry of slot lists per class name, a
generation on the instance, and a migration at the first access after
the definition moved. Driven from C because the event has no Flan
spelling — a class definition changes between two *modules*, so no
single program can see one change. test_dev.ml's daemon case is the
same protocol with a real editor at one end. *)
let code, out, err = run "classes" in
if code <> 0 || out <> "classes ok\n" then
fail "redefining a class\n got: %S (exit %d, err %S)" out code err;
let code, out, _ = run "nested" in let code, out, _ = run "nested" in
if code <> 0 || out <> "chain of 64 intact: yes\n" then if code <> 0 || out <> "chain of 64 intact: yes\n" then
fail "a chain of nested vecs\n got: %S (exit %d)" out code; fail "a chain of nested vecs\n got: %S (exit %d)" out code;
@ -247,7 +264,7 @@ let () =
(* A line on the way out, because a test that says nothing when it passes (* A line on the way out, because a test that says nothing when it passes
is a test nobody can tell from a test that did not run. *) is a test nobody can tell from a test that did not run. *)
if !failures = 0 then if !failures = 0 then
Printf.printf " ok the dyn runtime: %d refusals and nine runs\n" Printf.printf " ok the dyn runtime: %d refusals and ten runs\n"
(List.length refusals + List.length view_refusals) (List.length refusals + List.length view_refusals)
else exit 1 else exit 1
| _ -> print_endline "SKIP test_dyn: no clang" | _ -> print_endline "SKIP test_dyn: no clang"

108
test/test_flan.ml
View File

@ -4452,25 +4452,78 @@ let () =
"(defn f [] i32 (let [x 1] (.r x)))" "(defn f [] i32 (let [x 1] (.r x)))"
~needle:"i32 is not a struct, so it has no fields"; ~needle:"i32 is not a struct, so it has no fields";
(* A defn whose name is a builtin's is silently unreachable — the dispatch (* "Allow shadowing but warn": a defn whose name is a builtin's is legal,
reaches every builtin arm before it looks in the function table — and the it wins at the call sites of the file that wrote it, and the compiler
arity refusal was measured against the builtin while pointing at a call says so once at the definition.
the reader had written for their own. *)
(match diag_of "(defstruct P [x i32])\n(defn get [p P] i32 (.x p))\n (defn f [] i32 (let [p (P {.x 1})] (get p)))" with This used to be the other way round — the dispatch reached every builtin
| Some d -> arm before it looked in the function table, so the defn was silently
check "a shadowed builtin's arity has a kind" unreachable and the arity refusal carried a note saying so. That note
(d.Loc.kind = "check/builtin-arity"); described a resolution order this compiler no longer has, and the source
check "and says whose count it is" below, which used to be refused, is the one that proves it: (get p) is
(contains d.Loc.dmsg one argument, and the builtin get takes two. *)
"this is the builtin get, which a defn of the same name does not \ let shadow_src =
replace"); "(defstruct P [x i32])\n(defn get [p P] i32 (.x p))\n\
(match d.Loc.notes with (defn f [] i32 (let [p (P {.x 1})] (get p)))"
| [ n ] -> in
check "and notes the definition that is not being reached" (match Check.shadowed_builtins (program shadow_src) with
(n.Loc.nloc.Loc.line = 2 | [ d ] ->
&& contains n.Loc.nmsg "this call is not reaching it") check "a defn named after a builtin is warned about, at the definition"
| _ -> check "a shadowed builtin has one note" false) (d.Loc.kind = "check/shadows-builtin"
| None -> check "a shadowed builtin's call is refused" false); && d.Loc.dloc.Loc.line = 2 && d.Loc.dloc.Loc.col = 7);
check "and the warning says what the name now means"
(d.Loc.dmsg
= "get shadows the builtin get — every call in this program now \
reaches your definition");
check "and it carries no notes, being one sentence about one decision"
(d.Loc.notes = [])
| _ -> check "a shadowing defn is warned about exactly once" false);
check "and the call reaches the defn, at the defn's arity"
(match checked shadow_src with
| _ -> true
| exception Loc.Error _ -> false);
check "a program that shadows nothing is warned at not at all"
(Check.shadowed_builtins (program "(defn f [] i32 1)") = []);
(* An operator is a builtin like any other and shadows like any other.
Pinned in both halves because it is the case most likely to be thought
of as special and quietly excepted later: the warning is the same
sentence, and the call is a [Call] to the definition rather than the
[Add] prim it would otherwise have lowered to. *)
let plus_src = "(defn + [a i32 b i32] i32 99)\n(defn f [] i32 (+ 1 2))" in
(match Check.shadowed_builtins (program plus_src) with
| [ d ] ->
check "an operator shadowed by a defn warns like any other builtin"
(d.Loc.kind = "check/shadows-builtin"
&& d.Loc.dmsg
= "+ shadows the builtin + — every call in this program now \
reaches your definition")
| _ -> check "a shadowed operator warns exactly once" false);
(match checked plus_src with
| p ->
(match List.find_opt (fun (f : Tast.fn) -> f.Tast.name = "f") p.Tast.fns with
| Some { Tast.body = [ { Tast.e = Tast.Call ("+", _); _ } ]; _ } -> ()
| _ -> check "a shadowed operator's call reaches the defn" false)
| exception _ -> check "a shadowed operator's call reaches the defn" false);
(* And the file the definition was written in is what the shadow follows.
Same declaration list, a call whose location is another file: the
builtin, whose arity this call does not satisfy. This is the package
global-initialiser case at its smallest — an initialiser is checked with
no enclosing function, so the enclosing name cannot be what decides.
The shadowing defn takes *two* parameters and the builtin takes one, so
the two readings cannot produce the same sentence: reaching the builtin
is a refusal measured at one, and reaching the defn is no refusal at
all. With both at one argument this check passed under either
resolution, which is a check that cannot fail — found in review. *)
(match
Check.program
(program "(defn len [a string b string] i32 999)"
@ Parse.program (read ~file:"<elsewhere>" "(defn g [] i32 (len \"a\" \"b\"))"))
with
| _ -> check "a call in another file does not reach the shadow" false
| exception Loc.Error d ->
check "a call in another file reaches the builtin, at the builtin's arity"
(contains d.Loc.dmsg "len takes 1 argument, given 2"));
(* and's last operand is the then arm and the sentinel carrying the previous (* and's last operand is the then arm and the sentinel carrying the previous
operand's location is the else arm, so with no expectation in hand the operand's location is the else arm, so with no expectation in hand the
@ -4700,10 +4753,17 @@ let () =
There is no way to reflect over an OCaml match, so this reads the source There is no way to reflect over an OCaml match, so this reads the source
instead. The two regions are [named_call]'s arms and [var]'s, each from instead. The two regions are [named_call]'s arms and [var]'s, each from
its own [and] down to the first catch-all at the same indentation, and its own [and] down to the catch-all at the same indentation, and the
the names are the string literals in the arm heads. It is a regex over names are the string literals in the arm heads. It is a regex over one
one file and costs nothing, which is why it is in the default run rather file and costs nothing, which is why it is in the default run rather
than behind an alias. *) than behind an alias.
The catch-all is [ | _ ->] and not [ | _], because a guarded arm is
not one: [named_call] opens with [| _ when shadows_builtin ...], which
is a name the program defined taking its own call over, and stopping
there would read the region as empty and report every builtin as
undescribed. Guarded arms in between are skipped by the same rule that
skips a comment — they carry no string literal in the head. *)
let arm_names () = let arm_names () =
let src = let src =
In_channel.with_open_bin "../lib/check.ml" In_channel.input_all In_channel.with_open_bin "../lib/check.ml" In_channel.input_all
@ -4732,7 +4792,7 @@ let () =
let rec take = function let rec take = function
| [] -> [] | [] -> []
| l :: rest -> | l :: rest ->
if starts_with " | _" l then [] if starts_with " | _ ->" l then []
else if starts_with " | \"" l then quoted l @ take rest else if starts_with " | \"" l then quoted l @ take rest
else take rest else take rest
in in

10
test/test_sanitize.ml
View File

@ -306,7 +306,15 @@ let dyn_sweep () =
if reported text then fail "dyn %s: sanitizer report\n%s" mode text if reported text then fail "dyn %s: sanitizer report\n%s" mode text
else if code <> 0 then else if code <> 0 then
fail "dyn %s: exit %d under the sanitizers\n%s" mode code text) fail "dyn %s: exit %d under the sanitizers\n%s" mode code text)
[ "ops"; "gc"; "unrooted"; "desc"; "nested"; "sharing"; "park" ]; (* [classes] is in here for the reason this whole function is: it is
the one mode that *frees* an object's entry block while the object
stays live and reachable. A migration swaps an instance's storage
for a differently-sized one, and its last two thousand instances do
it with the collector running around them, so a marker that read
the old block, or a [len] that outlived the block it described,
is a use-after-free here and nothing anywhere else. *)
[ "ops"; "gc"; "unrooted"; "desc"; "nested"; "sharing"; "park";
"classes" ];
(try Sys.remove exe with Sys_error _ -> ()) (try Sys.remove exe with Sys_error _ -> ())
(* The positive controls, which are the only evidence that a clean sweep means (* The positive controls, which are the only evidence that a clean sweep means

113
test/test_session.ml
View File

@ -1061,4 +1061,117 @@ let () =
| exception Loc.Error { Loc.dmsg = m; _ } -> | exception Loc.Error { Loc.dmsg = m; _ } ->
fail "an expression that instantiates a generic: %s" m); fail "an expression that instantiates a generic: %s" m);
(* ── A class whose slots changed ────────────────────────────────
The dev loop's half of CLHS 4.3.6. Three things have to be true of the
session for the runtime's migration to ever be reached: a changed slot
list has to be *accepted*, the module has to carry the registration that
tells the runtime about it, and the case where accepting it would be
unsound has to stay refused. test_dev.ml runs the protocol against a
real program; these are the decisions taken before any of it is built.
Accepted first. A slot added is a constructor taking one more argument,
which is the signature change [compatible] refuses by default — and
rightly, since a call site compiled to pass two dyn words into a
three-parameter body leaves the third holding a register. Nothing in
dev-class.flan calls [point], so there is no such call site and the
refusal has nothing to protect. *)
(let t, _ = Session.create ~file:"programs/dev-class.flan" () in
match Session.eval t "(defclass point [x y z])" with
| c ->
if not (List.mem "point" c.Session.fns) then
fail "adding a slot to a class installed %s"
(String.concat " " c.Session.fns);
(* The registration, and the thunk that runs it. Without the first the
runtime never hears that the class changed; without the second the
module defines a function nothing calls.
[call void @] and not the bare symbol, which is the difference
between a pin and a decoration: [emit.ml]'s declare block names
every runtime entry point in every module it writes, so
"flan_dyn_class_def" on its own is in the text of a module that
registers nothing. Checked by mutation — the bare needle passes with
the thunk deleted. *)
if not (has c.Session.ir "call void @flan_dyn_class_def") then
fail "a redefined class did not register its slots";
if not (has c.Session.ir "define void @flan_reload_call") then
fail "the class registration had nothing to run it";
(* And the slot names, in the packed form the runtime splits — which is
what says the call carries *this* class's new list and not some
other module's leftovers. *)
if not (has c.Session.ir "c\"x\\0Ay\\0Az\"") then
fail "the registration did not carry the new slot list"
| exception Loc.Error { Loc.dmsg = m; _ } ->
fail "adding a slot to a class was refused: %s" m);
(* A slot removed is the same decision in the other direction, and it is
worth its own case: the refusal compares signatures and does not care
which way the arity moved. *)
(let t, _ = Session.create ~file:"programs/dev-class.flan" () in
match Session.eval t "(defclass point [y])" with
| c ->
if not (List.mem "point" c.Session.fns) then
fail "removing a slot from a class installed %s"
(String.concat " " c.Session.fns)
| exception Loc.Error { Loc.dmsg = m; _ } ->
fail "removing a slot from a class was refused: %s" m);
(* A class that did not change registers anyway — the runtime ignores a
re-registration of the same list, and something has to tell it the list
in the first place. This is the C-c C-k shape: every class in the file
arrives, whether or not any of them moved.
The needles are the same two discriminating ones, and the slot list is
the *old* one, which is what says the registration is of this class as
it currently stands rather than a leftover from the case above. That
the runtime then declines to bump the generation is flan_dyn.c's half
and is pinned where it happens: dyn_ops.c's [classes] mode writes a
value and re-registers the same list under it, and test_dev.ml does the
same against a live program. Neither is visible in IR text, which is
why neither is asserted here. *)
(let t, _ = Session.create ~file:"programs/dev-class.flan" () in
match Session.eval t "(defclass point [x y])" with
| c ->
if not (has c.Session.ir "call void @flan_dyn_class_def") then
fail "an unchanged class definition registered nothing";
if not (has c.Session.ir "c\"x\\0Ay\"") then
fail "an unchanged class registered some other slot list"
| exception Loc.Error { Loc.dmsg = m; _ } ->
fail "re-evaluating an unchanged class was refused: %s" m);
(* Now the refusal that stands, which is the whole reason the relaxation
above is safe. A function of the running program calls the constructor
and this evaluation is not recompiling it, so accepting the edit would
leave a call site passing two dyn words into a three-parameter body —
and the third would hold whatever was in the register, which is a wild
pointer rather than a wrong answer.
The sentence a reader gets is the *checker's*: the whole declaration
list is re-checked against the new constructor before the session's
compatibility rules are consulted at all, so the complaint lands at the
call site with a line number rather than at the class. [session.ml]'s
own walk over the callers is the backstop behind it and is not what
fires here; its comment says so. *)
(let t, _ = Session.create ~file:"programs/dev-class.flan" () in
ignore (Session.eval t "(defn origin [] dyn (point 0 0))");
match Session.eval t "(defclass point [x y z])" with
| _ -> fail "a class with a compiled caller changed its slots anyway"
| exception Loc.Error { Loc.dmsg = m; _ } ->
if not (has m "point takes 3 arguments") then
fail "the refusal was not about the call that would be left behind: %s"
m);
(* And the same edit accepted when the caller comes with it, which is what
C-c C-k sends: the class and everything that constructs one are
recompiled in the same module, so no call site is left passing the old
arguments. *)
(let t, _ = Session.create ~file:"programs/dev-class.flan" () in
ignore (Session.eval t "(defn origin [] dyn (point 0 0))");
match
Session.eval t
"(do (defclass point [x y z]) (defn origin [] dyn (point 0 0 0)))"
with
| c ->
if not (List.mem "point" c.Session.fns && List.mem "origin" c.Session.fns)
then
fail "a class and its caller together installed %s"
(String.concat " " c.Session.fns)
| exception Loc.Error { Loc.dmsg = m; _ } ->
fail "a class and its caller evaluated together: %s" m);
Test_support.report ~label:"session" () Test_support.report ~label:"session" ()