From 0c03550999014d408adae6953a1b727b6a37169e Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Mon, 21 Sep 2026 10:02:54 +0700 Subject: [PATCH] Evaluating a def assigns, because that is what defparameter means MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The author edited (def colors [4 u32] [...]) in his running game, pressed C-c C-c, and the colours did not change — the same complaint def was built to answer, one step further in. The reading behind it was that a re-evaluated def is a promise about the next re-run, so the session republished the lifted global/ and stopped; nothing called it. That is wrong for the reason the form is named after: def is Common Lisp's defparameter, and evaluating a defparameter assigns. The difference from defvar is not "one takes effect at restart", it is "one takes effect, the other does not touch the value at all". So a def now does both. The storage takes the new value at the next frame boundary, carried by the thunk a redefinition module already has — one Set per re-evaluated def, in the same flan_reload_call the class registrations use, run after the bodies are published and on the game thread. And the lifted initialiser is still republished, so the next re-run runs the edited one; dev-rerun.flan pins that half unchanged. A brand-new def gets its initialiser run too, which needed one thing from each backend: a lifted global/ asked for by name is neither a sibling nor one of the target's own lifted clauses, so it had no cell, and a dev call goes through a cell. Both now give an unknown one a slot of the module's own, filled from the registry by the installer. An initialiser that signals leaves the old value alone — the value is computed whole before it is stored — and offers abandon-evaluation like any other thunk. A retype is refused first, by the pass that names both types. defonce is untouched, which is its whole contract; defconst was already the immediate one, through consts. --- FIX.org | 144 +++++++++++++++++++++ docs/BUILT.md | 86 ++++++++++++- lib/check.ml | 7 +- lib/emit.ml | 35 +++-- lib/session.ml | 160 +++++++++++++++-------- lib/tast.ml | 7 +- lib/x86.ml | 11 +- test/programs/dev-defstore.flan | 85 ++++++++++++ test/test_dev.ml | 221 +++++++++++++++++++++++++++++++- 9 files changed, 689 insertions(+), 67 deletions(-) create mode 100644 test/programs/dev-defstore.flan diff --git a/FIX.org b/FIX.org index 9ea1c67..5373eb7 100644 --- a/FIX.org +++ b/FIX.org @@ -5049,3 +5049,147 @@ stops being one predicate per target and becomes a disjunction, ~numeric?~ or the reader meant. A cast *to* a variable bounded ~enum?~ is a second question with its own answer. Each of those is a decision, not a fill-in, and the author has not been asked. + +* Evaluating a def assigns, 2026-09-21 + +** The gap, hit dogfooding again +The author edited + +: (def colors [4 u32] [0xE6B800FF 0xFF0000FF 0xA83232FF 0xCC6B1FFF]) + +in his running game, pressed C-c C-c, and the colours did not change. That is +the same complaint the entry "def, and defvar renamed to defonce" above was +written to answer, one step further in: the form was right this time, and the +delivery stopped short. + +** Why it stopped short: the reading was wrong +[Session]'s [def_inits] said it outright — "a re-evaluated def is a promise +about the *next re-run*" — and did exactly that: republish the lifted +[global/] through its cell, and stop. Nothing called it. + +That reading is wrong, and it is wrong for the reason the form is named +after. ~def~ is Common Lisp's ~defparameter~, and **evaluating a defparameter +assigns**. That is precisely what distinguishes it from ~defvar~ — ~defonce~ +here. The difference between the two is not "one takes effect at restart"; +it is "one takes effect, the other does not touch an existing binding at +all". A promise about the next re-run is a promise ~defparameter~ does not +make and does not need: it assigns now, and it also re-initialises on the +next load. Both, not one or the other. + +** What is built +A re-evaluated ~def~ now does both: + +- the storage takes the new value at the next frame boundary; and +- the lifted [global/] is republished, so the edited initialiser is the + one the next re-run runs. Unchanged, and not regressed — dev-rerun.flan + still pins it. + +The store rides the thunk a redefinition module already carries. A module has +one optional [flan_reload_call], run by the agent after [flan_reload_install] +has published the bodies, at a frame boundary, on the game thread — the +mechanism C-x C-e uses. [Session.eval] puts one ~(Set (Pglobal n) ginit)~ per +re-evaluated def into it, which is the same store [Emit.startup_plan] writes +for the same global, without the [.init~once.] flag a defonce's carries. The +class registrations that already used the thunk share it, and come first. + +** The questions, answered +*** When +At the next frame boundary of a running program. A *parked* program drains +its ring when that sleep ends, so the store lands at the top of its next run, +ahead of main — and the run's own startup then computes the initialiser +again. Two runs of the initialiser, and that is right rather than a wart: +they are the two events ~defparameter~ has, an evaluation that assigns and a +load that re-initialises. A program that has not called ~(agent/start ...)~ +yet installs at its next ~(agent/poll)~ and never if it has none, which the +reply already says. Stopped at a break, it runs in the break loop like any +other evaluation. + +*** An initialiser that signals +The condition goes unhandled, the program stops, and the boundary restart the +agent establishes around every thunk is offered as ~abandon-evaluation~; +taking it unwinds past the store and the program carries on. The session is +not wedged — the next evaluation of the same name stores like any other. The +editor learns of it as a stop: the reply to the evaluation went out at +"queued", which is how every other condition in a running program arrives. + +The old value survives for a *scalar* on both backends, and for an aggregate +on LLVM only. A scalar comes back in a register and is stored after the +transfer guard, so the signal jumps past the store; LLVM does the same for an +aggregate, calling into a temporary and storing once. The x86 backend has no +[Set] arm of its own — it lowers a value straight into its place, and a call +returning an aggregate is handed the destination as its sret pointer — so + +: (def colors [4 u32] [9 9 (wreck) 9]) + +writes into the live global element by element: [colors] reads 9 in the first +element with the tail still old at the break, and all four elements zero after +the abandon. Measured on both backends, 2026-09-21. + +Pre-existing, and not about ~def~: a bare C-x C-e of ~(set colors (wreck))~ +does the same on x86, and the restart's own note already says "anything the +expression changed before it stopped is still changed". What is new is that +every def edit now travels that path, so it is written down rather than +promised away. Routing an x86 aggregate ~(Set (Pglobal …) (Call …))~ through a +temporary — and chasing why the abandon leaves zeroes rather than the partial +write — is a lane of its own and is not this one. + +*** A retype +[Session.compatible] fires before anything is built, and its sentence already +reads for this path: "speed changes type, from i64 to string; the running +program already laid that storage out. Restart to change it." + +*** A def the process has never seen +Gets its initialiser run too, where before it got [Emit.initial_image]'s +answer for a literal and calloc's zeroes for a computed one. This needed one +thing in each backend: a lifted [global/] asked for by name is neither a +sibling nor one of [lifted] — its [fparent] is the global, not a function in +[fns] — so it got no cell at all, and a call in a dev module goes through a +cell. Both backends now give an unknown one a slot of the module's own, +filled from the registry by the installer, exactly as a defn the host lacks +already was. + +*** defonce +Unchanged, and pinned: [kind = Once] never reaches the store list, so +re-evaluating one whose name the program already has builds no module and +answers "nothing to install". + +*** defconst +Unchanged, and it was already the immediate one — an unfolded constant is +republished by value at the frame boundary through [Session]'s [consts], the +same store by a shorter road since there is no initialiser to run. A folded +one is refused. Which means ~def~ had been the *worse* of the two on +immediacy, which is not a defensible place for the form that exists to follow +the source. + +** Tests +programs/dev-defstore.flan and its block in test_dev.ml: a native array (the +author's case, to the letter), a scalar, a struct, a dyn, a computed +initialiser, a brand-new name with a call and one with a literal, a defonce +control, an uninit def that stores nothing, a defconst re-evaluated live, a +defclass and a def constructing it sent as *one form* — which is what pins +the registrations-before-stores order inside the thunk — a retype refusal, +and an initialiser that signals, abandoned, with the same name edited again +afterwards. All read back out of a *running* program with no re-run anywhere. +Both backends: the default x86 and --llvm. + +Two things the fixture spells the way it does for a reason. The defconst is +an f64: the checker folds *integer* constants on the way in, and a folded one +is refused by name rather than published, so an i64 there would have pinned +the refusal instead of the store. And the class the def constructs is paired +with a *new* def rather than one the fixture declares — see below. + +** Left alone +- [Session.eval] takes no liveness argument, so it cannot skip the store for a + parked program. Deliberate, per the two-events reading above. +- A full-file C-c C-k now re-runs every def initialiser in the file. That is + what loading a file of defparameters means, and the state "edit the code, + keep the sand" is about lives in defonces. +- A class that gains or loses slots is checked against every caller of its + constructor in the running program, and the lifted [global/] of a def + whose initialiser constructs one *is* such a caller — so redefining a class + and re-evaluating an existing def that constructs it, in one form, is + refused naming ~global/origin~, a name the source never writes. Pre-existing + (the lifted function has been there since def landed) and against the house + rule for diagnostics, but it is [session.ml]'s stale-caller tripwire, which + is documented as unreachable and is not this lane's to loosen. A brand-new + def is unaffected, which is what the test uses. diff --git a/docs/BUILT.md b/docs/BUILT.md index 3c7a469..f794340 100644 --- a/docs/BUILT.md +++ b/docs/BUILT.md @@ -6238,10 +6238,13 @@ Three forms define a global, and the difference between them is entirely what a Lisp's `defvar` under Clojure's name, renamed because the old name said nothing — initialises once and keeps its value: the value the *program* built is the one that survives, which is "edit the code, keep the sand". `def` is CL's `defparameter`: its initialiser runs on every re-run, so the value the *source* spells is the one that wins — edit a -`(def colors [4 u32] [...])`, `C-c C-c`, `M-x flan-rerun`, and the colours change, which is the gap the author hit +`(def colors [4 u32] [...])`, `C-c C-c`, and the colours change on the next frame, which is the gap the author hit dogfooding and the reason the form exists. `defconst` is the image and no run reaches it. The old `defvar` spelling is refused with the rename and both new spellings, each of which compiles as written. +Evaluating a `def` also *assigns*, at once — that is the other half of what `defparameter` means, and it landed after +this section was first written. See "Evaluating a `def` stores the value" at the end of this file. + Both forms take the same spellings — `(def x Type)` zeroed, `(def x Type v)`, `(def x v)` a dyn global — through the same parse arm and the same third-element dispatch, and collide with a `defn` under the same Lisp-1 rule. The difference is one field, `Ast.reinit`, and two consequences of it. In `Emit.startup_plan` a `defonce`'s computed @@ -6251,6 +6254,7 @@ place — same storage, no reallocation, every reference sees the new bytes. And startup calls the initialiser through its function cell, so a re-evaluated `def` swaps the cell (`Session`'s `def_inits`, plus `Emit.redefinition` declaring the cell for a target whose `fparent` is a global) and the next re-run stores the edited value. A constant left inline would have baked the stale number into the host's startup for ever. +The same lifted function is what the immediate store calls; see the last section of this file. Two consequences of the always-lift that are worth stating, because both were bugs first. A global the *process was never built with* — a `def` typed fresh into a live session — gets its initial value from the image @@ -6334,3 +6338,83 @@ Arithmetic wraps in Flan (`emit.ml`, no `nsw`/`nuw`), and the counter is arithme carries `i` past `i32`'s range therefore wraps to the far end instead of trapping — defined, but the loop then runs far longer than it was meant to, and can fail to reach `stop` at all. Keep `stop` within one `step` of the width's limit; nothing checks it for you. +## Evaluating a `def` stores the value + +Editing `(def colors [4 u32] [...])` in a running game and pressing `C-c C-c` now changes the colours on the next +frame. It used to change them on the next *re-run*, which is the same complaint the form was built to answer, one step +further in. + +The reading that produced it was that a re-evaluated `def` is a promise about the next re-run. That is wrong, and it is +wrong for the reason the form is named after: `def` is Common Lisp's `defparameter`, and **evaluating a +`defparameter` assigns**. That is the whole difference from `defvar` — `defonce` here — which leaves an existing +binding alone. The difference was never "one takes effect at restart"; it is "one takes effect, the other does not +touch the value at all." + +So a `def` sent from the editor does both things it always meant: + +- the storage takes the new value **now**, so code already compiled against the global reads it at the next frame; and +- the lifted `global/` is republished through its cell, so the edited initialiser is the one the **next re-run** + runs, and every re-run after it. + +**The store is a thunk, and not a second delivery path.** A redefinition module already carries one optional +`flan_reload_call`, which the agent runs after `flan_reload_install` has published the bodies, at a frame boundary, on +the game thread — the mechanism `C-x C-e` uses. `Session.eval` now puts one `Set (Pglobal n, ginit)` per re-evaluated +`def` into that thunk, which is the same store `Emit.startup_plan` writes for the same global, without the +`.init~once.` flag a `defonce`'s carries. The class registrations that already used the thunk share it; they come +first, so a `def` whose initialiser constructs an instance of a class the same form redefined sees the slot list the +registry now holds. + +**When it happens.** At the next frame boundary of a running program. A parked program — one whose `main` has +finished — drains its ring when that sleep ends, so the store lands at the top of its next run, ahead of `main`; the +run's own startup then computes the initialiser again, which is not a wart but the two events `defparameter` has: an +evaluation assigns, and a re-run re-initialises. A program that has not called `(agent/start ...)` yet installs at its +next `(agent/poll)`, and never if it has none — the reply already says so. A program stopped at a break runs it in the +break loop, like any other evaluation. + +**A brand-new `def`** gets its initialiser run too. Its storage comes from `flan_dev_global` and nothing in the host's +`.init-globals` names it, so before this a new `(def n i64 (count-them))` came up as `calloc`'s zeroes and stayed +there; `Emit.initial_image` answers that for a literal initialiser and has nothing to say about a computed one. The +store answers both. Making it work needed one thing in each backend: a lifted `global/` asked for by name is +neither a sibling nor one of `lifted` — its `fparent` is the global, not a function in `fns` — so it got no cell, and +a call in a dev module goes through a cell. Both now give an unknown one a slot of this module's own, filled from the +registry by the installer, exactly as a `defn` the host lacks already was. + +**An initialiser that signals** stops the program rather than storing. The condition goes unhandled, the program stops +in a break loop, and the boundary restart the agent establishes around every thunk it runs is offered as +`abandon-evaluation`; taking it unwinds past the store and the program carries on. The session is not wedged — the +next evaluation of the same name stores like any other. The editor's reply to the evaluation has already gone out by +then, saying "queued", so the failure arrives as a stop, which is how every other condition in a running program +arrives. + +**What survives that is the scalar case on both backends, and the aggregate case on LLVM only.** A scalar initialiser +returns in a register and is stored after the transfer guard, so a signal jumps past the store and the global keeps +its old value; LLVM does the same for an aggregate, because it calls into a temporary and then stores once. The x86 +backend has no `Set` arm of its own — it lowers a value straight into its place (`x86.ml`'s `Set`), and a call +returning an aggregate is given the destination as its sret pointer — so `(def colors [4 u32] [9 9 (wreck) 9])` +writes into the live global element by element. At the break `colors[0]` is already `9` with the tail still old, and +after `abandon-evaluation` all four elements are zero. + +This is **pre-existing and not about `def`**: a plain `C-x C-e` of `(set colors (wreck))` does the same on x86, and +the runtime already says so in the restart's own note — "anything the expression changed before it stopped is still +changed". What is new is that every `def` edit now goes through that path, which is why it is written down here. +Routing an x86 aggregate `Set (Pglobal, Call)` through a temporary, and chasing the zeroing, is a lane of its own. + +**A `def` whose type changed** is refused before anything is stored, by `Session.compatible`, which already said the +right sentence for this path: *`speed` changes type, from `i64` to `string`; the running program already laid that +storage out. Restart to change it.* + +**`defonce` is untouched**, and that is its contract: `kind = Once` never reaches the store list, so re-evaluating one +whose name the program already has builds no module at all and answers "nothing to install". + +**`defconst` is unchanged too**, and was already the immediate one: a constant the checker never folded is republished +by value at the frame boundary through `Session`'s `consts` list — the same store by a shorter road, since there is no +initialiser to run. One the checker *did* fold is refused, because its value is in the shape of the program. + +`programs/dev-defstore.flan` is the fixture: a native array, a scalar, a struct, a dyn, a computed initialiser, a +brand-new name, a `defonce` control, an `uninit` `def` that stores nothing, a `defconst` re-evaluated live, a +`defclass` and a `def` constructing it sent as one form — which is what pins the ordering inside the thunk — and a +function that signals. All read back out of a running program with no re-run anywhere. Both backends, `flan dev`'s +default x86 and `--llvm`. `programs/dev-rerun.flan` keeps the other half. + +The `defconst` there is an `f64`, deliberately: the checker folds *integer* constants on the way in, and a folded one +is refused by name rather than published, so an `i64` would have pinned the refusal instead of the store. diff --git a/lib/check.ml b/lib/check.ml index fc60bfc..786af00 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -10326,8 +10326,11 @@ let check_global env (d : Ast.decl) : Tast.global option = its function cell, so a re-evaluated [def] swaps the cell and the next re-run stores the *edited* value. A constant left inline would be baked into the host's startup body, and every re-run would paint the - stale value back. [uninit] is the one exception on both forms: there - is nothing to run, so there is nothing to lift. *) + stale value back. The same lifted function is what [Session.eval]'s + store thunk calls to assign the new value straight away, which is the + other half of what [defparameter] means. [uninit] is the one exception + on both forms: there is nothing to run, so there is nothing to + lift. *) let lift_always = (match kind with Ast.Once -> false | Ast.Every -> true) in let ginit = match init with diff --git a/lib/emit.ml b/lib/emit.ml index a0cf554..95421a7 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -4508,11 +4508,21 @@ let redefinition ?(checks = true) ?(dev = false) ?(debug = false) an expression evaluated in a loop would exhaust them. Nothing pointing into the module is also what lets the agent unload it afterwards. *) let transient f = call = Some f in + (* A lifted initialiser asked for by name — [global/] when a [def] is + evaluated. It is not a sibling, because its [fparent] is the global it + initialises, so the cell machinery below would pass it over; and it is + not one of [lifted] either, because the thing it was lifted out of is a + global and not a function in [fns]. It still needs a cell like any other + published body: the thunk that stores the new value calls it, and a call + in a dev module goes through a cell. *) + let lifted_targets = + List.filter (fun (f : Tast.fn) -> f.Tast.fparent <> None) targets + in let new_fns = List.filter (fun (f : Tast.fn) -> (not (known f.Tast.name)) && not (transient f.Tast.name)) - siblings + (siblings @ lifted_targets) and new_globals = List.filter (fun (g : Tast.global) -> not (known g.Tast.gname)) p.Tast.globals in @@ -4554,18 +4564,23 @@ let redefinition ?(checks = true) ?(dev = false) ?(debug = false) (cellptr f.Tast.name))) siblings; (* A lifted initialiser handed in as a target — [global/] when a [def] - is re-evaluated — is not a sibling: its [fparent] is the global it - initialises, not a function in [fns]. Its cell is the host's like any - other (the host declares one per function, lifted ones included), so - the publish store below needs the declaration the sibling loop above - could not write. *) + is evaluated — is not a sibling: its [fparent] is the global it + initialises, not a function in [fns]. So the publish store below needs + the declaration the sibling loop above could not write, and it is the + same two: the host's cell for a name the host has, and a slot of this + module's own for one it does not, filled from the registry by the + installer. A brand-new [def] is the second case — nothing in the host + ever named its initialiser. *) List.iter (fun (f : Tast.fn) -> - if f.Tast.fparent <> None && known f.Tast.name - && not (transient f.Tast.name) then + if not (transient f.Tast.name) then Buffer.add_string m.out - (Printf.sprintf "%s = external global ptr\n" (cellname f.Tast.name))) - targets; + (if known f.Tast.name then + Printf.sprintf "%s = external global ptr\n" (cellname f.Tast.name) + else + Printf.sprintf "%s = internal global ptr null\n" + (cellptr f.Tast.name))) + lifted_targets; if new_fns <> [] || new_globals <> [] then Buffer.add_string m.out "\ndeclare ptr @flan_dev_cell(ptr)\n\ diff --git a/lib/session.ml b/lib/session.ml index 04e7555..8847b23 100644 --- a/lib/session.ml +++ b/lib/session.ml @@ -833,30 +833,60 @@ let eval ?(origin = "") ?pause t src : change = program.Tast.fns) names in - (* A re-evaluated [def] is a promise about the *next re-run*: its - initialiser runs every time, so the edited one has to be the one that - runs. The initialiser is the lifted [global/] — [Check.check_global] - lifts every [def] initialiser, constants included, for exactly this — - and the host's startup function calls it through its cell, so - republishing that one function is the whole delivery. Only when the - host already has the cell: a [def] the process has never seen gets its - storage from [flan_dev_global] like any new global, and there is no - startup call to swap. *) - let def_inits = - List.concat_map + (* ── What evaluating a [def] does to the value ──────────────────────── + [def] is Common Lisp's [defparameter], and evaluating a defparameter + assigns. That is the whole difference from [defvar] — [defonce] here — + which leaves an existing binding alone. So a [def] sent from the editor + has two effects and needs both: + + - the storage takes the new value *now*, so code already compiled against + the global reads it at the next frame; and + - the initialiser is the one that runs at the next re-run, and at every + one after it. + + The second is the lifted [global/] — [Check.check_global] lifts every + [def] initialiser, constants included, for exactly this — published + through its cell, which the host's startup function calls. Republishing + it is the whole of the re-run half. + + The first is the store below: the same [Set] the startup function makes, + run once, at a frame boundary, by the thunk this module carries. It is + not a second delivery path — it is the one [C-x C-e] already uses, which + is what makes "at a frame boundary, on the game thread" true of it. + + A [def] the process has never seen is in here too. Its storage comes + from [flan_dev_global] like any new global's and no startup call names + it, so without the store a brand-new [(def n i64 (count-them))] would + come up as calloc's zeroes and stay there for the life of the process. + [Emit.initial_image] answers that for a constant initialiser and cannot + for a computed one; the store answers both. + + [defonce] is not in here, and that is its contract: a value the program + already holds is not touched. A [defconst] is not either — a constant + the checker never consumed is republished by value, in [consts] below, + which is the same frame-boundary store by a shorter road. + + The filter is the lifted function's existence rather than the shape of + the initialiser: [uninit] is the one [def] with nothing to run, and it + is the one with nothing lifted. *) + let def_globals = + List.filter_map (fun (d : Ast.decl) -> match d.Ast.d with - | Ast.Defvar (n, _, _, Ast.Every) -> - let g = "global/" ^ n in - if known t g - && List.exists - (fun (f : Tast.fn) -> String.equal f.Tast.name g) - program.Tast.fns - then [ g ] - else [] - | _ -> []) + | Ast.Defvar (n, _, _, Ast.Every) + when List.exists + (fun (f : Tast.fn) -> + String.equal f.Tast.name ("global/" ^ n)) + program.Tast.fns -> + List.find_opt + (fun (g : Tast.global) -> String.equal g.Tast.gname n) + program.Tast.globals + | _ -> None) incoming in + let def_inits = + List.map (fun (g : Tast.global) -> "global/" ^ g.Tast.gname) def_globals + in let from_generics = List.concat_map (fun n -> @@ -891,7 +921,13 @@ let eval ?(origin = "") ?pause t src : change = program.Tast.globals) names in - (* ── Telling the runtime what the classes now are ───────────────────── + (* ── What the program has to run, and not merely load ───────────────── + Two things now, and one thunk for both, because a module carries one + [flan_reload_call] and the agent runs it once — after the bodies are + published, at a frame boundary, on the game thread. + + The first is the class registrations. + A (defclass ...) is compile-time sugar for a constructor [defn], so nothing about it reaches the running process except a function body — which is why redefining one used to be silent, and why the instances @@ -909,39 +945,63 @@ let eval ?(origin = "") ?pause t src : change = registry ignores a re-registration of the same list, so a C-c C-k costs a comparison per class and migrates nothing; and a class whose definition the registry has never seen has to arrive somehow. *) - let class_thunk = - if incoming_classes = [] then None - else begin + let class_body = + let str s : Tast.expr = { Tast.e = Tast.Str s; ty = Types.String; loc } in + List.map + (fun (n, slots) : Tast.expr -> + let kw : Tast.expr = + { Tast.e = Tast.Prim (Tast.Rt "flan_dyn_kw", [ str n ]); + ty = Types.Dyn; loc } + in + (* The slot names in one string, newline between: the runtime + splits them. A dyn vector would have been the obvious shape + and is the wrong one — it is a collector object, so the + registry would hold something the marker has to reach, where + a packed string reaches interned keywords that are immortal + already. *) + { Tast.e = + Tast.Prim (Tast.Rt "flan_dyn_class_def", + [ kw; str (String.concat "\n" slots) ]); + ty = Types.Unit; loc }) + incoming_classes + in + (* And the second: the [def] stores. One [Set] per re-evaluated [def], the + same one [Emit.startup_plan] writes for the same global and without the + guard flag a [defonce]'s carries. See the note at [def_globals] for why + this happens at all. + + What an initialiser that transfers out leaves behind is the backend's + answer and not this line's, and the two do not agree: a scalar comes + back in a register and is stored after the transfer guard on both, and + an aggregate is a temporary and one store on LLVM but is written into + the global in place on x86, which has no [Set] arm of its own. So an + x86 aggregate initialiser that signals half-way is a global half + written — the same thing [(set g (wreck))] has always done there, and + recorded in FIX.org rather than promised away here. + + After the registrations, not before: a [def] whose initialiser + constructs an instance of a class the same form redefined has to see the + slot list the registry now holds. *) + let store_body = + List.map + (fun (g : Tast.global) : Tast.expr -> + { Tast.e = Tast.Set (Tast.Pglobal g.Tast.gname, g.Tast.ginit); + ty = Types.Unit; loc = g.Tast.ginit.Tast.loc }) + def_globals + in + let run_thunk = + match class_body @ store_body with + | [] -> None + | body -> t.thunks <- t.thunks + 1; - let tname = Printf.sprintf "classdef/%d" t.thunks in - let str s : Tast.expr = { Tast.e = Tast.Str s; ty = Types.String; loc } in - let body = - List.map - (fun (n, slots) : Tast.expr -> - let kw : Tast.expr = - { Tast.e = Tast.Prim (Tast.Rt "flan_dyn_kw", [ str n ]); - ty = Types.Dyn; loc } - in - (* The slot names in one string, newline between: the runtime - splits them. A dyn vector would have been the obvious shape - and is the wrong one — it is a collector object, so the - registry would hold something the marker has to reach, where - a packed string reaches interned keywords that are immortal - already. *) - { Tast.e = - Tast.Prim (Tast.Rt "flan_dyn_class_def", - [ kw; str (String.concat "\n" slots) ]); - ty = Types.Unit; loc }) - incoming_classes - in Some - { Tast.name = tname; params = []; ret = Types.Unit; body; + { Tast.name = Printf.sprintf "install/%d" t.thunks; + params = []; ret = Types.Unit; body; fdefers = []; fparent = None; floc = loc; slots = [||]; snames = [||] } - end in let ir = - match class_thunk with + match run_thunk with | None -> redefinition t ~consts program ~fns | Some th -> redefinition t ~consts ~call:th.Tast.name @@ -966,12 +1026,12 @@ let eval ?(origin = "") ?pause t src : change = t.decls <- decls; t.program <- program; t.env <- env; - (* [class_thunk] counts: a form that is only a (defclass ...) already + (* [run_thunk] counts: a form that is only a (defclass ...) already installs its constructor, but a module carrying nothing but the registration still has something for the program to run. *) { ir; x86 = t.x86; names; fns; installs = - fns <> [] || allocates || consts <> [] || class_thunk <> None } + fns <> [] || allocates || consts <> [] || run_thunk <> None } (* ── Evaluating an expression ──────────────────────────────────────── *) diff --git a/lib/tast.ml b/lib/tast.ml index dccb51b..c2ff3fe 100644 --- a/lib/tast.ml +++ b/lib/tast.ml @@ -322,8 +322,11 @@ type global = { gconst : bool; gfolded : bool; (* [def] rather than [defonce]: the initialiser runs on every daemon - re-run instead of once behind a flag, so an edited initialiser takes - effect on the next re-run. [Emit.startup_plan] is the consumer. *) + re-run instead of once behind a flag. [Emit.startup_plan] is the + consumer of that. [Session.eval] is the other reader, for the half of + [defparameter] that is not about re-runs at all — evaluating one + assigns, so a re-evaluated [def] stores its new value at the next frame + boundary as well. *) grerun : bool; } diff --git a/lib/x86.ml b/lib/x86.ml index 107d32c..97cb40b 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -4742,11 +4742,20 @@ let redefinition ~checks ?(dev = true) ?(known = fun _ -> true) function's slot holds its cell's address and a global's holds its storage's, so the two are the same eight bytes and differ only in what fills them. *) + (* A lifted initialiser asked for by name -- [global/] when a [def] is + evaluated. Its [fparent] is the global it initialises, so it is neither a + sibling nor one of [lifted], and the slot machinery would pass it over; + but the thunk that stores the new value calls it, and a call in a dev + module goes through a cell. A brand-new [def]'s initialiser is the case: + the host never named it, so the cell is this module's own slot. *) + let lifted_targets = + List.filter (fun (f : Tast.fn) -> f.Tast.fparent <> None) targets + in let new_fns = List.filter (fun (f : Tast.fn) -> (not (known f.Tast.name)) && not (transient f.Tast.name)) - siblings + (siblings @ lifted_targets) and new_globals = List.filter (fun (g : Tast.global) -> not (known g.Tast.gname)) p.Tast.globals diff --git a/test/programs/dev-defstore.flan b/test/programs/dev-defstore.flan new file mode 100644 index 0000000..063bf07 --- /dev/null +++ b/test/programs/dev-defstore.flan @@ -0,0 +1,85 @@ +;;;; What evaluating a [def] does to a program that is running right now. +;;;; +;;;; dev-rerun.flan is the other half of the same form: it asks what the +;;;; *next run* sees. This asks what *this* run sees, and the answer is that +;;;; a [def] is Common Lisp's [defparameter] — evaluating one assigns. The +;;;; author's report is the first global here, to the letter: a palette in a +;;;; native array, edited in a live game, with the colours expected to change +;;;; on the next frame and not on the next restart. +;;;; +;;;; One global per shape the store has to reach: a native array, a scalar, a +;;;; struct, a dyn, and one whose initialiser is a call rather than a literal. +;;;; [kept] is the control — a [defonce] whose whole contract is that +;;;; evaluating it again leaves the value alone. +;;;; +;;;; main runs for as long as any of this takes and does nothing else. Every +;;;; assertion is read back out of the program's own storage by the daemon, +;;;; because a printed line is what a run computed and the question here is +;;;; what the storage holds *now*. +(import agent "vendor:agent") + +(defstruct Tuning [gain i32 bias i32]) + +(defstruct Boom [why i32]) + +;; A call, so that [computed]'s initialiser is lifted for a reason other than +;; the form — and so a brand-new def evaluated into this program has +;; something to call that the process was built with. +(defn twice [] i64 10) + +;; What a failed initialiser is made of. The condition goes unhandled, so a +;; def whose initialiser calls this stops the program in a break loop rather +;; than storing anything. +(defn blow [] i64 + (error (Boom {.why 1})) + 0) + +(def colors [4 u32] [0xE6B800FF 0xFF0000FF 0xA83232FF 0xCC6B1FFF]) + +(def speed i64 3) + +(def knobs Tuning {.gain 1 .bias 2}) + +;; A map and not a dyn number: a number is an immediate word and would read +;; back the same whether or not the store reached the heap. +(def label dyn {:n 1}) + +(def computed i64 (twice)) + +(defonce kept i64 7) + +;; A constant nothing consumes at compile time, so it is just bytes in the +;; program's memory and a re-evaluation can publish a new value into it. It +;; was the immediate one before [def] was, which is the fact this pins. +;; +;; A float rather than an integer, and that is the whole reason it is +;; written this way: the checker folds integer constants on the way in — +;; an array length has to be a number before any type resolves — and a +;; folded one is in the shape of the running program, where no store can +;; reach it. That one is refused by name, which is a different claim. +(defconst limit f64 40.0) + +;; The one [def] spelling with nothing to run: [uninit] lifts no initialiser, +;; so re-evaluating this form stores nothing, and the byte main wrote stays. +(def scratch [4 u8] uninit) + +;; For the one case where the thunk's two halves have to be in order. A +;; module carries one [flan_reload_call], and it holds both the class +;; registrations and the def stores; a def whose initialiser constructs an +;; instance of a class the same form redefined has to see the slot list the +;; registry now holds, so the registrations go first. +(defclass point [x y]) + +(defn main [] i32 + (agent/start "/tmp/flan-dev-defstore-fallback.sock") + ;; A byte of [scratch] with a value somebody chose, so that "the uninit + ;; form stored nothing" is a claim about a number and not about whatever + ;; the bytes happened to be. + (set (at scratch 0) (u8 77)) + ;; Long enough that the whole block below runs against a program that is + ;; still running: every assertion here is about what a *running* program + ;; holds, and a fixture that parked half way through would answer the + ;; second half of them from the park instead. + (dotimes [i 120000] + (agent/wait 5)) + 0) diff --git a/test/test_dev.ml b/test/test_dev.ml index a3afad9..f39ece6 100644 --- a/test/test_dev.ml +++ b/test/test_dev.ml @@ -5892,7 +5892,15 @@ let () = cell — [Session]'s [def_inits] — and the *next re-run* runs the edited initialiser: 9 repainted, then incremented, so 10. A [defonce] beside it keeps its value through the same re-run, which - is the pair the two forms exist to be. *) + is the pair the two forms exist to be. + + This is the re-run half of [def] and not the whole of it. The + evaluation also stores the new value straight away — see the + [dev-defstore.flan] block below, which is that half on a program + that is running — and a parked program takes that store when its + ring drains, which is at the top of this very re-run. Either way + the number below is the same one: 9 stored, 9 computed again by + the run's own startup, and 10 after main increments it. *) let r = request c "(:op \"eval\" :code \"(def c 9)\" :file \"programs/dev-rerun.flan\")" @@ -5931,6 +5939,217 @@ let () = List.iter (fun f -> try Sys.remove f with Sys_error _ -> ()) [ rsock; rout ]; + (* ── Evaluating a [def] stores the value, now ────────────────────── *) + + (* The author's report, and the whole of what [def] means: it is Common + Lisp's [defparameter], so evaluating one *assigns*. Editing a palette + in a running game and pressing C-c C-c has to change the colours on + the next frame — not on the next restart, which was the behaviour and + was the same complaint the form was built to answer, one step further + in. + + Read back rather than printed, because the question is what the + storage holds now: [programs/dev-defstore.flan] prints nothing and + runs a bare wait loop, so every line below is the daemon asking the + *running* program what it has. No re-run happens anywhere in here. + + Both backends. [flan dev] takes x86 unasked, so that is the first + pass; the second names [--llvm], and the two publish a lifted + initialiser through machinery each wrote for itself. *) + List.iter + (fun (tag, extra) -> + let dsock = tmp ("defstore-" ^ tag ^ ".sock") + and dout = tmp ("defstore-" ^ tag ^ ".out") in + (try Sys.remove dsock with Sys_error _ -> ()); + let dfd = + Unix.openfile dout [ Unix.O_WRONLY; Unix.O_CREAT; Unix.O_TRUNC ] 0o600 + in + let dpid = + Unix.create_process flan + (Array.append + [| flan; "dev"; "programs/dev-defstore.flan"; "-s"; dsock |] + extra) + Unix.stdin dfd Unix.stderr + in + Unix.close dfd; + if not (listening ~pid:dpid dsock) then begin + fail "the def-store daemon (%s) %s (%S)" tag !listen_why + (In_channel.with_open_bin dout In_channel.input_all); + (try Unix.kill dpid Sys.sigkill with Unix.Unix_error _ -> ()) + end + else begin + let c = connect dsock in + let said r = + Option.value ~default:"" (Wire.string_field r "message") + in + let file = " :file \"programs/dev-defstore.flan\")" in + let value code = + Wire.string_field + (request c + ("(:op \"eval-expr\" :code " ^ Wire.quote code ^ file)) + "value" + in + (* An evaluation is answered as soon as the module is queued, so + the read that follows is a second module behind it in the same + ring — which is an order and not a race, since the poll that + runs a thunk installs whatever is queued ahead of it. Polled + anyway, so that a failure here reads as the wrong value rather + than as a timing report nobody can act on. *) + let reads what code want = + if not (await ~ms:20000 (fun () -> value code = Some want)) then + fail "%s (%s): %s reads back as %S, wanted %S" what tag code + (Option.value ~default:"" (value code)) want + in + let evals what code = + let r = request c ("(:op \"eval\" :code " ^ Wire.quote code ^ file) in + if status r <> "ok" then fail "%s (%s): %s" what tag (said r); + r + in + (* The value the process was built with, so that every number + below is a change and not a coincidence. 0xE6B800FF. *) + reads "the palette as built" "(i64 (at colors 0))" "3870818559"; + (* The report, to the letter: a native array of colours. *) + ignore (evals "editing the palette" + "(def colors [4 u32] [1 2 3 4])"); + reads "the edited palette" "(i64 (at colors 0))" "1"; + (* The far element too: a store that reached only the first word + would pass the line above and be the bug in a smaller place. *) + reads "the edited palette's last element" + "(i64 (at colors 3))" "4"; + ignore (evals "editing a scalar" "(def speed i64 42)"); + reads "the edited scalar" "speed" "42"; + ignore (evals "editing a struct" + "(def knobs Tuning {.gain 9 .bias 8})"); + reads "the edited struct" "(i64 (.gain knobs))" "9"; + reads "the edited struct's second field" "(i64 (.bias knobs))" "8"; + (* A dyn, which is the one whose new value is on the heap: the + store has to land in storage the collector still roots. *) + ignore (evals "editing a dyn" "(def label dyn {:n 77})"); + reads "the edited dyn" "(i64 (get label :n))" "77"; + (* An initialiser that is a call rather than a literal, which is + the shape [Emit.initial_image] has no image for. *) + ignore (evals "editing a computed initialiser" + "(def computed i64 (+ (twice) 5))"); + reads "the edited computed initialiser" "computed" "15"; + (* The control, and it comes before the brand-new names below for + a reason of its own: once the session holds a global the host + lacks, every later evaluation has storage to allocate and so + has a module to build, and the sharpest form of this claim is + that there is nothing to send at all. + + [defonce] is Common Lisp's [defvar]: a value the program + already holds is not touched, which is the whole of what tells + the two forms apart. *) + let r = evals "re-evaluating a defonce" "(defonce kept i64 99)" in + if Wire.string_field r "note" <> Some "nothing to install" then + fail "re-evaluating a defonce built a module (%s): %s" tag + (Option.value ~default:(said r) (Wire.string_field r "note")); + reads "the defonce beside them" "kept" "7"; + (* A name the process was never built with. Its storage comes + from [flan_dev_global] and no startup call names its + initialiser, so without the store it would be calloc's zeroes + for the life of the process — and a computed one has no image + to fall back on. *) + ignore (evals "a brand-new def" "(def fresh i64 (twice))"); + reads "the brand-new def" "fresh" "10"; + ignore (evals "a brand-new def with a literal" + "(def plain i64 42)"); + reads "the brand-new def with a literal" "plain" "42"; + (* The two halves of the one thunk, in the order they have to be + in. A module carries one [flan_reload_call] and it holds the + class registrations and the def stores together; the + registrations go first, so an initialiser that constructs an + instance sees the slot list the registry now holds. Sent as + one form, which is what makes the ordering the only thing + that can decide the answer. + + The def is a new name rather than one the fixture declares: a + class that gains or loses slots is checked against every + caller the running program has, and the lifted initialiser of + an existing def would be one of them. *) + ignore (evals "a class and a def that constructs one, together" + "(defclass point [x y z]) (def anchor dyn (point 7 8 9))"); + reads "the slot the redefined class gained" + "(i64 (get anchor :z))" "9"; + reads "a slot the class always had" "(i64 (get anchor :x))" "7"; + (* [defconst], which this changes nothing about and which was + already the immediate one: a constant the checker never + consumed is just bytes in memory, so a new value is published + by value at the frame boundary. [dev-rerun.flan] pins the + re-run half; this is the live one. *) + ignore (evals "re-evaluating a defconst" + "(defconst limit f64 41.0)"); + reads "the re-evaluated defconst" "(i64 limit)" "41"; + (* And the one [def] spelling with nothing to run. [uninit] lifts + no initialiser, so there is nothing for the store to call and + nothing is stored — the byte the run wrote is still there. *) + reads "the byte the run wrote" "(i64 (at scratch 0))" "77"; + ignore (evals "re-evaluating an uninit def" + "(def scratch [4 u8] uninit)"); + reads "the byte an uninit def did not repaint" + "(i64 (at scratch 0))" "77"; + (* A retype is refused before anything is stored, by the pass + that can name both types. *) + let r = request c ("(:op \"eval\" :code " + ^ Wire.quote "(def speed string \"x\")" ^ file) in + if status r <> "error" then + fail "a def that changes type was accepted (%s)" tag + else if not (contains_sub (said r) "changes type") then + fail "the retype refusal reads %S (%s)" (said r) tag; + reads "the global a refused retype was about" "speed" "42"; + (* An initialiser is arbitrary code and can signal, and what has + to happen then is that the program stops rather than carrying + on. A scalar is the case where the old value survives on both + backends: it comes back in a register and is stored after the + transfer guard, so the signal jumps past the store. An + aggregate is not, on x86 — see FIX.org, 2026-09-21 — which is + why the probe here is an [i64] and the claim below is about + this shape and not about every one. *) + ignore (evals "a def whose initialiser signals" + "(def speed i64 (blow))"); + let stopped r = + match Wire.field r "stopped" with + | Some { Form.v = Form.Sym "t"; _ } -> true + | _ -> false + in + if not (await ~ms:20000 + (fun () -> stopped (request c "(:op \"describe\")"))) + then + fail "a signalling initialiser never stopped the program (%s)" tag + else begin + (* Named as the evaluation it is: the way out is the boundary + restart the agent establishes around every thunk it runs. *) + let r = request c "(:op \"break\")" in + (match Wire.field r "abandon" with + | Some { Form.v = Form.Int 0L; _ } -> () + | _ -> + fail "the break a signalling initialiser took offers no way \ + out (%s): %s" tag (said r)); + reads "the global after a failed initialiser" "speed" "42"; + let r = + request c + "(:op \"restart-at\" :index 0 :name \"abandon-evaluation\")" + in + if status r <> "ok" then + fail "abandoning a failed initialiser: %s (%s)" (said r) tag; + if not (await ~ms:20000 + (fun () -> + not (stopped (request c "(:op \"describe\")")))) + then + fail "the program never carried on after the abandon (%s)" tag; + (* And the session is not wedged: the next evaluation of the + same name stores like any other. *) + ignore (evals "editing the same def again" "(def speed i64 5)"); + reads "the def edited after a failed one" "speed" "5" + end; + ignore (request c "(:op \"close\")"); + (try Unix.close c with Unix.Unix_error _ -> ()); + (try ignore (Unix.waitpid [] dpid) with Unix.Unix_error _ -> ()) + end; + List.iter (fun f -> try Sys.remove f with Sys_error _ -> ()) + [ dsock; dout ]) + [ ("x86", [||]); ("llvm", [| "--llvm" |]) ]; + (* ── A daemon whose editor was killed ─────────────────────────────── *) (* The defect FIX.org recorded and PDEATHSIG does not reach: an editor that