From 97a2795cb2fcffbba74306a1e992c4bfd77ae698 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 11:24:34 +0700 Subject: [PATCH] A module that makes a closure stays mapped, and a closure held beside a collecting operand is rooted --- TODO.org | 5 +++-- docs/BUILT.md | 18 +++++++++++------- lib/emit.ml | 15 +++++++++++---- lib/x86.ml | 2 ++ plan.org | 3 ++- test/programs/fn-escape.flan | 7 +++++++ test/programs/fn-in-struct.flan | 8 +++----- test/programs/fn-values.flan | 5 ++--- test/test_acceptance.ml | 13 ++++++------- 9 files changed, 47 insertions(+), 29 deletions(-) diff --git a/TODO.org b/TODO.org index 4206414f..cad0ec66 100644 --- a/TODO.org +++ b/TODO.org @@ -507,8 +507,9 @@ environment is the collector's". ** TODO A stale Vec header is marked through A =Vec= copied by value keeps its old pointer after another copy's push reallocates, and the collector reads that old block when it marks a =Vec= of -function values. Harmless while the block stays mapped; a fault if the allocator -unmapped it. docs/BUILT.md, "Escape: the environment is the collector's". +function values. One level deep it reads freed words the collector rejects; a +=Vec= of =Vec=s reads freed elements as headers and dereferences their allocator +word, which can fault. docs/BUILT.md, "Escape: the environment is the collector's". ** TODO CFn and C's calling convention A Flan function's signature ends with the transfer channel and a C caller knows diff --git a/docs/BUILT.md b/docs/BUILT.md index 873ea162..af73e46d 100644 --- a/docs/BUILT.md +++ b/docs/BUILT.md @@ -4021,7 +4021,8 @@ implemented. **Refused, each with its own reason and its own program:** - **Capture does not exist.** *Superseded — see "Capture by value" below. It exists, the program that was this - refusal's witness now runs, and what is refused in its place is the **escape**.* + refusal's witness now runs. The escape refusal that replaced it is gone too; see "Escape: the environment is the + collector's".* - **An `fn` with nothing to say what it takes** (`fn-no-type.flan`), above. - **A position that would zero one** (`fn-in-struct.flan`): a struct field, a global, a fixed array's element, `(zeroed)`. ZII fills an omitted field with all-bytes-zero, and **a zeroed function value is a null pointer, which @@ -4281,12 +4282,15 @@ non-capturing `fn` never allocates; a `defn` and a `CFn` are unchanged. **Known gaps, not refused.** - A `Vec` header copied by value goes stale when another copy's push reallocates — that is ordinary `Vec` - behaviour — and the marker still reads the stale header's elements. On freed-but-mapped memory that is a harmless - read of words the set rejects; if the old block was large enough for the allocator to have unmapped it, a - collection there faults. It needs a stale header in a live frame and a collection before that frame ends. -- A value made by an expression the dev daemon evaluates in a module it then unloads carries a code address, and now - an environment descriptor, inside that module. Storing such a value somewhere that outlives the evaluation was - already a dangling code pointer; the descriptor is a second one. + behaviour — and the marker still reads the stale header's elements. For a `Vec` of function values that is a read + of freed memory whose words the set rejects, and faults only if the allocator unmapped the block. For a `Vec` of + `Vec`s the freed elements are read as headers and their allocator word is dereferenced for the epoch test, which + can fault with the block still mapped. It needs a stale header in a live frame and a collection before that frame + ends. + +**A module that makes a closure is never unloaded.** An environment points at its descriptor in the module that made +it, and unmapping that module would fault the next collection rather than the next call. So making one counts toward +the gate a string literal does (`nstr`), and a dev-loop expression that builds a closure keeps its mapping. - `wasm32`'s `SizeOf` for a type with pointers in it is x86-64's, so a `Vec` of `Fn` there has a 16-byte stride over 8-byte values. Consistent everywhere it is read, and older than this. diff --git a/lib/emit.ml b/lib/emit.ml index c0c0b88e..81f213cf 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -443,9 +443,9 @@ type m = { when the program makes a capturing [fn] anywhere, and always in a dev build — a redefinition can add the first one, and the frames of the running program would then hold function values nobody had rooted. When - it is false every [Fn] word is a code address or null and the output is - byte for byte what it was before closures could escape: the static side - does not pay for the dynamic one. See [gc_layout]. *) + it is false every [Fn] word is a code address or null, and nothing roots + one or starts the collector for it: the static side does not pay for the + dynamic one. See [gc_layout]. *) gcfn : bool; (* Was this name in the build the running process came from? False only in a redefinition module, and only for a name introduced since. *) @@ -497,7 +497,9 @@ type m = { the entries that named it came off when the frames that pushed them did, and no value of any type points at one. A redefinition module naming a type the base program already named therefore gets its own copy, which is - harmless — a descriptor is read-only and has no identity. *) + harmless — a descriptor is read-only and has no identity. A closure's + environment is the exception: it points at its descriptor for as long as + it lives, which is why making one counts in [nstr]. *) descs : (string, desc) Hashtbl.t; } @@ -2270,6 +2272,11 @@ and value_at f (e : Tast.expr) : string = while the allocation collects. The fresh object is in the runtime's allocation ring until this value reaches a root. *) | Tast.Closure (r, copies) -> + (* The environment will point at this module's descriptor, and the + value at this module's code, for as long as the collector keeps it. + Counted with the string literals so an expression thunk that makes + one keeps its mapping rather than being unloaded under it. *) + f.md.nstr <- f.md.nstr + 1; let v = value f copies in let ety = copies.Tast.ty in let desc = diff --git a/lib/x86.ml b/lib/x86.ml index 7adf2778..a5934b5a 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -1777,6 +1777,8 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = collect. The object is in the runtime's allocation ring until the value reaches a root. *) | Tast.Closure (r, copies) -> + (* See [Emit]'s arm: the environment points into this module. *) + f.md.Emit.nstr <- f.md.Emit.nstr + 1; let ety = copies.Tast.ty in let p = ptmp f in imm_into f ~reg:rdi (Int64.of_int (sizeof f.md ety)); diff --git a/plan.org b/plan.org index e8b45efd..060ad2dd 100644 --- a/plan.org +++ b/plan.org @@ -276,7 +276,8 @@ and on a managed ~class~ instance. An ordinary ~struct~ never carries one. pointer with no environment — the only kind that crosses FFI or sits in a reload cell; a *non-escaping* ~fn~ captures enclosing locals by value into a stack environment, which is what ~reduce~ callbacks and ~handler-bind~ handlers use; - an *escaping* closure needs a heap environment and is still an open decision. + an *escaping* closure's environment is allocated by the collector (built + 2026-09-25; every capturing ~fn~ takes that path now, see docs/BUILT.md). - No monads, no HKTs, no type classes. Effects are direct; error handling is conditions plus ~Option~ and ~or-else~. Monadic sequencing, if ever wanted, is a macro. diff --git a/test/programs/fn-escape.flan b/test/programs/fn-escape.flan index 4cf7f715..2d5a32aa 100644 --- a/test/programs/fn-escape.flan +++ b/test/programs/fn-escape.flan @@ -62,6 +62,10 @@ (defn double [x i64] i64 (* 2 x)) +;; A closure made as an argument and held while the next argument collects. +(defn apply-to [f (Fn [i64] i64) x i64] i64 (f x)) +(defn churn-1 [] i64 (churn) 1) + ;; A handler clause keeps its copies on the establishing frame, and may now ;; capture a dyn and a closure like an fn may. (defstruct Ping [n i64]) @@ -158,6 +162,9 @@ (stash (addr slot) (make-adder 7)) (churn) (println (f 1) (slot 1))) + ;; Held beside a sibling operand that collects. + (let [n 40] + (println (apply-to (fn [x] (+ x n)) (churn-1)))) ;; A Vec of function values, one capture per iteration plus a widened name. (let [fs (vec-new (Fn [i64] i64))] (let [i 0] diff --git a/test/programs/fn-in-struct.flan b/test/programs/fn-in-struct.flan index 4bfd32db..0baf66ce 100644 --- a/test/programs/fn-in-struct.flan +++ b/test/programs/fn-in-struct.flan @@ -5,11 +5,9 @@ ;; left to crash at the call, and the same rule covers a global, a fixed ;; array's element and (zeroed). ;; -;; Capture sharpened the reason behind this one without changing it. A struct -;; outlives the frame it was built on, so a field could not hold a value -;; carrying an environment either — see fn-escape-*.flan. The zero is still -;; what the message names, because it is the objection that applies to every -;; function value and not only to a capturing one. +;; The zero is the whole objection: a capturing value's environment belongs to +;; the collector and may be kept anywhere, and (Option (Fn ...)) is the field +;; that holds one — see fn-escape.flan. ;; ;; Which means a (CFn ...) field is refused too, and for the zero alone — ;; a table of function pointers is exactly what that type is for, and nothing diff --git a/test/programs/fn-values.flan b/test/programs/fn-values.flan index 52362f23..2a0f11ba 100644 --- a/test/programs/fn-values.flan +++ b/test/programs/fn-values.flan @@ -1,7 +1,6 @@ ;; Function values, and specifically the ones with no environment. Nothing -;; here captures, which is what makes every one of these safe to return and to -;; hand around — fn-capture.flan is the other half, and fn-escape-*.flan is -;; the line between them. +;; here captures — fn-capture.flan is the other half, and fn-escape.flan is a +;; capturing value outliving the frame that made it. ;; ;; Every signature below says (Fn ...), which is the wide one: it admits a ;; capturing value and so pays for a two-word value and a widening thunk where diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index f63bf14e..4aef3a86 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -3241,7 +3241,7 @@ let () = in (* And [programs/fn-escape.flan]'s, for the same reason. *) let fn_escape_out = - "15\n15\n21 8\n1007\n3\n3\n2\n1007\n53\n35 4\n5\n100000 true\n" + "15\n15\n21 8\n41\n1007\n3\n3\n2\n1007\n53\n35 4\n5\n100000 true\n" in (* ── wasm32 ──────────────────────────────────────────────────────── TODO.org, "The web target does not reach four things". @@ -4059,11 +4059,9 @@ level "1" outputs ~opt:"-O0" "the prelude's map, filter, reduce and sort-by, -O0" "programs/higher-order.flan" higher_order_out; - (* Capture by value into a stack environment — spec-memory.md's case 2. - Three opt levels for the reason the case above has them, and for one - more: the environment is a struct in the frame and the value carries - its address, which is exactly the shape -O2 is entitled to make - disappear. -O0 is what proves there is a real store and a real load + (* Capture by value. Three opt levels for the reason the case above has + them, and for one more: the value carries the address of the copies, + which is exactly the shape -O2 is entitled to make disappear. -O0 is what proves there is a real store and a real load behind it. A dev build is here because the value's code half still comes out of the indirection cell and the environment half must not have disturbed that. @@ -4139,7 +4137,8 @@ level "1" (* Closures that outlive their frame: spec-memory.md's case 3. The environment is allocated by the collector, so a capturing fn is - returned, passed through a function that hands it back, read back out + returned, passed through a function that hands it back, held as an + argument while the next one collects, read back out of another fn's environment, stored through a pointer, pushed into a Vec beside a widened name, kept in an Option field and an Option global, and called after a forced collection every time. The counter