A module that makes a closure stays mapped, and a closure held beside a collecting operand is rooted

This commit is contained in:
Joseph Ferano 2026-09-25 11:24:34 +07:00
parent 389e93ff09
commit 97a2795cb2
9 changed files with 47 additions and 29 deletions

View File

@ -507,8 +507,9 @@ environment is the collector's".
** TODO A stale Vec header is marked through ** TODO A stale Vec header is marked through
A =Vec= copied by value keeps its old pointer after another copy's push A =Vec= copied by value keeps its old pointer after another copy's push
reallocates, and the collector reads that old block when it marks a =Vec= of reallocates, and the collector reads that old block when it marks a =Vec= of
function values. Harmless while the block stays mapped; a fault if the allocator function values. One level deep it reads freed words the collector rejects; a
unmapped it. docs/BUILT.md, "Escape: the environment is the collector's". =Vec= of =Vec=s reads freed elements as headers and dereferences their allocator
word, which can fault. docs/BUILT.md, "Escape: the environment is the collector's".
** TODO CFn and C's calling convention ** TODO CFn and C's calling convention
A Flan function's signature ends with the transfer channel and a C caller knows A Flan function's signature ends with the transfer channel and a C caller knows

View File

@ -4021,7 +4021,8 @@ implemented.
**Refused, each with its own reason and its own program:** **Refused, each with its own reason and its own program:**
- **Capture does not exist.** *Superseded — see "Capture by value" below. It exists, the program that was this - **Capture does not exist.** *Superseded — see "Capture by value" below. It exists, the program that was this
refusal's witness now runs, and what is refused in its place is the **escape**.* refusal's witness now runs. The escape refusal that replaced it is gone too; see "Escape: the environment is the
collector's".*
- **An `fn` with nothing to say what it takes** (`fn-no-type.flan`), above. - **An `fn` with nothing to say what it takes** (`fn-no-type.flan`), above.
- **A position that would zero one** (`fn-in-struct.flan`): a struct field, a global, a fixed array's element, - **A position that would zero one** (`fn-in-struct.flan`): a struct field, a global, a fixed array's element,
`(zeroed)`. ZII fills an omitted field with all-bytes-zero, and **a zeroed function value is a null pointer, which `(zeroed)`. ZII fills an omitted field with all-bytes-zero, and **a zeroed function value is a null pointer, which
@ -4281,12 +4282,15 @@ non-capturing `fn` never allocates; a `defn` and a `CFn` are unchanged.
**Known gaps, not refused.** **Known gaps, not refused.**
- A `Vec` header copied by value goes stale when another copy's push reallocates — that is ordinary `Vec` - A `Vec` header copied by value goes stale when another copy's push reallocates — that is ordinary `Vec`
behaviour — and the marker still reads the stale header's elements. On freed-but-mapped memory that is a harmless behaviour — and the marker still reads the stale header's elements. For a `Vec` of function values that is a read
read of words the set rejects; if the old block was large enough for the allocator to have unmapped it, a of freed memory whose words the set rejects, and faults only if the allocator unmapped the block. For a `Vec` of
collection there faults. It needs a stale header in a live frame and a collection before that frame ends. `Vec`s the freed elements are read as headers and their allocator word is dereferenced for the epoch test, which
- A value made by an expression the dev daemon evaluates in a module it then unloads carries a code address, and now can fault with the block still mapped. It needs a stale header in a live frame and a collection before that frame
an environment descriptor, inside that module. Storing such a value somewhere that outlives the evaluation was ends.
already a dangling code pointer; the descriptor is a second one.
**A module that makes a closure is never unloaded.** An environment points at its descriptor in the module that made
it, and unmapping that module would fault the next collection rather than the next call. So making one counts toward
the gate a string literal does (`nstr`), and a dev-loop expression that builds a closure keeps its mapping.
- `wasm32`'s `SizeOf` for a type with pointers in it is x86-64's, so a `Vec` of `Fn` there has a 16-byte stride over - `wasm32`'s `SizeOf` for a type with pointers in it is x86-64's, so a `Vec` of `Fn` there has a 16-byte stride over
8-byte values. Consistent everywhere it is read, and older than this. 8-byte values. Consistent everywhere it is read, and older than this.

View File

@ -443,9 +443,9 @@ type m = {
when the program makes a capturing [fn] anywhere, and always in a dev when the program makes a capturing [fn] anywhere, and always in a dev
build — a redefinition can add the first one, and the frames of the build — a redefinition can add the first one, and the frames of the
running program would then hold function values nobody had rooted. When running program would then hold function values nobody had rooted. When
it is false every [Fn] word is a code address or null and the output is it is false every [Fn] word is a code address or null, and nothing roots
byte for byte what it was before closures could escape: the static side one or starts the collector for it: the static side does not pay for the
does not pay for the dynamic one. See [gc_layout]. *) dynamic one. See [gc_layout]. *)
gcfn : bool; gcfn : bool;
(* Was this name in the build the running process came from? False only in a (* Was this name in the build the running process came from? False only in a
redefinition module, and only for a name introduced since. *) redefinition module, and only for a name introduced since. *)
@ -497,7 +497,9 @@ type m = {
the entries that named it came off when the frames that pushed them did, the entries that named it came off when the frames that pushed them did,
and no value of any type points at one. A redefinition module naming a and no value of any type points at one. A redefinition module naming a
type the base program already named therefore gets its own copy, which is type the base program already named therefore gets its own copy, which is
harmless — a descriptor is read-only and has no identity. *) harmless — a descriptor is read-only and has no identity. A closure's
environment is the exception: it points at its descriptor for as long as
it lives, which is why making one counts in [nstr]. *)
descs : (string, desc) Hashtbl.t; descs : (string, desc) Hashtbl.t;
} }
@ -2270,6 +2272,11 @@ and value_at f (e : Tast.expr) : string =
while the allocation collects. The fresh object is in the while the allocation collects. The fresh object is in the
runtime's allocation ring until this value reaches a root. *) runtime's allocation ring until this value reaches a root. *)
| Tast.Closure (r, copies) -> | Tast.Closure (r, copies) ->
(* The environment will point at this module's descriptor, and the
value at this module's code, for as long as the collector keeps it.
Counted with the string literals so an expression thunk that makes
one keeps its mapping rather than being unloaded under it. *)
f.md.nstr <- f.md.nstr + 1;
let v = value f copies in let v = value f copies in
let ety = copies.Tast.ty in let ety = copies.Tast.ty in
let desc = let desc =

View File

@ -1777,6 +1777,8 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit =
collect. The object is in the runtime's allocation ring until the collect. The object is in the runtime's allocation ring until the
value reaches a root. *) value reaches a root. *)
| Tast.Closure (r, copies) -> | Tast.Closure (r, copies) ->
(* See [Emit]'s arm: the environment points into this module. *)
f.md.Emit.nstr <- f.md.Emit.nstr + 1;
let ety = copies.Tast.ty in let ety = copies.Tast.ty in
let p = ptmp f in let p = ptmp f in
imm_into f ~reg:rdi (Int64.of_int (sizeof f.md ety)); imm_into f ~reg:rdi (Int64.of_int (sizeof f.md ety));

View File

@ -276,7 +276,8 @@ and on a managed ~class~ instance. An ordinary ~struct~ never carries one.
pointer with no environment — the only kind that crosses FFI or sits in a reload pointer with no environment — the only kind that crosses FFI or sits in a reload
cell; a *non-escaping* ~fn~ captures enclosing locals by value into a stack cell; a *non-escaping* ~fn~ captures enclosing locals by value into a stack
environment, which is what ~reduce~ callbacks and ~handler-bind~ handlers use; environment, which is what ~reduce~ callbacks and ~handler-bind~ handlers use;
an *escaping* closure needs a heap environment and is still an open decision. an *escaping* closure's environment is allocated by the collector (built
2026-09-25; every capturing ~fn~ takes that path now, see docs/BUILT.md).
- No monads, no HKTs, no type classes. Effects are direct; error handling is - No monads, no HKTs, no type classes. Effects are direct; error handling is
conditions plus ~Option~ and ~or-else~. Monadic sequencing, if ever wanted, is a conditions plus ~Option~ and ~or-else~. Monadic sequencing, if ever wanted, is a
macro. macro.

View File

@ -62,6 +62,10 @@
(defn double [x i64] i64 (* 2 x)) (defn double [x i64] i64 (* 2 x))
;; A closure made as an argument and held while the next argument collects.
(defn apply-to [f (Fn [i64] i64) x i64] i64 (f x))
(defn churn-1 [] i64 (churn) 1)
;; A handler clause keeps its copies on the establishing frame, and may now ;; A handler clause keeps its copies on the establishing frame, and may now
;; capture a dyn and a closure like an fn may. ;; capture a dyn and a closure like an fn may.
(defstruct Ping [n i64]) (defstruct Ping [n i64])
@ -158,6 +162,9 @@
(stash (addr slot) (make-adder 7)) (stash (addr slot) (make-adder 7))
(churn) (churn)
(println (f 1) (slot 1))) (println (f 1) (slot 1)))
;; Held beside a sibling operand that collects.
(let [n 40]
(println (apply-to (fn [x] (+ x n)) (churn-1))))
;; A Vec of function values, one capture per iteration plus a widened name. ;; A Vec of function values, one capture per iteration plus a widened name.
(let [fs (vec-new (Fn [i64] i64))] (let [fs (vec-new (Fn [i64] i64))]
(let [i 0] (let [i 0]

View File

@ -5,11 +5,9 @@
;; left to crash at the call, and the same rule covers a global, a fixed ;; left to crash at the call, and the same rule covers a global, a fixed
;; array's element and (zeroed). ;; array's element and (zeroed).
;; ;;
;; Capture sharpened the reason behind this one without changing it. A struct ;; The zero is the whole objection: a capturing value's environment belongs to
;; outlives the frame it was built on, so a field could not hold a value ;; the collector and may be kept anywhere, and (Option (Fn ...)) is the field
;; carrying an environment either — see fn-escape-*.flan. The zero is still ;; that holds one — see fn-escape.flan.
;; what the message names, because it is the objection that applies to every
;; function value and not only to a capturing one.
;; ;;
;; Which means a (CFn ...) field is refused too, and for the zero alone — ;; Which means a (CFn ...) field is refused too, and for the zero alone —
;; a table of function pointers is exactly what that type is for, and nothing ;; a table of function pointers is exactly what that type is for, and nothing

View File

@ -1,7 +1,6 @@
;; Function values, and specifically the ones with no environment. Nothing ;; Function values, and specifically the ones with no environment. Nothing
;; here captures, which is what makes every one of these safe to return and to ;; here captures — fn-capture.flan is the other half, and fn-escape.flan is a
;; hand around — fn-capture.flan is the other half, and fn-escape-*.flan is ;; capturing value outliving the frame that made it.
;; the line between them.
;; ;;
;; Every signature below says (Fn ...), which is the wide one: it admits a ;; Every signature below says (Fn ...), which is the wide one: it admits a
;; capturing value and so pays for a two-word value and a widening thunk where ;; capturing value and so pays for a two-word value and a widening thunk where

View File

@ -3241,7 +3241,7 @@ let () =
in in
(* And [programs/fn-escape.flan]'s, for the same reason. *) (* And [programs/fn-escape.flan]'s, for the same reason. *)
let fn_escape_out = let fn_escape_out =
"15\n15\n21 8\n1007\n3\n3\n2\n1007\n53\n35 4\n5\n100000 true\n" "15\n15\n21 8\n41\n1007\n3\n3\n2\n1007\n53\n35 4\n5\n100000 true\n"
in in
(* ── wasm32 ──────────────────────────────────────────────────────── (* ── wasm32 ────────────────────────────────────────────────────────
TODO.org, "The web target does not reach four things". TODO.org, "The web target does not reach four things".
@ -4059,11 +4059,9 @@ level "1"
outputs ~opt:"-O0" "the prelude's map, filter, reduce and sort-by, -O0" outputs ~opt:"-O0" "the prelude's map, filter, reduce and sort-by, -O0"
"programs/higher-order.flan" higher_order_out; "programs/higher-order.flan" higher_order_out;
(* Capture by value into a stack environment — spec-memory.md's case 2. (* Capture by value. Three opt levels for the reason the case above has
Three opt levels for the reason the case above has them, and for one them, and for one more: the value carries the address of the copies,
more: the environment is a struct in the frame and the value carries which is exactly the shape -O2 is entitled to make disappear. -O0 is what proves there is a real store and a real load
its address, which is exactly the shape -O2 is entitled to make
disappear. -O0 is what proves there is a real store and a real load
behind it. A dev build is here because the value's code half still behind it. A dev build is here because the value's code half still
comes out of the indirection cell and the environment half must not comes out of the indirection cell and the environment half must not
have disturbed that. have disturbed that.
@ -4139,7 +4137,8 @@ level "1"
(* Closures that outlive their frame: spec-memory.md's case 3. The (* Closures that outlive their frame: spec-memory.md's case 3. The
environment is allocated by the collector, so a capturing fn is environment is allocated by the collector, so a capturing fn is
returned, passed through a function that hands it back, read back out returned, passed through a function that hands it back, held as an
argument while the next one collects, read back out
of another fn's environment, stored through a pointer, pushed into a of another fn's environment, stored through a pointer, pushed into a
Vec beside a widened name, kept in an Option field and an Option Vec beside a widened name, kept in an Option field and an Option
global, and called after a forced collection every time. The counter global, and called after a forced collection every time. The counter