Joseph Ferano
f6ab3b62fc
The crux was never where to put a descriptor; it was how an instance finds one. A bare struct on the stack has no header to hang a pointer off, and giving it one would change the layout C interop agrees on, change the stride of an array and change what embedding a struct in another costs. So it has none. The instance never carries a pointer to its type and the collector never derives one from the bytes: the pairing of an address with a descriptor is made at the *push*, by the code that put the value there and therefore knows its static type. That is the same trick the shadow stack has always used, and it makes the stack case the easy one rather than the impossible one. A descriptor is the size of an instance, a count, and a table of byte offsets, emitted once per type as private static data. Flattened, not a graph — a struct held by value contributes its offsets shifted by where it sits, and a fixed array contributes its element's once per element — so nesting costs nothing at run time and there is no recursion in the marker. The offsets of a big array would be a big table, and that is capped with a sentence rather than half of the repeat form item 3 will bring. Four places a value of such a type can live, and all four are rooted: a frame slot, a global, the temporary a call's by-value return is spilled into, and the slot a condition that is not a place is evaluated into. The last two are new and are the ones that were not obvious. A callee roots its dyn words and pops them in its epilogue, so between the return and the caller's store the only copy is a register, which a collector that finds its roots by address cannot see; the same hole was open for a Flan call answering a bare dyn and is closed here too. And a condition crosses as a pointer into the signalling frame while a handler allocates, which is exactly what the original refusal said could not be made safe. dyn_roots grows into root_plan and both backends read it, which is what the older note about one counter deciding both ends was always for. The aggregate temporaries are pooled by type rather than handed out in mint order: a positional supply that drifted would pair an address with another type's descriptor, and marking arbitrary offsets off a base is corruption where a missed root is only a bug. Pooled, the worst a drift can do is run out. What is still refused is a dyn no static offset can reach — inside a typed container, in a data type's payload or a union's members where the cases overlay, or under an Option where the payload exists only beneath the tag. A (Ptr S) and a [S] are deliberately not on that list: neither owns storage, and the only storage this compiler hands out for such a type is a frame slot, a global or a fixed array in one, all of them already rooted. That is what lets a handler clause take its (Ptr Cond) and read a dyn payload. test/programs/dyn-struct.flan is the evidence. It runs forty thousand rows past flan_dyn.c's one-megabyte floor, so marks and sweeps really happen, and it holds live values through them in all four places at once. It has teeth: with the descriptor walk stubbed out of the marker, the kept vector's length comes back 24 instead of 628 and its first element is a stale word. Clean under ASan and UBSan, same output at -O2, -O0 and --x86. dyn_ops.c grows an aggregate-root mode so the runtime half can be wrong on its own, with a header word holding a bit pattern that looks boxed and is not a dyn slot. --no-gc still refuses, and had to be told how: a struct with a dyn field is a collected value even when no expression in the program ever has the type dyn, because a zeroed one still has a word the collector is asked to mark. dune test --force: green, 0 failures across every suite.
115 lines
5.1 KiB
Plaintext
115 lines
5.1 KiB
Plaintext
;;;; A struct with a dyn field, under an actual collection.
|
|
;;;;
|
|
;;;; Until the per-type descriptors this program did not compile: the checker
|
|
;;;; refused a dyn field outright, because the collector's roots were frames
|
|
;;;; and a struct outlives the frame that built it, so the field's vector was
|
|
;;;; reachable only through memory the marker never walked.
|
|
;;;;
|
|
;;;; What lifted it: every type that holds dyn words at static offsets gets a
|
|
;;;; descriptor — a table of byte offsets, emitted once as static data — and
|
|
;;;; every place a value of that type can live goes on the collector's root
|
|
;;;; stack with the descriptor beside it. The instance never points at its
|
|
;;;; descriptor and the collector never derives one from the bytes; the pairing
|
|
;;;; is made at the push, by the code that knows the static type of what it put
|
|
;;;; there. runtime/flan_dyn.h's flan_dyn_root_push_desc argues that at length.
|
|
;;;;
|
|
;;;; So this file exercises the four places such a value lives, and does it
|
|
;;;; past flan_dyn.c's one-megabyte floor, which is the only way a mark and a
|
|
;;;; sweep actually run:
|
|
;;;;
|
|
;;;; - a frame slot, which is [keep] in [churn];
|
|
;;;; - a global, which is [registry], rooted before the startup function;
|
|
;;;; - the temporary a call's by-value return lands in, which is what
|
|
;;;; [make-row] hands back — the callee rooted that vector and popped it in
|
|
;;;; its epilogue, so between the return and the caller's store the only
|
|
;;;; copy is a register the collector cannot see;
|
|
;;;; - a condition's payload, which crosses a handler boundary as a pointer
|
|
;;;; into a live frame while the handler allocates.
|
|
;;;;
|
|
;;;; Nesting is in here twice over: Row holds a Tag by value, and Tag holds the
|
|
;;;; dyn. A descriptor is flattened, so Row's table names Tag's dyn word at
|
|
;;;; Row's offset plus Tag's, and there is no second descriptor to follow.
|
|
;;;;
|
|
;;;; What a lost root looks like here is not a wrong number. It is a use of
|
|
;;;; freed memory — a crash, or a word that decodes as another tag and traps
|
|
;;;; with a sentence about the wrong type.
|
|
|
|
(defstruct Tag [name dyn])
|
|
(defstruct Row [id i32 tag Tag rows dyn])
|
|
(defstruct Stalled [why dyn id i32])
|
|
|
|
;;; A global holding dyn words, which main roots before a line of the program
|
|
;;; runs and never pops. Zero until its field is set, and a zero word is not a
|
|
;;; value the collector follows.
|
|
(defvar registry Row)
|
|
|
|
;;; What the handler saw, read back after the handler had allocated.
|
|
(defvar echoed dyn)
|
|
(defvar stalls i64)
|
|
|
|
;;; The collector's own counters, so that "nothing leaks" is a fact this
|
|
;;; program states rather than one the absence of a crash implies.
|
|
(declare gc-collect [] () "flan_gc_collect")
|
|
(declare gc-count [] i64 "flan_gc_count")
|
|
|
|
;;; Returned by value. The vector is rooted in this frame and unrooted the
|
|
;;; instant the epilogue pops, so the caller's own root is the only thing
|
|
;;; between it and the next allocation.
|
|
(defn make-row [i i32] Row
|
|
(let [rows (vec-new dyn)]
|
|
(push rows i)
|
|
(push rows "row")
|
|
(push rows 2.5)
|
|
(Row {.id i .tag (Tag {.name "tag"}) .rows rows})))
|
|
|
|
;;; An aggregate parameter, which arrives in its slot before the roots are
|
|
;;; pushed. Zeroing its dyn words over the top of the argument would be a
|
|
;;; silent miscompile, so the count this returns is the check for it.
|
|
(defn row-len [r Row] i64 (i64 (len (.rows r))))
|
|
|
|
;;; The payload crosses as a pointer to a value in this frame, and the handler
|
|
;;; below allocates before it reads it.
|
|
(defn stall [i i32] ()
|
|
(let [why (vec-new dyn)]
|
|
(push why "stalled")
|
|
(push why i)
|
|
(signal (Stalled {.why why .id i}))))
|
|
|
|
;;; The garbage is a whole Row per iteration, kept by nothing. The live one
|
|
;;; grows *through* the collections rather than only between them.
|
|
(defn churn [n i32] i64
|
|
(let [keep (make-row 0)
|
|
total (i64 0)
|
|
i 0]
|
|
(while (< i n)
|
|
(let [junk (make-row i)]
|
|
(set total (+ total (row-len junk))))
|
|
(if (= 0 (% i 64)) (push (.rows keep) i))
|
|
(if (= 0 (% i 4096)) (stall i))
|
|
(set i (+ i 1)))
|
|
(set (.rows registry) (.rows keep))
|
|
total))
|
|
|
|
(defn main [] ()
|
|
(set (.name (.tag registry)) "registry")
|
|
(set (.rows registry) (vec-new dyn))
|
|
(handler-bind [(Stalled [c]
|
|
;; Allocate first, then read the payload: if the payload's
|
|
;; vector were unrooted across the transfer, this is the
|
|
;; allocation that would free it.
|
|
(let [noise (vec-new dyn)]
|
|
(push noise "noise"))
|
|
(set stalls (+ stalls 1))
|
|
(set echoed (at (.why c) 0)))]
|
|
(print (churn 40000)) (print "\n"))
|
|
(print stalls) (print "\n")
|
|
(print echoed) (print "\n")
|
|
(print (.name (.tag registry))) (print "\n")
|
|
(print (len (.rows registry))) (print "\n")
|
|
(print (at (.rows registry) 0)) (print "\n")
|
|
;; And the heap after a final collection, which is the leak question asked
|
|
;; rather than assumed. Everything the run built is unreachable by now except
|
|
;; the registry's vector and the handful of words it holds.
|
|
(gc-collect)
|
|
(if (< (gc-count) 2000) (print "bounded\n") (print "LEAKED\n")))
|