diff --git a/FIX.org b/FIX.org index 3e52a03..d6b3978 100644 --- a/FIX.org +++ b/FIX.org @@ -436,6 +436,41 @@ rename. typed-flan branch freezes the static language pre-dyn. tests, an acceptance row per backend, and a survey program (dyn-view.flan) proving the view against both a growing Vec and a fixed array/slice, plus its own two trap modes. + + REVIEW, 2026-09-20: relocation was proved sound but relocation was not + the hazard that mattered — a view can outlive the frame its Vec header + sits in, which nothing could reach before this lane because [box] + refused every container outright. Three routes, all newly constructible, + all stack-use-after-return: returning a view, stashing one in a dyn + global, leaving one behind across a condition transfer. AUTHOR'S RULE: + on the dynamic side Flan follows Clojure and Common Lisp, where holding a + value never hands you garbage, so a container may cross into dyn as a + view only when its own storage is permanent — a global's. [permanent_root] + in check.ml decides it: a global, a field or an array element of one, or + a slice cut directly from one at the call (the trace is lost the moment + it is bound to a name first). Everything else — a local, a parameter, a + temporary, anything behind a (Ptr T) — is refused by name, pointing at + the defvar spelling that works. A heap-held header is not expressible + soundly at this milestone for a structural reason rather than a missing + feature: a (Ptr (Vec i64)) taken off a heap block and one taken off a + local are the same type, so admitting a Ptr as permanent would readmit + the exact hole this closes. An arena-held header is not a separate case + at all — an arena changes where a Vec's elements live, never where its + own header (the binding) lives, so it is already covered by the cases + above. + + Three more, all in the runtime rather than the boundary: [view_vec_check] + recursed into itself rendering the very view it had just declared unsafe + to read (fixed by never rendering it — the sentence names the epochs and + nothing else); [dyn_equal]'s VEC arm read raw [len]/[items] regardless of + kind, so two views with different contents compared equal and a map keyed + by a view collided with every other view (fixed with view-aware + length/element readers, [vecish_len]/[vecish_at]); and the three + restatements of flan_vec's layout (flan_rt.c, flan_dyn.c, dyn_ops.c) had + nothing tying them together despite a comment's claim that they did — a + [layout] probe on each, compared field by field in dyn_ops.c's new + "layout" mode, makes a disagreement a FAIL line instead of a silent + corruption. 4. nil: arrives with maps. nil <-> None at (Option T) boundaries, trap at bare T, (Some nil) unconstructible. — LANDED, 3c1fb1b. The bare-T trap is split: a literal nil the checker can see is refused at compile time, in diff --git a/lib/check.ml b/lib/check.ml index ec640e3..fb4e056 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -1468,12 +1468,77 @@ let view_not_yet loc (container : Types.t) (elem : Types.t) = no_dyn_yet loc ~into:true container (Printf.sprintf ". A container view at this milestone holds i64, f64 or bool \ - elements — a %s element's dyn form is a pointer into the \ - collector's heap, and a typed container's storage is memory the \ - collector never scans, so a write through the view could plant a \ - pointer where nothing will ever trace it" + elements only — %s is neither a width this milestone's view carries \ + nor the case the restriction exists for, which is a string element: \ + a dyn string's form is a pointer into the collector's heap, and a \ + typed container's storage is memory the collector never scans, so \ + a write through a view over strings could plant a pointer where \ + nothing will ever trace it" (Types.to_string elem)) +(* M2 item 3's second guard, added on review: a view's descriptor holds an + address into the container's own storage, chased fresh on every + operation, which is what makes a Vec's growth safe — but it is also what + makes a *dangling* container's storage a live hazard nothing catches + until somebody reads through the view. A view returned from the function + whose frame the Vec lived in, stashed in a global and read after that + frame is gone, or left behind when a condition transfer unwinds it, are + all stack-use-after-return once box stopped refusing containers outright + — reachable now for the first time, not a pre-existing hole this lane + merely inherited. + + On the dynamic side Flan follows Clojure and Common Lisp: holding a value + can never hand you garbage. Treating a view as a bare pointer and calling + the lifetime the programmer's problem is the Odin answer, and neither + Odin nor C stops it — but a dyn value that can dangle is the wrong trade + on this side of the language, so this is refused rather than merely + documented. + + [permanent_root] asks whether an expression's own address — the one a + view's pointer will chase — is guaranteed to outlive every frame, which is + true of exactly one thing at this milestone: a global. A field of a + permanent value is permanent at the same fixed offset from it, and an + array element of one likewise; both are still inside the global's own + storage. A slice built directly from [(slice T lo hi)] inherits the + permanence of the [T] it was cut from — unwrapped here because that is + the one shape still carrying the trace back to it; once a slice has been + bound to a name the trace is gone and it is refused; the spelling that + keeps it is to view the slice expression directly, the way this file's + own survey program does. + + Everything else — a local, a parameter, a temporary, anything reached + through a [Ptr] — answers false. A [Ptr] is refused rather than trusted + because a heap-allocated block and a frame slot are the same type: a + [(Ptr (Vec i64))] taken from a heap allocation would be sound to view, but + the same type is what [(addr some-local)] answers too, and the checker + cannot tell the two apart. Admitting one admits the other, which is the + whole hazard this guard exists to close — so until a Flan type exists + that says "durably heap-owned" and a [Ptr] does not, a container reached + through one is refused rather than guessed at. An arena-held container is + not a separate case: an arena changes where a Vec's *elements* live, never + where its own header — the value a name is bound to — lives, so a Vec + grown from an arena is exactly as permanent as the binding that holds it, + already covered by the cases above. *) +let rec permanent_root (e : Tast.expr) : bool = + match e.Tast.e with + | Tast.Global _ -> true + | Tast.Field (target, _) -> permanent_root target + | Tast.Prim (Tast.At, target :: _) -> permanent_root target + | Tast.Prim (Tast.Slice, [ target; _; _ ]) -> permanent_root target + | _ -> false + +let view_not_permanent loc (container : Types.t) = + Loc.failk "check/dyn-view-lifetime" loc + "%s does not cross into dyn as a view here — its storage is not known \ + to outlive the view, and a dyn value that can dangle is not one this \ + language will hand back. A global's does: (defvar g %s ...) viewed \ + from anywhere is sound, because a global's address is fixed for the \ + process. A local, a parameter, a temporary, or anything reached \ + through a (Ptr T) is refused for the same reason — the checker cannot \ + tell a heap-durable pointer from a frame's own, and admitting one \ + admits the other" + (Types.to_string container) (Types.to_string container) + let box loc (e : Tast.expr) : Tast.expr = let dyn sym args = rt loc Types.Dyn sym args in match e.Tast.ty with @@ -1509,19 +1574,25 @@ let box loc (e : Tast.expr) : Tast.expr = cannot grow, so a snapshot taken once at the crossing is sound for both, and they share [flan_dyn_view_flat]. *) | Types.Vec elem -> - (match view_elem elem with - | Some k -> dyn "flan_dyn_view_vec" [ e; view_elem_lit loc k ] - | None -> view_not_yet loc e.Tast.ty elem) + if not (permanent_root e) then view_not_permanent loc e.Tast.ty + else + (match view_elem elem with + | Some k -> dyn "flan_dyn_view_vec" [ e; view_elem_lit loc k ] + | None -> view_not_yet loc e.Tast.ty elem) | Types.Slice elem -> - (match view_elem elem with - | Some k -> dyn "flan_dyn_view_flat" [ e; view_elem_lit loc k ] - | None -> view_not_yet loc e.Tast.ty elem) + if not (permanent_root e) then view_not_permanent loc e.Tast.ty + else + (match view_elem elem with + | Some k -> dyn "flan_dyn_view_flat" [ e; view_elem_lit loc k ] + | None -> view_not_yet loc e.Tast.ty elem) | Types.Array (n, elem) -> - (match view_elem elem with - | Some k -> - dyn "flan_dyn_view_flat" - [ e; mk loc dyn_i64 (Tast.Int (n, Types.I64)); view_elem_lit loc k ] - | None -> view_not_yet loc e.Tast.ty elem) + if not (permanent_root e) then view_not_permanent loc e.Tast.ty + else + (match view_elem elem with + | Some k -> + dyn "flan_dyn_view_flat" + [ e; mk loc dyn_i64 (Tast.Int (n, Types.I64)); view_elem_lit loc k ] + | None -> view_not_yet loc e.Tast.ty elem) | Types.Map _ -> no_dyn_yet loc ~into:true e.Tast.ty ". The dyn container at this milestone is the runtime's own, from \ @@ -2150,7 +2221,7 @@ let rec check ctx ?want (e : Ast.expr) : Tast.expr = if target.Tast.ty = Types.Dyn then let i = check ctx ~want:Types.Dyn idx in let v = check ctx ~want:Types.Dyn v in - expect loc ~want + expect ctx loc ~want (rt loc Types.Unit "flan_dyn_set_at" [ target; i; v ]) else begin let p, pty = @@ -2163,7 +2234,7 @@ let rec check ctx ?want (e : Ast.expr) : Tast.expr = Tast.Pindex (target, iidx), ty in let v = check ctx ~want:pty v in - expect loc ~want (mk loc Types.Unit (Tast.Set (p, v))) + expect ctx loc ~want (mk loc Types.Unit (Tast.Set (p, v))) end | Ast.Set (p, v) -> let p, pty = check_place ctx loc p in diff --git a/runtime/flan_dyn.c b/runtime/flan_dyn.c index 294bf5e..0e82c9c 100644 --- a/runtime/flan_dyn.c +++ b/runtime/flan_dyn.c @@ -34,6 +34,7 @@ */ #include +#include #include #include #include @@ -194,6 +195,19 @@ typedef struct flan_dyn_vec_hdr { int64_t epoch; } flan_dyn_vec_hdr; +/* This mirror's own layout, reported the same way flan_rt.c's + * [flan_vec_layout] reports the original's — see that function's comment + * for what ties the two together and why nothing at compile time otherwise + * does. */ +void flan_dyn_vec_hdr_layout(int64_t out[6]) { + out[0] = (int64_t)sizeof(flan_dyn_vec_hdr); + out[1] = (int64_t)offsetof(flan_dyn_vec_hdr, ptr); + out[2] = (int64_t)offsetof(flan_dyn_vec_hdr, len); + out[3] = (int64_t)offsetof(flan_dyn_vec_hdr, cap); + out[4] = (int64_t)offsetof(flan_dyn_vec_hdr, alloc); + out[5] = (int64_t)offsetof(flan_dyn_vec_hdr, epoch); +} + /* flan_allocator's prefix, far enough to read the one word a stale-container * check needs. The struct has more fields after [epoch]; this file never * touches them; and the alignment of a leading same-typed prefix is the same @@ -234,8 +248,18 @@ typedef struct flan_obj { } flan_obj; /* How many dyn words hang off an object's items block — the count the marker - * walks and the sweep charges. A map holds two per entry. */ + * walks and the sweep charges. A map holds two per entry. + * + * OBJ_VIEW answers 0 explicitly rather than falling into the [o->len] arm. + * [mark_push] never puts a view on the mark stack — it traces only + * OBJ_VEC/OBJ_MAP — so this is not reachable today, but [o->u.view.base] + * aliases [o->u.v.items] in the union, and a native array of i64 or f64 + * reinterpreted as dyn words is exactly the kind of thing this file's + * roots contract exists to prevent happening by accident. Answering 0 here + * is what keeps a future change to the marking gate from silently trusting + * this function's default arm instead of failing loudly. */ static inline int64_t obj_words(flan_obj *o) { + if (o->kind == OBJ_VIEW) return 0; return o->kind == OBJ_MAP ? o->len * 2 : o->len; } @@ -459,11 +483,17 @@ static double dyn_num_value(flan_dyn v); /* forward: the view helpers, needed by [render] and [say_render] above where * they are defined, alongside the container operations below */ -static int64_t view_len(const char *op, flan_dyn v, flan_obj *o); +static int64_t view_len(const char *op, flan_obj *o); static void *view_base(flan_obj *o); static flan_dyn view_box(int32_t elem, const uint8_t *p); static int64_t view_elem_size(int32_t elem); +/* forward: needed by [dyn_equal] below, defined alongside the view helpers + * further down — a length and an element reader that answer correctly + * whether [o] is an ordinary heap vec or a view over a typed container. */ +static int64_t vecish_len(flan_obj *o); +static flan_dyn vecish_at(flan_obj *o, int64_t i); + static void render(flan_dyn v, int depth, int nested) { char buf[64]; int32_t t = flan_dyn_tag(v); @@ -522,7 +552,7 @@ static void render(flan_dyn v, int depth, int nested) { } default: { flan_obj *o = dyn_obj(v); - int64_t i, n = o->kind == OBJ_VIEW ? view_len("print", v, o) : o->len; + int64_t i, n = o->kind == OBJ_VIEW ? view_len("print", o) : o->len; emit("["); for (i = 0; i < n; i++) { emit(" "); @@ -618,7 +648,7 @@ static void say_render(sayer *s, flan_dyn v, int depth) { } default: { flan_obj *o = dyn_obj(v); - int64_t i, n = o->kind == OBJ_VIEW ? view_len("print", v, o) : o->len; + int64_t i, n = o->kind == OBJ_VIEW ? view_len("print", o) : o->len; if (depth >= 2) { say_puts(s, "[...]"); return; } say_puts(s, "["); for (i = 0; i < n && s->n < s->cap - 8; i++) { @@ -1275,12 +1305,23 @@ static int dyn_equal(flan_dyn a, flan_dyn b, int depth) { } if (ta == FLAN_DYN_TAG_VEC) { flan_obj *x = dyn_obj(a), *y = dyn_obj(b); - int64_t i; + int64_t i, xn, yn; if (x == y) return 1; if (depth >= EQ_DEPTH) return 0; - if (x->len != y->len) return 0; - for (i = 0; i < x->len; i++) - if (!dyn_equal(x->u.v.items[i], y->u.v.items[i], depth + 1)) return 0; + /* [x]/[y] may each be an ordinary heap vec or a view (M2 item 3) — the + tag does not say which, so [vecish_len]/[vecish_at] below read either + shape correctly. Reading raw through [x->u.v.items] the way this arm + used to is wrong for a view: nothing sets [len] for OBJ_VIEW, so it + reads back 0, and the elements alias [u.view.base] reinterpreted as + dyn words — two views with different contents would compare equal, a + view and an equal heap vec would compare unequal, and a map keyed by + any view would collide with every other view, silently, with nothing + to crash. */ + xn = vecish_len(x); + yn = vecish_len(y); + if (xn != yn) return 0; + for (i = 0; i < xn; i++) + if (!dyn_equal(vecish_at(x, i), vecish_at(y, i), depth + 1)) return 0; return 1; } /* Two maps are equal when they hold the same keys and each key answers an @@ -1346,19 +1387,26 @@ static int64_t view_elem_size(int32_t elem) { * doctrine's dyn side gets its own spelling (docs/SPIKE-DUPLICITY.md), and a * dyn program that hits this wants the same park-and-inspect [flan_trap] * gives every other dyn mistake, not the typed side's [rt_die]. A Vec with - * no allocator yet — one nobody has pushed to — has nothing to check. */ -static void view_vec_check(const char *op, flan_dyn v, flan_dyn_vec_hdr *h) { + * no allocator yet — one nobody has pushed to — has nothing to check. + * + * The message never renders the view it just declared unsafe to read — + * review's second finding, and it was not a decoration this dropped for + * safety's sake, it was a real infinite recursion: [say] on a view calls + * [say_render]'s view branch, which calls [view_len], which calls back in + * here, unconditionally, because the epoch is still stale. Every render of + * this same view would hit the same check and take the same branch, so + * nothing about depth or a visited set closes it — the fix is that a + * stale-container check must never read the container it has just refused + * to trust, not even to describe it in the sentence explaining why. */ +static void view_vec_check(const char *op, flan_dyn_vec_hdr *h) { if (h->alloc) { flan_dyn_alloc_hdr *a = (flan_dyn_alloc_hdr *)h->alloc; if ((int64_t)a->epoch != h->epoch) { - char sv[SAY_MAX]; - say(sv, SAY_MAX, v); fflush(stdout); fprintf(stderr, "dyn %s: this view's container's allocator was released — the " - "Vec was made at epoch %lld and the allocator is at %lld now " - "— %s\n", - op, (long long)h->epoch, (long long)(int64_t)a->epoch, sv); + "Vec was made at epoch %lld and the allocator is at %lld now\n", + op, (long long)h->epoch, (long long)(int64_t)a->epoch); flan_trap((const uint8_t *)"DynRange", 8); } } @@ -1367,10 +1415,10 @@ static void view_vec_check(const char *op, flan_dyn v, flan_dyn_vec_hdr *h) { /* [len] and [base], read live for a Vec view (so a push that grows and * moves the underlying Vec is seen the very next operation) and read from * the snapshot for a flat one. */ -static int64_t view_len(const char *op, flan_dyn v, flan_obj *o) { +static int64_t view_len(const char *op, flan_obj *o) { if (o->u.view.is_vec) { flan_dyn_vec_hdr *h = (flan_dyn_vec_hdr *)o->u.view.base; - view_vec_check(op, v, h); + view_vec_check(op, h); return h->len; } return o->u.view.len; @@ -1392,6 +1440,27 @@ static flan_dyn view_box(int32_t elem, const uint8_t *p) { } } +/* A length and an element reader that answer correctly whether [o] is an + * ordinary heap vec (OBJ_VEC, elements are dyn words) or a view over a + * typed container (OBJ_VIEW, elements are native bytes boxed on the way + * out) — the pair [dyn_equal]'s VEC arm needs so that a view compares + * correctly against another view and against an ordinary vec alike. Reading + * [o->len]/[o->u.v.items] directly, the way that arm used to, answers 0 and + * garbage for a view: nothing sets [len] for OBJ_VIEW, and its elements + * alias [u.view.base] reinterpreted as dyn words rather than the native + * bytes they are. */ +static int64_t vecish_len(flan_obj *o) { + return o->kind == OBJ_VIEW ? view_len("=", o) : o->len; +} + +static flan_dyn vecish_at(flan_obj *o, int64_t i) { + if (o->kind == OBJ_VIEW) + return view_box(o->u.view.elem, + (const uint8_t *)view_base(o) + + i * view_elem_size(o->u.view.elem)); + return o->u.v.items[i]; +} + /* Writes tag-check on the way in: the dyn value's tag must be the one this * view's element type wants, or this traps by name and never coerces or * truncates a mismatched value into the slot. [v] is the view, for the @@ -1448,7 +1517,7 @@ flan_dyn flan_dyn_len(flan_dyn v) { if (is_text(v) || is_map(v)) return flan_dyn_from_i64(dyn_obj(v)->len); if (is_vec(v)) { flan_obj *o = dyn_obj(v); - if (o->kind == OBJ_VIEW) return flan_dyn_from_i64(view_len("len", v, o)); + if (o->kind == OBJ_VIEW) return flan_dyn_from_i64(view_len("len", o)); return flan_dyn_from_i64(o->len); } trap1(TYPE_TRAP, "len", "only a text, a vec or a map has one", v); @@ -1474,7 +1543,7 @@ flan_dyn flan_dyn_at(flan_dyn v, flan_dyn i) { k = need_index("at", v, i); o = dyn_obj(v); if (o->kind == OBJ_VIEW) { - int64_t len = view_len("at", v, o); + int64_t len = view_len("at", o); if (k < 0 || k >= len) trap_range("at", v, k, len); return view_box(o->u.view.elem, (const uint8_t *)view_base(o) + k * view_elem_size(o->u.view.elem)); @@ -1494,7 +1563,7 @@ void flan_dyn_set_at(flan_dyn v, flan_dyn i, flan_dyn x) { k = need_index("set-at", v, i); o = dyn_obj(v); if (o->kind == OBJ_VIEW) { - int64_t len = view_len("set-at", v, o); + int64_t len = view_len("set-at", o); uint8_t *p; if (k < 0 || k >= len) trap_range("set-at", v, k, len); p = (uint8_t *)view_base(o) + k * view_elem_size(o->u.view.elem); diff --git a/runtime/flan_dyn.h b/runtime/flan_dyn.h index 329705e..2cd0a5e 100644 --- a/runtime/flan_dyn.h +++ b/runtime/flan_dyn.h @@ -309,6 +309,14 @@ const char *flan_dyn_tag_name(int32_t tag); int64_t flan_gc_count(void); void flan_gc_set_floor(int64_t bytes); +/* Reports flan_dyn.c's own mirror of flan_rt.c's [flan_vec] — [size, then + * the offset of ptr, len, cap, alloc, epoch] — for test/dyn_ops.c's + * "layout" mode to compare against flan_rt.c's [flan_vec_layout] and + * against its own hand-built mirror. See [flan_vec_layout]'s comment in + * flan_rt.c for what this ties together and why nothing at compile time + * otherwise does. */ +void flan_dyn_vec_hdr_layout(int64_t out[6]); + #ifdef __cplusplus } #endif diff --git a/runtime/flan_rt.c b/runtime/flan_rt.c index f080aa4..db240b1 100644 --- a/runtime/flan_rt.c +++ b/runtime/flan_rt.c @@ -12,6 +12,7 @@ */ #include +#include #include #include #include @@ -1490,6 +1491,26 @@ typedef struct flan_vec { int64_t epoch; } flan_vec; +/* This struct's layout is restated twice more in the tree — flan_dyn.c's + * [flan_dyn_vec_hdr], for a typed container's view (M2 item 3), and + * test/dyn_ops.c's [hand_vec], which builds one by hand with no [flan_rt.c] + * linked in to call [flan_vec_init] for it. None of the three can [#include] + * this file (see [Build.compile_c]), so nothing at compile time ties them + * together — a reordered field here links and runs, and corrupts whichever + * of the other two disagrees. [flan_vec_layout] is the tie: it reports this + * struct's real size and field offsets, and test/dyn_ops.c's "layout" mode + * compares them against its own [hand_vec]'s and against flan_dyn.c's + * [flan_dyn_vec_hdr_layout], so a disagreement is a FAIL line in `dune test` + * rather than a silent corruption the next line over. */ +void flan_vec_layout(int64_t out[6]) { + out[0] = (int64_t)sizeof(flan_vec); + out[1] = (int64_t)offsetof(flan_vec, ptr); + out[2] = (int64_t)offsetof(flan_vec, len); + out[3] = (int64_t)offsetof(flan_vec, cap); + out[4] = (int64_t)offsetof(flan_vec, alloc); + out[5] = (int64_t)offsetof(flan_vec, epoch); +} + /* The request that did not fit, for the condition the compiler builds at the * failing site. A pair of globals rather than out-parameters because the * condition is a value struct on the signalling frame's stack with fixed diff --git a/test/dyn_ops.c b/test/dyn_ops.c index ca25ab2..6030516 100644 --- a/test/dyn_ops.c +++ b/test/dyn_ops.c @@ -34,6 +34,7 @@ void flan_rt_init(int32_t argc, char **argv); void flan_vec_free(void *v, int64_t size, int64_t align, const uint8_t *loc, int64_t loclen); +void flan_vec_layout(int64_t out[6]); static int failures; @@ -351,11 +352,19 @@ static void ops(void) { /* A typed container's own header, restated a third time — flan_rt.c's * [flan_vec], flan_dyn.c's [flan_dyn_vec_hdr], and this. The three must - * agree on layout, and this file is exactly the mechanism that makes a - * disagreement a compile or link error rather than a silent corruption: - * there is no [flan_vec_init] to call from here (flan_rt.c is not linked - * against this main), so the header is built by hand, the same five words - * [flan_vec_grow] would leave behind after a few pushes. */ + * agree on layout, and none of them can [#include] another's to say so at + * compile time (see [Build.compile_c]): there is no [flan_vec_init] to call + * from here, so the header below is built by hand, the same five words + * [flan_vec_grow] would leave behind after a few pushes. + * + * What actually ties the three together is [layout], further down: it reads + * flan_rt.c's [flan_vec_layout] and flan_dyn.c's [flan_dyn_vec_hdr_layout] + * and compares both against [offsetof] on this very struct, so a field + * reordered in any one of the three is a FAIL line here rather than a + * silent corruption at whatever call site next dereferences the wrong + * offset. A declared-as-[void*] prototype on its own proves nothing about + * layout — it was named as if it did in an earlier version of this + * comment, which was wrong, and [layout] is what makes the claim true. */ typedef struct { void *ptr; int64_t len; @@ -389,6 +398,38 @@ static void view(void) { "the array's own write reaches the view — it is not a copy"); prints(flat, "[ 10 20 99 7]"); + /* Structural equality, view-aware — review's third finding. [dyn_equal]'s + VEC arm used to read [x->len]/[x->u.v.items] regardless of kind, which + for a view answers 0 and garbage: two views with different contents + compared equal, a view and an equal heap vec compared unequal, and a + map keyed by any view collided with every other view. [buf] now reads + [ 10 20 99 7]; [same] is a second, independent view over the identical + bytes, and [other] a view over one differing element. */ + { + int64_t same_buf[4] = { 10, 20, 99, 7 }; + int64_t diff_buf[4] = { 10, 20, 99, 8 }; + flan_dyn same = flan_dyn_view_flat(same_buf, 4, FLAN_VIEW_I64); + flan_dyn other = flan_dyn_view_flat(diff_buf, 4, FLAN_VIEW_I64); + flan_dyn heap = flan_dyn_vec_new(); + flan_dyn_root_push(&same); + flan_dyn_root_push(&other); + flan_dyn_root_push(&heap); + check(truth(flan_dyn_eq(flat, same)), + "two views over equal bytes are equal"); + check(!truth(flan_dyn_eq(flat, other)), + "two views over different bytes are not equal"); + flan_dyn_push(heap, flan_dyn_from_i64(10)); + flan_dyn_push(heap, flan_dyn_from_i64(20)); + flan_dyn_push(heap, flan_dyn_from_i64(99)); + flan_dyn_push(heap, flan_dyn_from_i64(7)); + check(truth(flan_dyn_eq(flat, heap)), + "a view and an equal heap vec are equal"); + flan_dyn_set_at(heap, flan_dyn_from_i64(3), flan_dyn_from_i64(0)); + check(!truth(flan_dyn_eq(flat, heap)), + "a view and a differing heap vec are not equal"); + flan_dyn_root_pop(3); + } + /* A vec view: points at the header's own address, so a push that grows and moves it is seen on the very next read — there is no snapshot to go stale. */ @@ -466,6 +507,43 @@ static void refuse_view(const char *what) { exit(3); } +/* The three restatements of flan_vec's layout, compared — see [hand_vec]'s + * comment for why nothing at compile time otherwise ties them together. + * [offsetof] on [hand_vec] itself is this file's half; [flan_vec_layout] + * and [flan_dyn_vec_hdr_layout] are the other two's. */ +static void layout(void) { + int64_t rt[6], dyn[6]; + int64_t here[6] = { + (int64_t)sizeof(hand_vec), + (int64_t)offsetof(hand_vec, ptr), + (int64_t)offsetof(hand_vec, len), + (int64_t)offsetof(hand_vec, cap), + (int64_t)offsetof(hand_vec, alloc), + (int64_t)offsetof(hand_vec, epoch) + }; + static const char *const names[6] = + { "sizeof", "offset of ptr", "offset of len", "offset of cap", + "offset of alloc", "offset of epoch" }; + int i; + char msg[128]; + flan_vec_layout(rt); + flan_dyn_vec_hdr_layout(dyn); + for (i = 0; i < 6; i++) { + if (rt[i] != here[i]) { + snprintf(msg, sizeof msg, "flan_vec vs. hand_vec's %s: %lld vs. %lld", + names[i], (long long)rt[i], (long long)here[i]); + fail(msg); + } + if (dyn[i] != here[i]) { + snprintf(msg, sizeof msg, + "flan_dyn_vec_hdr vs. hand_vec's %s: %lld vs. %lld", + names[i], (long long)dyn[i], (long long)here[i]); + fail(msg); + } + } + printf(failures == 0 ? "layout ok\n" : "layout failed\n"); +} + /* ── The collector ─────────────────────────────────────────────────────*/ /* Allocate a great many, hold a few, and assert the heap does not grow. The @@ -857,6 +935,10 @@ int main(int argc, char **argv) { view(); return failures == 0 ? 0 : 1; } + if (strcmp(argv[1], "layout") == 0) { + layout(); + return failures == 0 ? 0 : 1; + } if (strncmp(argv[1], "refuse:", 7) == 0) { refuse(argv[1] + 7); return 0; } if (strncmp(argv[1], "refuseview:", 11) == 0) { refuse_view(argv[1] + 11); diff --git a/test/programs/dyn-view.flan b/test/programs/dyn-view.flan index fe59f65..a76cc5b 100644 --- a/test/programs/dyn-view.flan +++ b/test/programs/dyn-view.flan @@ -3,19 +3,30 @@ ;;;; [as-dyn]'s parameter is unannotated dyn and its argument is a typed ;;;; (Vec i64), a fixed array or a slice — the box happens at the call, on the ;;;; caller's own value, which is what makes [dv] below the SAME storage [v] -;;;; is and not a copy of it. (Boxing a value AFTER passing it through an -;;;; ordinary by-value parameter would view that parameter's own copy instead -;;;; — value semantics, not a hole in this feature — so every view here is -;;;; taken where the container already lives.) +;;;; is and not a copy of it. +;;;; +;;;; Every container viewed below is a GLOBAL, and that is not incidental to +;;;; this program — it is the lifetime guard review added after the first +;;;; landing: a view's descriptor chases the container's own address on every +;;;; operation, which is what makes a Vec's growth safe, but it is also what +;;;; makes a DANGLING container's address a live hazard. box refuses a Vec, a +;;;; slice or a fixed array whose storage is not known to outlive the view — +;;;; a local's, a parameter's, a temporary's — and a global's is the one +;;;; storage this milestone can prove permanent: fixed in .data for the +;;;; process. test_flan.ml's checker tests carry the refusal side of this +;;;; (a local Vec, a Vec parameter, a Vec behind a Ptr, a slice rebound to a +;;;; local); this program is the acceptance side, over storage the guard +;;;; allows. ;;;; ;;;; Mode 0 is the survey: a read through the view boxes the element ;;;; correctly, a write through either side is seen through the other, and a ;;;; push through the view — which can only mean the Vec case, since neither ;;;; a slice nor a fixed array can grow — moves the Vec's backing storage and -;;;; the typed side still sees the grown length and the new element. That -;;;; last one is the design's central claim: the view's descriptor points AT -;;;; the Vec's own header rather than snapshotting its pointer and length, so -;;;; there is no snapshot for the growth to invalidate. +;;;; the typed side still sees the grown length and the new element. That is +;;;; the design's central claim: the view's descriptor points AT the Vec's +;;;; own header rather than snapshotting its pointer and length, so there is +;;;; no snapshot for the growth to invalidate — and the header itself is the +;;;; global's, which never moves even though the buffer behind it does. ;;;; ;;;; Modes 1 and 2 are the two traps a view can throw: an index outside its ;;;; length, and a write whose dyn tag does not match the element type the @@ -25,87 +36,88 @@ (defn as-dyn [d dyn] dyn d) +(defvar v (Vec i64) (vec-new i64)) +(defvar a [4 i64]) +(defvar a2 [3 f64]) +(defvar bv (Vec bool) (vec-new bool)) + (defn main [args [string]] i32 (let [n (i32 (bytes->i64 (bytes (at args 1))))] (cond (= n 0) (do - ;; A (Vec i64) view. - (let [v (vec-new i64)] - (push v 10) - (push v 20) - (push v 30) - (let [dv (as-dyn v)] - (print dv) - (print "\n") - ;; Write through the view, read through the typed side. - (set (at dv 1) 999) - (print (at v 1)) - (print "\n") - ;; Write through the typed side, read through the view. - (set (at v 2) 777) - (print (at dv 2)) - (print "\n") - ;; Grow through the view. flan_vec_grow reallocates v's backing - ;; storage and overwrites v's own header in place, which is the - ;; same header the view points at — so the typed side, asked - ;; afterwards, already agrees with the push it never made itself. - (push dv 40) - (print (len v)) - (print "\n") - (print (at v 3)) - (print "\n"))) + ;; A (Vec i64) view, over the global. + (push v 10) + (push v 20) + (push v 30) + (let [dv (as-dyn v)] + (print dv) + (print "\n") + ;; Write through the view, read through the typed side. + (set (at dv 1) 999) + (print (at v 1)) + (print "\n") + ;; Write through the typed side, read through the view. + (set (at v 2) 777) + (print (at dv 2)) + (print "\n") + ;; Grow through the view. flan_vec_grow reallocates v's backing + ;; storage and overwrites v's own header in place, which is the + ;; same header the view points at — so the typed side, asked + ;; afterwards, already agrees with the push it never made itself. + (push dv 40) + (print (len v)) + (print "\n") + (print (at v 3)) + (print "\n")) ;; A fixed array's view: nothing here can grow, so a snapshot taken ;; once at the crossing is sound — there is no move to go stale over. - (let [a (array 4 i64)] - (set (at a 0) 1) - (set (at a 1) 2) - (set (at a 2) 3) - (set (at a 3) 4) - (let [da (as-dyn a)] - (print da) - (print "\n") - (set (at da 0) 100) - (print (at a 0)) - (print "\n") - (set (at a 3) 400) - (print (at da 3)) - (print "\n"))) + (set (at a 0) 1) + (set (at a 1) 2) + (set (at a 2) 3) + (set (at a 3) 4) + (let [da (as-dyn a)] + (print da) + (print "\n") + (set (at da 0) 100) + (print (at a 0)) + (print "\n") + (set (at a 3) 400) + (print (at da 3)) + (print "\n")) ;; A slice's view, over f64 elements, and a bool Vec's view — the - ;; other two of the three element kinds a view can hold. - (let [a2 (array 3 f64)] - (set (at a2 0) 1.5) - (set (at a2 1) 2.5) - (set (at a2 2) 3.5) - (let [ds (as-dyn (slice a2 0 3))] - (print ds) - (print "\n") - (set (at ds 0) 9.5) - (print (at a2 0)) - (print "\n"))) - (let [bv (vec-new bool)] - (push bv true) - (push bv false) - (let [db (as-dyn bv)] - (print db) - (print "\n") - (set (at db 1) true) - (print (at bv 1)) - (print "\n"))) + ;; other two of the three element kinds a view can hold. The slice + ;; is cut directly from the global at the call, which is what keeps + ;; its trace back to permanent storage visible to the checker. + (set (at a2 0) 1.5) + (set (at a2 1) 2.5) + (set (at a2 2) 3.5) + (let [ds (as-dyn (slice a2 0 3))] + (print ds) + (print "\n") + (set (at ds 0) 9.5) + (print (at a2 0)) + (print "\n")) + (push bv true) + (push bv false) + (let [db (as-dyn bv)] + (print db) + (print "\n") + (set (at db 1) true) + (print (at bv 1)) + (print "\n")) 0) (= n 1) ;; Out of range. The runtime's own message names the length. - (do (let [v (vec-new i64)] - (push v 1) - (let [dv (as-dyn v)] - (print (at dv 5)))) + (do (push v 1) + (let [dv (as-dyn v)] + (print (at dv 5))) 0) (= n 2) ;; Wrong type on write: a text where the view holds i64. Tag-checked ;; and refused, never coerced and never silently stored. - (do (let [v (vec-new i64)] - (push v 1) - (let [dv (as-dyn v)] - (set (at dv 0) "nope"))) + (do (push v 1) + (let [dv (as-dyn v)] + (set (at dv 0) "nope")) 0) :else (do (println "?") 1)))) diff --git a/test/test_dyn.ml b/test/test_dyn.ml index 0969a9c..a5c4fa1 100644 --- a/test/test_dyn.ml +++ b/test/test_dyn.ml @@ -23,7 +23,14 @@ view M2 item 3: a typed container's view, driven directly over a hand-built flan_vec header and a plain C array — the runtime half of "typed containers into dyn as views", with no - compiler in the loop + compiler in the loop. Also carries the view-aware equality + review's third finding asked for: two views, a view against + a heap vec, equal contents and differing ones + layout the three restatements of flan_vec's layout — flan_rt.c's + real one, flan_dyn.c's mirror, and this file's [hand_vec] — + compared field by field, which is what turns a struct any one + of the three reorders into a FAIL line here instead of a + silent corruption at whichever view next reads through it refuse:* twenty-four refusals, one process each, asserted on the sentence as well as on the status: a process that died some other way is not the guard firing, and the exit code cannot tell them apart @@ -31,7 +38,7 @@ mismatched write on each of the three element kinds, and a push against a flat (slice or array) view - One binary, built once, run thirty-five times. The build is the expensive + One binary, built once, run thirty-six times. The build is the expensive part and the runs are milliseconds, which is what keeps this inside `dune test` rather than behind an alias. *) @@ -142,17 +149,28 @@ let () = (* M2 item 3, the runtime's half: a flat view over a fixed C array (reads box, writes tag-check, and a write through the view is the array's own - write and vice versa — proving it is a view and not a copy), and a - Vec view over a hand-built header, pushed through twenty times so the + write and vice versa — proving it is a view and not a copy), a Vec + view over a hand-built header, pushed through twenty times so the header's own [ptr] moves under it — the case that says the descriptor pointing AT the header rather than snapshotting it is what survives a - growth. And a bool view and a float view, so the element dispatch is - exercised on all three kinds dyn_ops.c's [view] carries. *) + growth — a bool view and a float view, so the element dispatch is + exercised on all three kinds dyn_ops.c's [view] carries, and + [dyn_equal] made view-aware: two views over equal bytes, two views + over different bytes, a view against an equal heap vec and against a + differing one. *) let code, out, err = run "view" in if code <> 0 || out <> "view ok\n" then fail "a typed container's view\n got: %S (exit %d, err %S)" out code err; + (* The three restatements of flan_vec's layout, compared field by field — + see dyn_ops.c's [layout] and [hand_vec]'s comment for what ties them + together and why nothing at compile time otherwise does. *) + let code, out, err = run "layout" in + if code <> 0 || out <> "layout ok\n" then + fail "flan_vec's three restatements\n got: %S (exit %d, err %S)" + out code err; + (* Every refusal. The pair is (mode, a phrase the sentence must contain); the phrase is chosen to be the part that says *which* mistake it was, so a message that named the wrong operation or the wrong tag would not @@ -223,7 +241,7 @@ let () = (* A line on the way out, because a test that says nothing when it passes is a test nobody can tell from a test that did not run. *) if !failures = 0 then - Printf.printf " ok the dyn runtime: %d refusals and seven runs\n" + Printf.printf " ok the dyn runtime: %d refusals and eight runs\n" (List.length refusals + List.length view_refusals) else exit 1 | _ -> print_endline "SKIP test_dyn: no clang" diff --git a/test/test_flan.ml b/test/test_flan.ml index db6e3b0..a453f82 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -876,40 +876,90 @@ let () = (* M2 item 3 lifted the container-into-dyn refusal: a [(Vec T)], a slice or a fixed array with an i64/f64/bool element now crosses as a VIEW rather - than refusing. A [(Map K V)] still refuses — it rides a different + than refusing — but only when its storage is permanent, a global's, + which review added after the first landing: a view's descriptor chases + the container's own address on every operation, and a container whose + address dies with a frame is exactly the dangling dyn value the dynamic + side refuses to hand back. Every accepting row below views a global. A + [(Map K V)] still refuses regardless of storage — it rides a representation this milestone does not give a view — and so does any container whose element is outside the three the view can hold. *) accepts "a typed Vec boxed into dyn is a view, not a refusal" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [v (vec-new i64)] (take v)))"; + "(defvar v (Vec i64) (vec-new i64))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take v))"; accepts "a slice boxed into dyn is a view" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [xs (array 3 i64)] (take (slice xs 0 3))))"; + "(defvar xs [3 i64])\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take (slice xs 0 3)))"; accepts "a fixed array boxed into dyn is a view" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [a (array 4 i64)] (take a)))"; + "(defvar a [4 i64])\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take a))"; accepts "a bool Vec's view" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [v (vec-new bool)] (take v)))"; + "(defvar v (Vec bool) (vec-new bool))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take v))"; accepts "an f64 Vec's view" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [v (vec-new f64)] (take v)))"; + "(defvar v (Vec f64) (vec-new f64))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take v))"; (* The element restriction is still refused, and by name: a string element would need a dyn string's own boxing, whose payload is a pointer into the collector's heap, planted where nothing will ever trace it. *) rejects_check "a Vec of strings does not view into dyn yet" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [v (vec-new string)] (take v)))" + "(defvar v (Vec string) (vec-new string))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take v))" ~needle:"does not cross into dyn yet"; rejects_check "an i32 element is not one of the view's three" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [v (vec-new i32)] (take v)))" + "(defvar v (Vec i32) (vec-new i32))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take v))" ~needle:"does not cross into dyn yet"; (* A typed (Map K V) is unrelated to item 3 and keeps its own refusal. *) rejects_check "a typed Map still refuses into dyn" - "(defn take [d dyn] i32 1)\n\ - (defn main [] i32 (let [m (map-new i64 i64)] (take m)))" + "(defvar m (Map i64 i64) (map-new i64 i64))\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take m))" ~needle:"does not cross into dyn yet"; + (* ── The lifetime guard, added on review ───────────────────────── + A local, a parameter and a temporary all answer false to + [permanent_root], and each gets the same message rather than "cannot be + indexed" or some other accident of which path noticed. *) + rejects_check "a local Vec does not view into dyn — its frame ends" + "(defn take [d dyn] i32 1)\n\ + (defn main [] i32 (let [v (vec-new i64)] (take v)))" + ~needle:"does not cross into dyn as a view here"; + rejects_check "a Vec parameter does not view into dyn" + "(defn take [d dyn] i32 1)\n\ + (defn give [v (Vec i64)] i32 (take v))\n\ + (defn main [] i32 0)" + ~needle:"does not cross into dyn as a view here"; + rejects_check "a fixed array local does not view into dyn" + "(defn take [d dyn] i32 1)\n\ + (defn main [] i32 (let [a (array 4 i64)] (take a)))" + ~needle:"does not cross into dyn as a view here"; + (* A slice cut from a global is permanent; the same slice expression + rebound to a local first loses the trace back to it and is refused — + conservative rather than wrong, and the message says what does work. *) + accepts "a slice cut from a global inline is still permanent" + "(defvar xs [3 i64])\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (take (slice xs 0 3)))"; + rejects_check "a slice rebound to a local loses the trace and is refused" + "(defvar xs [3 i64])\n\ + (defn take [d dyn] i32 1)\n\ + (defn main [] i32 (let [s (slice xs 0 3)] (take s)))" + ~needle:"does not cross into dyn as a view here"; + (* A Vec behind a Ptr is refused even though some Ptrs really are + heap-durable — the checker cannot tell this one from a Ptr taken off a + local, and admitting one admits the other. *) + rejects_check "a Vec behind a Ptr does not view into dyn" + "(defn take [d dyn] i32 1)\n\ + (defn use [p (Ptr (Vec i64))] i32 (take (deref p)))\n\ + (defn main [] i32 0)" + ~needle:"does not cross into dyn as a view here"; (* A bracket *literal* is not a typed container yet, and where a dyn is wanted it builds the runtime's own vec instead — the lowering the map literal's values ride on, and what makes {:xs [1 2]} mean what it