Joseph Ferano
3f7c42257f
Relocation was proved sound and stayed sound — a Vec view holding the header's own address survives a push that grows and moves it, because there is no snapshot to invalidate. That was never the whole of the hazard. Refusing every container into dyn outright, before this lane, meant a dangling view was unreachable; the moment box stopped refusing, three routes opened at once — a view returned from the function whose frame the Vec lived in, one stashed in a dyn global and read after that frame is gone, and one left behind when a condition transfer unwinds it. All three are stack-use-after-return, reachable for the first time. The rule: a typed container crosses into dyn as a view only when its own storage is permanent — a global's. On the dynamic side Flan follows Clojure and Common Lisp, where holding a value can never hand you garbage; treating a view as a bare pointer and calling the lifetime the programmer's problem is the Odin answer, and it is the wrong trade on this side of the language. check.ml's permanent_root walks the checked expression back to its root: a global is permanent, a field or an array element of one is permanent at the same fixed offset, and a slice cut directly from one at the call site inherits it — the trace is what a slice carries, and it is lost the moment the slice is bound to a name first, so that case is refused too rather than guessed at. Everything else answers false: a local, a parameter, a temporary, and anything reached through a (Ptr T), because a heap-durable pointer and a frame's own are the same type and the checker cannot tell them apart — admitting one admits the other, which is the whole hazard this closes. An arena-held header turns out not to be a separate case at all: an arena changes where a Vec's elements live, never where its own header — the binding — lives, so it is already covered by the storage-class check above. Both directions of the F1 escape were reproduced before the fix (a genuine ASan stack-use-after-return, reproduced by building the pre-fix tree) and confirmed refused at check time after it, for all three routes. Three more findings, all in the runtime rather than the boundary: view_vec_check, on finding a stale container, rendered the very view it had just declared unsafe to read — which called back into the same check, unconditionally, an infinite recursion rather than the intended trap. Fixed by never rendering the container in the stale message at all; the sentence names the two epochs and nothing else, which is everything a reader needs and the one thing that was safe to read. dyn_equal's VEC arm read x->len and x->u.v.items regardless of kind, which for a view answers 0 and the union's other member reinterpreted as dyn words: two views with different contents compared equal, a view and an equal heap vec compared unequal, and a map keyed by any view collided with every other view, silently. vecish_len and vecish_at read either shape correctly and the arm now goes through them. obj_words gets the same explicit OBJ_VIEW case on the same reasoning, unreachable today only because mark_push's own gate already excludes the kind — this is the belt next to that brace. The three restatements of flan_vec's layout — flan_rt.c's real struct, flan_dyn.c's mirror, and dyn_ops.c's hand-built one — had a comment claiming a reorder would not compile or link, which was never true of a void*-typed forward declaration. flan_vec_layout and flan_dyn_vec_hdr_layout each report their struct's size and field offsets; dyn_ops.c's new "layout" mode compares both against offsetof on its own hand_vec, so a disagreement is a FAIL line in dune test instead of a silent corruption at whichever view reads through the wrong offset next. Also: the survey program's comment excusing a by-value parameter's view as "value semantics, not a hole" was wrong on its own terms — a write through such a view does reach the caller's storage, only growth diverges — but the question is moot now: every container the program views is a global, and the file was rewritten around that rather than patched. And an i32 element does not cross into a view either, but the refusal used to say why in words that were true only of a string element; it now says what i32 actually is and what the restriction is actually for. Rebased onto dev-loop's item-4 landing (221df5a).
325 lines
17 KiB
C
325 lines
17 KiB
C
/* flan_dyn — the dynamic-value runtime's ABI.
|
|
*
|
|
* This header is not compiled into a program. The build embeds the runtime's
|
|
* .c files as strings and hands each one to clang on its own, with no include
|
|
* path (see [Build.compile_c]), so runtime/flan_dyn.c declares everything it
|
|
* defines and this file declares it a second time. That is the same standing
|
|
* arrangement flan_escape_bytes and flan_dev_emit_str already live under —
|
|
* "if either table changes, change both" — and it is made mechanical rather
|
|
* than hopeful: test/dyn_ops.c includes this header and names every function
|
|
* below, so a signature that drifts from the implementation is a link error in
|
|
* `dune test` rather than a surprise at someone else's call site.
|
|
*
|
|
* Who reads it: the compiler lane, which emits calls to these names, and the
|
|
* C tests. The whole of the boundary is here. What is behind it — the value
|
|
* representation, the heap layout, the collector — is flan_dyn.c's business
|
|
* and is argued in docs/SPIKE-DYNAMIC.md.
|
|
*/
|
|
|
|
#ifndef FLAN_DYN_H
|
|
#define FLAN_DYN_H
|
|
|
|
#include <stdint.h>
|
|
|
|
#ifdef __cplusplus
|
|
extern "C" {
|
|
#endif
|
|
|
|
/* A dynamic value. One machine word, always — the whole point of the type is
|
|
* that a dyn local is a register or a stack slot and never a struct the ABI
|
|
* has to agree about. NaN-boxed; see the design doc. */
|
|
typedef uint64_t flan_dyn;
|
|
|
|
/* A type's dyn map — where the dyn words are inside one instance of it.
|
|
*
|
|
* The compiler emits one of these as static data for every type that holds a
|
|
* dyn anywhere: a struct with a dyn field, a struct holding such a struct by
|
|
* value, a fixed array of either. The offsets are flattened at compile time,
|
|
* so nesting costs nothing here — an inner struct's dyn word appears at the
|
|
* outer offset plus the inner one, and there is no walking of a type graph at
|
|
* run time and no second descriptor to follow.
|
|
*
|
|
* [size] is the stride of one instance. The collector does not read it; the
|
|
* typed-container view will, which is the reason it is here now rather than
|
|
* being added later to data both lanes already emit.
|
|
*
|
|
* Nothing in this ABI ever writes a descriptor, and no value ever points at
|
|
* one. See [flan_dyn_root_push_desc]. */
|
|
typedef struct flan_desc {
|
|
int64_t size;
|
|
int64_t n;
|
|
const int64_t *offs;
|
|
} flan_desc;
|
|
|
|
/* ── Constructors ──────────────────────────────────────────────────── */
|
|
|
|
flan_dyn flan_dyn_nil(void);
|
|
flan_dyn flan_dyn_from_i64(int64_t x);
|
|
flan_dyn flan_dyn_from_f64(double x);
|
|
flan_dyn flan_dyn_from_bool(uint8_t b);
|
|
|
|
/* Copies the bytes into the GC heap. The result is a text value, immutable
|
|
* from that moment: nothing in this ABI writes into one. [p] may point
|
|
* anywhere — a literal in .rodata, a frame slot, a slice the caller is about
|
|
* to drop — because the bytes are copied before this returns. */
|
|
flan_dyn flan_dyn_from_bytes(const uint8_t *p, int64_t n);
|
|
|
|
flan_dyn flan_dyn_vec_new(void);
|
|
flan_dyn flan_dyn_map_new(void);
|
|
|
|
/* A keyword: :foo as a run-time value. Interned — the runtime keeps one entry
|
|
* per distinct name forever, so two keywords with the same bytes are the same
|
|
* word and equality is an identity compare, never a memcmp. The entries are
|
|
* immortal by construction and the collector never traces or frees one.
|
|
* [p] may point anywhere; the bytes are copied on the first interning. The
|
|
* name is the bytes after the colon: flan_dyn_kw("a", 1) is :a. */
|
|
flan_dyn flan_dyn_kw(const uint8_t *p, int64_t n);
|
|
|
|
/* ── Operations ────────────────────────────────────────────────────────
|
|
*
|
|
* Every one of these may trap, and a trap does not return: it prints a
|
|
* sentence naming the operation, the tags it was given and the values, and
|
|
* then takes flan_rt.c's [flan_trap] — which parks the program for inspection
|
|
* in a dev session and ends it in a standalone build. The three that cannot
|
|
* trap say so on their own line. */
|
|
|
|
flan_dyn flan_dyn_add(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_sub(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_mul(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_div(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_rem(flan_dyn a, flan_dyn b);
|
|
|
|
/* Answer a bool dyn. Numbers compare as numbers and text compares bytewise;
|
|
* a mixture of the two, or anything else, traps. */
|
|
flan_dyn flan_dyn_lt(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_le(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_gt(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_ge(flan_dyn a, flan_dyn b);
|
|
|
|
/* Structural, and the one operation in this file that never traps: two values
|
|
* of unrelated tags are not an error, they are unequal. */
|
|
flan_dyn flan_dyn_eq(flan_dyn a, flan_dyn b);
|
|
|
|
/* Bytes of a text, elements of a vec. Anything else traps. */
|
|
flan_dyn flan_dyn_len(flan_dyn v);
|
|
|
|
/* Element of a vec, or the byte of a text as an int. Out of range traps. */
|
|
flan_dyn flan_dyn_at(flan_dyn v, flan_dyn i);
|
|
|
|
/* Vec only — a text is immutable and says so rather than being copied. */
|
|
void flan_dyn_set_at(flan_dyn v, flan_dyn i, flan_dyn x);
|
|
void flan_dyn_push(flan_dyn v, flan_dyn x);
|
|
|
|
/* Map only; anything else traps by name. Keys and values are both dyn and a
|
|
* key is compared structurally, so a keyword, a text, an int, or a whole map
|
|
* may key one. [get] on an absent key answers nil — absence is an answer, the
|
|
* same line [eq] takes about unrelated tags — and [contains] is the question
|
|
* to ask when nil might also be stored. [set] replaces the value of an equal
|
|
* key in place, so a key occurs once and insertion order is print order. */
|
|
flan_dyn flan_dyn_map_get(flan_dyn m, flan_dyn k);
|
|
void flan_dyn_map_set(flan_dyn m, flan_dyn k, flan_dyn v);
|
|
flan_dyn flan_dyn_map_contains(flan_dyn m, flan_dyn k);
|
|
|
|
/* Structural, and per type it renders what typed [print] renders. Never
|
|
* traps: every tag has a rendering, including nil. */
|
|
void flan_dyn_print(flan_dyn v);
|
|
|
|
/* ── The typed boundary ────────────────────────────────────────────────
|
|
*
|
|
* What an annotated parameter does with a dyn argument, and what a dyn
|
|
* expression does where the checker wants a machine value. A tag that is not
|
|
* the one asked for traps; there is no widening and no coercion here, which
|
|
* is deliberate — see the doc's boundary section. */
|
|
|
|
int64_t flan_dyn_need_i64(flan_dyn v);
|
|
double flan_dyn_need_f64(flan_dyn v);
|
|
uint8_t flan_dyn_need_bool(flan_dyn v);
|
|
|
|
/* nil <-> None at an (Option T) boundary, and (Some nil)'s refusal — M2 item
|
|
* 4. [flan_dyn_is_nil] is the tag test the boundary's runtime half needs and
|
|
* does not want to build out of [flan_dyn_tag] and a comparison at every call
|
|
* site; it answers 1 for nil and 0 for every other tag, and cannot trap.
|
|
* [flan_dyn_need_not_nil] is the other half: it answers [v] unchanged when
|
|
* [v] is not nil, and traps when it is — the run-time case of (Some nil),
|
|
* for a dyn value that is not known to be nil until the program runs. */
|
|
int32_t flan_dyn_is_nil(flan_dyn v);
|
|
flan_dyn flan_dyn_need_not_nil(flan_dyn v);
|
|
|
|
/* ── Typed containers as views — M2 item 3 ─────────────────────────────
|
|
*
|
|
* A [(Vec T)], a [T] slice, or a fixed [n T] array crossing into dyn is a
|
|
* VIEW, not a copy: the box holds a small heap record naming where the
|
|
* elements live and what one of them is, and every read or write goes
|
|
* straight through to the container's own storage. [flan_dyn_at] boxes an
|
|
* element on the way out; [flan_dyn_set_at] tag-checks the dyn value it is
|
|
* given against the element type on the way in and traps, by [flan_trap],
|
|
* on a mismatch — never a silent coercion.
|
|
*
|
|
* T is restricted to i64, f64 and bool — exactly the set [flan_dyn_need_i64]
|
|
* and friends already treat as crossing the typed boundary both ways. That
|
|
* is not an arbitrary cut: the excluded case that matters is a string
|
|
* element, whose dyn form is a pointer into this collector's heap, while a
|
|
* typed container's storage is arena or stack memory the collector never
|
|
* scans. Writing such a pointer into that memory would be a live reference
|
|
* nothing ever traces — a use-after-free the collector cannot see coming,
|
|
* not a bug in this file but a hazard the type admits. i64, f64 and bool
|
|
* carry no such pointer, so a view restricted to them cannot manufacture
|
|
* it. [box] in lib/check.ml keeps the "does not cross into dyn yet" refusal
|
|
* for every other element type, and this paragraph is why.
|
|
*
|
|
* Two kinds, because the containers split exactly here: a [(Vec T)] can grow
|
|
* and move (a push may reallocate), a slice and a fixed array cannot.
|
|
*
|
|
* [flan_dyn_view_vec] takes the address of the Vec's own header — the
|
|
* struct [flan_vec] in flan_rt.c, restated in flan_dyn.c under the same
|
|
* "if either table changes, change both" rule this whole boundary already
|
|
* lives under. That address is the Vec's home: a local's slot, a global, a
|
|
* field, fixed for as long as the Vec exists. Every operation re-reads that
|
|
* header's [ptr] and [len] fresh, so a push that grows and moves the Vec is
|
|
* never seen as stale — [flan_vec_grow] overwrites the SAME header's [ptr]
|
|
* field in place, and there is no snapshot anywhere to go stale. That is
|
|
* what makes the failure the open design question worried about
|
|
* (a push through dyn holding a dangling pointer) impossible rather than
|
|
* merely unlikely: there is nothing captured at the crossing for a later
|
|
* push to invalidate.
|
|
*
|
|
* [flan_dyn_view_flat] takes a data address and a length captured once, at
|
|
* the crossing — sound for a slice and for a fixed array because neither
|
|
* ever moves or grows. Note the asymmetry is not an oversight: pointing
|
|
* *this* case at the value's own slot instead would be worse than a
|
|
* snapshot, because a slot's lifetime is not the slice's, and a slice taken
|
|
* from a Vec is already one push away from dangling on its own account
|
|
* (flan_vec_grow's own comment says so) — the view is exactly as
|
|
* stale-safe as the thing it is a view of, no more and no less.
|
|
*/
|
|
#define FLAN_VIEW_I64 0
|
|
#define FLAN_VIEW_F64 1
|
|
#define FLAN_VIEW_BOOL 2
|
|
|
|
flan_dyn flan_dyn_view_vec(void *hdr, int32_t elem);
|
|
flan_dyn flan_dyn_view_flat(void *data, int64_t len, int32_t elem);
|
|
|
|
/* ── The collector ─────────────────────────────────────────────────────
|
|
*
|
|
* Mark-sweep, precise, and never moving. [flan_gc_init] is idempotent, and the
|
|
* first allocation calls it if nobody else has — deliberately, because
|
|
* flan_rt_init calling it would be flan_rt.c naming a symbol in flan_dyn.c,
|
|
* and the whole of the droppability argument is that the dependency runs one
|
|
* way only. So an emitted program need not call it at all; it is exported
|
|
* because a test that wants a heap in a known state wants to say so.
|
|
*
|
|
* [flan_gc_collect] is a full collection on demand, which nothing in an
|
|
* emitted program needs — collection happens inside allocation — and which
|
|
* the tests and a break loop want. [flan_gc_live_bytes] is what the heap holds
|
|
* after the last sweep, counted the way the trigger counts it. */
|
|
|
|
void flan_gc_init(void);
|
|
void flan_gc_collect(void);
|
|
int64_t flan_gc_live_bytes(void);
|
|
|
|
/* Roots, shadow-stack style, exactly as flan_dev.c's frame chain is: the
|
|
* compiler emits a push per dyn local on entry and one pop for the lot on the
|
|
* way out. The address is remembered, not the value, so a local that is
|
|
* reassigned needs no second push.
|
|
*
|
|
* **The slot must hold a valid flan_dyn before it is pushed.** The collector
|
|
* reads every registered address on every mark, and an uninitialised slot is a
|
|
* word of stack garbage that will be decoded as a pointer. Storing nil first
|
|
* is the whole of the contract; the compiler lane zeroes a slot at its
|
|
* declaration anyway.
|
|
*
|
|
* A zeroed slot satisfies it, and this used to be the one thing in this header
|
|
* decided by one side alone. It is checkable now that both sides exist:
|
|
* flan_dyn.c's mark walks a value only when it is boxed, and boxed means the
|
|
* quiet-NaN prefix is set, which the zero word does not have. So zero decodes
|
|
* as the double 0.0 — an ordinary value, and never an address anything
|
|
* follows. Both backends zero, and they are right to.
|
|
*
|
|
* Globals go through the same pair, pushed once at startup and never popped.
|
|
*
|
|
* [flan_dyn_root_pop] takes a count rather than an address because that is
|
|
* what a function epilogue knows cheaply. Popping more than are pushed is
|
|
* clamped at empty rather than being a second failure on top of the first. */
|
|
void flan_dyn_root_push(flan_dyn *slot);
|
|
void flan_dyn_root_pop(int64_t n);
|
|
|
|
/* The same stack, for a slot that holds an aggregate rather than a dyn word:
|
|
* a struct with a dyn field, a struct holding one of those by value, an array
|
|
* of either. [base] is the first byte of the instance and [d] says where the
|
|
* dyn words are inside it.
|
|
*
|
|
* The question this answers, and it is the only interesting one about the
|
|
* whole mechanism: how does the collector get from a run of bytes to the
|
|
* descriptor for the type at those bytes? It does not. **The instance never
|
|
* carries a pointer to its descriptor, and the collector never derives one.**
|
|
* The pairing is made here, at the push, by the code that put the value there
|
|
* and therefore knows its static type. That is what makes a bare struct on the
|
|
* stack the easy case rather than the impossible one, and it is why no Flan
|
|
* struct grows a header word: a header would change the layout C interop
|
|
* agrees on, change the stride of an array, and change what embedding a struct
|
|
* in another one costs.
|
|
*
|
|
* The same contract as the dyn form: **the dyn words named by [d] must hold
|
|
* valid flan_dyn values before the push**, which zero satisfies. The compiler
|
|
* zeroes those words and not the whole instance — the rest of the bytes are
|
|
* never read through this stack.
|
|
*
|
|
* One entry, so one pop takes it off like any other, and a function's pop
|
|
* count is still the number of pushes it made. [d] is static data with the
|
|
* lifetime of the program; nothing copies it. */
|
|
void flan_dyn_root_push_desc(void *base, const flan_desc *d);
|
|
|
|
/* ── Extensions ────────────────────────────────────────────────────────
|
|
*
|
|
* Additions to the agreed ABI, none of which the compiler lane has to emit.
|
|
* They are here because something in this repository needs them; each says
|
|
* what. */
|
|
|
|
/* The root stack, emptied. The counterpart of flan_dev.c's
|
|
* [flan_dev_frames_reset] and there for its one caller: the merged dev build's
|
|
* [main] is re-entered by longjmp, which pops no frame, so every root the
|
|
* finished run pushed still points into stack the next run is about to write
|
|
* over. Marking through those addresses would decode whatever the new run put
|
|
* there. Called between runs, on the thread that runs them. */
|
|
void flan_dyn_root_reset(void);
|
|
|
|
/* The tag of a value, as a number and as the word that number is printed as.
|
|
* The numbers are FLAN_DYN_TAG_* below. The break loop and the inspector want
|
|
* both — a value's tag is the first thing anyone asks a stopped dyn program —
|
|
* and the trap messages in flan_dyn.c are written from the same table, so a
|
|
* message and an inspector cannot disagree about what to call a value. */
|
|
#define FLAN_DYN_TAG_NIL 0
|
|
#define FLAN_DYN_TAG_BOOL 1
|
|
#define FLAN_DYN_TAG_INT 2
|
|
#define FLAN_DYN_TAG_FLOAT 3
|
|
#define FLAN_DYN_TAG_TEXT 4
|
|
#define FLAN_DYN_TAG_VEC 5
|
|
#define FLAN_DYN_TAG_KEYWORD 6
|
|
#define FLAN_DYN_TAG_MAP 7
|
|
|
|
int32_t flan_dyn_tag(flan_dyn v);
|
|
const char *flan_dyn_tag_name(int32_t tag);
|
|
|
|
/* How many objects the heap holds, and the floor under the collection
|
|
* trigger. Both are the tests': a live-bytes figure alone cannot tell a heap
|
|
* that is collecting from one whose objects happen to be small, and a floor of
|
|
* a megabyte would make the million-allocation case a megabyte of arithmetic
|
|
* before it proved anything. Setting the floor takes effect at the next
|
|
* allocation and never shrinks a heap by itself; pass 0 for the default. */
|
|
int64_t flan_gc_count(void);
|
|
void flan_gc_set_floor(int64_t bytes);
|
|
|
|
/* Reports flan_dyn.c's own mirror of flan_rt.c's [flan_vec] — [size, then
|
|
* the offset of ptr, len, cap, alloc, epoch] — for test/dyn_ops.c's
|
|
* "layout" mode to compare against flan_rt.c's [flan_vec_layout] and
|
|
* against its own hand-built mirror. See [flan_vec_layout]'s comment in
|
|
* flan_rt.c for what this ties together and why nothing at compile time
|
|
* otherwise does. */
|
|
void flan_dyn_vec_hdr_layout(int64_t out[6]);
|
|
|
|
#ifdef __cplusplus
|
|
}
|
|
#endif
|
|
|
|
#endif /* FLAN_DYN_H */
|