flan_merged_park called flan_dyn_root_reset, which emptied the collector's root stack. The frames' roots had to go — main is left by longjmp, so they name stack the next run overwrites — but the dyn globals' roots are on that same stack, pushed once by the emitted main and never popped, and the park took them with the frames. The park is not a quiet state. It services evaluated thunks, a thunk allocates, and an allocation collects. So a program with (defvar config dyn) answered (get config :s) with its string before any thunk ran and with nil after one that allocated past the heap's floor — a read of memory the sweep had freed, answering nil by luck of what the freed words decoded as. The emitted main now brackets its global pushes: flan_dyn_root_globals_begin empties the stack, the pushes go on, flan_dyn_root_globals_end records how many of them there are, and the park resets to that line instead of to zero. Nothing between the two allocates, which is what keeps the globals from being swept in the window where they are unrooted — and [begin] emptying the stack rather than adding to it is what makes a re-entered main re-root the same globals rather than push a second copy of each, which also closes the other half: a re-run used to re-push roots over slots left dangling by the park. Both emitters, because the dev loop's default backend is x86 and a fix in one lowering is not a fix. A program with no dyn globals emits neither call and its root stack still resets to empty, which is what an empty push list should leave behind. flan_dyn_root_pop now clamps at the globals rather than at zero. An over-popping frame eating the globals is the one way that clamp could turn a miscount into this same use-after-free. Covered twice. test/dyn_ops.c's park mode is the runtime's half — a run, a park with a collecting thunk in it, and another run, three times over, asserting both that the global survives and that the frame's five hundred objects do not. Under ASan the old reset reports heap-use-after-free in flan_dyn_tag with the free in gc_sweep; under memcheck it reports 24 errors and still prints the right answer, which is the shape of the bug. test_dev.ml drives the whole daemon over its socket on both backends against programs/dev-dyn-global.flan. Not touched, and it wants a decision rather than a patch: a re-run re-enters flan_program_main, which re-runs the lifted startup function, so every global with a computed initialiser is reset by a re-run. That contradicts dev.ml's own note and FIX.org item 1. It is independent of this — the roots are right whether or not the values are re-initialised. Nor is this the reload path. A defvar added by an evaluation gets its storage from flan_dev_global (emit.ml's new_globals, x86.ml's counterpart) and there is no flan_dyn_root_push anywhere on that path in either backend, so a dyn global added to a live session is unrooted. That is a separate defect with a separate fix, and nothing here makes it better or worse.
289 lines
15 KiB
C
289 lines
15 KiB
C
/* flan_dyn — the dynamic-value runtime's ABI.
|
|
*
|
|
* This header is not compiled into a program. The build embeds the runtime's
|
|
* .c files as strings and hands each one to clang on its own, with no include
|
|
* path (see [Build.compile_c]), so runtime/flan_dyn.c declares everything it
|
|
* defines and this file declares it a second time. That is the same standing
|
|
* arrangement flan_escape_bytes and flan_dev_emit_str already live under —
|
|
* "if either table changes, change both" — and it is made mechanical rather
|
|
* than hopeful: test/dyn_ops.c includes this header and names every function
|
|
* below, so a signature that drifts from the implementation is a link error in
|
|
* `dune test` rather than a surprise at someone else's call site.
|
|
*
|
|
* Who reads it: the compiler lane, which emits calls to these names, and the
|
|
* C tests. The whole of the boundary is here. What is behind it — the value
|
|
* representation, the heap layout, the collector — is flan_dyn.c's business
|
|
* and is argued in docs/SPIKE-DYNAMIC.md.
|
|
*/
|
|
|
|
#ifndef FLAN_DYN_H
|
|
#define FLAN_DYN_H
|
|
|
|
#include <stdint.h>
|
|
|
|
#ifdef __cplusplus
|
|
extern "C" {
|
|
#endif
|
|
|
|
/* A dynamic value. One machine word, always — the whole point of the type is
|
|
* that a dyn local is a register or a stack slot and never a struct the ABI
|
|
* has to agree about. NaN-boxed; see the design doc. */
|
|
typedef uint64_t flan_dyn;
|
|
|
|
/* A type's dyn map — where the dyn words are inside one instance of it.
|
|
*
|
|
* The compiler emits one of these as static data for every type that holds a
|
|
* dyn anywhere: a struct with a dyn field, a struct holding such a struct by
|
|
* value, a fixed array of either. The offsets are flattened at compile time,
|
|
* so nesting costs nothing here — an inner struct's dyn word appears at the
|
|
* outer offset plus the inner one, and there is no walking of a type graph at
|
|
* run time and no second descriptor to follow.
|
|
*
|
|
* [size] is the stride of one instance. The collector does not read it; the
|
|
* typed-container view will, which is the reason it is here now rather than
|
|
* being added later to data both lanes already emit.
|
|
*
|
|
* Nothing in this ABI ever writes a descriptor, and no value ever points at
|
|
* one. See [flan_dyn_root_push_desc]. */
|
|
typedef struct flan_desc {
|
|
int64_t size;
|
|
int64_t n;
|
|
const int64_t *offs;
|
|
} flan_desc;
|
|
|
|
/* ── Constructors ──────────────────────────────────────────────────── */
|
|
|
|
flan_dyn flan_dyn_nil(void);
|
|
flan_dyn flan_dyn_from_i64(int64_t x);
|
|
flan_dyn flan_dyn_from_f64(double x);
|
|
flan_dyn flan_dyn_from_bool(uint8_t b);
|
|
|
|
/* Copies the bytes into the GC heap. The result is a text value, immutable
|
|
* from that moment: nothing in this ABI writes into one. [p] may point
|
|
* anywhere — a literal in .rodata, a frame slot, a slice the caller is about
|
|
* to drop — because the bytes are copied before this returns. */
|
|
flan_dyn flan_dyn_from_bytes(const uint8_t *p, int64_t n);
|
|
|
|
flan_dyn flan_dyn_vec_new(void);
|
|
flan_dyn flan_dyn_map_new(void);
|
|
|
|
/* A keyword: :foo as a run-time value. Interned — the runtime keeps one entry
|
|
* per distinct name forever, so two keywords with the same bytes are the same
|
|
* word and equality is an identity compare, never a memcmp. The entries are
|
|
* immortal by construction and the collector never traces or frees one.
|
|
* [p] may point anywhere; the bytes are copied on the first interning. The
|
|
* name is the bytes after the colon: flan_dyn_kw("a", 1) is :a. */
|
|
flan_dyn flan_dyn_kw(const uint8_t *p, int64_t n);
|
|
|
|
/* ── Operations ────────────────────────────────────────────────────────
|
|
*
|
|
* Every one of these may trap, and a trap does not return: it prints a
|
|
* sentence naming the operation, the tags it was given and the values, and
|
|
* then takes flan_rt.c's [flan_trap] — which parks the program for inspection
|
|
* in a dev session and ends it in a standalone build. The three that cannot
|
|
* trap say so on their own line. */
|
|
|
|
flan_dyn flan_dyn_add(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_sub(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_mul(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_div(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_rem(flan_dyn a, flan_dyn b);
|
|
|
|
/* Answer a bool dyn. Numbers compare as numbers and text compares bytewise;
|
|
* a mixture of the two, or anything else, traps. */
|
|
flan_dyn flan_dyn_lt(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_le(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_gt(flan_dyn a, flan_dyn b);
|
|
flan_dyn flan_dyn_ge(flan_dyn a, flan_dyn b);
|
|
|
|
/* Structural, and the one operation in this file that never traps: two values
|
|
* of unrelated tags are not an error, they are unequal. */
|
|
flan_dyn flan_dyn_eq(flan_dyn a, flan_dyn b);
|
|
|
|
/* Bytes of a text, elements of a vec. Anything else traps. */
|
|
flan_dyn flan_dyn_len(flan_dyn v);
|
|
|
|
/* Element of a vec, or the byte of a text as an int. Out of range traps. */
|
|
flan_dyn flan_dyn_at(flan_dyn v, flan_dyn i);
|
|
|
|
/* Vec only — a text is immutable and says so rather than being copied. */
|
|
void flan_dyn_set_at(flan_dyn v, flan_dyn i, flan_dyn x);
|
|
void flan_dyn_push(flan_dyn v, flan_dyn x);
|
|
|
|
/* Map only; anything else traps by name. Keys and values are both dyn and a
|
|
* key is compared structurally, so a keyword, a text, an int, or a whole map
|
|
* may key one. [get] on an absent key answers nil — absence is an answer, the
|
|
* same line [eq] takes about unrelated tags — and [contains] is the question
|
|
* to ask when nil might also be stored. [set] replaces the value of an equal
|
|
* key in place, so a key occurs once and insertion order is print order. */
|
|
flan_dyn flan_dyn_map_get(flan_dyn m, flan_dyn k);
|
|
void flan_dyn_map_set(flan_dyn m, flan_dyn k, flan_dyn v);
|
|
flan_dyn flan_dyn_map_contains(flan_dyn m, flan_dyn k);
|
|
|
|
/* Structural, and per type it renders what typed [print] renders. Never
|
|
* traps: every tag has a rendering, including nil. */
|
|
void flan_dyn_print(flan_dyn v);
|
|
|
|
/* ── The typed boundary ────────────────────────────────────────────────
|
|
*
|
|
* What an annotated parameter does with a dyn argument, and what a dyn
|
|
* expression does where the checker wants a machine value. A tag that is not
|
|
* the one asked for traps; there is no widening and no coercion here, which
|
|
* is deliberate — see the doc's boundary section. */
|
|
|
|
int64_t flan_dyn_need_i64(flan_dyn v);
|
|
double flan_dyn_need_f64(flan_dyn v);
|
|
uint8_t flan_dyn_need_bool(flan_dyn v);
|
|
|
|
/* nil <-> None at an (Option T) boundary, and (Some nil)'s refusal — M2 item
|
|
* 4. [flan_dyn_is_nil] is the tag test the boundary's runtime half needs and
|
|
* does not want to build out of [flan_dyn_tag] and a comparison at every call
|
|
* site; it answers 1 for nil and 0 for every other tag, and cannot trap.
|
|
* [flan_dyn_need_not_nil] is the other half: it answers [v] unchanged when
|
|
* [v] is not nil, and traps when it is — the run-time case of (Some nil),
|
|
* for a dyn value that is not known to be nil until the program runs. */
|
|
int32_t flan_dyn_is_nil(flan_dyn v);
|
|
flan_dyn flan_dyn_need_not_nil(flan_dyn v);
|
|
|
|
/* Truthiness for a dyn used where a typed value would need a strict bool —
|
|
* an [if]'s condition when the scrutinee's own type is dyn. Clojure's rule:
|
|
* nil and false are falsey, every other value is truthy, including 0, 0.0,
|
|
* "", an empty vec, an empty map, and any keyword. Never traps. */
|
|
uint8_t flan_dyn_truthy(flan_dyn v);
|
|
|
|
/* ── The collector ─────────────────────────────────────────────────────
|
|
*
|
|
* Mark-sweep, precise, and never moving. [flan_gc_init] is idempotent, and the
|
|
* first allocation calls it if nobody else has — deliberately, because
|
|
* flan_rt_init calling it would be flan_rt.c naming a symbol in flan_dyn.c,
|
|
* and the whole of the droppability argument is that the dependency runs one
|
|
* way only. So an emitted program need not call it at all; it is exported
|
|
* because a test that wants a heap in a known state wants to say so.
|
|
*
|
|
* [flan_gc_collect] is a full collection on demand, which nothing in an
|
|
* emitted program needs — collection happens inside allocation — and which
|
|
* the tests and a break loop want. [flan_gc_live_bytes] is what the heap holds
|
|
* after the last sweep, counted the way the trigger counts it. */
|
|
|
|
void flan_gc_init(void);
|
|
void flan_gc_collect(void);
|
|
int64_t flan_gc_live_bytes(void);
|
|
|
|
/* Roots, shadow-stack style, exactly as flan_dev.c's frame chain is: the
|
|
* compiler emits a push per dyn local on entry and one pop for the lot on the
|
|
* way out. The address is remembered, not the value, so a local that is
|
|
* reassigned needs no second push.
|
|
*
|
|
* **The slot must hold a valid flan_dyn before it is pushed.** The collector
|
|
* reads every registered address on every mark, and an uninitialised slot is a
|
|
* word of stack garbage that will be decoded as a pointer. Storing nil first
|
|
* is the whole of the contract; the compiler lane zeroes a slot at its
|
|
* declaration anyway.
|
|
*
|
|
* A zeroed slot satisfies it, and this used to be the one thing in this header
|
|
* decided by one side alone. It is checkable now that both sides exist:
|
|
* flan_dyn.c's mark walks a value only when it is boxed, and boxed means the
|
|
* quiet-NaN prefix is set, which the zero word does not have. So zero decodes
|
|
* as the double 0.0 — an ordinary value, and never an address anything
|
|
* follows. Both backends zero, and they are right to.
|
|
*
|
|
* Globals go through the same pair, pushed once at startup and never popped.
|
|
*
|
|
* [flan_dyn_root_pop] takes a count rather than an address because that is
|
|
* what a function epilogue knows cheaply. Popping more than are pushed is
|
|
* clamped at empty rather than being a second failure on top of the first. */
|
|
void flan_dyn_root_push(flan_dyn *slot);
|
|
void flan_dyn_root_pop(int64_t n);
|
|
|
|
/* The same stack, for a slot that holds an aggregate rather than a dyn word:
|
|
* a struct with a dyn field, a struct holding one of those by value, an array
|
|
* of either. [base] is the first byte of the instance and [d] says where the
|
|
* dyn words are inside it.
|
|
*
|
|
* The question this answers, and it is the only interesting one about the
|
|
* whole mechanism: how does the collector get from a run of bytes to the
|
|
* descriptor for the type at those bytes? It does not. **The instance never
|
|
* carries a pointer to its descriptor, and the collector never derives one.**
|
|
* The pairing is made here, at the push, by the code that put the value there
|
|
* and therefore knows its static type. That is what makes a bare struct on the
|
|
* stack the easy case rather than the impossible one, and it is why no Flan
|
|
* struct grows a header word: a header would change the layout C interop
|
|
* agrees on, change the stride of an array, and change what embedding a struct
|
|
* in another one costs.
|
|
*
|
|
* The same contract as the dyn form: **the dyn words named by [d] must hold
|
|
* valid flan_dyn values before the push**, which zero satisfies. The compiler
|
|
* zeroes those words and not the whole instance — the rest of the bytes are
|
|
* never read through this stack.
|
|
*
|
|
* One entry, so one pop takes it off like any other, and a function's pop
|
|
* count is still the number of pushes it made. [d] is static data with the
|
|
* lifetime of the program; nothing copies it. */
|
|
void flan_dyn_root_push_desc(void *base, const flan_desc *d);
|
|
|
|
/* ── Extensions ────────────────────────────────────────────────────────
|
|
*
|
|
* Additions to the agreed ABI, none of which the compiler lane has to emit.
|
|
* They are here because something in this repository needs them; each says
|
|
* what. */
|
|
|
|
/* The *frames*' roots, dropped. The counterpart of flan_dev.c's
|
|
* [flan_dev_frames_reset] and there for its one caller: the merged dev build's
|
|
* [main] is re-entered by longjmp, which pops no frame, so every root the
|
|
* finished run pushed still points into stack the next run is about to write
|
|
* over. Marking through those addresses would decode whatever the new run put
|
|
* there. Called between runs, on the thread that runs them.
|
|
*
|
|
* The globals' roots are not dropped with them, and that is the whole of the
|
|
* distinction: a finished run's frames are gone but its globals are not — the
|
|
* dev daemon parks with them readable, and runs evaluated thunks against them
|
|
* that allocate and therefore collect. Emptying the stack outright unrooted
|
|
* every dyn global for the whole of the park. This resets to the line
|
|
* [flan_dyn_root_globals_end] recorded. */
|
|
void flan_dyn_root_reset(void);
|
|
|
|
/* Where that line comes from. The emitted [main] brackets its global pushes
|
|
* with these: [begin] immediately before the first, [end] immediately after
|
|
* the last, with nothing but the pushes in between — nothing there may
|
|
* allocate, because between the two the globals are unrooted and still hold
|
|
* whatever a previous run left in them. [begin] empties the stack rather than
|
|
* adding to it, so a [main] entered a second time re-roots the same globals
|
|
* instead of pushing a second copy of each.
|
|
*
|
|
* A program with no dyn globals need not call either: the line starts at zero,
|
|
* which is what an empty push list should leave it at. */
|
|
void flan_dyn_root_globals_begin(void);
|
|
void flan_dyn_root_globals_end(void);
|
|
|
|
/* The tag of a value, as a number and as the word that number is printed as.
|
|
* The numbers are FLAN_DYN_TAG_* below. The break loop and the inspector want
|
|
* both — a value's tag is the first thing anyone asks a stopped dyn program —
|
|
* and the trap messages in flan_dyn.c are written from the same table, so a
|
|
* message and an inspector cannot disagree about what to call a value. */
|
|
#define FLAN_DYN_TAG_NIL 0
|
|
#define FLAN_DYN_TAG_BOOL 1
|
|
#define FLAN_DYN_TAG_INT 2
|
|
#define FLAN_DYN_TAG_FLOAT 3
|
|
#define FLAN_DYN_TAG_TEXT 4
|
|
#define FLAN_DYN_TAG_VEC 5
|
|
#define FLAN_DYN_TAG_KEYWORD 6
|
|
#define FLAN_DYN_TAG_MAP 7
|
|
|
|
int32_t flan_dyn_tag(flan_dyn v);
|
|
const char *flan_dyn_tag_name(int32_t tag);
|
|
|
|
/* How many objects the heap holds, and the floor under the collection
|
|
* trigger. Both are the tests': a live-bytes figure alone cannot tell a heap
|
|
* that is collecting from one whose objects happen to be small, and a floor of
|
|
* a megabyte would make the million-allocation case a megabyte of arithmetic
|
|
* before it proved anything. Setting the floor takes effect at the next
|
|
* allocation and never shrinks a heap by itself; pass 0 for the default. */
|
|
int64_t flan_gc_count(void);
|
|
void flan_gc_set_floor(int64_t bytes);
|
|
|
|
#ifdef __cplusplus
|
|
}
|
|
#endif
|
|
|
|
#endif /* FLAN_DYN_H */
|