693 lines
38 KiB
Markdown
693 lines
38 KiB
Markdown
# Spec 1 — Ownership, containers, and copies
|
|
|
|
Status: **frozen**, with one amendment: **the repeal of 2026-09-18** (see "The
|
|
repeal", below), which removed the static flow analysis — use-after-move and
|
|
double-free are no longer compile errors. Everything structural in this
|
|
document still governs. Closes plan.org open decisions #6 and #10, and resolves the
|
|
contradiction between "value structs copy on assignment" and owning containers.
|
|
The Allocators section additionally settles the four things that had to be
|
|
decided before `Vec` and `Map` are written: when storage is released, the `drop`
|
|
hook, alignment, and allocation failure. One question there is left open on
|
|
purpose and says so.
|
|
Everything else in the design references this vocabulary. It governs plain
|
|
fixed-layout `struct` values, not the separately planned managed `class`
|
|
facility (see plan.org, "Managed classes").
|
|
|
|
## The four container types
|
|
|
|
| Notation | Layout | Assignment | Owns storage | Allocator |
|
|
|-----------|-------------------|------------|--------------|-----------|
|
|
| `[n T]` | n contiguous `T` | copies | no (inline) | — |
|
|
| `[T]` | ptr + len | copies the *view* | no | — |
|
|
| `(Vec T)` | ptr + len + cap | **moves** | yes | stored |
|
|
| `(Map K V)` | open-addressed, flat key/value arrays | **moves** | yes | stored |
|
|
|
|
- `[n T]` is a value. It lives wherever it is declared, copies on assignment and
|
|
on pass-by-value, and is what `defconst colors [4 u32] ...` and
|
|
`(defvar grid [rows [cols u32]] ...)` are.
|
|
- `[T]` is a **non-owning slice**: a borrowed window into a `[n T]`, a `(Vec T)`,
|
|
or a literal in read-only memory. Copying a slice copies ptr+len, never the
|
|
elements. A slice may be `const`-qualified; freeing through one is not possible
|
|
because a slice has no allocator and no `cap`.
|
|
- `(Vec T)` and `(Map K V)` are **move-only**. Assignment hands over the one
|
|
header rather than copying it — there is no implicit shallow copy, so two
|
|
owners never arise from an assignment; `(clone x)` is the only spelling of a
|
|
second, independent one. Since the repeal, using the source binding again is
|
|
not a compile error: the header is still there, and a program that frees
|
|
through it twice or reads through it after a free misbehaves at run time,
|
|
where the allocator and the dev build's generation word are the net.
|
|
|
|
## Maps — first implementation
|
|
|
|
Every map is homogeneous: `(Map K V)` has one key type and one value type. The
|
|
first implementation accepts only built-in structural key types: integers,
|
|
enums, strings, fixed arrays, and value structs composed recursively from those
|
|
types. Tuples and triples join that set when they are introduced. `Ptr`, slices,
|
|
`Vec`, and `Map` are not map keys yet.
|
|
|
|
Equality and hashing for those keys are compiler-provided structural operations
|
|
and not type classes. They are not available to an unconstrained type variable
|
|
either; a variable that means to key a map declares `hashable?` in the signature
|
|
that binds it, and the refusal then lands at the call site that names an
|
|
unhashable key. An empty map names its key and value types, because the position
|
|
it is usually written in — a `let` binding — has no type slot for it to take
|
|
them from:
|
|
|
|
```
|
|
(let [enemies (map-new string Enemy)]
|
|
...)
|
|
```
|
|
|
|
`(get m k)` returns `(Option V)`: absence is `None`, not an untyped `nil`.
|
|
`(put m k v)` is the upsert operation and returns `()`; it either inserts or
|
|
replaces. `(set (get m k) v)` is not map syntax.
|
|
|
|
The first Map implementation admits copyable keys and values only, so `get`
|
|
returns a copy. Move-aware lookup and removal are deferred; the map itself
|
|
remains an owning, move-only container.
|
|
|
|
A value that **owns storage** is admitted, and only in a region — see "A
|
|
container of owning elements lives in a region" below, which is also where what
|
|
`get` hands back in that case is settled. The key half is not relaxed and will
|
|
not be: a key that owned storage would hash its header rather than what it
|
|
points at.
|
|
|
|
## Copying is always explicit
|
|
|
|
`(clone x)` produces an independent deep copy of a `Vec`/`Map` using the current
|
|
allocator; `(clone x alloc)` names one. Value types (`[n T]`, structs of value
|
|
types, primitives) need no `clone` — assignment already copies them.
|
|
|
|
A struct containing a `Vec` field is itself move-only. Ownership is structural,
|
|
not declared: a type is a value type iff all of its fields are **and it declares
|
|
no `drop` hook** (see Allocators). A `drop` hook makes a type move-only for the
|
|
same reason a `Vec` field does — exactly one owner, so the hook fires exactly
|
|
once — and a type with one cannot be `clone`d.
|
|
|
|
**With one exception, and it is the one that makes a recursive dynamic value
|
|
expressible.** A field whose container holds *owning* elements does not make
|
|
its struct or `defdata` case move-only: such an aggregate stays **copyable**,
|
|
and copies alias into the region rather than duplicating anything. That is
|
|
sound for the reason the whole arrangement is — the container can only have
|
|
been built against a region, so neither copy owns the blocks and the region
|
|
does. Transitive move-only is what recursive teardown would have needed, and
|
|
there is no recursive teardown; see "A container of owning elements lives in a
|
|
region" for the rule that stands in its place and for what it costs.
|
|
|
|
## Borrowing
|
|
|
|
- `(as-slice v)` / `(as-slice v lo hi)` view a `Vec` or fixed array as `[T]`.
|
|
- **The first implementation follows Zig/Odin's explicit model, not Rust's
|
|
borrow checker.** A slice is invalidated by any operation that may reallocate
|
|
its owner (`push`, `put`, `reserve`); its user is responsible for respecting
|
|
that contract. Dev builds carry a generation word on `Vec` and trap on use of
|
|
a stale slice. `Ptr` is the explicit lower-level escape hatch and has the
|
|
same lifetime contract.
|
|
- A future lightweight provenance pass may reject the obvious mistakes (a
|
|
borrow of a local escaping, use after an owner moves, and reallocation with a
|
|
live borrow). It must not require Rust-style lifetime annotations or dictate
|
|
an ECS-shaped object model. Long-lived graph links use `(Handle a)`; temporary
|
|
graphs may use explicitly managed, stable region storage.
|
|
- Cross-referencing long-lived objects uses `(Handle a)` into a pool, never a
|
|
raw pointer or slice. A stale handle is detectable.
|
|
|
|
## Globals of move-only type
|
|
|
|
A global may be a `Vec` or a `Map`. Its intended lifetime is the process's — it
|
|
is loaded once and never released — and before the repeal a flow rule enforced
|
|
that: reading one was always a borrow, so nothing could take or free it. Since
|
|
the repeal the intent is unchanged and the enforcement is manners: passing,
|
|
binding, or freeing a global type-checks, and a program that frees one while
|
|
other code still reads it has the ordinary use-after-free it would have with
|
|
any other value. `(clone g)` remains the way to get something another owner
|
|
may have.
|
|
|
|
Such a global is **mutable in place**: `push`, `put`, `reserve` and `set` all
|
|
take their target as a borrow, so a global `(Vec u8)` is filled and grown where
|
|
it stands. The aliasing contract is the one above and nothing more — a push that
|
|
reallocates invalidates a slice taken before it, and that is the programmer's
|
|
whether the owner is a local or a global. Globals get no borrow rule locals do
|
|
not have; the dev build's generation word lives on the `Vec`, so the stale-slice
|
|
trap works the same either way.
|
|
|
|
A move-only global **starts zeroed** — a zeroed `Vec` is an empty `Vec` — and is
|
|
loaded by whichever function loads it, with an ordinary assignment:
|
|
|
|
```
|
|
(defvar the-data (Vec u8))
|
|
(defn load [] () (set the-data (slurp "game-data.edn")))
|
|
```
|
|
|
|
A computed initialiser on the declaration is refused: a global's initialiser is
|
|
a link-time constant, and the load has to be something the program does rather
|
|
than something that happens before `main`. That is also what makes the data
|
|
outlive `main` — nothing re-runs between one entry and the next, so a re-entered
|
|
`main` finds the global as it left it. Assigning a second time overwrites the
|
|
first block and leaks it; there is no `drop`, and freeing is a thing you write.
|
|
|
|
## The repeal — 2026-09-18
|
|
|
|
The first implementation carried a static flow analysis: a per-function dead
|
|
set recording moved-out bindings, a borrow flag over the container-reading
|
|
operations, an iteration diff for loops, and a borrowed-never-moved rule for
|
|
globals. It made use-after-move and double-free compile errors. It was removed,
|
|
and removed rather than repaired, after one day's bug hunt found four
|
|
structural holes in it (a move hidden in a borrowed target's subtree, the same
|
|
hole through a global, a `while` condition outside the loop rule, and the
|
|
region guard never asking about elements — docs/BUGS-2026-09-18.md). Each was
|
|
fixable; the shape of the four together said the analysis would keep growing
|
|
holes, and a checker that sometimes misses is worse than none, because it is
|
|
believed.
|
|
|
|
What the language promises after the repeal is Odin's contract, which is the
|
|
model this memory design was built from in the first place:
|
|
|
|
- **The types** still decide what may be copied and what may own what: move-only
|
|
assignment hands over the header, `clone` is the only copy, and the
|
|
struct/union/pool ownership rules all stand.
|
|
- **The allocator** still decides what a free means: `can-free`, regions,
|
|
`free-all`, and the region-only rules for containers of owning elements all
|
|
stand.
|
|
- **Which frees run, and when, is the program's.** `defer` is the tool, and a
|
|
double free or use-after-free is a run-time misbehaviour, not a compile
|
|
error.
|
|
- **The dev build detects.** The generation word on `Vec`, the region epoch
|
|
trap, and the block registry are the net, where a game is actually run.
|
|
|
|
A future provenance pass (the "Open" section at the end of this document, and
|
|
plan.org #3) remains the door back to static checking. It is additive: nothing
|
|
removed here changed what an accepted program means, so a stricter pass can
|
|
return without changing the language, only its acceptances.
|
|
|
|
## Taking an address
|
|
|
|
`(addr x)` yields `(Ptr T)` for any assignable place `x` — a local, a global, a
|
|
field, an element. The pointer is non-owning and does not extend anything's
|
|
lifetime, so `addr` of a local is only valid while that frame lives. This is the
|
|
same escape question as case 3 below. The first implementation leaves it as an
|
|
explicit lifetime contract; a future provenance pass can check it.
|
|
|
|
`addr` is how a value struct is shared mutably without an allocator — recursive
|
|
descent over a cursor, an entity passed down a call chain — and it is why
|
|
milestone 2 needs no heap at all. In the first implementation its non-escape
|
|
rule is an explicit programmer contract, aided by dev checks; the future
|
|
provenance pass above may enforce it.
|
|
|
|
## Places — what `set` accepts
|
|
|
|
A fixed set of assignable forms, not a `setf`-style extensible place mechanism:
|
|
|
|
```
|
|
(set x v) ; a local or a defvar
|
|
(set (.field x) v) ; struct field; x may be a struct, (Ptr S) or (Handle S)
|
|
(set (at a i ...) v) ; fixed array, slice, or Vec element
|
|
(set (deref p) v) ; whole-object store through a pointer
|
|
```
|
|
|
|
`.field` and `at` auto-deref exactly one pointer or handle level, which is what
|
|
makes `(set (.hp e) ...)` legal when `e : (Ptr Enemy)` and illegal when
|
|
`e : Enemy` bound by value.
|
|
|
|
**Mutating something you matched.** Pattern bindings bind *values*, so a matched
|
|
struct is a copy. To mutate in place, obtain a pointer first — the pointer is
|
|
visible in the type:
|
|
|
|
```
|
|
(match (resolve w h) ; (Option (Ptr Enemy))
|
|
(Some e) (set (.hp e) ...) ; e : (Ptr Enemy), field access derefs
|
|
None ...)
|
|
```
|
|
|
|
`deref` yields a value; `resolve` yields a pointer. Both are overloaded on
|
|
`(Ptr a)` and `(Handle a)` and resolve at compile time.
|
|
|
|
## Generics
|
|
|
|
Parametric polymorphism is monomorphisation, with **no type classes**. A type
|
|
variable is written `$t` wherever a *type* goes — a parameter, the return type,
|
|
or nested as `[$t]` or `(Vec $t)` — and bare `t` where a type's *name* is an
|
|
argument in expression position, as in `(vec-new t)` and the cast `(t x)`. A
|
|
generic body is checked **abstractly**, with nothing substituted, so the rule
|
|
below bites at the definition rather than at whichever call site first
|
|
instantiates it:
|
|
|
|
> A type variable `$t` supports only what every type supports: move, `clone`,
|
|
> field-free storage. It does **not** support `=`, `<`, `+`, or `hash`.
|
|
|
|
What makes that liveable is a `where` clause of compile-time type predicates,
|
|
written as a map at the head of the body. There are five — `ordered?`, `equal?`,
|
|
`hashable?`, `numeric?`, `copyable?` — they are not type classes because a
|
|
predicate carries no implementations and merely gates a builtin the compiler
|
|
already has, and they entail one another in one direction, so one clause usually
|
|
does. A variable is move-only by default and `copyable?` is the opt-out, because
|
|
whether a variable moves is not decidable abstractly. plan.org's Types section
|
|
has the full account.
|
|
|
|
```
|
|
(defn sort! [s [$t]] ()
|
|
{:where (ordered? $t)}
|
|
...)
|
|
```
|
|
|
|
Without such a clause the operator is rejected where it is written, not silently
|
|
instantiated, and the operation is passed in explicitly as a function value
|
|
instead:
|
|
|
|
```
|
|
(defn largest [xs [$t] gt (Fn [$t $t] bool)] (Option $t) ...)
|
|
```
|
|
|
|
The value handed to such a parameter is a named `defn`. An `fn` cannot be written
|
|
inline into it, because the generic body is checked with nothing substituted and
|
|
there is no concrete type yet for the `fn`'s own parameters to come from; that
|
|
restriction lifts at a monomorphic call site, where `reduce`'s and `filter`'s
|
|
callbacks are ordinary inline `fn`s.
|
|
|
|
The alternatives to predicates — compile-time interfaces, or intrinsics
|
|
restricted to primitives — remain deliberately deferred until the base checker is
|
|
stable (build sequence milestone 4). The ceiling is that nobody can supply a
|
|
user-defined `<`.
|
|
|
|
`println` is the deliberate exception. It is a compiler-provided,
|
|
type-directed intrinsic: monomorphisation selects or emits a structural printer
|
|
for each concrete instantiation, so `(println x)` is legal for `x : $t` without
|
|
introducing a `Printable` type class. Structs, fixed arrays, options and,
|
|
eventually, Vecs and Maps print structurally. `Ptr` and `Handle` print their
|
|
address or identity rather than recursively dereferencing, and collection
|
|
printers impose depth and length limits. `any` and `Error` use their runtime type
|
|
metadata. User generic code still passes an explicit function for every other
|
|
operation that depends on a type's structure.
|
|
|
|
Type arguments are **inferred at call sites** from the argument types; there is
|
|
no explicit instantiation syntax in the first implementation. A type variable
|
|
that appears only in the return type is therefore an error.
|
|
|
|
## Function values
|
|
|
|
Three cases, split by whether the value escapes the frame that made it.
|
|
|
|
**1. `(Fn [T1 T2] R)` — a plain, stable function pointer.** No captured
|
|
environment or allocation. In a dev build, a reference to a top-level `defn`
|
|
is the address of a stable trampoline that loads that function version's
|
|
indirection cell and calls its current body; it is never the address of a
|
|
particular body. Thus stored callbacks and ordinary calls observe a later
|
|
*body* redefinition, as in Common Lisp. Release builds may call the body
|
|
directly because it cannot be redefined.
|
|
|
|
A signature-changing redefinition makes a new internal function version and a
|
|
new trampoline ABI. Newly compiled code resolves the source name to that new
|
|
version. Existing callers and stored `Fn` values keep their old trampoline and
|
|
therefore safely call the old version. The session immediately warns at every
|
|
tracked caller source location that still targets the old signature; recompiling
|
|
one either retargets it successfully or reports an ordinary type error. This is
|
|
what raylib callbacks, hot-reload cells, and function parameters use. A
|
|
top-level `defn` is one, so `(largest hps >)` passes `>` at `i32` directly. This
|
|
is the only function type that may cross an FFI boundary or sit in a reload
|
|
cell.
|
|
|
|
**2. Non-escaping `fn` — captures by value into a stack environment.** A `fn`
|
|
whose value provably does not outlive the frame that created it gets an
|
|
environment allocated in that frame and captures the named locals **by value**
|
|
at the point of creation. No heap, no allocator, no lifetime question. This
|
|
covers essentially every lambda in practice:
|
|
|
|
- callbacks to `reduce` / `filter` / `each` / `map`, which consume them and return
|
|
- comparators passed to a function that does not store them
|
|
- `handler-bind` handler bodies
|
|
|
|
That last one is not a convenience. A handler must be able to see the enclosing
|
|
locals — `(fn [c] (push errors c) (invoke-restart 'skip-form))` capturing a local
|
|
`(Vec ParseError)` *is* the accumulation pattern, and conditions are not worth
|
|
building without it. Handlers are strictly non-escaping: the `handler-bind` frame
|
|
outlives every call to them.
|
|
|
|
Captured `Vec`/`Map` are captured **by pointer**, not moved, since the capture
|
|
does not outlive the owner. A non-escaping `fn` is therefore not itself an owner.
|
|
|
|
**3. Escaping closures — deferred.** A `fn` stored in a struct, pushed into a
|
|
container, or returned needs a heap environment and an answer to "which allocator
|
|
owns it, and what happens when the frame arena resets". Do not settle this until
|
|
a concrete use case requires it; revisit it with the optional lightweight
|
|
provenance work.
|
|
|
|
**Early exit inside a `fn`.** `try`, `some`, and `return` in a `fn` body exit the
|
|
`fn`, not the enclosing function — a `fn` is a function. Code that wants to
|
|
propagate out of a loop uses an imperative loop form, not a callback.
|
|
|
|
## Allocators
|
|
|
|
The allocator is part of the calling convention (`context/allocator`,
|
|
`context/temp`). `Vec` and `Map` record the allocator they were created with, so
|
|
`free` and `clone` never need it named again. Allocation uses the current
|
|
implicit allocator by default, as in Odin; an operation never falls back to a
|
|
hidden global allocator, and an explicit allocator can override the context.
|
|
|
|
### The allocator is one type-erased procedure
|
|
|
|
As in Odin (`base/runtime/core.odin:422`, `Allocator_Proc`), an allocator is a
|
|
procedure plus an opaque data pointer, and every operation takes `size` and
|
|
`align` as parameters:
|
|
|
|
| Operation | Meaning |
|
|
|------------------------------------|----------------------------------------|
|
|
| `alloc size align` | new block |
|
|
| `resize p old-size new-size align` | grow or shrink |
|
|
| `free p` | release one block |
|
|
| `free-all` | release everything the allocator holds |
|
|
|
|
It is type-erased on purpose. `Vec` and `Map` are one runtime over `(size,
|
|
align)` and, for `Map`, a compiler-emitted hash and equality pair passed as
|
|
arguments — Odin's `Map_Info` (`base/runtime/core.odin:369`). No generics are
|
|
involved, and none are needed.
|
|
|
|
An allocator declares which operations it implements. Odin's arena answers
|
|
`.Free` with `.Mode_Not_Implemented` (`core/mem/allocators.odin:307`); Flan's
|
|
equivalent is a **capability set** on the allocator value, readable at run time.
|
|
The one that is load-bearing below is `can-free`.
|
|
|
|
### When storage is released
|
|
|
|
There are exactly two release points, and neither of them is a scope.
|
|
|
|
1. **`(free v)`** — explicit. `v` is any move-only value: a `Vec`, a `Map`, a
|
|
struct that owns one, or a struct that owns a resource rather than storage
|
|
(a `Texture2D`, a socket, a file handle — see `drop` below). For a value that
|
|
holds a resource and no storage, `free` runs `drop` and nothing else; it is
|
|
still the release operation, and it is how a `Texture2D` in a local is
|
|
released. Since the repeal, `free` consumes at run time only: nothing marks
|
|
the binding dead, and a second `free` or a later read through it is a
|
|
run-time misbehaviour the allocator or the dev build catches, not a compile
|
|
error.
|
|
2. **Region release** — `(free-all a)` on an allocator, which releases
|
|
everything made from it at once, including storage reachable from bindings
|
|
that are still in scope. The per-frame `(free-all context/temp)` at the top
|
|
of a game loop *is* the frame arena, and it is the normal way arena-tier
|
|
storage dies.
|
|
|
|
**Nothing is released at scope exit.** Not at the end of a `let`, not at the end
|
|
of a function, not at the end of a `with-allocator` body. `with-allocator`
|
|
rebinds the current allocator for its dynamic extent and releases nothing; the
|
|
region it names is released, if ever, by an explicit `free-all` somewhere else.
|
|
|
|
This is deliberate, and it is the point on which the two obvious precedents were
|
|
rejected:
|
|
|
|
- **Odin's `defer delete`** cannot be written here. `defer` is function-scoped
|
|
(`check.ml:505` refuses it in a `let`, a loop or a branch) and, because `let`
|
|
is a block, a top-level `defer` is checked in a scope containing only the
|
|
parameters and globals (`check.ml:1670`). `(defer (free v))` for a `let`-bound
|
|
`v` is **not expressible today**. It becomes expressible with either
|
|
block-scoped `defer` or a sequential top-of-body binder; until one of those
|
|
exists, no idiom in this spec may depend on it.
|
|
- **Carp's scope-end frees** are a whole-program linear analysis that inserts a
|
|
teardown call at every binding's last use (`Memory.hs`, and `Info.hs`'s
|
|
`Deleter`). Carp could not reconcile that with an arena and therefore has no
|
|
allocator abstraction at all. A release point the programmer cannot see is
|
|
exactly what makes a frame arena unstateable.
|
|
|
|
**Leaking is defined behaviour.** Storage that is never freed and whose
|
|
allocator is never released is leaked, and for the permanent arena (symbols,
|
|
code) and the dev/REPL tier that is the correct program. "Did you forget to
|
|
free" is not a type question here; it is an allocator-tier question, and dev
|
|
builds answer it by reporting a general-purpose allocator's outstanding blocks
|
|
when it is destroyed.
|
|
|
|
**`free` applies to a whole owner.** It recurses structurally into owning
|
|
fields. A field is never freed on its own: `(free (.textures e))` is refused,
|
|
because it would leave `e` partly dead with no way to say so. The recursion is
|
|
the half of this that is not built — see the next section, which says what is
|
|
built instead and what `free` does where the recursion would have been.
|
|
|
|
### A container of owning elements lives in a region
|
|
|
|
A `(Vec Value)` where a `Value` may itself hold a `(Vec Value)` — the recursive
|
|
dynamic value an EDN reader has to answer with when it is handed no target
|
|
struct type — was refused outright while this spec described only the recursion
|
|
above. The reason given was always the same one: the container runtime is
|
|
type-erased, so it copies and releases slots **bytewise** and cannot reach
|
|
inside a slot. A `free` would release the slots and leave every block they
|
|
point at stranded.
|
|
|
|
That reason is about **teardown**, and it does not hold for a region. `free-all`
|
|
never releases an individual slot; it takes the whole region, and every block
|
|
the elements own is in it, because they came out of it. So the refusal was
|
|
over-broad, and the rule is now narrower:
|
|
|
|
> **A container whose element type owns storage, transitively, may only
|
|
> allocate from an allocator that lacks `can-free`.** Checked at the point of
|
|
> construction and at any later growth: one branch per container, never per
|
|
> element.
|
|
|
|
Three things follow, and each is a decision rather than a consequence:
|
|
|
|
- **It is a run-time branch, not a compile-time refusal.** `can-free` is a
|
|
capability on an allocator *value*, and `with-allocator` rebinds a dynamic
|
|
variable, so which tier a `(vec-new)` will meet is not a property of the
|
|
place it is written. What is decided at compile time is only *whether to
|
|
ask* — that is a property of the element type. The question is asked of
|
|
`can-free` rather than of "is this an arena" so that a fixed backing buffer
|
|
written later answers it the same way for the same reason.
|
|
- **`free` on such a container is refused, not silently shallow.** It cannot
|
|
recurse — that is the whole premise — and releasing the outer block alone
|
|
would be "I freed it" written over a program that stranded everything
|
|
inside. A reader will assume recursion, so the refusal names `free-all`
|
|
instead, which is reachable by construction: such a container is
|
|
region-allocated or it does not exist.
|
|
- **`clone` on such a container is not possible and stays refused.** This is
|
|
the half a region does *not* answer, and it is a different failure from the
|
|
teardown one the two were once stated together as. `clone` promises a deep,
|
|
independent copy; a bytewise one hands back a second container whose elements
|
|
still point into the first one's blocks. Cloning into a *second* region is
|
|
worse rather than better, because releasing that region leaves the copy's
|
|
elements pointing into a region that is still live. `at` and `get` are
|
|
untouched: they promise nothing, and the alias they hand back is an alias
|
|
into storage nobody individually owns, which is the bargain a region is.
|
|
|
|
A **struct field or a `defdata` case's field** of such a container type is
|
|
admitted for the same reason and only for that reason — the field's container
|
|
can only have been built against a region, so the aggregate's whole graph is
|
|
released by one `free-all`. The aggregate itself **stays copyable**: ownership
|
|
is not made transitive through a name, because that is the model recursive
|
|
teardown would have needed. So a `Value` is copied bytewise like any other
|
|
value, and two copies share the inner blocks. In a region that is aliasing and
|
|
not a double free — and mutating through one alias is visible through the
|
|
other, which is a logic bug and the price of the bargain, exactly as it is in
|
|
Odin. A field holding a container whose elements own
|
|
*nothing* stays refused: nothing would force that one into a region, and two
|
|
copies of the aggregate would be two headers over one heap block. An untagged
|
|
`defunion` is not part of this at all: a move-only member in one is refused for
|
|
a reason that is not teardown — nothing records which member is live, so there
|
|
is no fact a release could read — and a region answers the teardown question
|
|
without touching that one.
|
|
|
|
A `Pool` is admitted with the others, and it is the one where the trade is not
|
|
identical, because `(release p h)` recycles a slot while the pool lives on.
|
|
Whatever the dead element owned stays allocated until `free-all`. That is a
|
|
region leak bounded by the region, which is what a region already is; it is not
|
|
a use-after-free, because nothing was released.
|
|
|
|
### Dev builds detect a released region
|
|
|
|
A `Vec` or `Map` records its allocator (see above). In a dev build it also
|
|
records that allocator's **epoch** — a counter the allocator bumps on every
|
|
`free-all`. Any operation on a container whose recorded epoch has moved traps,
|
|
naming the allocation site and the release site. This is a second and separate
|
|
counter from the per-`Vec` generation word that catches stale slices; the two
|
|
answer different questions and must not be conflated. Both are dev-only: the
|
|
release layout of a `Vec` is `ptr + len + cap + allocator` and nothing more.
|
|
|
|
This is what covers a use after `free-all`, including the case the section
|
|
above makes reachable: an inner container's header copied *out* of an
|
|
arena-held element into a local before the release still traps on the next
|
|
operation. It works because an `Allocator` is a **pointer** to the allocator
|
|
and not a copy of one — a copied-by-value allocator would give each copy its
|
|
own epoch, and a copy taken before the bump would never notice it.
|
|
|
|
### `drop` — owning something that is not memory
|
|
|
|
A type may name one hook:
|
|
|
|
```
|
|
(drop Texture [t (Ptr Texture)] ...)
|
|
```
|
|
|
|
It takes a **pointer, not the value**, which is Carp's shape and for Carp's
|
|
reason. Carp shipped `delete` — auto-generated per type, consuming, and
|
|
responsible for the recursive teardown of every field — and then had to add
|
|
`drop` separately, because a user who redefined `delete` to close a file had to
|
|
re-implement that whole teardown by hand. Carp's `drop` is looked up per
|
|
teardown site (`Memory.hs:806`, `getDropFunc`, at `RefTy t` where `delete` is
|
|
`FuncTy [t]`) and emitted immediately before the teardown call
|
|
(`Emit.hs:1042`), so the hook *composes with* compiler-generated teardown rather
|
|
than replacing it. Flan takes that arrangement unchanged.
|
|
|
|
- `(free v)` runs `drop` on `v` first, then tears down `v`'s owning fields in
|
|
declaration order, each by the same rule.
|
|
- A `drop` hook may read and mutate through its pointer. It may **not** move out
|
|
of the value, and it may not `free` it.
|
|
- A type has a `drop` hook transitively: a struct any of whose fields has one,
|
|
has one.
|
|
- **A type with a `drop` hook is move-only and cannot be `clone`d.** Move-only,
|
|
because a value type copies on assignment and two copies of one socket would
|
|
each run `drop`; the same argument that makes a `Vec` field move-only. Not
|
|
`clone`able, because duplicating a texture id or a file descriptor is not the
|
|
compiler's decision to make — Carp needed a separate `copy` interface for
|
|
exactly this. A type that *can* be duplicated says so with an ordinary named
|
|
function.
|
|
|
|
**Nothing runs `drop` when an arena resets — because such a value cannot be in
|
|
an arena.** Constructing a container whose element type transitively has a
|
|
`drop` hook, or allocating such a value, against an allocator that lacks
|
|
`can-free` is **refused at the point of construction**: one branch per
|
|
container, not per element. This is the rule the section on containers of
|
|
owning elements already states and already implements, with one difference that
|
|
matters if `drop` is ever built: a hook is the case a region *cannot* answer —
|
|
releasing the region does not close the socket — so a hooked element would have
|
|
to be refused against an arena rather than *required* to be in one. The two
|
|
questions must stay apart. Asking "does this element own anything" where "does
|
|
it have a hook" was meant would refuse nested containers in the frame tier,
|
|
which is precisely the case the frame tier exists for. `free-all` therefore never has to walk a list of
|
|
registered destructors, which is what keeps the frame tier's reset genuinely
|
|
free (plan.org's memory table) and keeps a destructor list — an allocation
|
|
nobody wrote — out of the core.
|
|
|
|
The consequence, stated plainly because a reader will assume otherwise:
|
|
|
|
> **`drop` is not a destructor.** A `Texture2D` held in a local, a parameter, or
|
|
> a plain stack struct never has `drop` run, because Flan has no scope-end
|
|
> anything. `drop` fires at exactly one place — inside `free` — and resources in
|
|
> locals are released explicitly, exactly as memory is. Carp's `drop` fires at
|
|
> scope end only because Carp has scope-end frees, which the section above
|
|
> rejects.
|
|
|
|
### Alignment
|
|
|
|
Alignment is a property of the **type**, computed at the **call site**, and
|
|
passed as a **parameter** to the type-erased allocator. All three, and they are
|
|
not alternatives.
|
|
|
|
Odin arranges it exactly this way: `elem_align` is threaded through every
|
|
type-erased dynamic-array entry point (`base/runtime/dynamic_array_internal.odin`
|
|
— `__dynamic_array_reserve`, `__dynamic_array_resize`, `__dynamic_array_append`),
|
|
and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:350`). The
|
|
monomorphised wrapper is the only place the concrete type is known, so it is the
|
|
only place that can produce the number.
|
|
|
|
Alignment is **not stored** in the `Vec` or `Map` header. That is safe because of
|
|
a condition worth writing down: every operation that needs it — `push`,
|
|
`reserve`, `resize`, `clone`, `free` — is compiler-emitted at a site where the
|
|
concrete element type is known. Any future type-erased teardown path would break
|
|
that condition; there is not to be one. (This is the second reason the `drop`
|
|
registry above was rejected: it would have been exactly such a path.)
|
|
|
|
The natural alignment of `T` is `align-of T`. Raising it above natural — 16
|
|
bytes for `#soa` and for component-wise fixed arrays — is declared **on the
|
|
type**, so that every site computing `align-of T` gets the raised number with no
|
|
further plumbing. The surface syntax for that declaration is deliberately not
|
|
fixed here; nothing is built that needs it yet.
|
|
|
|
### Untagged unions and what a read of one means
|
|
|
|
`defunion` is C's union: the members overlay one storage, the size is the
|
|
largest of them, the alignment the strictest, and **nothing records which
|
|
member was written**. It is not `defdata`, which is the tagged sum — a case, its
|
|
fields, and a tag that steers every `match`.
|
|
|
|
**Reading a member that was not the one last written is defined**, and it is
|
|
the one place in this language where bytes win over safety on purpose. It reads
|
|
the storage through that member's type: the layout is the target's, the bytes
|
|
are the bytes, and the read is a reinterpretation of them. C leaves this to the
|
|
implementation; Flan does not, because both uses the type exists for *are* that
|
|
read. Binding a C header means holding the union the library holds and reading
|
|
whichever member the library's own tag says is live — a tag the compiler cannot
|
|
see, since the rule relating them is prose in a manual. Overlaying an `f32` on a
|
|
`u32` to look at its bits is the same read. A rule that refused it would be
|
|
refusing the type.
|
|
|
|
What is **not** promised is anything about bytes nobody wrote. A member wider
|
|
than the one last stored reads its own size, and the tail is indeterminate
|
|
exactly as a struct's padding is. ZII narrows that to almost nothing in
|
|
practice: a union is all-bytes-zero unless `uninit` says otherwise.
|
|
|
|
Three things a union may not do, and each for a reason that does not expire:
|
|
|
|
- **No move-only member.** Nothing knows which member is live, so nothing can
|
|
tear one down. Unlike the struct and `defdata` refusals, this is not waiting
|
|
on recursive teardown — there is no fact for teardown to read, and freeing
|
|
the wrong member is a free of a pointer that was an `f64` a moment ago.
|
|
- **No `bool` member, at any depth.** An `i1` loaded out of a byte that is
|
|
neither 0 nor 1 is a value the optimiser is entitled to assume cannot exist,
|
|
and a union is the only type that can produce one. Hold a `u8` and compare it.
|
|
- **Not a map key.** A member narrower than the union leaves the rest of the
|
|
bytes indeterminate, so two values agreeing about everything written would
|
|
still hash apart.
|
|
|
|
`uninit` on a union **is** allowed, unlike on a `defdata`. The refusal there is
|
|
not about garbage: it is that a tag no case names falls past every comparison in
|
|
a `match` into a block the optimiser may treat as unreachable. An untagged union
|
|
steers nothing, so `uninit` makes its bytes arbitrary and changes nothing else —
|
|
which is what it means on an `i64`.
|
|
|
|
### Allocation failure
|
|
|
|
**No allocating operation returns an error, and none can fail silently.** When
|
|
the allocator cannot satisfy a request, the operation signals
|
|
|
|
```
|
|
(StorageExhausted {.bytes n .align a .allocator id})
|
|
```
|
|
|
|
with `error`, whose type is `Never` (spec-conditions.md §2), inside a
|
|
`restart-case` offering `retry`. This is one rule over *every* allocating
|
|
operation — `vec-new`, `map-new`, `push`, `put`, `reserve`, `clone` — so their
|
|
result types stay `(Vec T)`, `()`, `()` and so on, with no `Result` and no
|
|
out-parameter anywhere.
|
|
|
|
What that buys, against the alternative: Odin's `append` returns an ignorable
|
|
`Allocator_Error` (`base/runtime/core_builtin.odin:767`,
|
|
`#optional_allocator_error`), and the type-erased path underneath returns the old
|
|
length on a failed reserve, marked
|
|
`// TODO(bill): Better error handling for failed reservation`
|
|
(`base/runtime/dynamic_array_internal.odin`). An `append` that appends nothing
|
|
and says nothing is the outcome this rule exists to make impossible.
|
|
|
|
- The condition is a value struct on the signalling frame's stack, with fixed
|
|
numeric fields and **no rendered message**, because formatting would allocate
|
|
and this is the one path that must not. Rendering happens in the handler or the
|
|
break loop, where a working allocator is known.
|
|
- Unhandled, `error` enters the dev break loop or aborts in release
|
|
(spec-conditions.md §2). It is never a no-op; `signal` is not used here.
|
|
- A handler that frees something, releases a scratch region, or grows the arena
|
|
and then invokes `retry` re-attempts the same request. A handler that wants a
|
|
*different* allocator needs a restart taking an argument, which does not exist
|
|
yet; until it does, such a handler rebinds the context allocator and retries.
|
|
- Because an allocating operation can transfer, every caller of one checks the
|
|
transfer channel after the call (spec-conditions.md §6). `push` is not a leaf
|
|
call, and that per-call-site check is the price of not being Odin.
|
|
|
|
**This is the named exception to plan.org's "restarts go at the resync point,
|
|
once".** That rule is right for program-level errors and wrong here: a restart
|
|
established at a parser's top-level loop cannot re-attempt an allocation, and
|
|
only the allocation site can. Compiler-emitted restarts at the point of failure
|
|
are the exception, in the same way Common Lisp's runtime establishes
|
|
`store-value` at an unbound-variable error rather than at a resync point. No
|
|
*user* code establishes restarts below a resync point.
|
|
|
|
### Open: catching a use-after-release statically
|
|
|
|
Both release points above are dynamic, and the frame arena is the reason. A
|
|
static rule — "a move-only value constructed under a given allocator may not
|
|
outlive it" — needs to know statically which allocator a construction used, and
|
|
`with-allocator` plus `context/allocator` are precisely the mechanisms that deny
|
|
that knowledge. The lexical subset (a value made inside a `with-allocator` body
|
|
and returned out of it) is checkable; the general case is not; and shipping only
|
|
the subset would teach a rule that silently stops applying at the loop where it
|
|
matters most.
|
|
|
|
Until a provenance pass exists (plan.org open decision #3), the answer is the
|
|
dev-build epoch trap above: detection, loud and immediate, rather than
|
|
prevention. Settling this needs one thing that does not exist yet — real Flan
|
|
programs using arenas, to say whether the escapes that actually occur are
|
|
lexical. It is not settleable from the design alone, and it is not papered over
|
|
here.
|