(i32 (at xs i)) inside a body bounded integer? was refused with "i32 converts a number, found t". Arithmetic, comparison, min/max, the bitwise fold and the shifts all ask the where clause; the conversions were the family nobody had gone back to, and the cast block held three arms of it. The machine-type target asked Types.is_numeric of its operand and the enum target asked Types.Int _, so a variable fell through both to the refusal however it was bounded. The variable target had the opposite defect: it asked the bound of the target and then took any generic operand, so a second variable declared only ordered? passed the abstract pass on the strength of a sentence about a different one. Nothing wrong was ever emitted through it — ordered? admits numbers and enums and both convert at the instantiation — which is exactly why it is worth closing: the hole opens the day ordered? admits a type that does not. The rule is the repo's own, applied to a set instead of a type: a conversion is legal at a bounded variable exactly when it is legal at every type the bound admits. A machine-type target needs numeric?, an enum target needs integer? because numeric? admits the floats the concrete arm refuses, and ordered?/equal?/hashable? admit nothing — the last by what the predicate says rather than by the set it denotes, since hashable? already admits strings and ordered? may. Both float targets and the narrowing i32 stay legal: (f64 i64-x) rounds above 2^53 and (i32 f64-x) truncates where the types are written, and a generic that refused what its copies accept would be the fork the rule forbids. FIX.org, 2026-09-21, has the account, and records what this costs: no predicate now licenses a generic enum to integer conversion, and enum? is the eventual answer. The refusal says what the variable is known to be and what to write, in the clause spelling unconstrained already uses: a body with no clause gets the clause, a body that has one is told which predicate to add.
771 lines
42 KiB
Markdown
771 lines
42 KiB
Markdown
# Spec 1 — Ownership, containers, and copies
|
|
|
|
Status: **frozen**, with one amendment: **the repeal of 2026-09-18** (see "The
|
|
repeal", below), which removed the static flow analysis — use-after-move and
|
|
double-free are no longer compile errors. Everything structural in this
|
|
document still governs. Closes plan.org open decisions #6 and #10, and resolves the
|
|
contradiction between "value structs copy on assignment" and owning containers.
|
|
The Allocators section additionally settles the four things that had to be
|
|
decided before `Vec` and `Map` are written: when storage is released, the `drop`
|
|
hook, alignment, and allocation failure. One question there is left open on
|
|
purpose and says so.
|
|
Everything else in the design references this vocabulary. It governs plain
|
|
fixed-layout `struct` values, not the separately planned managed `class`
|
|
facility (see plan.org, "Managed classes").
|
|
|
|
## The four container types
|
|
|
|
| Notation | Layout | Assignment | Owns storage | Allocator |
|
|
|-----------|-------------------|------------|--------------|-----------|
|
|
| `[n T]` | n contiguous `T` | copies | no (inline) | — |
|
|
| `[T]` | ptr + len | copies the *view* | no | — |
|
|
| `(Vec T)` | ptr + len + cap | **moves** | yes | stored |
|
|
| `(Map K V)` | open-addressed, flat key/value arrays | **moves** | yes | stored |
|
|
|
|
- `[n T]` is a value. It lives wherever it is declared, copies on assignment and
|
|
on pass-by-value, and is what `defconst colors [4 u32] ...` and
|
|
`(defonce grid [rows [cols u32]] ...)` are.
|
|
- `[T]` is a **non-owning slice**: a borrowed window into a `[n T]`, a `(Vec T)`,
|
|
or a literal in read-only memory. Copying a slice copies ptr+len, never the
|
|
elements. A slice may be `const`-qualified; freeing through one is not possible
|
|
because a slice has no allocator and no `cap`.
|
|
- `(Vec T)` and `(Map K V)` **were move-only**; the second repeal removed the
|
|
concept. Assignment copies the header, the copies alias one buffer, and
|
|
`(clone x)` is the spelling of an independent one. Using a binding after
|
|
assigning it away is ordinary: the header is still there, and a program that frees
|
|
through it twice or reads through it after a free misbehaves at run time,
|
|
where the allocator and the dev build's generation word are the net.
|
|
|
|
## Maps — first implementation
|
|
|
|
Every map is homogeneous: `(Map K V)` has one key type and one value type. The
|
|
first implementation accepts only built-in structural key types: integers,
|
|
enums, strings, fixed arrays, and value structs composed recursively from those
|
|
types. Tuples and triples join that set when they are introduced. `Ptr`, slices,
|
|
`Vec`, and `Map` are not map keys yet.
|
|
|
|
Equality and hashing for those keys are compiler-provided structural operations
|
|
and not type classes. They are not available to an unconstrained type variable
|
|
either; a variable that means to key a map declares `hashable?` in the signature
|
|
that binds it, and the refusal then lands at the call site that names an
|
|
unhashable key. An empty map names its key and value types, because the position
|
|
it is usually written in — a `let` binding — has no type slot for it to take
|
|
them from:
|
|
|
|
```
|
|
(let [enemies (map-new string Enemy)]
|
|
...)
|
|
```
|
|
|
|
`(get m k)` returns `(Option V)`: absence is `None`, not an untyped `nil`.
|
|
`(put m k v)` is the upsert operation and returns `()`; it either inserts or
|
|
replaces. `(set (get m k) v)` is not map syntax.
|
|
|
|
The first Map implementation admits copyable keys and values only, so `get`
|
|
returns a copy. Move-aware lookup and removal are deferred; the map itself
|
|
remains an owning, move-only container.
|
|
|
|
A value that **owns storage** is admitted, and only in a region — see "A
|
|
container of owning elements lives in a region" below, which is also where what
|
|
`get` hands back in that case is settled. The key half is not relaxed and will
|
|
not be: a key that owned storage would hash its header rather than what it
|
|
points at.
|
|
|
|
## Copying is always explicit
|
|
|
|
`(clone x)` produces an independent deep copy of a `Vec`/`Map` using the current
|
|
allocator; `(clone x alloc)` names one. Value types (`[n T]`, structs of value
|
|
types, primitives) need no `clone` — assignment already copies them.
|
|
|
|
A struct containing a `Vec` field was itself move-only until the second
|
|
repeal; since it, the field is admitted and the struct copies like any other
|
|
(see "The repeal"). The paragraphs below record the rule as designed.
|
|
Ownership is structural,
|
|
not declared: a type is a value type iff all of its fields are **and it declares
|
|
no `drop` hook** (see Allocators). A `drop` hook makes a type move-only for the
|
|
same reason a `Vec` field does — exactly one owner, so the hook fires exactly
|
|
once — and a type with one cannot be `clone`d.
|
|
|
|
**With one exception, and it is the one that makes a recursive dynamic value
|
|
expressible.** A field whose container holds *owning* elements does not make
|
|
its struct or `defdata` case move-only: such an aggregate stays **copyable**,
|
|
and copies alias into the region rather than duplicating anything. That is
|
|
sound for the reason the whole arrangement is — the container can only have
|
|
been built against a region, so neither copy owns the blocks and the region
|
|
does. Transitive move-only is what recursive teardown would have needed, and
|
|
there is no recursive teardown; see "A container of owning elements lives in a
|
|
region" for the rule that stands in its place and for what it costs.
|
|
|
|
## Borrowing
|
|
|
|
- `(as-slice v)` / `(as-slice v lo hi)` view a `Vec` or fixed array as `[T]`.
|
|
- **The first implementation follows Zig/Odin's explicit model, not Rust's
|
|
borrow checker.** A slice is invalidated by any operation that may reallocate
|
|
its owner (`push`, `put`, `reserve`); its user is responsible for respecting
|
|
that contract. Dev builds carry a generation word on `Vec` and trap on use of
|
|
a stale slice. `Ptr` is the explicit lower-level escape hatch and has the
|
|
same lifetime contract.
|
|
- A future lightweight provenance pass may reject the obvious mistakes (a
|
|
borrow of a local escaping, use after an owner moves, and reallocation with a
|
|
live borrow). It must not require Rust-style lifetime annotations or dictate
|
|
an ECS-shaped object model.
|
|
- Cross-referencing long-lived objects used `(Handle a)` into a `Pool` until
|
|
the second repeal (below) removed both: two containers proved enough, and a
|
|
program that wants generational indirection builds it over a `Vec`, as an
|
|
Odin program does.
|
|
|
|
## Container globals
|
|
|
|
A global may be a `Vec` or a `Map`. Its intended lifetime is the process's — it
|
|
is loaded once and never released — and before the repeal a flow rule enforced
|
|
that: reading one was always a borrow, so nothing could take or free it. Since
|
|
the repeal the intent is unchanged and the enforcement is manners: passing,
|
|
binding, or freeing a global type-checks, and a program that frees one while
|
|
other code still reads it has the ordinary use-after-free it would have with
|
|
any other value. `(clone g)` remains the way to get something another owner
|
|
may have.
|
|
|
|
Such a global is **mutable in place**: `push`, `put`, `reserve` and `set` all
|
|
take their target as a borrow, so a global `(Vec u8)` is filled and grown where
|
|
it stands. The aliasing contract is the one above and nothing more — a push that
|
|
reallocates invalidates a slice taken before it, and that is the programmer's
|
|
whether the owner is a local or a global. Globals get no borrow rule locals do
|
|
not have.
|
|
|
|
A container global **starts zeroed** — a zeroed `Vec` is an empty `Vec` — and is
|
|
loaded by whichever function loads it, with an ordinary assignment:
|
|
|
|
```
|
|
(defonce the-data (Vec u8))
|
|
(defn load [] () (set the-data (slurp "game-data.edn")))
|
|
```
|
|
|
|
A computed initialiser on the declaration is refused: a global's initialiser is
|
|
a link-time constant, and the load has to be something the program does rather
|
|
than something that happens before `main`. That is also what makes the data
|
|
outlive `main` — nothing re-runs between one entry and the next, so a re-entered
|
|
`main` finds the global as it left it. Assigning a second time overwrites the
|
|
first block and leaks it; there is no `drop`, and freeing is a thing you write.
|
|
|
|
## The repeal — 2026-09-18
|
|
|
|
(Extended the same day by a second round; see the amendments at the end of
|
|
this section.)
|
|
|
|
The first implementation carried a static flow analysis: a per-function dead
|
|
set recording moved-out bindings, a borrow flag over the container-reading
|
|
operations, an iteration diff for loops, and a borrowed-never-moved rule for
|
|
globals. It made use-after-move and double-free compile errors. It was removed,
|
|
and removed rather than repaired, after one day's bug hunt found four
|
|
structural holes in it (a move hidden in a borrowed target's subtree, the same
|
|
hole through a global, a `while` condition outside the loop rule, and the
|
|
region guard never asking about elements — docs/BUGS-2026-09-18.md). Each was
|
|
fixable; the shape of the four together said the analysis would keep growing
|
|
holes, and a checker that sometimes misses is worse than none, because it is
|
|
believed.
|
|
|
|
What the language promises after the repeal is Odin's contract, which is the
|
|
model this memory design was built from in the first place:
|
|
|
|
- **The allocator** decides what a free means: `can-free`, regions,
|
|
`free-all`, and the region-only rules for containers of owning elements all
|
|
stand.
|
|
- **Which frees run, and when, is the program's.** `defer` is the tool, and a
|
|
double free or use-after-free is a run-time misbehaviour, not a compile
|
|
error.
|
|
- **The dev build detects.** The region epoch trap and the block registry are
|
|
the net, where a game is actually run.
|
|
|
|
The same day, the rest of the concept followed, on the same argument carried
|
|
to its end:
|
|
|
|
- **The move-only concept itself is gone.** Everything copies — a container as
|
|
its header, the copies aliasing one buffer. The `copyable?` predicate went
|
|
with it (four predicates remain), as did the struct, data-type and union
|
|
owning-field refusals: a struct may hold a `Vec`, and the two-headers-one-
|
|
buffer aliasing that buys is the program's, exactly as it is in Odin.
|
|
- **`Pool` and `(Handle a)` are gone.** Two containers are enough; a slab with
|
|
generational handles is a library a program writes over a `Vec` when it
|
|
wants one, which is where Odin keeps it too.
|
|
- **The `gen` word is gone from both headers.** It was bumped on every
|
|
reallocation and read by nothing — the stale-slice check it promised needs a
|
|
third word on every slice, not a word here. A `Vec` is now
|
|
`ptr len cap allocator epoch`; the epoch word stays because free-all
|
|
detection reads it on every operation.
|
|
|
|
A future provenance pass (the "Open" section at the end of this document, and
|
|
plan.org #3) remains the door back to static checking. It is additive: nothing
|
|
removed here changed what an accepted program means, so a stricter pass can
|
|
return without changing the language, only its acceptances.
|
|
|
|
## Taking an address
|
|
|
|
`(addr x)` yields `(Ptr T)` for any assignable place `x` — a local, a global, a
|
|
field, an element. The pointer is non-owning and does not extend anything's
|
|
lifetime, so `addr` of a local is only valid while that frame lives. This is the
|
|
same escape question as case 3 below. The first implementation leaves it as an
|
|
explicit lifetime contract; a future provenance pass can check it.
|
|
|
|
`addr` is how a value struct is shared mutably without an allocator — recursive
|
|
descent over a cursor, an entity passed down a call chain — and it is why
|
|
milestone 2 needs no heap at all. In the first implementation its non-escape
|
|
rule is an explicit programmer contract, aided by dev checks; the future
|
|
provenance pass above may enforce it.
|
|
|
|
## Places — what `set` accepts
|
|
|
|
A fixed set of assignable forms, not a `setf`-style extensible place mechanism:
|
|
|
|
```
|
|
(set x v) ; a local or a defonce
|
|
(set (.field x) v) ; struct field; x may be a struct, (Ptr S) or (Handle S)
|
|
(set (at a i ...) v) ; fixed array, slice, or Vec element
|
|
(set (deref p) v) ; whole-object store through a pointer
|
|
```
|
|
|
|
`.field` and `at` auto-deref exactly one pointer or handle level, which is what
|
|
makes `(set (.hp e) ...)` legal when `e : (Ptr Enemy)` and illegal when
|
|
`e : Enemy` bound by value.
|
|
|
|
**Mutating something you matched.** Pattern bindings bind *values*, so a matched
|
|
struct is a copy. To mutate in place, obtain a pointer first — the pointer is
|
|
visible in the type:
|
|
|
|
```
|
|
(match (resolve w h) ; (Option (Ptr Enemy))
|
|
(Some e) (set (.hp e) ...) ; e : (Ptr Enemy), field access derefs
|
|
None ...)
|
|
```
|
|
|
|
`deref` yields a value; `resolve` yields a pointer. Both are overloaded on
|
|
`(Ptr a)` and `(Handle a)` and resolve at compile time.
|
|
|
|
## Generics
|
|
|
|
Parametric polymorphism is monomorphisation, with **no type classes**. A type
|
|
variable is written `$t` wherever a *type* goes — a parameter, the return type,
|
|
or nested as `[$t]` or `(Vec $t)` — and bare `t` where a type's *name* is an
|
|
argument in expression position, as in `(vec-new t)` and the cast `(t x)`. A
|
|
generic body is checked **abstractly**, with nothing substituted, so the rule
|
|
below bites at the definition rather than at whichever call site first
|
|
instantiates it:
|
|
|
|
> A type variable `$t` supports only what every type supports: move, `clone`,
|
|
> field-free storage. It does **not** support `=`, `<`, `+`, or `hash`.
|
|
|
|
What makes that liveable is a `where` clause of compile-time type predicates,
|
|
written as a map at the head of the body. There are five — `ordered?`,
|
|
`equal?`, `hashable?`, `numeric?`, `integer?` — they are not type classes
|
|
because a predicate carries no implementations and merely gates a builtin the
|
|
compiler already has, and they entail one another in one direction, so one
|
|
clause usually does: `integer?` admits every integer kind and no float, and
|
|
entails `numeric?`, which entails `ordered?`, which entails `equal?`.
|
|
`integer?` is what admits the bitwise operators, the shifts and an
|
|
integer-only body like `abs`'s — under `numeric?` those bodies would be
|
|
instantiated at the floats too (FIX.org 2026-09-20). (`copyable?` was once a
|
|
sixth until the second repeal removed the move concept it opted out of.)
|
|
plan.org's Types section has the full account.
|
|
|
|
```
|
|
(defn sort [s [$t]] ()
|
|
{:where (ordered? $t)}
|
|
...)
|
|
```
|
|
|
|
Without such a clause the operator is rejected where it is written, not silently
|
|
instantiated, and the operation is passed in explicitly as a function value
|
|
instead:
|
|
|
|
```
|
|
(defn largest [xs [$t] gt (Fn [$t $t] bool)] (Option $t) ...)
|
|
```
|
|
|
|
The value handed to such a parameter is a named `defn`. An `fn` cannot be written
|
|
inline into it, because the generic body is checked with nothing substituted and
|
|
there is no concrete type yet for the `fn`'s own parameters to come from; that
|
|
restriction lifts at a monomorphic call site, where `reduce`'s and `filter`'s
|
|
callbacks are ordinary inline `fn`s.
|
|
|
|
The alternatives to predicates — compile-time interfaces, or intrinsics
|
|
restricted to primitives — remain deliberately deferred until the base checker is
|
|
stable (build sequence milestone 4). The ceiling is that nobody can supply a
|
|
user-defined `<`.
|
|
|
|
`println` is the deliberate exception. It is a compiler-provided,
|
|
type-directed intrinsic: monomorphisation selects or emits a structural printer
|
|
for each concrete instantiation, so `(println x)` is legal for `x : $t` without
|
|
introducing a `Printable` type class. Structs, fixed arrays, options and,
|
|
eventually, Vecs and Maps print structurally. `Ptr` and `Handle` print their
|
|
address or identity rather than recursively dereferencing, and collection
|
|
printers impose depth and length limits. `any` and `Error` use their runtime type
|
|
metadata. User generic code still passes an explicit function for every other
|
|
operation that depends on a type's structure.
|
|
|
|
Type arguments are **inferred at call sites** from the argument types; there is
|
|
no explicit instantiation syntax in the first implementation. A type variable
|
|
that appears only in the return type is therefore an error.
|
|
|
|
Three further rules about what a call site may pass, all decided 2026-09-20 and
|
|
written up in FIX.org, "Milestone 5, and the sweep behind it":
|
|
|
|
**A written number may stand where a type variable stands, under `numeric?`.**
|
|
`(defn pos? [x $t] bool {:where (numeric? $t)} (> x 0))` is the family the
|
|
feature was asked for, and the bound is what makes the `0` sound rather than
|
|
optimistic: every type `numeric?` admits is an integer or a float, and an
|
|
untyped integer constant is usable at all of them. Nothing weaker admits it —
|
|
`ordered?` admits an enum, which holds no number. A *float* literal is refused
|
|
at a type variable even under `numeric?`, because `numeric?` covers the
|
|
integers too and a float literal is never usable where an integer is wanted.
|
|
The range check belongs to each copy, not to the definition.
|
|
|
|
**Mixed widths at one type variable join at the wider type.** The first rule
|
|
here refused the pair both ways and recorded the join as the loosening that
|
|
could be added later; the author added it on 2026-09-20 (FIX.org, the
|
|
integer? entry). Numeric scalars bound to one `$t` resolve it to whichever
|
|
type every one of them widens into — value-preserving widening only, and in
|
|
any argument order, so both orders produce the identical copy. A pair with no
|
|
join (u64 against i64) is still refused, and a variable the signature also
|
|
reaches through a container or function type is still bound exactly, because
|
|
a slice's elements cannot be rewritten. An untyped literal is unaffected: it
|
|
has no type of its own to keep.
|
|
|
|
**A conversion under a bound is legal exactly when it is legal at every type
|
|
the bound admits.** `(i32 x)` asks whether `x` is a number, and inside a
|
|
generic body no type is there to answer; the `where` clause answers for every
|
|
copy at once. `numeric?` is what a conversion to a machine type needs —
|
|
`integer?` entails it, so an `integer?` body converts too — and that covers
|
|
narrowing and widening alike: `(i32 x)` on a written `f64` truncates towards
|
|
zero and `(f64 x)` on a written `i64` rounds above 2^53, neither is refused
|
|
where the type is written, so neither is refused at the variable. A conversion
|
|
was never a claim that the value survives. The conversion *to* an enum needs
|
|
`integer?` exactly, because an enum is an `i32` and a float has no enum
|
|
reading, and `numeric?` would admit an `f32` copy the concrete rule refuses.
|
|
`ordered?`, `equal?` and `hashable?` admit no conversion at all: they say what
|
|
can be compared or keyed, not what is a number — and that is a claim about
|
|
what the predicate says, not about the set it denotes today, which currently
|
|
does admit only numbers and enums. The refusal names the predicate to write
|
|
(FIX.org 2026-09-21). One consequence is recorded there as open: no predicate
|
|
now licenses a generic enum → integer conversion.
|
|
|
|
**A type variable is not instantiated at `dyn`.** Two models answer "one body,
|
|
many types" and they are not rivals: this one copies per written type at
|
|
compile time, `defgeneric`/`defmethod` dispatch at run time on a value that
|
|
carries its own. A dyn argument asks the second question of the first
|
|
machinery, and the refusal says so. Whether dyn should ever flow through a
|
|
generic is open.
|
|
|
|
## Function values
|
|
|
|
Three cases, split by whether the value escapes the frame that made it.
|
|
|
|
**1. `(Fn [T1 T2] R)` — a plain, stable function pointer.** No captured
|
|
environment or allocation. In a dev build, a reference to a top-level `defn`
|
|
is the address of a stable trampoline that loads that function version's
|
|
indirection cell and calls its current body; it is never the address of a
|
|
particular body. Thus stored callbacks and ordinary calls observe a later
|
|
*body* redefinition, as in Common Lisp. Release builds may call the body
|
|
directly because it cannot be redefined.
|
|
|
|
A signature-changing redefinition makes a new internal function version and a
|
|
new trampoline ABI. Newly compiled code resolves the source name to that new
|
|
version. Existing callers and stored `Fn` values keep their old trampoline and
|
|
therefore safely call the old version. The session immediately warns at every
|
|
tracked caller source location that still targets the old signature; recompiling
|
|
one either retargets it successfully or reports an ordinary type error. This is
|
|
what raylib callbacks, hot-reload cells, and function parameters use. A
|
|
top-level `defn` is one, so `(largest hps >)` passes `>` at `i32` directly. This
|
|
is the only function type that may cross an FFI boundary or sit in a reload
|
|
cell.
|
|
|
|
**2. Non-escaping `fn` — captures by value into a stack environment.** A `fn`
|
|
whose value provably does not outlive the frame that created it gets an
|
|
environment allocated in that frame and captures the named locals **by value**
|
|
at the point of creation. No heap, no allocator, no lifetime question. This
|
|
covers essentially every lambda in practice:
|
|
|
|
- callbacks to `reduce` / `filter` / `each` / `map`, which consume them and return
|
|
- comparators passed to a function that does not store them
|
|
- `handler-bind` handler bodies
|
|
|
|
That last one is not a convenience. A handler must be able to see the enclosing
|
|
locals — `(fn [c] (push errors c) (invoke-restart 'skip-form))` capturing a local
|
|
`(Vec ParseError)` *is* the accumulation pattern, and conditions are not worth
|
|
building without it. Handlers are strictly non-escaping: the `handler-bind` frame
|
|
outlives every call to them.
|
|
|
|
Captured `Vec`/`Map` are captured **by pointer**, not moved, since the capture
|
|
does not outlive the owner. A non-escaping `fn` is therefore not itself an owner.
|
|
|
|
**3. Escaping closures — deferred.** A `fn` stored in a struct, pushed into a
|
|
container, or returned needs a heap environment and an answer to "which allocator
|
|
owns it, and what happens when the frame arena resets". Do not settle this until
|
|
a concrete use case requires it; revisit it with the optional lightweight
|
|
provenance work.
|
|
|
|
**Early exit inside a `fn`.** `try`, `some`, and `return` in a `fn` body exit the
|
|
`fn`, not the enclosing function — a `fn` is a function. Code that wants to
|
|
propagate out of a loop uses an imperative loop form, not a callback.
|
|
|
|
## Allocators
|
|
|
|
The allocator is part of the calling convention (`context/allocator`,
|
|
`context/temp`). `Vec` and `Map` record the allocator they were created with, so
|
|
`free` and `clone` never need it named again. Allocation uses the current
|
|
implicit allocator by default, as in Odin; an operation never falls back to a
|
|
hidden global allocator, and an explicit allocator can override the context.
|
|
|
|
### The allocator is one type-erased procedure
|
|
|
|
As in Odin (`base/runtime/core.odin:422`, `Allocator_Proc`), an allocator is a
|
|
procedure plus an opaque data pointer, and every operation takes `size` and
|
|
`align` as parameters:
|
|
|
|
| Operation | Meaning |
|
|
|------------------------------------|----------------------------------------|
|
|
| `alloc size align` | new block |
|
|
| `resize p old-size new-size align` | grow or shrink |
|
|
| `free p` | release one block |
|
|
| `free-all` | release everything the allocator holds |
|
|
|
|
It is type-erased on purpose. `Vec` and `Map` are one runtime over `(size,
|
|
align)` and, for `Map`, a compiler-emitted hash and equality pair passed as
|
|
arguments — Odin's `Map_Info` (`base/runtime/core.odin:369`). No generics are
|
|
involved, and none are needed.
|
|
|
|
An allocator declares which operations it implements. Odin's arena answers
|
|
`.Free` with `.Mode_Not_Implemented` (`core/mem/allocators.odin:307`); Flan's
|
|
equivalent is a **capability set** on the allocator value, readable at run time.
|
|
The one that is load-bearing below is `can-free`.
|
|
|
|
### When storage is released
|
|
|
|
There are exactly two release points, and neither of them is a scope.
|
|
|
|
1. **`(free v)`** — explicit. `v` is any move-only value: a `Vec`, a `Map`, a
|
|
struct that owns one, or a struct that owns a resource rather than storage
|
|
(a `Texture2D`, a socket, a file handle — see `drop` below). For a value that
|
|
holds a resource and no storage, `free` runs `drop` and nothing else; it is
|
|
still the release operation, and it is how a `Texture2D` in a local is
|
|
released. Since the repeal, `free` consumes at run time only: nothing marks
|
|
the binding dead, and a second `free` or a later read through it is a
|
|
run-time misbehaviour the allocator or the dev build catches, not a compile
|
|
error.
|
|
2. **Region release** — `(free-all a)` on an allocator, which releases
|
|
everything made from it at once, including storage reachable from bindings
|
|
that are still in scope. The per-frame `(free-all context/temp)` at the top
|
|
of a game loop *is* the frame arena, and it is the normal way arena-tier
|
|
storage dies.
|
|
|
|
**Nothing is released at scope exit.** Not at the end of a `let`, not at the end
|
|
of a function, not at the end of a `with-allocator` body. `with-allocator`
|
|
rebinds the current allocator for its dynamic extent and releases nothing; the
|
|
region it names is released, if ever, by an explicit `free-all` somewhere else.
|
|
|
|
This is deliberate, and it is the point on which the two obvious precedents were
|
|
rejected:
|
|
|
|
- **Odin's `defer delete`** cannot be written here. `defer` is function-scoped
|
|
(`check.ml:505` refuses it in a `let`, a loop or a branch) and, because `let`
|
|
is a block, a top-level `defer` is checked in a scope containing only the
|
|
parameters and globals (`check.ml:1670`). `(defer (free v))` for a `let`-bound
|
|
`v` is **not expressible today**. It becomes expressible with either
|
|
block-scoped `defer` or a sequential top-of-body binder; until one of those
|
|
exists, no idiom in this spec may depend on it.
|
|
- **Carp's scope-end frees** are a whole-program linear analysis that inserts a
|
|
teardown call at every binding's last use (`Memory.hs`, and `Info.hs`'s
|
|
`Deleter`). Carp could not reconcile that with an arena and therefore has no
|
|
allocator abstraction at all. A release point the programmer cannot see is
|
|
exactly what makes a frame arena unstateable.
|
|
|
|
**Leaking is defined behaviour.** Storage that is never freed and whose
|
|
allocator is never released is leaked, and for the permanent arena (symbols,
|
|
code) and the dev/REPL tier that is the correct program. "Did you forget to
|
|
free" is not a type question here; it is an allocator-tier question, and dev
|
|
builds answer it by reporting a general-purpose allocator's outstanding blocks
|
|
when it is destroyed.
|
|
|
|
**`free` applies to a whole owner.** It recurses structurally into owning
|
|
fields. A field is never freed on its own: `(free (.textures e))` is refused,
|
|
because it would leave `e` partly dead with no way to say so. The recursion is
|
|
the half of this that is not built — see the next section, which says what is
|
|
built instead and what `free` does where the recursion would have been.
|
|
|
|
### A container of owning elements lives in a region
|
|
|
|
A `(Vec Value)` where a `Value` may itself hold a `(Vec Value)` — the recursive
|
|
dynamic value an EDN reader has to answer with when it is handed no target
|
|
struct type — was refused outright while this spec described only the recursion
|
|
above. The reason given was always the same one: the container runtime is
|
|
type-erased, so it copies and releases slots **bytewise** and cannot reach
|
|
inside a slot. A `free` would release the slots and leave every block they
|
|
point at stranded.
|
|
|
|
That reason is about **teardown**, and it does not hold for a region. `free-all`
|
|
never releases an individual slot; it takes the whole region, and every block
|
|
the elements own is in it, because they came out of it. So the refusal was
|
|
over-broad, and the rule is now narrower:
|
|
|
|
> **A container whose element type owns storage, transitively, may only
|
|
> allocate from an allocator that lacks `can-free`.** Checked at the point of
|
|
> construction and at any later growth: one branch per container, never per
|
|
> element.
|
|
|
|
Three things follow, and each is a decision rather than a consequence:
|
|
|
|
- **It is a run-time branch, not a compile-time refusal.** `can-free` is a
|
|
capability on an allocator *value*, and `with-allocator` rebinds a dynamic
|
|
variable, so which tier a `(vec-new)` will meet is not a property of the
|
|
place it is written. What is decided at compile time is only *whether to
|
|
ask* — that is a property of the element type. The question is asked of
|
|
`can-free` rather than of "is this an arena" so that a fixed backing buffer
|
|
written later answers it the same way for the same reason.
|
|
- **`free` on such a container is refused, not silently shallow.** It cannot
|
|
recurse — that is the whole premise — and releasing the outer block alone
|
|
would be "I freed it" written over a program that stranded everything
|
|
inside. A reader will assume recursion, so the refusal names `free-all`
|
|
instead, which is reachable by construction: such a container is
|
|
region-allocated or it does not exist.
|
|
- **`clone` on such a container is not possible and stays refused.** This is
|
|
the half a region does *not* answer, and it is a different failure from the
|
|
teardown one the two were once stated together as. `clone` promises a deep,
|
|
independent copy; a bytewise one hands back a second container whose elements
|
|
still point into the first one's blocks. Cloning into a *second* region is
|
|
worse rather than better, because releasing that region leaves the copy's
|
|
elements pointing into a region that is still live. `at` and `get` are
|
|
untouched: they promise nothing, and the alias they hand back is an alias
|
|
into storage nobody individually owns, which is the bargain a region is.
|
|
|
|
A **struct field or a `defdata` case's field** of such a container type is
|
|
admitted for the same reason and only for that reason — the field's container
|
|
can only have been built against a region, so the aggregate's whole graph is
|
|
released by one `free-all`. The aggregate itself **stays copyable**: ownership
|
|
is not made transitive through a name, because that is the model recursive
|
|
teardown would have needed. So a `Value` is copied bytewise like any other
|
|
value, and two copies share the inner blocks. In a region that is aliasing and
|
|
not a double free — and mutating through one alias is visible through the
|
|
other, which is a logic bug and the price of the bargain, exactly as it is in
|
|
Odin. A field holding a container whose elements own
|
|
*nothing* stays refused: nothing would force that one into a region, and two
|
|
copies of the aggregate would be two headers over one heap block. An untagged
|
|
`defunion` is not part of this at all: a move-only member in one is refused for
|
|
a reason that is not teardown — nothing records which member is live, so there
|
|
is no fact a release could read — and a region answers the teardown question
|
|
without touching that one.
|
|
|
|
A `Pool` was admitted with the others while it existed (the second repeal
|
|
removed the type), and it was the one where the trade is not
|
|
identical, because `(release p h)` recycles a slot while the pool lives on.
|
|
Whatever the dead element owned stays allocated until `free-all`. That is a
|
|
region leak bounded by the region, which is what a region already is; it is not
|
|
a use-after-free, because nothing was released.
|
|
|
|
### Dev builds detect a released region
|
|
|
|
A `Vec` or `Map` records its allocator (see above). In a dev build it also
|
|
records that allocator's **epoch** — a counter the allocator bumps on every
|
|
`free-all`. Any operation on a container whose recorded epoch has moved traps,
|
|
naming the allocation site and the release site. This is a second and separate
|
|
counter from the per-`Vec` generation word that catches stale slices; the two
|
|
answer different questions and must not be conflated. Both are dev-only: the
|
|
release layout of a `Vec` is `ptr + len + cap + allocator` and nothing more.
|
|
|
|
This is what covers a use after `free-all`, including the case the section
|
|
above makes reachable: an inner container's header copied *out* of an
|
|
arena-held element into a local before the release still traps on the next
|
|
operation. It works because an `Allocator` is a **pointer** to the allocator
|
|
and not a copy of one — a copied-by-value allocator would give each copy its
|
|
own epoch, and a copy taken before the bump would never notice it.
|
|
|
|
### `drop` — owning something that is not memory
|
|
|
|
A type may name one hook:
|
|
|
|
```
|
|
(drop Texture [t (Ptr Texture)] ...)
|
|
```
|
|
|
|
It takes a **pointer, not the value**, which is Carp's shape and for Carp's
|
|
reason. Carp shipped `delete` — auto-generated per type, consuming, and
|
|
responsible for the recursive teardown of every field — and then had to add
|
|
`drop` separately, because a user who redefined `delete` to close a file had to
|
|
re-implement that whole teardown by hand. Carp's `drop` is looked up per
|
|
teardown site (`Memory.hs:806`, `getDropFunc`, at `RefTy t` where `delete` is
|
|
`FuncTy [t]`) and emitted immediately before the teardown call
|
|
(`Emit.hs:1042`), so the hook *composes with* compiler-generated teardown rather
|
|
than replacing it. Flan takes that arrangement unchanged.
|
|
|
|
- `(free v)` runs `drop` on `v` first, then tears down `v`'s owning fields in
|
|
declaration order, each by the same rule.
|
|
- A `drop` hook may read and mutate through its pointer. It may **not** move out
|
|
of the value, and it may not `free` it.
|
|
- A type has a `drop` hook transitively: a struct any of whose fields has one,
|
|
has one.
|
|
- **A type with a `drop` hook is move-only and cannot be `clone`d.** Move-only,
|
|
because a value type copies on assignment and two copies of one socket would
|
|
each run `drop`; the same argument that makes a `Vec` field move-only. Not
|
|
`clone`able, because duplicating a texture id or a file descriptor is not the
|
|
compiler's decision to make — Carp needed a separate `copy` interface for
|
|
exactly this. A type that *can* be duplicated says so with an ordinary named
|
|
function.
|
|
|
|
**Nothing runs `drop` when an arena resets — because such a value cannot be in
|
|
an arena.** Constructing a container whose element type transitively has a
|
|
`drop` hook, or allocating such a value, against an allocator that lacks
|
|
`can-free` is **refused at the point of construction**: one branch per
|
|
container, not per element. This is the rule the section on containers of
|
|
owning elements already states and already implements, with one difference that
|
|
matters if `drop` is ever built: a hook is the case a region *cannot* answer —
|
|
releasing the region does not close the socket — so a hooked element would have
|
|
to be refused against an arena rather than *required* to be in one. The two
|
|
questions must stay apart. Asking "does this element own anything" where "does
|
|
it have a hook" was meant would refuse nested containers in the frame tier,
|
|
which is precisely the case the frame tier exists for. `free-all` therefore never has to walk a list of
|
|
registered destructors, which is what keeps the frame tier's reset genuinely
|
|
free (plan.org's memory table) and keeps a destructor list — an allocation
|
|
nobody wrote — out of the core.
|
|
|
|
The consequence, stated plainly because a reader will assume otherwise:
|
|
|
|
> **`drop` is not a destructor.** A `Texture2D` held in a local, a parameter, or
|
|
> a plain stack struct never has `drop` run, because Flan has no scope-end
|
|
> anything. `drop` fires at exactly one place — inside `free` — and resources in
|
|
> locals are released explicitly, exactly as memory is. Carp's `drop` fires at
|
|
> scope end only because Carp has scope-end frees, which the section above
|
|
> rejects.
|
|
|
|
### Alignment
|
|
|
|
Alignment is a property of the **type**, computed at the **call site**, and
|
|
passed as a **parameter** to the type-erased allocator. All three, and they are
|
|
not alternatives.
|
|
|
|
Odin arranges it exactly this way: `elem_align` is threaded through every
|
|
type-erased dynamic-array entry point (`base/runtime/dynamic_array_internal.odin`
|
|
— `__dynamic_array_reserve`, `__dynamic_array_resize`, `__dynamic_array_append`),
|
|
and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:350`). The
|
|
monomorphised wrapper is the only place the concrete type is known, so it is the
|
|
only place that can produce the number.
|
|
|
|
Alignment is **not stored** in the `Vec` or `Map` header. That is safe because of
|
|
a condition worth writing down: every operation that needs it — `push`,
|
|
`reserve`, `resize`, `clone`, `free` — is compiler-emitted at a site where the
|
|
concrete element type is known. Any future type-erased teardown path would break
|
|
that condition; there is not to be one. (This is the second reason the `drop`
|
|
registry above was rejected: it would have been exactly such a path.)
|
|
|
|
The natural alignment of `T` is `align-of T`. Raising it above natural — 16
|
|
bytes for `#soa` and for component-wise fixed arrays — is declared **on the
|
|
type**, so that every site computing `align-of T` gets the raised number with no
|
|
further plumbing. The surface syntax for that declaration is deliberately not
|
|
fixed here; nothing is built that needs it yet.
|
|
|
|
### Untagged unions and what a read of one means
|
|
|
|
`defunion` is C's union: the members overlay one storage, the size is the
|
|
largest of them, the alignment the strictest, and **nothing records which
|
|
member was written**. It is not `defdata`, which is the tagged sum — a case, its
|
|
fields, and a tag that steers every `match`.
|
|
|
|
**Reading a member that was not the one last written is defined**, and it is
|
|
the one place in this language where bytes win over safety on purpose. It reads
|
|
the storage through that member's type: the layout is the target's, the bytes
|
|
are the bytes, and the read is a reinterpretation of them. C leaves this to the
|
|
implementation; Flan does not, because both uses the type exists for *are* that
|
|
read. Binding a C header means holding the union the library holds and reading
|
|
whichever member the library's own tag says is live — a tag the compiler cannot
|
|
see, since the rule relating them is prose in a manual. Overlaying an `f32` on a
|
|
`u32` to look at its bits is the same read. A rule that refused it would be
|
|
refusing the type.
|
|
|
|
What is **not** promised is anything about bytes nobody wrote. A member wider
|
|
than the one last stored reads its own size, and the tail is indeterminate
|
|
exactly as a struct's padding is. ZII narrows that to almost nothing in
|
|
practice: a union is all-bytes-zero unless `uninit` says otherwise.
|
|
|
|
Three things a union may not do, and each for a reason that does not expire:
|
|
|
|
- **No owning member** — until the second repeal, which admits one: nothing
|
|
records which member is live, and since nothing (the program included) can
|
|
free the right one through the union itself, the tag the program keeps
|
|
beside it is the way, C's way. The rest of this entry records the rule as
|
|
designed. Nothing knows which member is live, so nothing can
|
|
tear one down. Unlike the struct and `defdata` refusals, this is not waiting
|
|
on recursive teardown — there is no fact for teardown to read, and freeing
|
|
the wrong member is a free of a pointer that was an `f64` a moment ago.
|
|
- **No `bool` member, at any depth.** An `i1` loaded out of a byte that is
|
|
neither 0 nor 1 is a value the optimiser is entitled to assume cannot exist,
|
|
and a union is the only type that can produce one. Hold a `u8` and compare it.
|
|
- **Not a map key.** A member narrower than the union leaves the rest of the
|
|
bytes indeterminate, so two values agreeing about everything written would
|
|
still hash apart.
|
|
|
|
`uninit` on a union **is** allowed, unlike on a `defdata`. The refusal there is
|
|
not about garbage: it is that a tag no case names falls past every comparison in
|
|
a `match` into a block the optimiser may treat as unreachable. An untagged union
|
|
steers nothing, so `uninit` makes its bytes arbitrary and changes nothing else —
|
|
which is what it means on an `i64`.
|
|
|
|
### Allocation failure
|
|
|
|
**No allocating operation returns an error, and none can fail silently.** When
|
|
the allocator cannot satisfy a request, the operation signals
|
|
|
|
```
|
|
(StorageExhausted {.bytes n .align a .allocator id})
|
|
```
|
|
|
|
with `error`, whose type is `Never` (spec-conditions.md §2), inside a
|
|
`restart-case` offering `retry`. This is one rule over *every* allocating
|
|
operation — `vec-new`, `map-new`, `push`, `put`, `reserve`, `clone`, and
|
|
`bytes` since it became a copy (2026-09-20; `bytes-view` is the free view) —
|
|
so their result types stay `(Vec T)`, `()`, `()` and so on, with no `Result`
|
|
and no out-parameter anywhere.
|
|
|
|
What that buys, against the alternative: Odin's `append` returns an ignorable
|
|
`Allocator_Error` (`base/runtime/core_builtin.odin:767`,
|
|
`#optional_allocator_error`), and the type-erased path underneath returns the old
|
|
length on a failed reserve, marked
|
|
`// TODO(bill): Better error handling for failed reservation`
|
|
(`base/runtime/dynamic_array_internal.odin`). An `append` that appends nothing
|
|
and says nothing is the outcome this rule exists to make impossible.
|
|
|
|
- The condition is a value struct on the signalling frame's stack, with fixed
|
|
numeric fields and **no rendered message**, because formatting would allocate
|
|
and this is the one path that must not. Rendering happens in the handler or the
|
|
break loop, where a working allocator is known.
|
|
- Unhandled, `error` enters the dev break loop or aborts in release
|
|
(spec-conditions.md §2). It is never a no-op; `signal` is not used here.
|
|
- A handler that frees something, releases a scratch region, or grows the arena
|
|
and then invokes `retry` re-attempts the same request. A handler that wants a
|
|
*different* allocator needs a restart taking an argument, which does not exist
|
|
yet; until it does, such a handler rebinds the context allocator and retries.
|
|
- Because an allocating operation can transfer, every caller of one checks the
|
|
transfer channel after the call (spec-conditions.md §6). `push` is not a leaf
|
|
call, and that per-call-site check is the price of not being Odin.
|
|
|
|
**This is the named exception to plan.org's "restarts go at the resync point,
|
|
once".** That rule is right for program-level errors and wrong here: a restart
|
|
established at a parser's top-level loop cannot re-attempt an allocation, and
|
|
only the allocation site can. Compiler-emitted restarts at the point of failure
|
|
are the exception, in the same way Common Lisp's runtime establishes
|
|
`store-value` at an unbound-variable error rather than at a resync point. No
|
|
*user* code establishes restarts below a resync point.
|
|
|
|
### Open: catching a use-after-release statically
|
|
|
|
Both release points above are dynamic, and the frame arena is the reason. A
|
|
static rule — "a move-only value constructed under a given allocator may not
|
|
outlive it" — needs to know statically which allocator a construction used, and
|
|
`with-allocator` plus `context/allocator` are precisely the mechanisms that deny
|
|
that knowledge. The lexical subset (a value made inside a `with-allocator` body
|
|
and returned out of it) is checkable; the general case is not; and shipping only
|
|
the subset would teach a rule that silently stops applying at the loop where it
|
|
matters most.
|
|
|
|
Until a provenance pass exists (plan.org open decision #3), the answer is the
|
|
dev-build epoch trap above: detection, loud and immediate, rather than
|
|
prevention. Settling this needs one thing that does not exist yet — real Flan
|
|
programs using arenas, to say whether the escapes that actually occur are
|
|
lexical. It is not settleable from the design alone, and it is not papered over
|
|
here.
|