flan/spec-memory.md

# Spec 1 — Ownership, containers, and copies

Status: **frozen**. Closes plan.org open decisions #6 and #10, and resolves the
contradiction between "value structs copy on assignment" and owning containers.
The Allocators section additionally settles the four things that had to be
decided before `Vec` and `Map` are written: when storage is released, the `drop`
hook, alignment, and allocation failure. One question there is left open on
purpose and says so.
Everything else in the design references this vocabulary. It governs plain
fixed-layout `struct` values, not the separately planned managed `class`
facility (see plan.org, "Managed classes").

## The four container types

| Notation  | Layout            | Assignment | Owns storage | Allocator |
|-----------|-------------------|------------|--------------|-----------|
| `[n T]`   | n contiguous `T`  | copies     | no (inline)  | —         |
| `[T]`     | ptr + len         | copies the *view* | no    | —         |
| `(Vec T)` | ptr + len + cap   | **moves**  | yes          | stored    |
| `(Map K V)` | open-addressed, flat key/value arrays | **moves** | yes | stored |

- `[n T]` is a value. It lives wherever it is declared, copies on assignment and
  on pass-by-value, and is what `defconst colors [4 u32] ...` and
  `(defvar grid [rows [cols u32]] ...)` are.
- `[T]` is a **non-owning slice**: a borrowed window into a `[n T]`, a `(Vec T)`,
  or a literal in read-only memory. Copying a slice copies ptr+len, never the
  elements. A slice may be `const`-qualified; freeing through one is not possible
  because a slice has no allocator and no `cap`.
- `(Vec T)` and `(Map K V)` are **move-only**. Binding, passing, or returning one
  transfers ownership; the source binding is dead afterwards and using it is a
  compile error. There is no shallow copy, so there is no double free.

## Maps — first implementation

Every map is homogeneous: `(Map K V)` has one key type and one value type. The
first implementation accepts only built-in structural key types: integers,
enums, strings, fixed arrays, and value structs composed recursively from those
types. Tuples and triples join that set when they are introduced. `Ptr`, slices,
`Vec`, and `Map` are not map keys yet.

Equality and hashing for those keys are compiler-provided structural operations
and not type classes. They are not available to an unconstrained type variable
either; a variable that means to key a map declares `hashable?` in the signature
that binds it, and the refusal then lands at the call site that names an
unhashable key. An empty map names its key and value types, because a global
cannot hold one and there is therefore no declaration for it to take a type from:

```
(let [enemies (map-new string Enemy)]
  ...)
```

`(get m k)` returns `(Option V)`: absence is `None`, not an untyped `nil`.
`(put m k v)` is the upsert operation and returns `()`; it either inserts or
replaces. `(set (get m k) v)` is not map syntax.

The first Map implementation admits copyable keys and values only, so `get`
returns a copy. Move-aware lookup, removal, and owned entries are deferred until
`Vec`/`Map` values are supported in maps; the map itself remains an owning,
move-only container.

## Copying is always explicit

`(clone x)` produces an independent deep copy of a `Vec`/`Map` using the current
allocator; `(clone x alloc)` names one. Value types (`[n T]`, structs of value
types, primitives) need no `clone` — assignment already copies them.

A struct containing a `Vec` field is itself move-only. Ownership is structural,
not declared: a type is a value type iff all of its fields are **and it declares
no `drop` hook** (see Allocators). A `drop` hook makes a type move-only for the
same reason a `Vec` field does — exactly one owner, so the hook fires exactly
once — and a type with one cannot be `clone`d.

## Borrowing

- `(as-slice v)` / `(as-slice v lo hi)` view a `Vec` or fixed array as `[T]`.
- **The first implementation follows Zig/Odin's explicit model, not Rust's
  borrow checker.** A slice is invalidated by any operation that may reallocate
  its owner (`push`, `put`, `reserve`); its user is responsible for respecting
  that contract. Dev builds carry a generation word on `Vec` and trap on use of
  a stale slice. `Ptr` is the explicit lower-level escape hatch and has the
  same lifetime contract.
- A future lightweight provenance pass may reject the obvious mistakes (a
  borrow of a local escaping, use after an owner moves, and reallocation with a
  live borrow). It must not require Rust-style lifetime annotations or dictate
  an ECS-shaped object model. Long-lived graph links use `(Handle a)`; temporary
  graphs may use explicitly managed, stable region storage.
- Cross-referencing long-lived objects uses `(Handle a)` into a pool, never a
  raw pointer or slice. A stale handle is detectable.

## Taking an address

`(addr x)` yields `(Ptr T)` for any assignable place `x` — a local, a global, a
field, an element. The pointer is non-owning and does not extend anything's
lifetime, so `addr` of a local is only valid while that frame lives. This is the
same escape question as case 3 below. The first implementation leaves it as an
explicit lifetime contract; a future provenance pass can check it.

`addr` is how a value struct is shared mutably without an allocator — recursive
descent over a cursor, an entity passed down a call chain — and it is why
milestone 2 needs no heap at all. In the first implementation its non-escape
rule is an explicit programmer contract, aided by dev checks; the future
provenance pass above may enforce it.

## Places — what `set` accepts

A fixed set of assignable forms, not a `setf`-style extensible place mechanism:

```
(set x v)              ; a local or a defvar
(set (.field x) v)     ; struct field; x may be a struct, (Ptr S) or (Handle S)
(set (at a i ...) v)   ; fixed array, slice, or Vec element
(set (deref p) v)      ; whole-object store through a pointer
```

`.field` and `at` auto-deref exactly one pointer or handle level, which is what
makes `(set (.hp e) ...)` legal when `e : (Ptr Enemy)` and illegal when
`e : Enemy` bound by value.

**Mutating something you matched.** Pattern bindings bind *values*, so a matched
struct is a copy. To mutate in place, obtain a pointer first — the pointer is
visible in the type:

```
(match (resolve w h)          ; (Option (Ptr Enemy))
  (Some e) (set (.hp e) ...)  ; e : (Ptr Enemy), field access derefs
  None     ...)
```

`deref` yields a value; `resolve` yields a pointer. Both are overloaded on
`(Ptr a)` and `(Handle a)` and resolve at compile time.

## Generics

Parametric polymorphism is monomorphisation, with **no type classes**. A type
variable is written `$t` wherever a *type* goes — a parameter, the return type,
or nested as `[$t]` or `(Vec $t)` — and bare `t` where a type's *name* is an
argument in expression position, as in `(vec-new t)` and the cast `(t x)`. A
generic body is checked **abstractly**, with nothing substituted, so the rule
below bites at the definition rather than at whichever call site first
instantiates it:

> A type variable `$t` supports only what every type supports: move, `clone`,
> field-free storage. It does **not** support `=`, `<`, `+`, or `hash`.

What makes that liveable is a `where` clause of compile-time type predicates,
written as a map at the head of the body. There are five — `ordered?`, `equal?`,
`hashable?`, `numeric?`, `copyable?` — they are not type classes because a
predicate carries no implementations and merely gates a builtin the compiler
already has, and they entail one another in one direction, so one clause usually
does. A variable is move-only by default and `copyable?` is the opt-out, because
whether a variable moves is not decidable abstractly. plan.org's Types section
has the full account.

```
(defn sort! [s [$t]] ()
  {:where (ordered? $t)}
  ...)
```

Without such a clause the operator is rejected where it is written, not silently
instantiated, and the operation is passed in explicitly as a function value
instead:

```
(defn largest [xs [$t]  gt (Fn [$t $t] bool)] (Option $t) ...)
```

The value handed to such a parameter is a named `defn`. An `fn` cannot be written
inline into it, because the generic body is checked with nothing substituted and
there is no concrete type yet for the `fn`'s own parameters to come from; that
restriction lifts at a monomorphic call site, where `reduce`'s and `filter`'s
callbacks are ordinary inline `fn`s.

The alternatives to predicates — compile-time interfaces, or intrinsics
restricted to primitives — remain deliberately deferred until the base checker is
stable (build sequence milestone 4). The ceiling is that nobody can supply a
user-defined `<`.

`println` is the deliberate exception. It is a compiler-provided,
type-directed intrinsic: monomorphisation selects or emits a structural printer
for each concrete instantiation, so `(println x)` is legal for `x : $t` without
introducing a `Printable` type class. Structs, fixed arrays, options and,
eventually, Vecs and Maps print structurally. `Ptr` and `Handle` print their
address or identity rather than recursively dereferencing, and collection
printers impose depth and length limits. `any` and `Error` use their runtime type
metadata. User generic code still passes an explicit function for every other
operation that depends on a type's structure.

Type arguments are **inferred at call sites** from the argument types; there is
no explicit instantiation syntax in the first implementation. A type variable
that appears only in the return type is therefore an error.

## Function values

Three cases, split by whether the value escapes the frame that made it.

**1. `(Fn [T1 T2] R)` — a plain, stable function pointer.** No captured
environment or allocation. In a dev build, a reference to a top-level `defn`
is the address of a stable trampoline that loads that function version's
indirection cell and calls its current body; it is never the address of a
particular body. Thus stored callbacks and ordinary calls observe a later
*body* redefinition, as in Common Lisp. Release builds may call the body
directly because it cannot be redefined.

A signature-changing redefinition makes a new internal function version and a
new trampoline ABI. Newly compiled code resolves the source name to that new
version. Existing callers and stored `Fn` values keep their old trampoline and
therefore safely call the old version. The session immediately warns at every
tracked caller source location that still targets the old signature; recompiling
one either retargets it successfully or reports an ordinary type error. This is
what raylib callbacks, hot-reload cells, and function parameters use. A
top-level `defn` is one, so `(largest hps >)` passes `>` at `i32` directly. This
is the only function type that may cross an FFI boundary or sit in a reload
cell.

**2. Non-escaping `fn` — captures by value into a stack environment.** A `fn`
whose value provably does not outlive the frame that created it gets an
environment allocated in that frame and captures the named locals **by value**
at the point of creation. No heap, no allocator, no lifetime question. This
covers essentially every lambda in practice:

- callbacks to `reduce` / `filter` / `each` / `map`, which consume them and return
- comparators passed to a function that does not store them
- `handler-bind` handler bodies

That last one is not a convenience. A handler must be able to see the enclosing
locals — `(fn [c] (push errors c) (invoke-restart 'skip-form))` capturing a local
`(Vec ParseError)` *is* the accumulation pattern, and conditions are not worth
building without it. Handlers are strictly non-escaping: the `handler-bind` frame
outlives every call to them.

Captured `Vec`/`Map` are captured **by pointer**, not moved, since the capture
does not outlive the owner. A non-escaping `fn` is therefore not itself an owner.

**3. Escaping closures — deferred.** A `fn` stored in a struct, pushed into a
container, or returned needs a heap environment and an answer to "which allocator
owns it, and what happens when the frame arena resets". Do not settle this until
a concrete use case requires it; revisit it with the optional lightweight
provenance work.

**Early exit inside a `fn`.** `try`, `some`, and `return` in a `fn` body exit the
`fn`, not the enclosing function — a `fn` is a function. Code that wants to
propagate out of a loop uses an imperative loop form, not a callback.

## Allocators

The allocator is part of the calling convention (`context/allocator`,
`context/temp`). `Vec` and `Map` record the allocator they were created with, so
`free` and `clone` never need it named again. Allocation uses the current
implicit allocator by default, as in Odin; an operation never falls back to a
hidden global allocator, and an explicit allocator can override the context.

### The allocator is one type-erased procedure

As in Odin (`base/runtime/core.odin:422`, `Allocator_Proc`), an allocator is a
procedure plus an opaque data pointer, and every operation takes `size` and
`align` as parameters:

| Operation                          | Meaning                                |
|------------------------------------|----------------------------------------|
| `alloc size align`                 | new block                              |
| `resize p old-size new-size align` | grow or shrink                         |
| `free p`                           | release one block                      |
| `free-all`                         | release everything the allocator holds |

It is type-erased on purpose. `Vec` and `Map` are one runtime over `(size,
align)` and, for `Map`, a compiler-emitted hash and equality pair passed as
arguments — Odin's `Map_Info` (`base/runtime/core.odin:369`). No generics are
involved, and none are needed.

An allocator declares which operations it implements. Odin's arena answers
`.Free` with `.Mode_Not_Implemented` (`core/mem/allocators.odin:307`); Flan's
equivalent is a **capability set** on the allocator value, readable at run time.
The one that is load-bearing below is `can-free`.

### When storage is released

There are exactly two release points, and neither of them is a scope.

1. **`(free v)`** — explicit. `v` is any move-only value: a `Vec`, a `Map`, a
   struct that owns one, or a struct that owns a resource rather than storage
   (a `Texture2D`, a socket, a file handle — see `drop` below). For a value that
   holds a resource and no storage, `free` runs `drop` and nothing else; it is
   still the release operation, and it is how a `Texture2D` in a local is
   released. `free` consumes its argument exactly as any other move does: the source
   binding is dead afterwards and using it is a compile error. That rule is
   already what makes a double free unrepresentable, so `free` needs no new
   analysis.
2. **Region release** — `(free-all a)` on an allocator, which releases
   everything made from it at once, including storage reachable from bindings
   that are still in scope. The per-frame `(free-all context/temp)` at the top
   of a game loop *is* the frame arena, and it is the normal way arena-tier
   storage dies.

**Nothing is released at scope exit.** Not at the end of a `let`, not at the end
of a function, not at the end of a `with-allocator` body. `with-allocator`
rebinds the current allocator for its dynamic extent and releases nothing; the
region it names is released, if ever, by an explicit `free-all` somewhere else.

This is deliberate, and it is the point on which the two obvious precedents were
rejected:

- **Odin's `defer delete`** cannot be written here. `defer` is function-scoped
  (`check.ml:505` refuses it in a `let`, a loop or a branch) and, because `let`
  is a block, a top-level `defer` is checked in a scope containing only the
  parameters and globals (`check.ml:1670`). `(defer (free v))` for a `let`-bound
  `v` is **not expressible today**. It becomes expressible with either
  block-scoped `defer` or a sequential top-of-body binder; until one of those
  exists, no idiom in this spec may depend on it.
- **Carp's scope-end frees** are a whole-program linear analysis that inserts a
  teardown call at every binding's last use (`Memory.hs`, and `Info.hs`'s
  `Deleter`). Carp could not reconcile that with an arena and therefore has no
  allocator abstraction at all. A release point the programmer cannot see is
  exactly what makes a frame arena unstateable.

**Leaking is defined behaviour.** Storage that is never freed and whose
allocator is never released is leaked, and for the permanent arena (symbols,
code) and the dev/REPL tier that is the correct program. "Did you forget to
free" is not a type question here; it is an allocator-tier question, and dev
builds answer it by reporting a general-purpose allocator's outstanding blocks
when it is destroyed.

**`free` applies to a whole owner.** It recurses structurally into owning
fields. A field is never freed on its own: `(free (.textures e))` is refused,
because it would leave `e` partly dead with no way to say so.

### Dev builds detect a released region

A `Vec` or `Map` records its allocator (see above). In a dev build it also
records that allocator's **epoch** — a counter the allocator bumps on every
`free-all`. Any operation on a container whose recorded epoch has moved traps,
naming the allocation site and the release site. This is a second and separate
counter from the per-`Vec` generation word that catches stale slices; the two
answer different questions and must not be conflated. Both are dev-only: the
release layout of a `Vec` is `ptr + len + cap + allocator` and nothing more.

### `drop` — owning something that is not memory

A type may name one hook:

```
(drop Texture [t (Ptr Texture)] ...)
```

It takes a **pointer, not the value**, which is Carp's shape and for Carp's
reason. Carp shipped `delete` — auto-generated per type, consuming, and
responsible for the recursive teardown of every field — and then had to add
`drop` separately, because a user who redefined `delete` to close a file had to
re-implement that whole teardown by hand. Carp's `drop` is looked up per
teardown site (`Memory.hs:806`, `getDropFunc`, at `RefTy t` where `delete` is
`FuncTy [t]`) and emitted immediately before the teardown call
(`Emit.hs:1042`), so the hook *composes with* compiler-generated teardown rather
than replacing it. Flan takes that arrangement unchanged.

- `(free v)` runs `drop` on `v` first, then tears down `v`'s owning fields in
  declaration order, each by the same rule.
- A `drop` hook may read and mutate through its pointer. It may **not** move out
  of the value, and it may not `free` it.
- A type has a `drop` hook transitively: a struct any of whose fields has one,
  has one.
- **A type with a `drop` hook is move-only and cannot be `clone`d.** Move-only,
  because a value type copies on assignment and two copies of one socket would
  each run `drop`; the same argument that makes a `Vec` field move-only. Not
  `clone`able, because duplicating a texture id or a file descriptor is not the
  compiler's decision to make — Carp needed a separate `copy` interface for
  exactly this. A type that *can* be duplicated says so with an ordinary named
  function.

**Nothing runs `drop` when an arena resets — because such a value cannot be in
an arena.** Constructing a container whose element type transitively has a
`drop` hook, or allocating such a value, against an allocator that lacks
`can-free` is **refused at the point of construction**: one branch per
container, not per element. `free-all` therefore never has to walk a list of
registered destructors, which is what keeps the frame tier's reset genuinely
free (plan.org's memory table) and keeps a destructor list — an allocation
nobody wrote — out of the core.

The consequence, stated plainly because a reader will assume otherwise:

> **`drop` is not a destructor.** A `Texture2D` held in a local, a parameter, or
> a plain stack struct never has `drop` run, because Flan has no scope-end
> anything. `drop` fires at exactly one place — inside `free` — and resources in
> locals are released explicitly, exactly as memory is. Carp's `drop` fires at
> scope end only because Carp has scope-end frees, which the section above
> rejects.

### Alignment

Alignment is a property of the **type**, computed at the **call site**, and
passed as a **parameter** to the type-erased allocator. All three, and they are
not alternatives.

Odin arranges it exactly this way: `elem_align` is threaded through every
type-erased dynamic-array entry point (`base/runtime/dynamic_array_internal.odin`
— `__dynamic_array_reserve`, `__dynamic_array_resize`, `__dynamic_array_append`),
and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:350`). The
monomorphised wrapper is the only place the concrete type is known, so it is the
only place that can produce the number.

Alignment is **not stored** in the `Vec` or `Map` header. That is safe because of
a condition worth writing down: every operation that needs it — `push`,
`reserve`, `resize`, `clone`, `free` — is compiler-emitted at a site where the
concrete element type is known. Any future type-erased teardown path would break
that condition; there is not to be one. (This is the second reason the `drop`
registry above was rejected: it would have been exactly such a path.)

The natural alignment of `T` is `align-of T`. Raising it above natural — 16
bytes for `#soa` and for component-wise fixed arrays — is declared **on the
type**, so that every site computing `align-of T` gets the raised number with no
further plumbing. The surface syntax for that declaration is deliberately not
fixed here; nothing is built that needs it yet.

### Allocation failure

**No allocating operation returns an error, and none can fail silently.** When
the allocator cannot satisfy a request, the operation signals

```
(StorageExhausted {.bytes n .align a .allocator id})
```

with `error`, whose type is `Never` (spec-conditions.md §2), inside a
`restart-case` offering `retry`. This is one rule over *every* allocating
operation — `vec-new`, `map-new`, `push`, `put`, `reserve`, `clone` — so their
result types stay `(Vec T)`, `()`, `()` and so on, with no `Result` and no
out-parameter anywhere.

What that buys, against the alternative: Odin's `append` returns an ignorable
`Allocator_Error` (`base/runtime/core_builtin.odin:767`,
`#optional_allocator_error`), and the type-erased path underneath returns the old
length on a failed reserve, marked
`// TODO(bill): Better error handling for failed reservation`
(`base/runtime/dynamic_array_internal.odin`). An `append` that appends nothing
and says nothing is the outcome this rule exists to make impossible.

- The condition is a value struct on the signalling frame's stack, with fixed
  numeric fields and **no rendered message**, because formatting would allocate
  and this is the one path that must not. Rendering happens in the handler or the
  break loop, where a working allocator is known.
- Unhandled, `error` enters the dev break loop or aborts in release
  (spec-conditions.md §2). It is never a no-op; `signal` is not used here.
- A handler that frees something, releases a scratch region, or grows the arena
  and then invokes `retry` re-attempts the same request. A handler that wants a
  *different* allocator needs a restart taking an argument, which does not exist
  yet; until it does, such a handler rebinds the context allocator and retries.
- Because an allocating operation can transfer, every caller of one checks the
  transfer channel after the call (spec-conditions.md §6). `push` is not a leaf
  call, and that per-call-site check is the price of not being Odin.

**This is the named exception to plan.org's "restarts go at the resync point,
once".** That rule is right for program-level errors and wrong here: a restart
established at a parser's top-level loop cannot re-attempt an allocation, and
only the allocation site can. Compiler-emitted restarts at the point of failure
are the exception, in the same way Common Lisp's runtime establishes
`store-value` at an unbound-variable error rather than at a resync point. No
*user* code establishes restarts below a resync point.

### Open: catching a use-after-release statically

Both release points above are dynamic, and the frame arena is the reason. A
static rule — "a move-only value constructed under a given allocator may not
outlive it" — needs to know statically which allocator a construction used, and
`with-allocator` plus `context/allocator` are precisely the mechanisms that deny
that knowledge. The lexical subset (a value made inside a `with-allocator` body
and returned out of it) is checkable; the general case is not; and shipping only
the subset would teach a rule that silently stops applying at the loop where it
matters most.

Until a provenance pass exists (plan.org open decision #3), the answer is the
dev-build epoch trap above: detection, loud and immediate, rather than
prevention. Settling this needs one thing that does not exist yet — real Flan
programs using arenas, to say whether the escapes that actually occur are
lexical. It is not settleable from the design alone, and it is not papered over
here.