flan/spec-memory.md

# Spec 1 — Ownership, containers, and copies

Status: **frozen**. Closes plan.org open decisions #6 and #10, and resolves the
contradiction between "value structs copy on assignment" and owning containers.
The Allocators section additionally settles the four things that had to be
decided before `Vec` and `Map` are written: when storage is released, the `drop`
hook, alignment, and allocation failure. One question there is left open on
purpose and says so.
Everything else in the design references this vocabulary. It governs plain
fixed-layout `struct` values, not the separately planned managed `class`
facility (see plan.org, "Managed classes").

## The four container types

| Notation  | Layout            | Assignment | Owns storage | Allocator |
|-----------|-------------------|------------|--------------|-----------|
| `[n T]`   | n contiguous `T`  | copies     | no (inline)  | —         |
| `[T]`     | ptr + len         | copies the *view* | no    | —         |
| `(Vec T)` | ptr + len + cap   | **moves**  | yes          | stored    |
| `(Map K V)` | open-addressed, flat key/value arrays | **moves** | yes | stored |

- `[n T]` is a value. It lives wherever it is declared, copies on assignment and
  on pass-by-value, and is what `defconst colors [4 u32] ...` and
  `(defvar grid [rows [cols u32]] ...)` are.
- `[T]` is a **non-owning slice**: a borrowed window into a `[n T]`, a `(Vec T)`,
  or a literal in read-only memory. Copying a slice copies ptr+len, never the
  elements. A slice may be `const`-qualified; freeing through one is not possible
  because a slice has no allocator and no `cap`.
- `(Vec T)` and `(Map K V)` are **move-only**. Binding, passing, or returning one
  transfers ownership; the source binding is dead afterwards and using it is a
  compile error. There is no shallow copy, so there is no double free.

## Maps — first implementation

Every map is homogeneous: `(Map K V)` has one key type and one value type. The
first implementation accepts only built-in structural key types: integers,
enums, strings, fixed arrays, and value structs composed recursively from those
types. Tuples and triples join that set when they are introduced. `Ptr`, slices,
`Vec`, and `Map` are not map keys yet.

Equality and hashing for those keys are compiler-provided structural operations
and not type classes. They are not available to an unconstrained type variable
either; a variable that means to key a map declares `hashable?` in the signature
that binds it, and the refusal then lands at the call site that names an
unhashable key. An empty map names its key and value types, because the position
it is usually written in — a `let` binding — has no type slot for it to take
them from:

```
(let [enemies (map-new string Enemy)]
  ...)
```

`(get m k)` returns `(Option V)`: absence is `None`, not an untyped `nil`.
`(put m k v)` is the upsert operation and returns `()`; it either inserts or
replaces. `(set (get m k) v)` is not map syntax.

The first Map implementation admits copyable keys and values only, so `get`
returns a copy. Move-aware lookup, removal, and owned entries are deferred until
`Vec`/`Map` values are supported in maps; the map itself remains an owning,
move-only container.

## Copying is always explicit

`(clone x)` produces an independent deep copy of a `Vec`/`Map` using the current
allocator; `(clone x alloc)` names one. Value types (`[n T]`, structs of value
types, primitives) need no `clone` — assignment already copies them.

A struct containing a `Vec` field is itself move-only. Ownership is structural,
not declared: a type is a value type iff all of its fields are **and it declares
no `drop` hook** (see Allocators). A `drop` hook makes a type move-only for the
same reason a `Vec` field does — exactly one owner, so the hook fires exactly
once — and a type with one cannot be `clone`d.

## Borrowing

- `(as-slice v)` / `(as-slice v lo hi)` view a `Vec` or fixed array as `[T]`.
- **The first implementation follows Zig/Odin's explicit model, not Rust's
  borrow checker.** A slice is invalidated by any operation that may reallocate
  its owner (`push`, `put`, `reserve`); its user is responsible for respecting
  that contract. Dev builds carry a generation word on `Vec` and trap on use of
  a stale slice. `Ptr` is the explicit lower-level escape hatch and has the
  same lifetime contract.
- A future lightweight provenance pass may reject the obvious mistakes (a
  borrow of a local escaping, use after an owner moves, and reallocation with a
  live borrow). It must not require Rust-style lifetime annotations or dictate
  an ECS-shaped object model. Long-lived graph links use `(Handle a)`; temporary
  graphs may use explicitly managed, stable region storage.
- Cross-referencing long-lived objects uses `(Handle a)` into a pool, never a
  raw pointer or slice. A stale handle is detectable.

## Globals of move-only type

A global may be a `Vec` or a `Map`, and **reading one is always a borrow, never
a move**. Nothing can take ownership of it, so nothing can `free` it; its
lifetime is the process's and it is never released. That is one rule rather than
a general ownership model for globals, and it is sound for the reason a general
model would be needed and is not: the lifetime question has a constant answer.
Passing a global to a function that owns its parameter, binding it to a local,
returning it and freeing it are all refused at the read, which is exactly where
a move would have been recorded for a local. `(clone g)` is the one of these
that yields something another owner may have.

Such a global is **mutable in place**: `push`, `put`, `reserve` and `set` all
take their target as a borrow, so a global `(Vec u8)` is filled and grown where
it stands. The aliasing contract is the one above and nothing more — a push that
reallocates invalidates a slice taken before it, and that is the programmer's
whether the owner is a local or a global. Globals get no borrow rule locals do
not have; the dev build's generation word lives on the `Vec`, so the stale-slice
trap works the same either way.

A move-only global **starts zeroed** — a zeroed `Vec` is an empty `Vec` — and is
loaded by whichever function loads it, with an ordinary assignment:

```
(defvar the-data (Vec u8))
(defn load [] () (set the-data (slurp "game-data.edn")))
```

A computed initialiser on the declaration is refused: a global's initialiser is
a link-time constant, and the load has to be something the program does rather
than something that happens before `main`. That is also what makes the data
outlive `main` — nothing re-runs between one entry and the next, so a re-entered
`main` finds the global as it left it. Assigning a second time overwrites the
first block and leaks it; there is no `drop`, and freeing is a thing you write.

## Taking an address

`(addr x)` yields `(Ptr T)` for any assignable place `x` — a local, a global, a
field, an element. The pointer is non-owning and does not extend anything's
lifetime, so `addr` of a local is only valid while that frame lives. This is the
same escape question as case 3 below. The first implementation leaves it as an
explicit lifetime contract; a future provenance pass can check it.

`addr` is how a value struct is shared mutably without an allocator — recursive
descent over a cursor, an entity passed down a call chain — and it is why
milestone 2 needs no heap at all. In the first implementation its non-escape
rule is an explicit programmer contract, aided by dev checks; the future
provenance pass above may enforce it.

## Places — what `set` accepts

A fixed set of assignable forms, not a `setf`-style extensible place mechanism:

```
(set x v)              ; a local or a defvar
(set (.field x) v)     ; struct field; x may be a struct, (Ptr S) or (Handle S)
(set (at a i ...) v)   ; fixed array, slice, or Vec element
(set (deref p) v)      ; whole-object store through a pointer
```

`.field` and `at` auto-deref exactly one pointer or handle level, which is what
makes `(set (.hp e) ...)` legal when `e : (Ptr Enemy)` and illegal when
`e : Enemy` bound by value.

**Mutating something you matched.** Pattern bindings bind *values*, so a matched
struct is a copy. To mutate in place, obtain a pointer first — the pointer is
visible in the type:

```
(match (resolve w h)          ; (Option (Ptr Enemy))
  (Some e) (set (.hp e) ...)  ; e : (Ptr Enemy), field access derefs
  None     ...)
```

`deref` yields a value; `resolve` yields a pointer. Both are overloaded on
`(Ptr a)` and `(Handle a)` and resolve at compile time.

## Generics

Parametric polymorphism is monomorphisation, with **no type classes**. A type
variable is written `$t` wherever a *type* goes — a parameter, the return type,
or nested as `[$t]` or `(Vec $t)` — and bare `t` where a type's *name* is an
argument in expression position, as in `(vec-new t)` and the cast `(t x)`. A
generic body is checked **abstractly**, with nothing substituted, so the rule
below bites at the definition rather than at whichever call site first
instantiates it:

> A type variable `$t` supports only what every type supports: move, `clone`,
> field-free storage. It does **not** support `=`, `<`, `+`, or `hash`.

What makes that liveable is a `where` clause of compile-time type predicates,
written as a map at the head of the body. There are five — `ordered?`, `equal?`,
`hashable?`, `numeric?`, `copyable?` — they are not type classes because a
predicate carries no implementations and merely gates a builtin the compiler
already has, and they entail one another in one direction, so one clause usually
does. A variable is move-only by default and `copyable?` is the opt-out, because
whether a variable moves is not decidable abstractly. plan.org's Types section
has the full account.

```
(defn sort! [s [$t]] ()
  {:where (ordered? $t)}
  ...)
```

Without such a clause the operator is rejected where it is written, not silently
instantiated, and the operation is passed in explicitly as a function value
instead:

```
(defn largest [xs [$t]  gt (Fn [$t $t] bool)] (Option $t) ...)
```

The value handed to such a parameter is a named `defn`. An `fn` cannot be written
inline into it, because the generic body is checked with nothing substituted and
there is no concrete type yet for the `fn`'s own parameters to come from; that
restriction lifts at a monomorphic call site, where `reduce`'s and `filter`'s
callbacks are ordinary inline `fn`s.

The alternatives to predicates — compile-time interfaces, or intrinsics
restricted to primitives — remain deliberately deferred until the base checker is
stable (build sequence milestone 4). The ceiling is that nobody can supply a
user-defined `<`.

`println` is the deliberate exception. It is a compiler-provided,
type-directed intrinsic: monomorphisation selects or emits a structural printer
for each concrete instantiation, so `(println x)` is legal for `x : $t` without
introducing a `Printable` type class. Structs, fixed arrays, options and,
eventually, Vecs and Maps print structurally. `Ptr` and `Handle` print their
address or identity rather than recursively dereferencing, and collection
printers impose depth and length limits. `any` and `Error` use their runtime type
metadata. User generic code still passes an explicit function for every other
operation that depends on a type's structure.

Type arguments are **inferred at call sites** from the argument types; there is
no explicit instantiation syntax in the first implementation. A type variable
that appears only in the return type is therefore an error.

## Function values

Three cases, split by whether the value escapes the frame that made it.

**1. `(Fn [T1 T2] R)` — a plain, stable function pointer.** No captured
environment or allocation. In a dev build, a reference to a top-level `defn`
is the address of a stable trampoline that loads that function version's
indirection cell and calls its current body; it is never the address of a
particular body. Thus stored callbacks and ordinary calls observe a later
*body* redefinition, as in Common Lisp. Release builds may call the body
directly because it cannot be redefined.

A signature-changing redefinition makes a new internal function version and a
new trampoline ABI. Newly compiled code resolves the source name to that new
version. Existing callers and stored `Fn` values keep their old trampoline and
therefore safely call the old version. The session immediately warns at every
tracked caller source location that still targets the old signature; recompiling
one either retargets it successfully or reports an ordinary type error. This is
what raylib callbacks, hot-reload cells, and function parameters use. A
top-level `defn` is one, so `(largest hps >)` passes `>` at `i32` directly. This
is the only function type that may cross an FFI boundary or sit in a reload
cell.

**2. Non-escaping `fn` — captures by value into a stack environment.** A `fn`
whose value provably does not outlive the frame that created it gets an
environment allocated in that frame and captures the named locals **by value**
at the point of creation. No heap, no allocator, no lifetime question. This
covers essentially every lambda in practice:

- callbacks to `reduce` / `filter` / `each` / `map`, which consume them and return
- comparators passed to a function that does not store them
- `handler-bind` handler bodies

That last one is not a convenience. A handler must be able to see the enclosing
locals — `(fn [c] (push errors c) (invoke-restart 'skip-form))` capturing a local
`(Vec ParseError)` *is* the accumulation pattern, and conditions are not worth
building without it. Handlers are strictly non-escaping: the `handler-bind` frame
outlives every call to them.

Captured `Vec`/`Map` are captured **by pointer**, not moved, since the capture
does not outlive the owner. A non-escaping `fn` is therefore not itself an owner.

**3. Escaping closures — deferred.** A `fn` stored in a struct, pushed into a
container, or returned needs a heap environment and an answer to "which allocator
owns it, and what happens when the frame arena resets". Do not settle this until
a concrete use case requires it; revisit it with the optional lightweight
provenance work.

**Early exit inside a `fn`.** `try`, `some`, and `return` in a `fn` body exit the
`fn`, not the enclosing function — a `fn` is a function. Code that wants to
propagate out of a loop uses an imperative loop form, not a callback.

## Allocators

The allocator is part of the calling convention (`context/allocator`,
`context/temp`). `Vec` and `Map` record the allocator they were created with, so
`free` and `clone` never need it named again. Allocation uses the current
implicit allocator by default, as in Odin; an operation never falls back to a
hidden global allocator, and an explicit allocator can override the context.

### The allocator is one type-erased procedure

As in Odin (`base/runtime/core.odin:422`, `Allocator_Proc`), an allocator is a
procedure plus an opaque data pointer, and every operation takes `size` and
`align` as parameters:

| Operation                          | Meaning                                |
|------------------------------------|----------------------------------------|
| `alloc size align`                 | new block                              |
| `resize p old-size new-size align` | grow or shrink                         |
| `free p`                           | release one block                      |
| `free-all`                         | release everything the allocator holds |

It is type-erased on purpose. `Vec` and `Map` are one runtime over `(size,
align)` and, for `Map`, a compiler-emitted hash and equality pair passed as
arguments — Odin's `Map_Info` (`base/runtime/core.odin:369`). No generics are
involved, and none are needed.

An allocator declares which operations it implements. Odin's arena answers
`.Free` with `.Mode_Not_Implemented` (`core/mem/allocators.odin:307`); Flan's
equivalent is a **capability set** on the allocator value, readable at run time.
The one that is load-bearing below is `can-free`.

### When storage is released

There are exactly two release points, and neither of them is a scope.

1. **`(free v)`** — explicit. `v` is any move-only value: a `Vec`, a `Map`, a
   struct that owns one, or a struct that owns a resource rather than storage
   (a `Texture2D`, a socket, a file handle — see `drop` below). For a value that
   holds a resource and no storage, `free` runs `drop` and nothing else; it is
   still the release operation, and it is how a `Texture2D` in a local is
   released. `free` consumes its argument exactly as any other move does: the source
   binding is dead afterwards and using it is a compile error. That rule is
   already what makes a double free unrepresentable, so `free` needs no new
   analysis.
2. **Region release** — `(free-all a)` on an allocator, which releases
   everything made from it at once, including storage reachable from bindings
   that are still in scope. The per-frame `(free-all context/temp)` at the top
   of a game loop *is* the frame arena, and it is the normal way arena-tier
   storage dies.

**Nothing is released at scope exit.** Not at the end of a `let`, not at the end
of a function, not at the end of a `with-allocator` body. `with-allocator`
rebinds the current allocator for its dynamic extent and releases nothing; the
region it names is released, if ever, by an explicit `free-all` somewhere else.

This is deliberate, and it is the point on which the two obvious precedents were
rejected:

- **Odin's `defer delete`** cannot be written here. `defer` is function-scoped
  (`check.ml:505` refuses it in a `let`, a loop or a branch) and, because `let`
  is a block, a top-level `defer` is checked in a scope containing only the
  parameters and globals (`check.ml:1670`). `(defer (free v))` for a `let`-bound
  `v` is **not expressible today**. It becomes expressible with either
  block-scoped `defer` or a sequential top-of-body binder; until one of those
  exists, no idiom in this spec may depend on it.
- **Carp's scope-end frees** are a whole-program linear analysis that inserts a
  teardown call at every binding's last use (`Memory.hs`, and `Info.hs`'s
  `Deleter`). Carp could not reconcile that with an arena and therefore has no
  allocator abstraction at all. A release point the programmer cannot see is
  exactly what makes a frame arena unstateable.

**Leaking is defined behaviour.** Storage that is never freed and whose
allocator is never released is leaked, and for the permanent arena (symbols,
code) and the dev/REPL tier that is the correct program. "Did you forget to
free" is not a type question here; it is an allocator-tier question, and dev
builds answer it by reporting a general-purpose allocator's outstanding blocks
when it is destroyed.

**`free` applies to a whole owner.** It recurses structurally into owning
fields. A field is never freed on its own: `(free (.textures e))` is refused,
because it would leave `e` partly dead with no way to say so.

### Dev builds detect a released region

A `Vec` or `Map` records its allocator (see above). In a dev build it also
records that allocator's **epoch** — a counter the allocator bumps on every
`free-all`. Any operation on a container whose recorded epoch has moved traps,
naming the allocation site and the release site. This is a second and separate
counter from the per-`Vec` generation word that catches stale slices; the two
answer different questions and must not be conflated. Both are dev-only: the
release layout of a `Vec` is `ptr + len + cap + allocator` and nothing more.

### `drop` — owning something that is not memory

A type may name one hook:

```
(drop Texture [t (Ptr Texture)] ...)
```

It takes a **pointer, not the value**, which is Carp's shape and for Carp's
reason. Carp shipped `delete` — auto-generated per type, consuming, and
responsible for the recursive teardown of every field — and then had to add
`drop` separately, because a user who redefined `delete` to close a file had to
re-implement that whole teardown by hand. Carp's `drop` is looked up per
teardown site (`Memory.hs:806`, `getDropFunc`, at `RefTy t` where `delete` is
`FuncTy [t]`) and emitted immediately before the teardown call
(`Emit.hs:1042`), so the hook *composes with* compiler-generated teardown rather
than replacing it. Flan takes that arrangement unchanged.

- `(free v)` runs `drop` on `v` first, then tears down `v`'s owning fields in
  declaration order, each by the same rule.
- A `drop` hook may read and mutate through its pointer. It may **not** move out
  of the value, and it may not `free` it.
- A type has a `drop` hook transitively: a struct any of whose fields has one,
  has one.
- **A type with a `drop` hook is move-only and cannot be `clone`d.** Move-only,
  because a value type copies on assignment and two copies of one socket would
  each run `drop`; the same argument that makes a `Vec` field move-only. Not
  `clone`able, because duplicating a texture id or a file descriptor is not the
  compiler's decision to make — Carp needed a separate `copy` interface for
  exactly this. A type that *can* be duplicated says so with an ordinary named
  function.

**Nothing runs `drop` when an arena resets — because such a value cannot be in
an arena.** Constructing a container whose element type transitively has a
`drop` hook, or allocating such a value, against an allocator that lacks
`can-free` is **refused at the point of construction**: one branch per
container, not per element. `free-all` therefore never has to walk a list of
registered destructors, which is what keeps the frame tier's reset genuinely
free (plan.org's memory table) and keeps a destructor list — an allocation
nobody wrote — out of the core.

The consequence, stated plainly because a reader will assume otherwise:

> **`drop` is not a destructor.** A `Texture2D` held in a local, a parameter, or
> a plain stack struct never has `drop` run, because Flan has no scope-end
> anything. `drop` fires at exactly one place — inside `free` — and resources in
> locals are released explicitly, exactly as memory is. Carp's `drop` fires at
> scope end only because Carp has scope-end frees, which the section above
> rejects.

### Alignment

Alignment is a property of the **type**, computed at the **call site**, and
passed as a **parameter** to the type-erased allocator. All three, and they are
not alternatives.

Odin arranges it exactly this way: `elem_align` is threaded through every
type-erased dynamic-array entry point (`base/runtime/dynamic_array_internal.odin`
— `__dynamic_array_reserve`, `__dynamic_array_resize`, `__dynamic_array_append`),
and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:350`). The
monomorphised wrapper is the only place the concrete type is known, so it is the
only place that can produce the number.

Alignment is **not stored** in the `Vec` or `Map` header. That is safe because of
a condition worth writing down: every operation that needs it — `push`,
`reserve`, `resize`, `clone`, `free` — is compiler-emitted at a site where the
concrete element type is known. Any future type-erased teardown path would break
that condition; there is not to be one. (This is the second reason the `drop`
registry above was rejected: it would have been exactly such a path.)

The natural alignment of `T` is `align-of T`. Raising it above natural — 16
bytes for `#soa` and for component-wise fixed arrays — is declared **on the
type**, so that every site computing `align-of T` gets the raised number with no
further plumbing. The surface syntax for that declaration is deliberately not
fixed here; nothing is built that needs it yet.

### Allocation failure

**No allocating operation returns an error, and none can fail silently.** When
the allocator cannot satisfy a request, the operation signals

```
(StorageExhausted {.bytes n .align a .allocator id})
```

with `error`, whose type is `Never` (spec-conditions.md §2), inside a
`restart-case` offering `retry`. This is one rule over *every* allocating
operation — `vec-new`, `map-new`, `push`, `put`, `reserve`, `clone` — so their
result types stay `(Vec T)`, `()`, `()` and so on, with no `Result` and no
out-parameter anywhere.

What that buys, against the alternative: Odin's `append` returns an ignorable
`Allocator_Error` (`base/runtime/core_builtin.odin:767`,
`#optional_allocator_error`), and the type-erased path underneath returns the old
length on a failed reserve, marked
`// TODO(bill): Better error handling for failed reservation`
(`base/runtime/dynamic_array_internal.odin`). An `append` that appends nothing
and says nothing is the outcome this rule exists to make impossible.

- The condition is a value struct on the signalling frame's stack, with fixed
  numeric fields and **no rendered message**, because formatting would allocate
  and this is the one path that must not. Rendering happens in the handler or the
  break loop, where a working allocator is known.
- Unhandled, `error` enters the dev break loop or aborts in release
  (spec-conditions.md §2). It is never a no-op; `signal` is not used here.
- A handler that frees something, releases a scratch region, or grows the arena
  and then invokes `retry` re-attempts the same request. A handler that wants a
  *different* allocator needs a restart taking an argument, which does not exist
  yet; until it does, such a handler rebinds the context allocator and retries.
- Because an allocating operation can transfer, every caller of one checks the
  transfer channel after the call (spec-conditions.md §6). `push` is not a leaf
  call, and that per-call-site check is the price of not being Odin.

**This is the named exception to plan.org's "restarts go at the resync point,
once".** That rule is right for program-level errors and wrong here: a restart
established at a parser's top-level loop cannot re-attempt an allocation, and
only the allocation site can. Compiler-emitted restarts at the point of failure
are the exception, in the same way Common Lisp's runtime establishes
`store-value` at an unbound-variable error rather than at a resync point. No
*user* code establishes restarts below a resync point.

### Open: catching a use-after-release statically

Both release points above are dynamic, and the frame arena is the reason. A
static rule — "a move-only value constructed under a given allocator may not
outlive it" — needs to know statically which allocator a construction used, and
`with-allocator` plus `context/allocator` are precisely the mechanisms that deny
that knowledge. The lexical subset (a value made inside a `with-allocator` body
and returned out of it) is checkable; the general case is not; and shipping only
the subset would teach a rule that silently stops applying at the loop where it
matters most.

Until a provenance pass exists (plan.org open decision #3), the answer is the
dev-build epoch trap above: detection, loud and immediate, rather than
prevention. Settling this needs one thing that does not exist yet — real Flan
programs using arenas, to say whether the escapes that actually occur are
lexical. It is not settleable from the design alone, and it is not papered over
here.