flan/spec-memory.md

Spec 1 — Ownership, containers, and copies

Status: frozen. Closes plan.org open decisions #6 and #10, and resolves the contradiction between "value structs copy on assignment" and owning containers. The Allocators section additionally settles the four things that had to be decided before Vec and Map are written: when storage is released, the drop hook, alignment, and allocation failure. One question there is left open on purpose and says so. Everything else in the design references this vocabulary. It governs plain fixed-layout struct values, not the separately planned managed class facility (see plan.org, "Managed classes").

The four container types

Notation	Layout	Assignment	Owns storage	Allocator
[n T]	n contiguous T	copies	no (inline)	—
[T]	ptr + len	copies the view	no	—
(Vec T)	ptr + len + cap	moves	yes	stored
(Map K V)	open-addressed, flat key/value arrays	moves	yes	stored

• [n T] is a value. It lives wherever it is declared, copies on assignment and on pass-by-value, and is what defconst colors [4 u32] ... and (defvar grid [rows [cols u32]] ...) are.
• [T] is a non-owning slice: a borrowed window into a [n T], a (Vec T), or a literal in read-only memory. Copying a slice copies ptr+len, never the elements. A slice may be const-qualified; freeing through one is not possible because a slice has no allocator and no cap.
• (Vec T) and (Map K V) are move-only. Binding, passing, or returning one transfers ownership; the source binding is dead afterwards and using it is a compile error. There is no shallow copy, so there is no double free.

Maps — first implementation

Every map is homogeneous: (Map K V) has one key type and one value type. The first implementation accepts only built-in structural key types: integers, enums, strings, fixed arrays, and value structs composed recursively from those types. Tuples and triples join that set when they are introduced. Ptr, slices, Vec, and Map are not map keys yet.

Equality and hashing for those keys are compiler-provided structural operations, not type classes and not operations available to an unconstrained type variable. An empty map takes its type from its context:

(defvar enemies (Map string Enemy) (map-new))

(get m k) returns (Option V): absence is None, not an untyped nil. (put m k v) is the upsert operation and returns Unit; it either inserts or replaces. (set (get m k) v) is not map syntax.

The first Map implementation admits copyable keys and values only, so get returns a copy. Move-aware lookup, removal, and owned entries are deferred until Vec/Map values are supported in maps; the map itself remains an owning, move-only container.

Copying is always explicit

(clone x) produces an independent deep copy of a Vec/Map using the current allocator; (clone x alloc) names one. Value types ([n T], structs of value types, primitives) need no clone — assignment already copies them.

A struct containing a Vec field is itself move-only. Ownership is structural, not declared: a type is a value type iff all of its fields are and it declares no drop hook (see Allocators). A drop hook makes a type move-only for the same reason a Vec field does — exactly one owner, so the hook fires exactly once — and a type with one cannot be cloned.

Borrowing

• (as-slice v) / (as-slice v lo hi) view a Vec or fixed array as [T].
• The first implementation follows Zig/Odin's explicit model, not Rust's borrow checker. A slice is invalidated by any operation that may reallocate its owner (push, put, reserve); its user is responsible for respecting that contract. Dev builds carry a generation word on Vec and trap on use of a stale slice. Ptr is the explicit lower-level escape hatch and has the same lifetime contract.
• A future lightweight provenance pass may reject the obvious mistakes (a borrow of a local escaping, use after an owner moves, and reallocation with a live borrow). It must not require Rust-style lifetime annotations or dictate an ECS-shaped object model. Long-lived graph links use (Handle a); temporary graphs may use explicitly managed, stable region storage.
• Cross-referencing long-lived objects uses (Handle a) into a pool, never a raw pointer or slice. A stale handle is detectable.

Taking an address

(addr x) yields (Ptr T) for any assignable place x — a local, a global, a field, an element. The pointer is non-owning and does not extend anything's lifetime, so addr of a local is only valid while that frame lives. This is the same escape question as case 3 below. The first implementation leaves it as an explicit lifetime contract; a future provenance pass can check it.

addr is how a value struct is shared mutably without an allocator — recursive descent over a cursor, an entity passed down a call chain — and it is why milestone 2 needs no heap at all. In the first implementation its non-escape rule is an explicit programmer contract, aided by dev checks; the future provenance pass above may enforce it.

Places — what set accepts

A fixed set of assignable forms, not a setf-style extensible place mechanism:

(set x v)              ; a local or a defvar
(set (.field x) v)     ; struct field; x may be a struct, (Ptr S) or (Handle S)
(set (at a i ...) v)   ; fixed array, slice, or Vec element
(set (deref p) v)      ; whole-object store through a pointer

.field and at auto-deref exactly one pointer or handle level, which is what makes (set (.hp e) ...) legal when e : (Ptr Enemy) and illegal when e : Enemy bound by value.

Mutating something you matched. Pattern bindings bind values, so a matched struct is a copy. To mutate in place, obtain a pointer first — the pointer is visible in the type:

(match (resolve w h)          ; (Option (Ptr Enemy))
  (Some e) (set (.hp e) ...)  ; e : (Ptr Enemy), field access derefs
  None     ...)

deref yields a value; resolve yields a pointer. Both are overloaded on (Ptr a) and (Handle a) and resolve at compile time.

Generics

Parametric polymorphism is monomorphisation, with no type classes and no constraints. The consequence is a hard rule:

A type variable a supports only what every type supports: move, clone, field-free storage. It does not support =, <, +, or hash.

Anything else is passed in explicitly as a function value:

(defn largest [xs [a]  gt (Fn [a a] bool)] (Option a) ...)

Ordered/arithmetic operators over a are therefore rejected, not silently instantiated. The alternatives — compile-time interfaces, or intrinsics restricted to primitives — are deliberately deferred until the base checker is stable (build sequence milestone 4).

println is the deliberate exception. It is a compiler-provided, type-directed intrinsic: monomorphisation selects or emits a structural printer for each concrete instantiation, so (println x) is legal for x : a without introducing a Printable type class. Structs, fixed arrays, options and, eventually, Vecs and Maps print structurally. Ptr and Handle print their address or identity rather than recursively dereferencing, and collection printers impose depth and length limits. any and Error use their runtime type metadata. User generic code still passes an explicit function for every other operation that depends on a type's structure.

Type arguments are inferred at call sites from the argument types; there is no explicit instantiation syntax in the first implementation. A type variable that appears only in the return type is therefore an error.

Function values

Three cases, split by whether the value escapes the frame that made it.

1. (Fn [T1 T2] R) — a plain, stable function pointer. No captured environment or allocation. In a dev build, a reference to a top-level defn is the address of a stable trampoline that loads that function version's indirection cell and calls its current body; it is never the address of a particular body. Thus stored callbacks and ordinary calls observe a later body redefinition, as in Common Lisp. Release builds may call the body directly because it cannot be redefined.

A signature-changing redefinition makes a new internal function version and a new trampoline ABI. Newly compiled code resolves the source name to that new version. Existing callers and stored Fn values keep their old trampoline and therefore safely call the old version. The session immediately warns at every tracked caller source location that still targets the old signature; recompiling one either retargets it successfully or reports an ordinary type error. This is what raylib callbacks, hot-reload cells, and function parameters use. A top-level defn is one, so (largest hps >) passes > at i32 directly. This is the only function type that may cross an FFI boundary or sit in a reload cell.

2. Non-escaping fn — captures by value into a stack environment. A fn whose value provably does not outlive the frame that created it gets an environment allocated in that frame and captures the named locals by value at the point of creation. No heap, no allocator, no lifetime question. This covers essentially every lambda in practice:

• callbacks to reduce / filter / each / map, which consume them and return
• comparators passed to a function that does not store them
• handler-bind handler bodies

That last one is not a convenience. A handler must be able to see the enclosing locals — (fn [c] (push errors c) (invoke-restart 'skip-form)) capturing a local (Vec ParseError) is the accumulation pattern, and conditions are not worth building without it. Handlers are strictly non-escaping: the handler-bind frame outlives every call to them.

Captured Vec/Map are captured by pointer, not moved, since the capture does not outlive the owner. A non-escaping fn is therefore not itself an owner.

3. Escaping closures — deferred. A fn stored in a struct, pushed into a container, or returned needs a heap environment and an answer to "which allocator owns it, and what happens when the frame arena resets". Do not settle this until a concrete use case requires it; revisit it with the optional lightweight provenance work.

Early exit inside a fn. try, some, and return in a fn body exit the fn, not the enclosing function — a fn is a function. Code that wants to propagate out of a loop uses an imperative loop form, not a callback.

Allocators

The allocator is part of the calling convention (context/allocator, context/temp). Vec and Map record the allocator they were created with, so free and clone never need it named again. Allocation uses the current implicit allocator by default, as in Odin; an operation never falls back to a hidden global allocator, and an explicit allocator can override the context.

The allocator is one type-erased procedure

As in Odin (base/runtime/core.odin:422, Allocator_Proc), an allocator is a procedure plus an opaque data pointer, and every operation takes size and align as parameters:

Operation	Meaning
alloc size align	new block
resize p old-size new-size align	grow or shrink
free p	release one block
free-all	release everything the allocator holds

It is type-erased on purpose. Vec and Map are one runtime over (size, align) and, for Map, a compiler-emitted hash and equality pair passed as arguments — Odin's Map_Info (base/runtime/core.odin:369). No generics are involved, and none are needed.

An allocator declares which operations it implements. Odin's arena answers .Free with .Mode_Not_Implemented (core/mem/allocators.odin:307); Flan's equivalent is a capability set on the allocator value, readable at run time. The one that is load-bearing below is can-free.

When storage is released

There are exactly two release points, and neither of them is a scope.

1. (free v) — explicit. v is any move-only value: a Vec, a Map, a struct that owns one, or a struct that owns a resource rather than storage (a Texture2D, a socket, a file handle — see drop below). For a value that holds a resource and no storage, free runs drop and nothing else; it is still the release operation, and it is how a Texture2D in a local is released. free consumes its argument exactly as any other move does: the source binding is dead afterwards and using it is a compile error. That rule is already what makes a double free unrepresentable, so free needs no new analysis.
2. Region release — (free-all a) on an allocator, which releases everything made from it at once, including storage reachable from bindings that are still in scope. The per-frame (free-all context/temp) at the top of a game loop is the frame arena, and it is the normal way arena-tier storage dies.

Nothing is released at scope exit. Not at the end of a let, not at the end of a function, not at the end of a with-allocator body. with-allocator rebinds the current allocator for its dynamic extent and releases nothing; the region it names is released, if ever, by an explicit free-all somewhere else.

This is deliberate, and it is the point on which the two obvious precedents were rejected:

• Odin's defer delete cannot be written here. defer is function-scoped (check.ml:505 refuses it in a let, a loop or a branch) and, because let is a block, a top-level defer is checked in a scope containing only the parameters and globals (check.ml:1670). (defer (free v)) for a let-bound v is not expressible today. It becomes expressible with either block-scoped defer or a sequential top-of-body binder; until one of those exists, no idiom in this spec may depend on it.
• Carp's scope-end frees are a whole-program linear analysis that inserts a teardown call at every binding's last use (Memory.hs, and Info.hs's Deleter). Carp could not reconcile that with an arena and therefore has no allocator abstraction at all. A release point the programmer cannot see is exactly what makes a frame arena unstateable.

Leaking is defined behaviour. Storage that is never freed and whose allocator is never released is leaked, and for the permanent arena (symbols, code) and the dev/REPL tier that is the correct program. "Did you forget to free" is not a type question here; it is an allocator-tier question, and dev builds answer it by reporting a general-purpose allocator's outstanding blocks when it is destroyed.

free applies to a whole owner. It recurses structurally into owning fields. A field is never freed on its own: (free (.textures e)) is refused, because it would leave e partly dead with no way to say so.

Dev builds detect a released region

A Vec or Map records its allocator (see above). In a dev build it also records that allocator's epoch — a counter the allocator bumps on every free-all. Any operation on a container whose recorded epoch has moved traps, naming the allocation site and the release site. This is a second and separate counter from the per-Vec generation word that catches stale slices; the two answer different questions and must not be conflated. Both are dev-only: the release layout of a Vec is ptr + len + cap + allocator and nothing more.

drop — owning something that is not memory

A type may name one hook:

(drop Texture [t (Ptr Texture)] ...)

It takes a pointer, not the value, which is Carp's shape and for Carp's reason. Carp shipped delete — auto-generated per type, consuming, and responsible for the recursive teardown of every field — and then had to add drop separately, because a user who redefined delete to close a file had to re-implement that whole teardown by hand. Carp's drop is looked up per teardown site (Memory.hs:806, getDropFunc, at RefTy t where delete is FuncTy [t]) and emitted immediately before the teardown call (Emit.hs:1042), so the hook composes with compiler-generated teardown rather than replacing it. Flan takes that arrangement unchanged.

• (free v) runs drop on v first, then tears down v's owning fields in declaration order, each by the same rule.
• A drop hook may read and mutate through its pointer. It may not move out of the value, and it may not free it.
• A type has a drop hook transitively: a struct any of whose fields has one, has one.
• A type with a drop hook is move-only and cannot be cloned. Move-only, because a value type copies on assignment and two copies of one socket would each run drop; the same argument that makes a Vec field move-only. Not cloneable, because duplicating a texture id or a file descriptor is not the compiler's decision to make — Carp needed a separate copy interface for exactly this. A type that can be duplicated says so with an ordinary named function.

Nothing runs drop when an arena resets — because such a value cannot be in an arena. Constructing a container whose element type transitively has a drop hook, or allocating such a value, against an allocator that lacks can-free is refused at the point of construction: one branch per container, not per element. free-all therefore never has to walk a list of registered destructors, which is what keeps the frame tier's reset genuinely free (plan.org's memory table) and keeps a destructor list — an allocation nobody wrote — out of the core.

The consequence, stated plainly because a reader will assume otherwise:

drop is not a destructor. A Texture2D held in a local, a parameter, or a plain stack struct never has drop run, because Flan has no scope-end anything. drop fires at exactly one place — inside free — and resources in locals are released explicitly, exactly as memory is. Carp's drop fires at scope end only because Carp has scope-end frees, which the section above rejects.

Alignment

Alignment is a property of the type, computed at the call site, and passed as a parameter to the type-erased allocator. All three, and they are not alternatives.

Odin arranges it exactly this way: elem_align is threaded through every type-erased dynamic-array entry point (base/runtime/dynamic_array_internal.odin — __dynamic_array_reserve, __dynamic_array_resize, __dynamic_array_append), and align_of_type sits in Map_Cell_Info (base/runtime/core.odin:350). The monomorphised wrapper is the only place the concrete type is known, so it is the only place that can produce the number.

Alignment is not stored in the Vec or Map header. That is safe because of a condition worth writing down: every operation that needs it — push, reserve, resize, clone, free — is compiler-emitted at a site where the concrete element type is known. Any future type-erased teardown path would break that condition; there is not to be one. (This is the second reason the drop registry above was rejected: it would have been exactly such a path.)

The natural alignment of T is align-of T. Raising it above natural — 16 bytes for #soa and for component-wise fixed arrays — is declared on the type, so that every site computing align-of T gets the raised number with no further plumbing. The surface syntax for that declaration is deliberately not fixed here; nothing is built that needs it yet.

Allocation failure

No allocating operation returns an error, and none can fail silently. When the allocator cannot satisfy a request, the operation signals

(StorageExhausted {:bytes n :align a :allocator id})

with error, whose type is Never (spec-conditions.md §2), inside a restart-case offering retry. This is one rule over every allocating operation — vec-new, map-new, push, put, reserve, clone — so their result types stay (Vec T), Unit, Unit and so on, with no Result and no out-parameter anywhere.

What that buys, against the alternative: Odin's append returns an ignorable Allocator_Error (base/runtime/core_builtin.odin:767, #optional_allocator_error), and the type-erased path underneath returns the old length on a failed reserve, marked // TODO(bill): Better error handling for failed reservation (base/runtime/dynamic_array_internal.odin). An append that appends nothing and says nothing is the outcome this rule exists to make impossible.

• The condition is a value struct on the signalling frame's stack, with fixed numeric fields and no rendered message, because formatting would allocate and this is the one path that must not. Rendering happens in the handler or the break loop, where a working allocator is known.
• Unhandled, error enters the dev break loop or aborts in release (spec-conditions.md §2). It is never a no-op; signal is not used here.
• A handler that frees something, releases a scratch region, or grows the arena and then invokes retry re-attempts the same request. A handler that wants a different allocator needs a restart taking an argument, which does not exist yet; until it does, such a handler rebinds the context allocator and retries.
• Because an allocating operation can transfer, every caller of one checks the transfer channel after the call (spec-conditions.md §6). push is not a leaf call, and that per-call-site check is the price of not being Odin.

This is the named exception to plan.org's "restarts go at the resync point, once". That rule is right for program-level errors and wrong here: a restart established at a parser's top-level loop cannot re-attempt an allocation, and only the allocation site can. Compiler-emitted restarts at the point of failure are the exception, in the same way Common Lisp's runtime establishes store-value at an unbound-variable error rather than at a resync point. No user code establishes restarts below a resync point.

Open: catching a use-after-release statically

Both release points above are dynamic, and the frame arena is the reason. A static rule — "a move-only value constructed under a given allocator may not outlive it" — needs to know statically which allocator a construction used, and with-allocator plus context/allocator are precisely the mechanisms that deny that knowledge. The lexical subset (a value made inside a with-allocator body and returned out of it) is checkable; the general case is not; and shipping only the subset would teach a rule that silently stops applying at the loop where it matters most.

Until a provenance pass exists (plan.org open decision #3), the answer is the dev-build epoch trap above: detection, loud and immediate, rather than prevention. Settling this needs one thing that does not exist yet — real Flan programs using arenas, to say whether the escapes that actually occur are lexical. It is not settleable from the design alone, and it is not papered over here.