diff --git a/FIX.org b/FIX.org index 9ea1c67..79f72eb 100644 --- a/FIX.org +++ b/FIX.org @@ -5049,3 +5049,73 @@ stops being one predicate per target and becomes a disjunction, ~numeric?~ or the reader meant. A cast *to* a variable bounded ~enum?~ is a second question with its own answer. Each of those is a decision, not a fill-in, and the author has not been asked. +* Generic allocation, 2026-09-21 — the sigil, not the feature +Reported as "generic code cannot allocate a container of its own element type": +(vec-new $t) inside a generic body was refused with "nothing here says what +(vec-new) is a Vec of". + +The feature was already there. [type_named] and the cast arm asked +[List.mem n env.tyvars] / [List.mem_assoc n env.subst] of the name as *written*, +and those two tables are keyed on the *bare* name — [signature_tyvars] strips +the sigil when it records a variable, and [resolve_name] strips it again when +it answers one. So [(vec-new t)] worked and had worked since generics landed — +generics.flan's [one-of] and [bump] are written that way — and [(vec-new $t)] +fell past the guard into the no-element-type message, which then described a +missing annotation for a body that had written one. + +Three membership tests, one helper: [tyvar_bare] and [tyvar_in_scope] near +[resolve_name], used by [type_named] (which fronts vec-new and map-new) and by +the cast arm. [resolve_name] uses [tyvar_bare] for its own strip, so there is +one place that knows what the character means. A sigil on a name nothing binds +now reaches [resolve_name] too, so [(vec-new $u)] says the variable has no +binding site rather than blaming the element type. + +Already fine, both spellings: [(array n $t)], [(zeroed)], [(Some x)], +[(Option $t)], [(Ptr $t)], a [(Vec $t)] return, a [(Map $t i32)] parameter — +every type *position* goes through [resolve], which has always stripped. A +local declared [(Vec $t)] is not a thing in the language: parse gives a let +binding no type slot. + +Broken and fixed: [(vec-new $t)], [(vec-new $t a)], [(map-new $k $v)], +[(map-new $k $v a)], [($t x)]. + +Size and alignment come from the copy: the i32 instantiation of [sorted] emits +flan_vec_init with 4/4 and the f64 one with 8/8, and flan_dev_reg_note_vec +with 4 and 8. The abstract pass holds [Var t] and is never emitted — emit.ml +has no layout for a Var and would die if it were. + +The dyn question does not arise: a generic is not instantiated at dyn at all +any more, and the refusal says to reach for the dyn side instead. So no copy +of one of these bodies can reach the dyn container, and the branch in vec-new +that picks it is unreachable from here. + +test/programs/generic-alloc.flan is the motivating program end to end; +x86 matches LLVM on it. + +docs/SPIKE-GENERICS.md already specified this — "Both spellings are accepted +at a use" — so the doc was right and check.ml was the divergence. No doc +change; the tests are what now hold the claim up. + +Two diagnostics came with it, because the fix left the same mistake wearing +two faces. [($u x)] was an unknown function where [(vec-new $u)] in the same +body was an unbound variable, so the cast arm took the sigil clause too. And +the unbound-sigil message said "write the concrete type here" in a signature +that introduces one: it names the variables that *are* bound now, read from +[tyvars] abstractly and from [subst] inside an instantiation, so one run does +not answer the same mistake two ways. Where none is in scope — a struct field, +a global — it is still the rule, because there is no answer to give. + +Found while widening the cast arm and left alone: a *declared name* may carry +the sigil. [(defn $foo [x i32] i32 ...)] is accepted and [($foo 3)] calls it; +so is [(defstruct $S [a i32])], and [($S 3)] constructs one — though the type +[$S] cannot be written anywhere, so nothing can hold the result but a let. +The character is reserved in every type position and in no name, so the cast +arm declines a name a binding, a struct or a defn already claims rather than +assume it is a type. That is one decline per table a name can be declared in, +and the arm sits above every one of them: [ordinary_call] and, last in +[named_call], [positional_struct]. Refusing the sigil in a declared name +would close it properly; that is a decision about the spelling and not this +lane's to make. + +Left: docs/SPIKE-GENERICS.md still lists map-new, zeroed and the casts under +"Mechanical" as remaining work. They landed. diff --git a/lib/check.ml b/lib/check.ml index fc60bfc..0223653 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -911,6 +911,25 @@ let rec unfillable env seen (t : Types.t) : Types.t option = | None -> Some t) | _ -> Some t +(* The name under the sigil. [$t] is how a defn signature introduces a type + variable and [t] is how the body spells the same one, so the tables that + record which variables are in scope — [env.tyvars] and [env.subst] — are + keyed on the bare name and every membership test has to strip first. A name + with no sigil is its own bare name. *) +let tyvar_bare n = + if n <> "" && n.[0] = '$' then String.sub n 1 (String.length n - 1) else n + +(* Is this name, as written, a type variable that is in scope here? Both + spellings answer yes, because both denote the same variable: the sigil is + the binding site's and is redundant rather than wrong in the body. Every + test against [tyvars] or [subst] goes through this, so a caller cannot ask + the question of the raw name and miss the spelling with the sigil — which is + what made [(vec-new $t)] report a missing element type for a body that had + written one. *) +let tyvar_in_scope env n = + let bare = tyvar_bare n in + List.mem bare env.tyvars || List.mem_assoc bare env.subst + let rec resolve env ?(seen = []) (t : Ast.texpr) : Types.t = let loc = t.Ast.tloc in match t.Ast.t with @@ -1012,7 +1031,7 @@ and resolve_name env ~seen loc n = unknown-type error it always was. That is the point of the sigil: without one, a mistyped type name silently became a type parameter and made the function more permissive than it was written to be. *) - let bare = if n <> "" && n.[0] = '$' then String.sub n 1 (String.length n - 1) else n in + let bare = tyvar_bare n in match List.assoc_opt bare env.subst with (* Inside an instantiation: the variable is this concrete type, and every node checked under it is as concrete as if it had been written out. *) @@ -1020,12 +1039,34 @@ and resolve_name env ~seen loc n = | None -> if List.mem bare env.tyvars then Types.Var bare else if n <> bare then - (* A sigil somewhere that is not a [defn] signature: a struct field, a - global, a [let] annotation. There is nowhere for it to bind, so it is - the error rather than a variable with no scope. *) - Loc.failk "check/unbound-type-variable" loc - "%s introduces a type variable, and only a defn signature can — write \ - the concrete type here" n + (* A sigil on a name nothing binds. Two different mistakes wear the same + spelling, and which one it is turns on whether any variable is in scope + at all. Where none is — a struct field, a global, a [let] annotation — + there is nowhere for a variable to bind and the fix is a concrete type. + Where some are, the name is almost always a variable that was + introduced once and spelled differently the second time, and the fix is + one of the names that *is* bound. Naming them is the difference between + a rule and an answer. + + Which names those are is read from [tyvars] during the abstract pass and + from [subst] inside an instantiation, because the instantiation clears + the first and fills the second — and a body is checked under both, so + reading only one of them would answer the same mistake two ways in a + single run. *) + (match (match env.tyvars with [] -> List.map fst env.subst | vs -> vs) with + | [] -> + Loc.failk "check/unbound-type-variable" loc + "%s introduces a type variable, and only a defn signature can — write \ + the concrete type here" n + | [ v ] -> + Loc.failk "check/unbound-type-variable" loc + "nothing binds the type variable %s — this signature introduces %s, \ + so write %s here, or a concrete type" n v v + | vars -> + Loc.failk "check/unbound-type-variable" loc + "nothing binds the type variable %s — this signature introduces %s, \ + so write one of those here, or a concrete type" + n (String.concat " and " vars)) else match Types.ikind_of_name n with | Some k -> Types.Int k @@ -5959,12 +6000,16 @@ and file_guard ctx loc ~path_slot ~op mk_steps = them. *) and type_named ctx n = (* A type variable names a type here too, which is what lets [(vec-new t)] - be written in a generic body: inside an instantiation [resolve_name] - answers with the concrete element type, and during the abstract pass it - answers [Var t] and the [Vec] that comes back is a [(Vec t)] — generic, - and refused by anything that needs a size. *) - List.mem n ctx.env.tyvars - || List.mem_assoc n ctx.env.subst + and [(vec-new $t)] be written in a generic body: inside an instantiation + [resolve_name] answers with the concrete element type, and during the + abstract pass it answers [Var t] and the [Vec] that comes back is a + [(Vec t)] — generic, and refused by anything that needs a size. *) + tyvar_in_scope ctx.env n + (* A sigil is only ever written where a type goes, so a name carrying one is + answered here even when nothing binds it: [resolve_name] then says that a + variable has no binding site outside a defn signature, which is the + mistake, instead of this form reporting a missing element type. *) + || n <> tyvar_bare n || List.mem n Types.primitive_names || Hashtbl.mem ctx.env.structs n || Hashtbl.mem ctx.env.datas n @@ -8127,17 +8172,35 @@ and named_call ?(qualified = false) ctx ~want loc name args = float goes through (i32 x) first" name (Types.to_string other)); prim (Tast.Cast target) target [ a ] - (* A cast to a *type variable*: [(t x)] inside a generic body. The name is - not one [is_cast] knows, because [is_cast] asks whether the name is a - machine type and [t] is not — so this is its own arm, above the ordinary - one and below the enums, and it reaches the same [Cast] prim. + (* A cast to a *type variable*: [(t x)] or [($t x)] inside a generic body. + The name is not one [is_cast] knows, because [is_cast] asks whether the + name is a machine type and [t] is not — so this is its own arm, above the + ordinary one and below the enums, and it reaches the same [Cast] prim. Inside an instantiation [resolve_name] has already answered with the concrete target, so the copy casts to a real type and the emitter sees nothing unusual. During the abstract pass the target is [Var t] and the [where] clause is what says the cast means anything at all: a cast - produces a number, so [numeric?] is what admits it. *) - | _ when (List.mem name ctx.env.tyvars || List.mem_assoc name ctx.env.subst) + produces a number, so [numeric?] is what admits it. + + A sigil on a name nothing binds comes here too, for the reason + [type_named] takes one: the character is only ever written where a type + goes, so [resolve_name] gets to say that a variable has no binding site + outside a defn signature. Otherwise [($u x)] would be an unknown function + in the same body where [(vec-new $u)] is an unbound variable — one + mistake told two ways. + + Only where nothing else claims the name, though. Nothing stops a defn, a + struct or a binding from carrying the character, and a call to one is a + call and not a type: this arm sits above the arms that would have found + it — [ordinary_call] and, last of all, [positional_struct] — so it has to + decline first, once per table a name can be declared in. *) + | _ when (tyvar_in_scope ctx.env name + || (name <> tyvar_bare name + && lookup ctx name = None + && not (Hashtbl.mem ctx.env.structs name) + && not (Hashtbl.mem ctx.env.fns name) + && not (Hashtbl.mem ctx.env.gsigs name))) && List.length args = 1 -> let target = resolve_name ctx.env ~seen:[] loc name in unconstrained ctx.env loc ("a cast to " ^ name) ~needs:"numeric?" target; diff --git a/test/programs/generic-alloc.flan b/test/programs/generic-alloc.flan new file mode 100644 index 0000000..38c3851 --- /dev/null +++ b/test/programs/generic-alloc.flan @@ -0,0 +1,112 @@ +;;;; A generic body that allocates a container of its own element type. +;;;; +;;;; The caller-allocated spelling — the caller passes a destination slice and +;;;; the callee fills it — is the one a generic could always write. This is the +;;;; other half: [sorted] below asks for the storage itself, at whatever type +;;;; the instantiation turned out to be, and hands the container back. +;;;; +;;;; What makes that work is that a type variable names a type wherever a type +;;;; name goes, in both spellings: [$t] is how a defn signature introduces the +;;;; variable and [t] is how the body spells the same one, and the constructors +;;;; that read a type argument — vec-new, map-new, array, a cast — accept +;;;; either. The size and the alignment flan_vec_init is given are produced +;;;; from the element type at the point the copy is checked, so each copy +;;;; carries its own: 4 and 4 in the i32 one, 8 and 8 in the f64 one. The +;;;; abstract pass, which checks the body once with nothing substituted, is +;;;; never emitted, so no copy ever asks for the size of a variable. + +;; The motivating case: allocate, fill, sort, return. Two element types below, +;; so there are two copies and the storage in each is the instantiation's. +(defn sorted [xs [$t]] (Vec $t) + {:where (ordered? $t)} + (let [v (vec-new $t)] + (dotimes [i (len xs)] (push v (at xs i))) + (sort (as-slice v)) + v)) + +;; The same, against a named allocator rather than the context's. Both arities +;; read their element type the same way, so both take a variable. +(defn sorted-in [xs [$t] al Allocator] (Vec $t) + {:where (ordered? $t)} + (let [v (vec-new $t al)] + (dotimes [i (len xs)] (push v (at xs i))) + (sort (as-slice v)) + v)) + +;; A Map whose key is the variable. The predicate is what lets the hash and the +;; equality be deferred to the copy; see generics.flan for that half. +(defn distinct-count [ks [$k]] i32 + {:where (hashable? $k)} + (let [m (map-new $k i32)] + (dotimes [i (len ks)] + (put m (at ks i) 1)) + (let [n (len m)] (free m) n))) + +;; The zeroed fixed array. Its length is still a compile-time constant; only +;; the element type is the variable, and that is answered per copy. +(defn middle-of [x $t] $t + (let [a (array 3 $t)] + (set (at a 1) x) + (at a 1))) + +;; A cast to the variable, written with the sigil. +(defn widen [x i32 d $t] $t + {:where (numeric? $t)} + (do d ($t x))) + +;; There is no row here for a type variable that instantiates at dyn: a +;; generic is not instantiated at dyn at all, and the refusal says to reach +;; for the dyn side instead. So nothing a copy of one of these bodies does +;; can reach the dyn container. + +;; Allocation failure inside a generic copy, handled the way exhausted.flan +;; handles it: a ceiling, a handler that raises it, and retry re-attempting the +;; request that did not fit. The globals are because a handler cannot see the +;; locals of the function that established it. +(defonce tight Allocator) +(defonce failures i64) + +(defn main [] () + (let [ns [5 3 9 1] + fs [2.5 0.5 1.5] + a (sorted (slice ns 0 4)) + b (sorted (slice fs 0 3))] + (println (at (as-slice a) 0)) ; 1 + (println (at (as-slice a) 3)) ; 9 + (println (at (as-slice b) 0)) ; 0.5 + (println (len (as-slice b))) ; 3 + (free a) + (free b)) + + (let [ns [4 2 8] + ar (arena-new 4096) + c (sorted-in (slice ns 0 3) ar)] + (println (at (as-slice c) 0)) ; 2 + (free-all ar)) + + (let [ns [7 7 3] + ws ["a" "b" "a"]] + (println (distinct-count (slice ns 0 3))) ; 2 + (println (distinct-count (slice ws 0 3)))) ; 2 + + (println (middle-of 6)) ; 6 + (println (middle-of 1.5)) ; 1.5 + (println (widen 3 0.0)) ; 3 + + ;; The guard, the condition and the restart are the concrete form's, emitted + ;; into the copy unchanged. 32 bytes is four i32 and the sixteen this fills + ;; are not. + (set tight (heap-allocator)) + (set-alloc-budget tight 32) + (handler-bind + [(StorageExhausted [c] + (set failures (+ failures 1)) + (set-alloc-budget tight (* 4 (alloc-budget tight))) + (invoke-restart 'retry))] + (let [ns [9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4] + d (sorted-in (slice ns 0 16) tight)] + (println (len (as-slice d))) ; 16 + (println (at (as-slice d) 0)) ; 0 + (free d))) + (println (> failures 0)) ; true + (set-alloc-budget tight 0)) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index bd66636..e1da6bf 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -2782,6 +2782,19 @@ let () = outputs ~opt:"-O0" "integer? and the collapsed abs, -O0" "programs/int-generic.flan" int_generic_out; + (* A generic body that allocates a container of its own element type, + rather than filling one the caller allocated. Every line here is a copy + answering at a type the written body never named: the first four are one + [sorted] at i32 and at f64, and the 16 near the end is the same body + running under a ceiling it has to hit, signal and retry through. *) + let generic_alloc_out = + "1\n9\n0.5\n3\n2\n2\n2\n6\n1.5\n3\n16\n0\ntrue\n" + in + outputs "a generic allocates its own element type" + "programs/generic-alloc.flan" generic_alloc_out; + outputs ~opt:"-O0" "a generic allocates its own element type, -O0" + "programs/generic-alloc.flan" generic_alloc_out; + (* Reach's walk, edge by edge. Pruning is what makes the link follow the program, and the cost of getting it wrong is not a wrong answer: a function the walk fails to reach is not emitted, and the build dies in diff --git a/test/test_flan.ml b/test/test_flan.ml index 033f1c1..f3b1923 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -5790,6 +5790,49 @@ let () = ~needle:"is not a map key" "(defn f [k $t] () (let [m (map-new t i32)] (put m k 1) (free m)))"; + (* ── A type variable, spelled with the sigil, where a type name goes ── + [$t] is the signature's spelling and [t] is the body's, and they are the + same variable: the tables that record which variables are in scope are + keyed on the bare name, so every membership test has to strip the sigil + before asking. The ones that did not strip were the guards in front of + [vec-new] and [map-new] and the cast arm, which is why a body that wrote + [(vec-new $t)] was told it had not said what the Vec held. *) + accepts "vec-new over a type variable written with the sigil" + "(defn f [x $t] (Vec $t) (let [v (vec-new $t)] (push v x) v))"; + accepts "and against a named allocator" + "(defn f [x $t a Allocator] (Vec $t) \ + (let [v (vec-new $t a)] (push v x) v))"; + accepts "map-new over type variables written with the sigil" + "(defn f [k $t] i32 {:where (hashable? $t)} \ + (let [m (map-new $t i32)] (put m k 1) (let [n (len m)] (free m) n)))"; + accepts "a zeroed fixed array of a type variable" + "(defn f [x $t] $t (let [a (array 3 $t)] (set (at a 1) x) (at a 1)))"; + accepts "a cast to a type variable written with the sigil" + "(defn f [x i32 d $t] $t {:where (numeric? $t)} (do d ($t x)))"; + (* The message these were taking is still the message for the case it was + written for: nothing named, and nothing at the site that says. *) + rejects_check "vec-new with no element type and nothing to take one from" + ~needle:"nothing here says what (vec-new) is a Vec of" + "(defn f [x $t] i32 (do x (let [v (vec-new)] (free v) 0)))"; + (* And a sigil on a name nothing binds is answered as the unbound variable + it is, rather than as a missing element type — with the names that *are* + bound, because inside a signature that introduces one the mistake is + nearly always the second spelling of the first. *) + rejects_check "vec-new over a sigil that names no variable in scope" + ~needle:"this signature introduces t, so write t here" + "(defn f [x $t] i32 (do x (let [v (vec-new $u)] (free v) 0)))"; + rejects_check "and a cast over one tells the same story" + ~needle:"this signature introduces t, so write t here" + "(defn f [x i32 d $t] $t {:where (numeric? $t)} (do d ($u x)))"; + rejects_check "two variables in scope are both named" + ~needle:"introduces t and u, so write one of those" + "(defn f [a $t b $u] i32 (do a b (let [v (vec-new $w)] (free v) 0)))"; + (* Where no variable is in scope there is none to name, and the answer is + the rule: a sigil binds, and only a defn signature is a binding site. *) + rejects_check "a sigil in a struct field, where nothing can bind one" + ~needle:"only a defn signature can" + "(defstruct S [v $t])"; + (* ── The builtin table against the arms it describes ────────────── [Check.builtins] is what the editor's C-c C-v and M-. read for a name no program wrote — [arena-new] and the seventy-seven others. A table like