From 9194f918fceef2295e5fdf9b62eb3efb906f4e74 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:05:18 +0700 Subject: [PATCH 01/10] An aggregate that reads its own destination sees the old value, on x86 too x86 built a settling right-hand side straight into its destination, so (set p (P {.a (.b p) .b (.a p)})) wrote .a and read it back for .b. assign now also asks whether the value reads storage the destination lies in, and goes through the temporary when it does. The global-initialiser entry was already closed by startup_plan's Set; the unrooted-temporary entry turns out to be a both-backends rooting gap and stays open for a decision. --- TODO.org | 38 +++++++++------ docs/BUILT.md | 8 ++++ lib/x86.ml | 89 ++++++++++++++++++++++++++++++++++-- test/programs/self-read.flan | 62 +++++++++++++++++++++++++ test/test_acceptance.ml | 14 ++++++ 5 files changed, 193 insertions(+), 18 deletions(-) create mode 100644 test/programs/self-read.flan diff --git a/TODO.org b/TODO.org index fcf9257b..76bef6bc 100644 --- a/TODO.org +++ b/TODO.org @@ -1030,22 +1030,32 @@ is built into a frame temporary and copied over now, unless lowering is bound to reach the end. Separately, the transfer exit zeroed an aggregate return value, which for an aggregate is the caller's storage; it zeroes scalars only. -** TODO A global initialiser still builds an aggregate straight into its destination on x86 -The statement-level assignment goes through a frame temporary now, so the -half-write is closed where a program can observe it. The release build's -globals-init path was not converted and still writes in place. +** DONE A global initialiser builds an aggregate through the same temporary on x86 +CLOSED: [2026-09-25] +Already true when the entry was written. Every computed initialiser, release +build included, reaches =emit_globals_init= as =Emit.startup_plan='s +=(set g init)= and so goes through =assign=. The only initialisers lowered +straight into their symbol are =Tast.const_init= ones, which neither transfer +nor read anything. Nothing to change. -** TODO An aggregate built in place can read its own destination -=(set p (P {.a (.b p) .b (.a p)}))= answers =2 1= under LLVM and =2 2= under -=--x86=: lowering asks whether it can transfer and not whether it can alias. The -fix has a shape already — the same temporary, chosen by an aliasing question. -Whether the two become one predicate or two is the lane to decide. +** DONE An aggregate built in place never reads its own destination +CLOSED: [2026-09-25] +=assign= builds in place only when the right-hand side settles *and* reads no +storage the destination lies in; otherwise it goes through the temporary. Two +predicates, not one: =settles= is about leaving part-way, =reads= about a value +reading itself, and a read through a pointer counts as reading everything. +=test/programs/self-read.flan= runs on both backends. -** TODO The aggregate temporary is an unrooted buffer while it is filled -No root table names it. Harmless only while a dyn field in a struct was refused, -and per-type descriptors have since lifted that. Whoever relies on a dyn field -reaching this path has to root the buffer, or a collection running inside the -construction will not see what has been built so far. +** TODO A dyn value read out of a place is unrooted until it is stored, on both backends +First filed as the x86 aggregate temporary being unrooted. It is wider than +that: a dyn loaded from a place and held while a later sibling allocates is +unrooted on LLVM -O0 as well, in an aggregate literal or an argument list alike. +=(pick (.d other) (do (set other (T {.n 0})) (churn)))= prints a different +object's contents on both backends. =Emit.root_plan= roots dyn call results +only. Decision for the author: root such intermediates on both backends (a +=root_plan= change, plus LLVM spilling them to rooted allocas), or declare the +shape unsupported. Rooting only the x86 temporary would fork the backends and +leave the argument case open. ** TODO Marking through a descriptor an x86 reload module emitted The module links and runs. What is not proved is a collection running while a live diff --git a/docs/BUILT.md b/docs/BUILT.md index 6dfdd95c..230071a0 100644 --- a/docs/BUILT.md +++ b/docs/BUILT.md @@ -6104,6 +6104,14 @@ Scalars were never affected, on either backend, and `half-write.flan`'s last row being read as one: a scalar's store is a single instruction and it happens after the check for a transfer, so a call that signalled never reaches it. +**Settling is not enough on its own.** A right-hand side that reads its own destination sees whatever has been +written so far: `(set p (P {.a (.b p) .b (.a p)}))` built in place writes `.a` and then reads it back for `.b`, +answering `2 2` where LLVM answers `2 1`. So `assign` asks a second question, `X86.reads`: whether the value reads +storage the destination lies in, where the destination is named as one local, one global, or anywhere at all when it is +reached through a pointer. A read through a pointer counts as reading everything. The two questions stay separate +predicates because they are about different things, and in-place construction needs a no from both. +`test/programs/self-read.flan` crosses the destinations with that self-read on both backends. + ## Ghost text finds its anchor in the buffer, not in the table `M-x flan-watch-ghost-mode` paints each watched value inline, after the line holding the call that wrote it, as an diff --git a/lib/x86.ml b/lib/x86.ml index b01553ff..387b4bb4 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -1677,6 +1677,82 @@ and settles_place (p : Tast.place) = (* An index is bounds-checked, and the check signals. *) | Tast.Pindex _ -> false +(* The storage a destination lies in, as far as it can be named without + running anything: one local's slot, one global, or somewhere behind a + pointer, which could be any of them. + + Building an aggregate in its destination needs a second answer besides + [settles]. A right-hand side that reads the destination sees the fields + already written: [(set p (P {.a (.b p) .b (.a p)}))] writes [.a] and then + reads it back as the new [.b]. LLVM builds the value before it stores any + of it, so the read sees the old [p]. The two questions stay two predicates + because they are about different things — one about leaving part-way, the + other about the value reading itself — and [assign] asks both. *) +type root = Rlocal of int | Rglobal of string | Rany + +let rec expr_root (e : Tast.expr) = + match e.Tast.e with + | Tast.Local i -> Rlocal i + | Tast.Global n -> Rglobal n + | Tast.Field (x, _) | Tast.CaseField (x, _, _) -> through x + | Tast.Prim (Tast.At, a :: _ :: _) -> through a + | _ -> Rany + +(* A field or an element is in its container's storage unless the container + is reached through a pointer or is a slice, both of which are a pointer. *) +and through (x : Tast.expr) = + match x.Tast.ty with + | Types.Ptr _ | Types.Slice _ | Types.String -> Rany + | _ -> expr_root x + +let place_root (p : Tast.place) = + match p with + | Tast.Plocal i -> Rlocal i + | Tast.Pglobal n -> Rglobal n + | Tast.Pfield (x, _) | Tast.Pindex (x, _) -> through x + | Tast.Pderef _ -> Rany + +let overlaps a b = + match a, b with + | Rany, _ | _, Rany -> true + | Rlocal i, Rlocal j -> i = j + | Rglobal m, Rglobal n -> String.equal m n + | _ -> false + +(* Whether evaluating [e] can read storage in [root]. Only asked of an + expression that [settles], so the shapes are the ones listed there; a + shape this does not recognise answers yes. A read through a pointer can + reach anything. [Addr] is counted as a read of its place, which it is not, + because being wrong that way costs a copy. *) +let rec reads root (e : Tast.expr) = + match e.Tast.e with + | Tast.Int _ | Tast.Float _ | Tast.Bool _ | Tast.Str _ | Tast.Unit + | Tast.Zero _ | Tast.Uninit _ | Tast.None_ | Tast.FnAddr _ -> false + | Tast.Local i -> overlaps root (Rlocal i) + | Tast.Global n -> overlaps root (Rglobal n) + | Tast.Deref _ -> true + | Tast.Field (x, _) | Tast.CaseField (x, _, _) -> + (match x.Tast.ty with Types.Ptr _ -> true | _ -> false) || reads root x + | Tast.Some_ x -> reads root x + | Tast.Make (_, es) | Tast.MakeCase (_, _, es) | Tast.Arr es | Tast.Do es + | Tast.Prim (_, es) -> List.exists (reads root) es + | Tast.If (c, a, b) -> reads root c || reads root a || reads root b + | Tast.Addr p -> reads_place root p + | _ -> true + +and reads_place root (p : Tast.place) = + match p with + | Tast.Plocal i -> overlaps root (Rlocal i) + | Tast.Pglobal n -> overlaps root (Rglobal n) + | Tast.Pderef _ -> true + | Tast.Pfield (x, _) -> + (match x.Tast.ty with Types.Ptr _ -> true | _ -> false) || reads root x + | Tast.Pindex (x, is) -> + (match x.Tast.ty with + | Types.Ptr _ | Types.Slice _ | Types.String -> true + | _ -> false) + || reads root x || List.exists (reads root) is + let rec lower f (e : Tast.expr) (dst : loc) : unit = (* The one hook the line table needs, and it is here rather than at statement granularity on purpose: the same recursion that lowers a nested @@ -1817,7 +1893,8 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = nowhere else. *) scoped f (fun () -> let d = Lf f.slots.(slot) in - if f.loops = [] then lower f v d else assign f ~dst:d v); + if f.loops = [] then lower f v d + else assign f ~root:(Rlocal slot) ~dst:d v); bind_slot f slot) bs; block f body dst t @@ -1862,7 +1939,7 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = | None -> unsupported "continue %d outside a loop" n) | Tast.Set (p, v) -> let l = place f p in - scoped f (fun () -> assign f ~dst:l v) + scoped f (fun () -> assign f ~root:(place_root p) ~dst:l v) | Tast.Make (sn, xs) -> let offs = field_offsets f sn in List.iteri @@ -2386,11 +2463,15 @@ and block f body dst t = instruction and it happens after the transfer guard, so a call that signalled never reaches it. + The same temporary answers a right-hand side that reads its own + destination, which [reads] asks of [root]: built in place, the fields + written first would be what the later ones read. + [dst] is worked out by the caller and outlives this: [scoped] reclaims the temporary, not the destination. *) -and assign f ~(dst : loc) (v : Tast.expr) : unit = +and assign f ~(root : root) ~(dst : loc) (v : Tast.expr) : unit = let t = v.Tast.ty in - if (not (is_agg t)) || settles v then lower f v dst + if (not (is_agg t)) || (settles v && not (reads root v)) then lower f v dst else begin let o = Lf (tmp f t) in lower f v o; diff --git a/test/programs/self-read.flan b/test/programs/self-read.flan new file mode 100644 index 00000000..a468a19d --- /dev/null +++ b/test/programs/self-read.flan @@ -0,0 +1,62 @@ +;;;; An aggregate whose value reads the place it is assigned to sees the old +;;;; value of that place, in every field. +;;;; +;;;; (set p (P {.a (.b p) .b (.a p)})) swaps the two fields. It does only if +;;;; the whole right-hand side is read before any of it is stored. A backend +;;;; that builds the struct straight into [p] writes [.a] first, and the read +;;;; of [(.a p)] for [.b] then sees the new value: the answer comes back 2 2 +;;;; rather than 2 1. +;;;; +;;;; Each row is a different destination: a global, a field of a global, a +;;;; local, a field of a local, a place behind a pointer, a variable read +;;;; through a pointer that points at it, and a local rebound inside a loop +;;;; whose previous turn the new value reads. The last row reads a different +;;;; variable, which is the case still built in place. + +(defstruct P [a i32 b i32]) +(defstruct Q [tag i32 inner P]) + +(defonce g P (P {.a 1 .b 2})) +(defonce gq Q (Q {.tag 0 .inner (P {.a 1 .b 2})})) + +(defn show [p P] () + (print (.a p)) (print " ") (print (.b p)) (print "\n")) + +(defn main [] () + ;; A global. + (set g (P {.a (.b g) .b (.a g)})) + (show g) + ;; A field of a global. + (set (.inner gq) (P {.a (.b (.inner gq)) .b (.a (.inner gq))})) + (show (.inner gq)) + (let [p (P {.a 1 .b 2}) + q (Q {.tag 0 .inner (P {.a 1 .b 2})}) + r (P {.a 1 .b 2}) + s (P {.a 1 .b 2}) + t (P {.a 1 .b 2}) + other (P {.a 3 .b 4})] + ;; A local. + (set p (P {.a (.b p) .b (.a p)})) + (show p) + ;; A field of a local. + (set (.inner q) (P {.a (.b (.inner q)) .b (.a (.inner q))})) + (show (.inner q)) + ;; A place behind a pointer, reading the variable it points at. + (let [pr (addr r)] + (set (deref pr) (P {.a (.b r) .b (.a r)})) + (show r)) + ;; A variable, reading itself through a pointer. + (let [ps (addr s)] + (set s (P {.a (.b (deref ps)) .b (.a (deref ps))})) + (show s)) + ;; A local rebound in a loop, reading its own previous turn. + (let [pt (addr t) + i 0] + (while (< i 2) + (let [u (P {.a (.b (deref pt)) .b (.a (deref pt))})] + (show u) + (set pt (addr u))) + (set i (+ i 1)))) + ;; No self-read. + (set p (P {.a (.b other) .b (.a other)})) + (show p))) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 58a8cf4e..53273af4 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -2720,6 +2720,20 @@ let () = outputs ~x86:true "an assignment signalled through leaves the old value, --x86" "programs/half-write.flan" half_write_out; + (* The other half of building in place: a value that reads its own + destination has to see the old value in every field. x86 built + [(set p (P {.a (.b p) .b (.a p)}))] into [p] and answered 2 2. *) + let self_read_out = + "2 1\n2 1\n2 1\n2 1\n2 1\n2 1\n2 1\n1 2\n4 3\n" + in + outputs "an aggregate that reads its destination sees the old value" + "programs/self-read.flan" self_read_out; + outputs ~opt:"-O0" + "an aggregate that reads its destination sees the old value, -O0" + "programs/self-read.flan" self_read_out; + outputs ~x86:true + "an aggregate that reads its destination sees the old value, --x86" + "programs/self-read.flan" self_read_out; (* ── Packages: the link follows the program ──────────────────────── A package's C and linker arguments used to come with the import, From 8b354d4bc89d22219f2c6fd21a54a85bbc1d7f44 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:02:31 +0700 Subject: [PATCH 02/10] The notes say what is built: no pool, frames on x86, implicit widening, and the budget in the spec --- README.md | 3 +- TODO.org | 90 +++++++++++------- docs/BUILT.md | 186 +++++++------------------------------- docs/SPIKE-GENERICS.md | 2 +- emacs/MANUAL.md | 13 +-- lib/dev.ml | 33 +------ plan.org | 30 +++--- runtime/flan_dev.c | 4 +- spec-memory.md | 42 +++++++-- web/examples/numbers.flan | 2 +- web/index.html | 19 ++-- 11 files changed, 156 insertions(+), 268 deletions(-) diff --git a/README.md b/README.md index 94b0892a..368b8a7e 100644 --- a/README.md +++ b/README.md @@ -145,8 +145,7 @@ executable in a trench coat. `flan dev` is the one command that takes it unasked: a dev session is what it was written for, it halves the `C-c C-c` round trip, and nothing it builds outlives the session. `--llvm` is how to ask for the other one there — for a -program this backend refuses by name, for `--debug`, and for the inspector, -which walks a shadow stack it does not push. Every other command here is LLVM +program this backend refuses by name, and for `--debug`. Every other command here is LLVM by default and stays that way; `emacs/MANUAL.md` lists what the dev backend does not do. diff --git a/TODO.org b/TODO.org index fcf9257b..25ab769a 100644 --- a/TODO.org +++ b/TODO.org @@ -957,6 +957,11 @@ ignore order, writable access has to alias the real storage. Flexible field orde waits for classes deliberately, because a class owns its layout and a =Vector2= should not pay for identity and metadata. Not implemented. +** TODO An error in a called generic's body is reported twice +=(defn g [x $t] u64 (nosuch x))= called once from =main= prints "unknown +function nosuch" twice at the same place and counts 2 errors — once from the +abstract pass and once from the instantiation. + * Backends ** DONE The x86 backend tracks LLVM at -O0 @@ -1184,9 +1189,12 @@ for needs a slice that can carry the Vec's identity, and a slice is pointer and length — so either slices grow a word in a dev build or the trap does not exist. Today it does not. -** TODO The allocator's budget is not in the spec -=alloc-budget= and =set-alloc-budget= exist and the spec does not mention them. -Worth folding in or replacing with a growable arena. +** DONE The allocator's budget is not in the spec +CLOSED: [2026-09-25] +spec-memory.md has a Budget subsection under Allocators, as built: a ceiling on +live bytes, 0 for none, that a =retry= handler raises. The failure bullet says +"raises the allocator's budget" where it said "grows the arena", since no arena +grows. A growable arena is not ruled out; nothing here asks for one. ** TODO The Vec header is not the size the spec fixes Five words in every build rather than the spec's four, and for a stated reason: a @@ -1506,8 +1514,8 @@ before shadowed locals are even honest, and that buys locals in lldb rather than the break loop. ** DONE The x86 backend pushes shadow-stack frames -It does, in every dev build. The prose in the daemon that rewrites the agent's -reply to say otherwise is now false and should go with whoever next touches it. +It does, in every dev build. The daemon no longer rewrites the agent's reply to +say otherwise. ** DONE A restart is not a transaction If a frame mutates a global and then signals, taking a =retry= re-runs the @@ -1935,41 +1943,57 @@ Printing the stable kind at the end of the first line, the understood-then- conflicted clause order as a writing rule, and non-cascading multiple errors. Each needs a decision rather than work. -** TODO docs/BUILT.md still describes (Handle T) and the pool as built -It reads as shipped fact for a type the checker has no constructor for and the -runtime has no code for, including an enumeration primitive in the present tense. -The pool was removed on 2026-09-18. +** DONE docs/BUILT.md still describes (Handle T) and the pool as built +CLOSED: [2026-09-25] +The section is a short past-tense record: that neither exists, why they went, +the decisions a library version would face, and that classes landed without the +enumeration the pool was built to give. A stale runtime comment naming +=resolve= went with it. -** TODO The daemon still tells an editor the x86 backend pushes no frames -It does push them, in every dev build. The reply is rewritten from a claim that -stopped being true. +** DONE The daemon still tells an editor the x86 backend pushes no frames +CLOSED: [2026-09-25] +The rewrite in =Dev.ask= is gone, so an x86 session passes the agent's reply +through as an LLVM one does. The Emacs manual and the README no longer list the +inspector as something the x86 backend cannot do. -** TODO web/index.html still claims there is no implicit widening -Two places. Left alone deliberately — the website has its own rewrite lane. +** DONE web/index.html still claims there is no implicit widening +CLOSED: [2026-09-25] +Both sentences say what is true: a conversion that cannot change the number is +implicit, any other is written. The rest of the page is left for its rewrite. -** TODO plan.org's Types section lists a predicate that no longer exists -It still describes =copyable?= and "a type variable is move-only by default", -both of which the ownership repeal removed. Left alone deliberately by the -generics lane as the repeal lane's sentence to retire. +** DONE plan.org's Types section lists a predicate that no longer exists +CLOSED: [2026-09-25] +The five predicates are the checker's five, with =integer?= in place of +=copyable?=, and the section no longer names =(Handle $t)= or =pool-new=. -** TODO plan.org cites the wrong mechanism for jank's relinking bug -The real cause was a process-teardown race; jank calls through vars, which are -already indirection cells. We are safe from the repro because we compile out of -process, not because of cells. A normative document citing the wrong mechanism -protects the wrong invariant. +** TODO plan.org's Data model section still describes move-only containers +"Owning containers move rather than copy on assignment" and "one =Vec= field +makes it move-only" predate the repeal, under which everything copies. -** TODO docs/SPIKE-GENERICS.md lists landed work as remaining -Three items are under "Mechanical" as remaining work and have landed. A dated -report going stale at a live claim. +** DONE plan.org cites the wrong mechanism for jank's relinking bug +CLOSED: [2026-09-25] +plan.org now says the crash was a teardown race (the maintainer's diagnosis on +issue #947), that jank already calls through vars, citing the clone, and that +Flan avoids the repro by compiling out of process. -** TODO Two citations in spec-memory.md do not land where they say -One is off by a line in a reference clone; the other names an arm that is not the -one the claim is about. The claim behind the second is true and cited in the wrong -place, and a third companion citation in the same section is stale too. +** DONE docs/SPIKE-GENERICS.md lists landed work as remaining +CLOSED: [2026-09-25] +The row is marked as landed, with the test that covers it, rather than removed; +the report stays a dated record. -** TODO The sand hash is quoted as prose in five places besides its assertion -The assertion carries the current number; the prose copies do not all. Whoever -re-takes the number has that list. +** DONE Two citations in spec-memory.md do not land where they say +CLOSED: [2026-09-25] +=Map_Cell_Info= is cited at core.odin:351. The =defer= bullet cites the +=defer_ok= field and the =Ast.Defer= arm, and says what the checker accepts: a +=defer= in a top-level =let= is legal, so =(defer (free v))= for a =let=-bound +=v= is expressible and the spec no longer says otherwise. + +** DONE The sand hash is quoted as prose in five places besides its assertion +CLOSED: [2026-09-25] +Four prose copies were left, not five. BUILT.md points at =sand_out= in +test_acceptance.ml instead of quoting a number. The page's two are checked by +quotes.sh against a run, so they stay. The handoff report is dated and keeps the +number it recorded. ** DONE sand.flan is two programs CLOSED: [2026-09-13] diff --git a/docs/BUILT.md b/docs/BUILT.md index 6dfdd95c..9453e8a5 100644 --- a/docs/BUILT.md +++ b/docs/BUILT.md @@ -3316,164 +3316,42 @@ there and matching it from a program. `dev.ml`'s inspector still says "union val frame's locals, and `shim.ml`'s "a Flan union has no C layout" is now inaccurate as prose though the refusal it guards is still right: a union has a C layout and still may not cross to C by value, because the shim flattens aggregates. -## `(Handle T)` and the pool, which is what a stale reference answers with +## `(Handle T)` and the pool, which were built and removed -**Removed 2026-09-18**, in the second round of the repeal: two containers are -enough, the corpus's only pools were its own fixtures, and a slab behind -generational handles is a library over a `Vec` when a program wants one — -which is where Odin keeps it. The `gen` word on `Vec` and `Map` went the same -day (maintained, read by nothing; the header is `ptr len cap allocator -epoch` now), and so did the move-only concept itself — everything copies, and -`copyable?` left the predicate list. spec-memory.md, "The repeal", carries -all three amendments. The section below is kept as the record of what was -built. +Neither exists. The checker has no constructor for `(Handle T)` or `(Pool T)`, and the runtime has no code for either. +Both were built and then **removed on 2026-09-18**, in the second round of the repeal: two containers are enough, the +corpus's only pools were its own fixtures, and a slab behind generational handles is a library over a `Vec` when a +program wants one — which is where Odin keeps it. The `gen` word on `Vec` and `Map` went the same day, and so did the +move-only concept itself. `spec-memory.md`, "The repeal", carries the amendments. -A handle is a reference to something that can die, which reports that it died rather than silently resolving to -whatever reused its slot. `check.ml` refused `(Handle T)` by name as milestone 6; this is what it stood for, and the -pool came with it because the pool is what makes the report possible. +A handle was a reference to something that can die, which reported that it died rather than silently resolving to +whatever reused its slot. The decisions the built version made are the ones a library version faces, so they are kept +here: -The problem is concrete and is not about memory safety. Entities live in a pool; something holds a reference to one — -a projectile chasing it, the UI showing its health. The entity dies, the slot is reused, and a raw index now names a -different entity. Nothing crashes. The projectile chases the wrong thing, at full speed, for the rest of the game. +- **A handle was one i64**: slot index in the low 32 bits, the slot's generation in the high 32. It owned nothing; the +pool was the single owner. +- **Live was odd.** A slot's generation was bumped on every allocation and every release, so a zeroed handle +(generation 0) resolved to nothing and a liveness test needed no second array. +- **The generation wrapped by retiring the slot.** A release from `0xFFFFFFFF` left the slot dead forever rather than +letting a future handle collide with an old one. One leaked slot is a bounded price for making the collision +unrepresentable. +- **`resolve` answered `(Option (Ptr T))`**, because a matched struct is a copy and writing to the pooled entity in +place is what a pool is for. A pointer from `resolve` was invalidated by an `insert` that grew the pool, as a slice is +by a `push`. +- **`length` was the slot high-water and `live` the count**, so a loop bounded by `length` over `(pool-handle p i)` +visited every live entry. That pair was the whole of iteration. +- **A slot was released through the pool**, `(release p h)`, answering `bool` so a double release was an answer +rather than a trap. +- **A stale handle was an answer and a released pool was a trap**, through the epoch check a `Vec` gets. +- **Growth was transactional**: both blocks were allocated and copied before the old pair was released, because +`retry` re-attempts the same request. -It was built for two reasons, both already recorded. On its own terms, for entities referred to across frames. And -because **it is the real gate on managed classes**: plan.org's rule is that nothing starts on `defclass` until -ordinary `struct`, `Handle` and reload semantics work, and `Handle` was the only one of the three missing. That is not -an incidental precondition — `migrate-instances` has to *enumerate* live instances, and a pool behind generational -handles gives that by construction where a world arena and an owned region do not. plan.org presents the three storage -strategies as a free choice and they are not. +### Classes did not need it -`PORTING.md` found no customer for handles in the author's real game today, so this is deliberately the smallest -correct thing rather than a rich API: eight names, no iteration protocol, no cursor type, no `clone`. - -### A handle is one i64, and the halves are 32 and 32 - -Slot index in the low 32 bits, that slot's generation counter in the high 32. One machine word, so it copies, zeroes -and compares like the integer it is, and it **owns nothing** — the pool is the single owner. That is what lets a -handle sit in a struct field and in a global where a `Vec` may not, and it is the reason the ownership rules needed no -new case: `Types.is_move_only` says yes to `Pool` and no to `Handle`, and the three existing refusals (a struct field, -a union case, a global) picked the pool up unchanged with the messages they already had. - -The index is 32 bits because a `Vec`'s index is an `i32` here and widening indices is one change across every -container, not a pool question. - -### Live is odd, and two things fall out of it - -A slot's generation starts at 0 and is bumped on every allocation *and* on every release, so an odd generation means -live and an even one means dead. Both consequences are load-bearing: - -- **A zeroed handle resolves to nothing.** Generation 0 is even, so ZII gives a `(Handle T)` field the right meaning -for free instead of pointing it at slot 0. `handles.flan` prints one: ``, and resolving it answers `None`. -- **Iteration needs no second array and no spare bit.** Asking whether a slot is live is asking whether its generation -is odd. - -### The generation wraps by retiring the slot - -32 bits is 2^31 allocate/release pairs on one slot — every frame at 60fps for a year and a bit. "Rare" is not an -answer when the failure it produces is the silent wrong one this type exists to prevent, so a release from generation -`0xFFFFFFFF` bumps to 0 and does **not** put the slot back on the free list. The slot is retired: dead forever, its -payload leaked, and no future handle can collide with an old one. Leaking is defined behaviour here, and one slot is a -bounded price for making the collision unrepresentable rather than unlikely. - -### `resolve` answers `(Option (Ptr T))`, and the spec settled that, not this lane - -The task asked whether a lookup should answer `(Option T)`, matching `(get m k)`. It answers `(Option (Ptr T))`, and -`spec-memory.md` already writes it out — its worked example under "Mutating something you matched" is annotated -`(Option (Ptr Enemy))` — for the reason given one line above it: *pattern bindings bind values, so a matched struct is -a copy*. A copy cannot be written back, and writing to the pooled entity in place is what a pool is for. `(Option T)` -would answer a question nobody asked. - -The `Option` half is `get`'s shape and for `get`'s reason: absence is an answer, not a failure. A trap would be wrong -here — the entity dying is the *expected* case, not a bug. - -**The hole, said plainly.** A `(Ptr T)` from `resolve` is invalidated by any `insert` that grows the pool, exactly as -a slice is invalidated by a `push`. The handle survives that and the pointer does not. This is `spec-memory.md`'s -explicit Zig/Odin borrowing contract one level down, and it is worth naming rather than implying, because it -reintroduces the silent-wrong-answer mode the handle just removed for anyone who keeps a resolved pointer across an -insert. Chunked never-moving storage is the fix and it costs code; taking the contract is the smaller correct thing, -given `slice` already established it. - -### `length` is the slot high-water and `live` is the count, in that direction - -`(length p)` is how many slots have ever been handed out. `(live p)` is how many of them are live now. It had to be that -way round: `0..(length p)` are the indices `(pool-handle p i)` accepts, so a loop bounded by `length` visits every live -entry. Bounded by the live count instead, it would silently skip entries the moment anything had been released — -which is exactly the quiet wrong answer the whole type exists to remove. - -`(pool-handle p i)` answers `(Option (Handle T))`: the handle of slot `i`, or `None` if that slot is dead. That plus -`length` is the whole of iteration. An index outside `0..(length p)` **traps**, exactly as `(at v i)` traps: an index is an -index here, and answering `None` for one would hide a bug rather than a death. - -### A slot is released through the pool, and that is not a third release point - -`free` consumes its argument as a move, and a handle is a copyable number that owns nothing — consuming one copy would -say nothing about the others. So `(free h)` is refused by name and the release operation is on the owner: -`(release p h)`. `spec-memory.md`'s two release points are untouched: `(free p)` is release point 1 applied to the -pool, and a `free-all` of the region takes the pool with everything else. `release` recycles a slot inside storage the -pool still owns, which is not a release of storage at all. - -It answers `bool` rather than `()`: true if this call released it, false if the handle was already gone. The -generational scheme makes a double release **detectable**, and that is worth handing to the caller — this is the one -place in the language where freeing something twice is an answer instead of a refusal. - -### Growth is transactional, because `retry` re-attempts the same call - -`StorageExhausted`'s restart re-attempts the *same* request, so a failed grow has to leave the pool byte for byte as -it was — including a `cap` that still agrees with the real block sizes, since the next attempt passes `cap` as the -allocator's `old_size`. A pool grows two blocks together (payloads and slot headers), so resizing the first in place -and then failing on the second would leave `cap` describing neither. So the runtime allocates both, copies, and only -then releases the old pair: nothing is mutated after the last thing that can fail. An allocator without `can-free` -leaks the first block when the second fails, which is the defined outcome and not a new one — the request failed -because the region is exhausted, and the region is about to be released whole or its ceiling raised. - -### Two failures, kept apart - -A stale handle is an **answer**: `resolve` says `None` and the program carries on. A pool whose allocator was released -**traps**, through the same epoch check a `Vec` gets — the slot array went with the storage and there is nothing left -to ask. `test/programs/pool-stale-region.flan` is that case, and keeping the two apart is the same rule that keeps a -`Vec`'s generation word and its epoch word apart: they answer different questions and must not be conflated. - -### Two amendments to a frozen spec - -Both are places where `spec-memory.md` describes a handle doing something that cannot answer "gone", which is the one -thing the type exists to do. **This amends it: both are deferred, not built.** - -**1. `.field` and `at` do not auto-deref a handle.** The Places table says `x` may be a struct, a `(Ptr S)` or a -`(Handle S)`, and that the two forms auto-deref exactly one pointer *or handle* level. They auto-deref one pointer -level and nothing else. A `(set (.hp h) ...)` through a handle has two possible meanings when the entity is dead — trap, -or do nothing — and both are worse than the third option, which is the spec's own worked example: resolve first, match, -and the compiler makes you handle the `None`. The spec contradicts itself here and the example is the half that is -right. - -**2. `deref` is not overloaded on `(Handle a)`.** The Generics section says "`deref` yields a value; `resolve` yields a -pointer. Both are overloaded on `(Ptr a)` and `(Handle a)`." `deref` is `(Ptr a)` only. Same reason: `deref` returns a -value and has nowhere to put "gone". - -### What this does not have, and one of the gaps is not a pool question - -- **No `clone`.** Refused by name. A copied pool would carry the same slot generations, so one handle would resolve in -both copies and name two different things — the exact confusion the type removes. A program that wants a second world -builds one and inserts into it, and the new handles say they are new. -- **No pool of an owning element.** `(Pool (Vec i32))` is refused where `(Vec (Vec i32))` is refused and for the same -reason: the type-erased runtime copies and releases slots bytewise. Recursive teardown arrives with `drop`. -- **A handle is not a map key.** For the reason a `Ptr` is not: hashing an identity is a different operation from -hashing what it names, and a stale handle hashes the same as it always did while naming nothing. -- **Handles compare with `=` and not with `<`.** `Types` grew a second predicate, `is_equatable`, beside -`is_comparable`. Two handles are equal exactly when they name the same slot at the same generation, so a stale handle -is never equal to the live one that replaced it — that is worth one integer compare. Ordering them would order a slot -index, which is a free-list artefact and means nothing. -- **A pool passed to a helper is consumed**, because a pool is move-only exactly as a `Vec` is and there is no -borrowing parameter in the language. `test/programs/handles.flan` is one long function for that reason, and it does -not work around it. This is a pre-existing gap and not a pool question: the same sentence is true of every `Vec` in -the tree. - -### What classes still need - -The enumeration primitive is the piece migration was blocked on, and it exists now. What is left is `defclass` itself -and its runtime shape metadata; `migrate-instances`, which is a walk over `(length p)` and `(pool-handle p i)`; generic -functions and method dispatch, whose expensive half is already built and tested (a generic function is an indirection -cell whose body is a dispatch table, and a reload extends the table); and the rule that `Enemy@1` stays resolvable for -as long as any instance holds it — the same rule as "nothing is ever `dlclose`d". +The pool was built as the gate on managed classes, on the reading that `migrate-instances` has to enumerate live +instances. Classes landed without it: a `defclass` instance is a dyn map with its class in the object header, and a +redefined class migrates each instance lazily at its next touch, so nothing is enumerated (TODO.org, "A redefined +defclass migrates its instances lazily"). ## Macros: the compiler dlopens the program @@ -4839,7 +4717,7 @@ and changing one's type is a silent mismatch against storage the host already la Deferred until after the dev loop: 6. ~~**wasm32.**~~ **Done, with one glued joint.** `flan build --target=wasm32-wasi` produces a module, and -`test/programs/sand-headless.flan` prints `15595743031174623232` under it — the same hash as native, byte for byte, at +`test/programs/sand-headless.flan` prints the same hash under it as native, byte for byte (the number is `sand_out` in `test/test_acceptance.ml`), at `-O2` and at `-O0`. That is the milestone: the RNG is ours rather than libc's precisely so that number can be compared across targets, and it compares equal. `values.flan` and `machine.flan` run there too, which is where a 32-bit pointer would have shown. The acceptance table runs all four, and skips them by *probing* — it builds the smallest program and diff --git a/docs/SPIKE-GENERICS.md b/docs/SPIKE-GENERICS.md index c3c04da7..1533da17 100644 --- a/docs/SPIKE-GENERICS.md +++ b/docs/SPIKE-GENERICS.md @@ -228,7 +228,7 @@ with where predicates", in buckets: | | | |---|---| | **Done in the spike** | one or more type variables in a signature; binding through `[t]`, `(Ptr t)`, `(Option t)`, `(Fn [t] t)` and nesting; return types mentioning a variable; left-to-right binding with substitution into later parameters so an `fn` literal gets its types; the Odin-keyed instantiation cache; generic calling generic, transitively; `(vec-new t)`; the abstract refusal pass for `+`, `-`, `*`, `/`, `%`, `=`, `!=`, `<`, `<=`, `>`, `>=`, the bitwise operators and the shifts; instantiations as ordinary `Tast.fn`s with dev cells and `Reach` edges for free; a depth cap so runaway instantiation refuses instead of hanging | -| **Mechanical** | the remaining builtins that take a *type name* as an argument — `pool-new`, `map-new`, `zeroed`, `uninit`, the casts — each of which reaches `type_named`/`resolve_name` by its own path, exactly as `vec-new` did (one line there fixed `vec-new`; the others are one line each); `hash` and the map key-pair path, which must refuse a `Var` rather than assume; the `fns` expansion in `session.ml` described above | +| **Mechanical** (all three have since landed, and `test/programs/generics.flan` exercises the first two; `pool-new` was removed with the pool) | the remaining builtins that take a *type name* as an argument — `pool-new`, `map-new`, `zeroed`, `uninit`, the casts — each of which reaches `type_named`/`resolve_name` by its own path, exactly as `vec-new` did (one line there fixed `vec-new`; the others are one line each); `hash` and the map key-pair path, which must refuse a `Var` rather than assume; the `fns` expansion in `session.ml` described above | | **Bulky, not hard** | error messages that say *where* an instantiation came from — Odin's "in instantiation of" note. Today a refusal inside an instantiated body points at the generic's source with no indication which call site asked for that type, and with three or four instantiations that is the difference between a readable refusal and a puzzle. It is a context stack in `ctx` and a `Loc.note` per frame, and it touches every `fail` under an instantiation | | **Fiddly** | move-only and ownership. `Types.is_move_only (Var _)` is false, but the same variable at `(Vec i32)` is move-only — so the abstract pass **cannot decide ownership at all**, and the dead-set analysis is only sound per instantiation. Today that means a generic body that moves its parameter type-checks abstractly and is caught, if at all, at one instantiation and not another. The rule has to be stated: either ownership is checked only per copy (and the abstract pass skips it, so a generic may be accepted and its instantiation refused), or type variables carry a move-only constraint, which is a constraint system and plan.org says not yet. One smaller thing in the same bucket, found and left alone: the abstract pass over a generic body that calls *another* generic at a concrete type generates that copy and keeps it, so plain `flan emit` can carry a body no call site asked for. It is a valid instantiation and `Reach.link` drops it, so `flan build` and `flan run` are unaffected — but the abstract pass is meant to leave nothing behind and this is the one thing it does | | **No plan** | (1) **Unbounded instantiation.** `(defn grow [x $t] () (grow [x x]))` asks for a copy at `[2 t]`, which asks for one at `[2 [2 t]]`, forever. Before the cap it did not fail, it *hung* — and since `Session.eval` runs this same code, the thing that hangs is `C-c C-c`, with the dev daemon wedged behind it and no error to show. That is the project's stated priority hanging on three lines of ordinary-looking Flan, so the spike stops it: a depth counter in `env`, refusing past 32 and naming the type it had reached (`spike/generics/runaway.flan`). **The number is arbitrary and the designed refusal — one that names the chain of instantiations rather than the depth it gave up at — is still open.** **Odin has no cap of its own**, so there is no implementation to copy. (2) **Generic structs and containers.** `Types.Named` is a bare string with no parameters, so `(defstruct Pair [a $t b $t])` cannot be spelled at all — a parameterised named type is a change to `Types.t` and therefore to every backend, `Render`, DWARF and the layout calculator. (3) **`println` over a type variable.** plan.org's one compiler-provided exception; the abstract pass rejects it (`no printer for t`), see question 6. (4) **Generics across packages.** `Load` flattens imports into one namespace before checking, so it happens to work here, but a package boundary that is ever a real compilation-unit boundary would need the generic's *body* to cross it — the thing separate compilation cannot do and the reason C++ puts templates in headers | diff --git a/emacs/MANUAL.md b/emacs/MANUAL.md index 0ab2ecb5..e08994ed 100644 --- a/emacs/MANUAL.md +++ b/emacs/MANUAL.md @@ -97,13 +97,8 @@ the default, and it is the reason `C-c C-c` is fast: about 28ms at the socket against about 63ms through LLVM, because `as` does in 8ms what `llc` does in 45ms. On a project you are iterating in, that is the difference you feel. -It is not the whole compiler. Three things a session built by it cannot do: +It is not the whole compiler. Two things a session built by it cannot do: -- **No backtrace, no locals, no globals.** The inspector — `C-c C-b`, and - everything the break loop shows you about *where* a stopped program is — walks - a shadow stack of frames, and this backend does not push them. A program built - by it still stops on an unhandled error, still takes a restart, still answers - `C-x C-e` at the break. It just cannot tell you where it stopped. - **No `C-u C-c C-a`.** That view shows the LLVM IR a body was built from, and there is none; the listing this backend produced is assembly. Plain `C-c C-a` disassembles the object and works exactly as before. @@ -128,7 +123,7 @@ command line after the file, and it is how anything that is not the file or the socket reaches `flan dev` from Emacs: ```elisp -(setq flan-daemon-args '("--llvm")) ; the inspector, the IR view, every form +(setq flan-daemon-args '("--llvm")) ; the IR view, every form (setq flan-daemon-args '("--debug")) ; breakpoints, through flan-dape ``` @@ -142,8 +137,8 @@ honest shape of the thing — which backend your sessions use is a property of the project you are working on, not of the keystroke that started this one. **If you were relying on the old default:** every `flan dev` before this one -was an LLVM session, so a workflow built around `C-c C-b` or `C-u C-c C-a` will -find them refused now. One line in your init puts it back. +was an LLVM session, so a workflow built around `C-u C-c C-a` will find it +refused now. One line in your init puts it back. --- diff --git a/lib/dev.ml b/lib/dev.ml index 693575a2..0ae23018 100644 --- a/lib/dev.ml +++ b/lib/dev.ml @@ -295,38 +295,7 @@ let result t = thread on the frame that erred and waits. The agent is where that shows, and the session is the only thing holding it — so an editor asks here or not at all. *) -(* One sentence the agent cannot write, rewritten in the one place every verb - passes through. - - [vendor/agent/flan_agent.c] answers a backtrace, a locals or a globals - request from a program with no shadow-stack frames with "this program was - not built with --dev". That was true of exactly one thing when it was - written -- a release build, which has no frames because it has no dev - machinery at all -- and it stopped being true when [lib/x86.ml] became the - default for [flan dev]. An x86 dev host has its cells, its globals and its - registry; what it has not got is the frame push, so it stops on an error and - then cannot say where. The agent has no way to tell the two apart: it sees - an empty chain either way, and it is inside the program, which knows nothing - about the backend that compiled it. - - The session does know, so it is the one that corrects the sentence. Rewriting - the reply rather than teaching the agent a new environment variable keeps the - claim where the fact is -- and a message that names [--llvm] is the whole of - what the reader needs, where "not built with --dev" sends them to look for a - flag they did not leave off. *) -let mentions hay needle = - let n = String.length hay and m = String.length needle in - let rec go i = i + m <= n && (String.sub hay i m = needle || go (i + 1)) in - m = 0 || go 0 - -let ask t verb = - let text = request t verb in - if t.session.Session.x86 && mentions text "was not built with --dev" then - "err the x86 dev backend pushes no shadow-stack frames, so a stopped \ - program built by it cannot say where it is -- no backtrace, no locals and \ - no globals. It is the default for flan dev because it is about twice as \ - fast; restart the daemon with flan dev --llvm to inspect frames.\n" - else text +let ask t verb = request t verb type state = | Running diff --git a/plan.org b/plan.org index 73d2be6d..1498b8da 100644 --- a/plan.org +++ b/plan.org @@ -218,7 +218,7 @@ and on a managed ~class~ instance. An ordinary ~struct~ never carries one. names; the guess was silently wrong twice, so the slot is mandatory. - Every type notation reads as exactly one data item: ~[f32]~, ~[4 f32]~, ~(Vec f32)~, ~(Map string i32)~, ~(Ptr World)~, ~(Fn [f32] bool)~, - ~(Option $t)~, ~(Handle $t)~. The map spelling was ~{string i32}~ once and is + ~(Option $t)~. The map spelling was ~{string i32}~ once and is not any more: braces in type position are refused by name. The ~where~ clause below is a brace form at the head of a body, so keeping the brace type would have put ~(defn f [...] {string i32} {:where ...} body)~ in the language — two @@ -229,7 +229,7 @@ and on a managed ~class~ instance. An ordinary ~struct~ never carries one. HKTs). A type variable is written ~$t~ wherever a *type* goes — parameter, return type, or nested as ~[$t]~ or ~(Vec $t)~ — and bare ~t~ wherever a type's *name* is an argument in expression position: ~(vec-new t)~, ~(map-new t i32)~, - ~(pool-new t)~, and the cast ~(t x)~. This is what makes + and the cast ~(t x)~. This is what makes ~map-in-place~/~filter~/~reduce~ and the monomorphic containers work; it collapsed the prelude's per-type families into one function each. A generic body is checked *abstractly*, with nothing substituted, so ~=~, ~<~, @@ -241,23 +241,16 @@ and on a managed ~class~ instance. An ordinary ~struct~ never carries one. predicates, which is Odin's (~core/slice/slice.odin:289~, ~where intrinsics.type_is_ordered(T)~). It is written as a Clojure-style map at the head of the body — ~{:where (ordered? $t)}~, or a vector for more than one, - ~{:where [(copyable? $t) (copyable? $u)]}~ — on the precedent of Clojure's + ~{:where [(ordered? $t) (hashable? $u)]}~ — on the precedent of Clojure's ~{:pre ... :post ...}~, and because a bare ~{}~ in expression position is already refused so nothing else it could be. ~sort~ declares ~ordered?~ of its variable, the abstract pass then allows ~<~ in the body, and each instantiation checks the concrete type satisfies the predicate and refuses the call site if it does not. There are *five* predicates — ~ordered?~, ~equal?~, ~hashable?~, - ~numeric?~, ~copyable?~ — against Odin's forty-one, and they entail one another - in one direction, so one clause usually does: ~numeric?~ gives ~ordered?~, - ~ordered?~ gives ~equal?~, and any of the four gives ~copyable?~. - ~copyable?~ has no Odin counterpart, because Odin has no move semantics and a - ~$T~ there never has to answer the question. *A type variable is move-only by - default* and ~copyable?~ is the opt-out: whether a variable is move-only is not - decidable abstractly — ~i32~ at one instantiation, ~(Vec i32)~ at the next — so - the checker takes the stricter rule, which can only refuse a program that would - have been fine and never admit one that double-frees. The prior art is Rust's - ~T: Copy~, differing in that the compiler answers the question rather than a - user implementing a trait. + ~numeric?~, ~integer?~ — against Odin's forty-one, and they entail one another + in one direction, so one clause usually does: ~integer?~ gives ~numeric?~, + ~numeric?~ gives ~ordered?~, and ~ordered?~ gives ~equal?~. ~integer?~ exists + because ~numeric?~ admits floats. ~hashable?~ is what lets a variable *key a map*: without it the type ~(Map $t i32)~ is refused where it is written, and with it the refusal moves to the call site that names an unhashable key. @@ -695,8 +688,13 @@ values safely retain the old version. The session immediately warns at each tracked old caller site, and recompiling that caller either updates it or gives a normal type error. This preserves live running code without hiding stale calls. -This is the fix for jank issue #947 (segfault redefining a running loop's -function plus its callee — their JIT relinks and unloads under a running thread). +jank issue #947 (a segfault after redefining a running loop's function and its +callee) is not what cells protect against. jank already calls through vars, +which are indirection cells (~compiler+runtime/src/cpp/jank/codegen/cpp_processor.cpp:565~ +derefs the var at the call). The crash was a teardown race: the REPL's piped +input reached EOF and jank shut LLVM down while another thread was still +JIT-compiling. Flan is safe from that repro because it compiles out of process, +in the daemon, not because of cells. ** What redefinition cannot do Patch a mid-execution frame and continue at the same PC — its register diff --git a/runtime/flan_dev.c b/runtime/flan_dev.c index 2fc3bfe3..fbedac8a 100644 --- a/runtime/flan_dev.c +++ b/runtime/flan_dev.c @@ -1116,8 +1116,8 @@ void *flan_dev_frame_slot(const void *frame, int32_t i) { * What a note records is a *block*, not a value: base, extent, and the size of * one element. So a lookup is containment rather than equality, and that is * not an optimisation — every pointer a program can hold into heap storage is - * interior. (at v i) is v->ptr + i*size and (resolve p h) is an item in the - * middle of a pool's array; neither is ever a base address. A table that + * interior. (at v i) is v->ptr + i*size, which is a base address only at + * i = 0. A table that * answered only exact hits would answer nothing anybody can ask it. * * Dead entries are kept, which is the second thing this buys: an address that diff --git a/spec-memory.md b/spec-memory.md index 19e9fcaf..578dcc11 100644 --- a/spec-memory.md +++ b/spec-memory.md @@ -450,6 +450,27 @@ An allocator declares which operations it implements. Odin's arena answers equivalent is a **capability set** on the allocator value, readable at run time. The one that is load-bearing below is `can-free`. +### The budget + +Every allocator also carries a ceiling on the bytes it has live, read and set at +run time: + +| Operation | Meaning | +|---------------------------|------------------------------------------------| +| `(alloc-budget a)` | the ceiling, an `i64`; 0 means there is none | +| `(set-alloc-budget a n)` | sets it; a negative `n` sets 0 | + +A request that would take the live total over the ceiling fails as an exhausted +allocator does, and signals `StorageExhausted` (see "Allocation failure"). An +arena is bounded by its backing buffer as well, and whichever limit is lower +applies. Setting a ceiling below what is already live releases nothing; it +refuses the next request. + +The budget is what a `retry` handler raises. Releasing the region a container +lives in invalidates the container, so for a fixed store the handler that makes +the same request succeed is the one that raises the ceiling. It is also how a +program exhausts an allocator on purpose, as `test/programs/exhausted.flan` does. + ### When storage is released There are exactly two release points, and neither of them is a scope. @@ -477,13 +498,14 @@ region it names is released, if ever, by an explicit `free-all` somewhere else. This is deliberate, and it is the point on which the two obvious precedents were rejected: -- **Odin's `defer delete`** cannot be written here. `defer` is function-scoped - (`check.ml:505` refuses it in a `let`, a loop or a branch) and, because `let` - is a block, a top-level `defer` is checked in a scope containing only the - parameters and globals (`check.ml:1670`). `(defer (free v))` for a `let`-bound - `v` is **not expressible today**. It becomes expressible with either - block-scoped `defer` or a sequential top-of-body binder; until one of those - exists, no idiom in this spec may depend on it. +- **Odin's `defer delete`** can be written only where its scope is the whole + function. `defer` is function-scoped: it is accepted at the top level of a + function body and in a `let` that is itself at the top level, to any depth, + and refused in a branch or a loop body (`check.ml:477`, the `defer_ok` field, + and the `Ast.Defer` arm at `check.ml:3661`). So `(defer (free v))` for a + `let`-bound `v` works when that `let` is at the top level of the body, and + runs at function exit rather than at the end of the `let`. There is no + block-scoped `defer`, and no idiom in this spec may depend on one. - **Carp's scope-end frees** are a whole-program linear analysis that inserts a teardown call at every binding's last use (`Memory.hs`, and `Info.hs`'s `Deleter`). Carp could not reconcile that with an arena and therefore has no @@ -655,7 +677,7 @@ not alternatives. Odin arranges it exactly this way: `elem_align` is threaded through every type-erased dynamic-array entry point (`base/runtime/dynamic_array_internal.odin` — `__dynamic_array_reserve`, `__dynamic_array_resize`, `__dynamic_array_append`), -and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:350`). The +and `align_of_type` sits in `Map_Cell_Info` (`base/runtime/core.odin:351`). The monomorphised wrapper is the only place the concrete type is known, so it is the only place that can produce the number. @@ -748,8 +770,8 @@ and says nothing is the outcome this rule exists to make impossible. break loop, where a working allocator is known. - Unhandled, `error` enters the dev break loop or aborts in release (spec-conditions.md §2). It is never a no-op; `signal` is not used here. -- A handler that frees something, releases a scratch region, or grows the arena - and then invokes `retry` re-attempts the same request. A handler that wants a +- A handler that frees something, releases a scratch region, or raises the + allocator's budget and then invokes `retry` re-attempts the same request. A handler that wants a *different* allocator needs a restart taking an argument, which does not exist yet; until it does, such a handler rebinds the context allocator and retries. - Because an allocating operation can transfer, every caller of one checks the diff --git a/web/examples/numbers.flan b/web/examples/numbers.flan index 0fa0a5fb..9cbe216f 100644 --- a/web/examples/numbers.flan +++ b/web/examples/numbers.flan @@ -1,6 +1,6 @@ (defn main [] i32 (let [n 40 ; i32, inferred - big (i64 n) ; every widening is written + big (i64 n) ; a cast is a call named for the type x 1.5] ; f64 (print (+ big 2)) (println "") (print (* x 2.5)) (println "") diff --git a/web/index.html b/web/index.html index ea157cfe..aac03f46 100644 --- a/web/index.html +++ b/web/index.html @@ -539,12 +539,15 @@ notation reads as exactly one data item.

-

There is no implicit widening. Every operand of an arithmetic or -comparison form has one type, and every conversion is written as a cast:

+

A conversion that cannot change the number is implicit; any other is +written. An i32 goes where an i64 or an f64 +is wanted, and an f32 where an f64 is. An i64 into an +i32, or an i32 into an f32, is an error until it is +written as a cast:

(defn main [] i32
   (let [n 40                       ; i32, inferred
-        big (i64 n)                ; every widening is written
+        big (i64 n)                ; a cast is a call named for the type
         x 1.5]                     ; f64
     (print (+ big 2)) (println "")
     (print (* x 2.5)) (println "")
@@ -1367,11 +1370,11 @@ quoted or it could not be told from the punctuation around it.

since the walk is unrolled at compile time, rather than putting ten thousand printing sites in the module.

-

That the walk takes the argument's own type matters more here than it would in a -language that widens implicitly. Nothing widens implicitly in this one, so a printer -that named a type would need a cast written at every call — and a u64 -above 263 put through a signed one comes out negative. print -takes the value as it is and prints the number it holds.

+

That the walk takes the argument's own type matters for the types that do not +widen into one another. A printer that named one type, say i64, would take +an i32 as it is but need a cast for a u64 — and a +u64 above 263 put through that cast comes out negative. +print takes the value as it is and prints the number it holds.

The prelude

From 991e4423aae0748f8c7c5dcc2af6770a66f065b6 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:19:52 +0700 Subject: [PATCH 03/10] A dev session removes its build directory when it closes, and the next session sweeps the ones whose process is gone Every daemon left about 8MB in $TMPDIR/flan-dev-. A test_dev run left 300MB, and a few concurrent runs filled the tmpfs /tmp, after which every daemon died on its link before binding. The half-write test's abort also goes through [aborted] now, since an abort that worked can arrive as the socket closing. --- lib/dev.ml | 82 +++++++++++++++++++++++++++++++++++++++++------- test/test_dev.ml | 48 ++++++++++++++++++++++++++-- 2 files changed, 116 insertions(+), 14 deletions(-) diff --git a/lib/dev.ml b/lib/dev.ml index 693575a2..ca813039 100644 --- a/lib/dev.ml +++ b/lib/dev.ml @@ -4241,6 +4241,72 @@ let accept_loop ?grace t ls = in go () +(* ── The session's directory ───────────────────────────────────────── *) + +(* Every session builds into [$TMPDIR/flan-dev-]: the host executable, + its IR or listing, one module per evaluation. About 8MB before the first + evaluation, and nothing used to remove it, so every daemon that ever ran + left one behind. On a machine whose /tmp is a tmpfs that is memory, and a + test run starts some forty daemons: enough of them at once filled /tmp, and + the daemons after that died before binding, on the link, with "No space + left on device" — the "fail to bind under load" flake. + + Two halves. A session that ends through [close] removes its own directory + on the way out. One that does not — a signal, a program that died where it + stood through [_exit], a failed build — cannot, so the next session to start + removes every [flan-dev-] whose process no longer exists. A pid that + is alive keeps its directory whoever it is now: a reused pid costs one + directory kept too long, never one removed from under a session. + + Only ESRCH counts as gone. EPERM is a live process that belongs to someone + else, and a directory in a shared /tmp that is not ours fails to remove + anyway. *) +let dir_prefix = "flan-dev-" + +let session_dir_of pid = + Filename.concat (Filename.get_temp_dir_name ()) + (Printf.sprintf "%s%d" dir_prefix pid) + +(* Best effort throughout: what cannot be removed stays. The chmod is for a + directory a session left read-only, which test_dev's cannot-write-a-module + case does on purpose. *) +let rec remove_tree path = + match Unix.lstat path with + | { Unix.st_kind = Unix.S_DIR; _ } -> + (try Unix.chmod path 0o700 with Unix.Unix_error _ -> ()); + (match Sys.readdir path with + | names -> Array.iter (fun n -> remove_tree (Filename.concat path n)) names + | exception Sys_error _ -> ()); + (try Unix.rmdir path with Unix.Unix_error _ -> ()) + | _ -> (try Unix.unlink path with Unix.Unix_error _ -> ()) + | exception Unix.Unix_error _ -> () + +let sweep_dead_sessions () = + let tmp = Filename.get_temp_dir_name () in + let own = Unix.getpid () and plen = String.length dir_prefix in + match Sys.readdir tmp with + | exception Sys_error _ -> () + | names -> + Array.iter + (fun n -> + if String.length n > plen && String.sub n 0 plen = dir_prefix then + match int_of_string_opt (String.sub n plen (String.length n - plen)) with + | Some pid when pid > 0 && pid <> own -> + (match Unix.kill pid 0 with + | () -> () + | exception Unix.Unix_error (Unix.ESRCH, _, _) -> + remove_tree (Filename.concat tmp n) + | exception Unix.Unix_error _ -> ()) + | _ -> ()) + names + +(* This process's directory, after clearing out the dead ones. *) +let session_dir () = + sweep_dead_sessions (); + let dir = session_dir_of (Unix.getpid ()) in + (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); + dir + (* [debug] is off by default, which keeps [flan dev] exactly what it was: a -O2 host and -O2 modules. It is opt-in rather than always-on because a debug build is an -O0 build — [llvm.dbg.declare] describes an alloca and mem2reg @@ -4256,11 +4322,7 @@ let two_process ?(debug = false) ?(x86 = true) ~file ~sock () = directory it was relative to. *) let file = try Unix.realpath file with Unix.Unix_error _ -> file in let session, l = Session.create ~debug ~x86 ~file () in - let dir = - Filename.concat (Filename.get_temp_dir_name ()) - (Printf.sprintf "flan-dev-%d" (Unix.getpid ())) - in - (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); + let dir = session_dir () in let exe = Filename.concat dir "program" in (* [keep] so the host's own IR survives the build. It is the text [llc] was actually given, not a second emission of it, which is the difference @@ -4351,7 +4413,8 @@ let two_process ?(debug = false) ?(x86 = true) ~file ~sock () = (try Unix.kill child Sys.sigterm with Unix.Unix_error _ -> ()); (try Unix.close ls with Unix.Unix_error _ -> ()); (try Unix.close rd with Unix.Unix_error _ -> ()); - (try Unix.unlink sock with Unix.Unix_error _ -> ())) + (try Unix.unlink sock with Unix.Unix_error _ -> ()); + remove_tree dir) (fun () -> accept_loop t ls) (* ── One process: the program and the compiler in the same binary ──── *) @@ -5234,6 +5297,7 @@ let merged_serve () = Printf.eprintf "flan dev: %s\n%!" (Printexc.to_string e)); (try Unix.close ls with Unix.Unix_error _ -> ()); (try Unix.unlink sock with Unix.Unix_error _ -> ()); + remove_tree t.dir; (* [close] from the editor ends the session, and so now does an editor that stopped being there; in one process either means the program too — which is what the daemon did by killing its child. [_exit] for the loader-lock @@ -5250,11 +5314,7 @@ let start_merged ?(debug = false) ?(x86 = true) ~file ~sock () = let t0 = Unix.gettimeofday () in let file = try Unix.realpath file with Unix.Unix_error _ -> file in let session, l = Session.create ~debug ~x86 ~file () in - let dir = - Filename.concat (Filename.get_temp_dir_name ()) - (Printf.sprintf "flan-dev-%d" (Unix.getpid ())) - in - (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); + let dir = session_dir () in let exe = Filename.concat dir "program" in (* The host's IR goes straight to its final home rather than being written into the build's working directory and moved: the merged link is spelled diff --git a/test/test_dev.ml b/test/test_dev.ml index 5be18307..7ee6b241 100644 --- a/test/test_dev.ml +++ b/test/test_dev.ml @@ -216,6 +216,33 @@ let () = rewrites: a true statement about the backend and no test of the verb. The default itself is checked further down, on a daemon that does not need frames. *) + (* A session builds into [$TMPDIR/flan-dev-], about 8MB before its + first evaluation. Left behind by every daemon, those filled a tmpfs /tmp + under a few concurrent runs of this file, and every daemon after that + died on its link before binding. So: a dead session's directory is + swept by the next session to start, a live one's is not, and a session + that ends through [close] takes its own with it. The dead pid is a + process this test ran and reaped; this test's own pid is the live one. *) + let session_dir p = + Filename.concat (Filename.get_temp_dir_name ()) + (Printf.sprintf "flan-dev-%d" p) + in + let dead = + let p = + Unix.create_process "true" [| "true" |] Unix.stdin Unix.stdout + Unix.stderr + in + ignore (Unix.waitpid [] p); + p + in + let plant p = + let d = session_dir p in + (try Unix.mkdir d 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); + Out_channel.with_open_bin (Filename.concat d "program") (fun oc -> + output_string oc "left behind") + in + plant dead; + plant (Unix.getpid ()); let pid = Unix.create_process flan [| flan; "dev"; "programs/dev-loop.flan"; "-s"; sock; "--llvm" |] @@ -226,6 +253,15 @@ let () = if not (listening ~pid sock) then fail "the daemon %s" !listen_why else begin + if Sys.file_exists (session_dir dead) then + fail "a dead session's directory %s outlived the next session's start" + (session_dir dead); + if not (Sys.file_exists (session_dir (Unix.getpid ()))) then + fail "a live process's session directory was swept"; + (try Sys.remove (Filename.concat (session_dir (Unix.getpid ())) "program") + with Sys_error _ -> ()); + (try Unix.rmdir (session_dir (Unix.getpid ())) + with Unix.Unix_error _ -> ()); (* The daemon owns the program's lifetime and kills it on [close], so every step waits for the program to have got there. "ok" from an eval means the module was queued, not that it has been installed. *) @@ -797,6 +833,9 @@ let () = transcript is the claim — after everything the first run printed and before anything the second did. *) ignore (Unix.waitpid [] pid); + if Sys.file_exists (session_dir pid) then + fail "a session that ended through close left %s behind" + (session_dir pid); let text = Buffer.contents output in let wanted = "1\n5\n105\n777\npk\n106\n" in if text <> wanted then @@ -5605,9 +5644,12 @@ let () = abort is refused by a program that is *running*, which is what a failure above would leave behind — and this fixture then polls for twenty seconds and parks, so the wait below would be a hang rather - than a report. The signal is for that case only. *) - if status (request c "(:op \"abort\")") <> "ok" then - (try Unix.kill hpid Sys.sigkill with Unix.Unix_error _ -> ()); + than a report. The signal is for that case only. Through [aborted], + because an abort that worked can arrive as the socket closing. *) + (match aborted c with + | Some r when status r <> "ok" -> + (try Unix.kill hpid Sys.sigkill with Unix.Unix_error _ -> ()) + | _ -> ()); (try Unix.close c with Unix.Unix_error _ -> ()); (try ignore (Unix.waitpid [] hpid) with Unix.Unix_error _ -> ()) end; From 2250548b0773e6e59d3cee04c7cda16031448cf7 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:21:44 +0700 Subject: [PATCH 04/10] A test that sees a process die on a signal names the signal, not OCaml's number for it OCaml numbers the signals negatively and in its own order: SIGTERM is -11. The globals daemon's "transient signal 11" was a SIGTERM, not a segfault. --- test/test_acceptance.ml | 8 ++++---- test/test_agent.ml | 12 ++++++------ test/test_dev.ml | 21 +++++++++++++++++++-- test/test_emacs.ml | 7 ++++--- test/test_repl.ml | 6 ++++-- test/test_support.ml | 22 ++++++++++++++++++++-- 6 files changed, 57 insertions(+), 19 deletions(-) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 58a8cf4e..283af120 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -186,8 +186,8 @@ module Pool = struct name (match status with | Unix.WEXITED n -> Printf.sprintf "with code %d" n - | Unix.WSIGNALED n -> Printf.sprintf "on signal %d" n - | Unix.WSTOPPED n -> Printf.sprintf "stopped on signal %d" n)) + | Unix.WSIGNALED n -> "on " ^ Test_support.signal_name n + | Unix.WSTOPPED n -> "stopped on " ^ Test_support.signal_name n)) | None, _ -> Some (Printf.sprintf @@ -195,8 +195,8 @@ module Pool = struct name (match status with | Unix.WEXITED n -> Printf.sprintf "exit %d" n - | Unix.WSIGNALED n -> Printf.sprintf "signal %d" n - | Unix.WSTOPPED n -> Printf.sprintf "stopped, signal %d" n)) + | Unix.WSIGNALED n -> Test_support.signal_name n + | Unix.WSTOPPED n -> "stopped on " ^ Test_support.signal_name n)) in ignore (Queue.pop inflight); match msg with diff --git a/test/test_agent.ml b/test/test_agent.ml index 62a24bef..4d04b60a 100644 --- a/test/test_agent.ml +++ b/test/test_agent.ml @@ -149,8 +149,8 @@ let () = fail "agent reload\n got: %S (%s)\n wanted: %S" text (match status with | Unix.WEXITED c -> Printf.sprintf "exit %d" c - | Unix.WSIGNALED c -> Printf.sprintf "signal %d" c - | Unix.WSTOPPED c -> Printf.sprintf "stopped %d" c) + | Unix.WSIGNALED c -> Test_support.signal_name c + | Unix.WSTOPPED c -> "stopped on " ^ Test_support.signal_name c) "1\n1000\n1007\n" end; @@ -309,8 +309,8 @@ let () = %S (exit 1)" text (match !bstat with | Unix.WEXITED c -> Printf.sprintf "exit %d" c - | Unix.WSIGNALED c -> Printf.sprintf "signal %d" c - | Unix.WSTOPPED c -> Printf.sprintf "stopped %d" c) + | Unix.WSIGNALED c -> Test_support.signal_name c + | Unix.WSTOPPED c -> "stopped on " ^ Test_support.signal_name c) "cannot listen\n" end; @@ -766,8 +766,8 @@ let () = fail "abort left status %s, wanted exit 134" (match !lstatus with | Unix.WEXITED c -> Printf.sprintf "exit %d" c - | Unix.WSIGNALED c -> Printf.sprintf "signal %d" c - | Unix.WSTOPPED c -> Printf.sprintf "stopped %d" c) + | Unix.WSIGNALED c -> Test_support.signal_name c + | Unix.WSTOPPED c -> "stopped on " ^ Test_support.signal_name c) (* The other way out, and the one that skips atexit on purpose: [abort] leaves by [_exit] so that it cannot hang on the loader lock, and the socket is therefore unlinked by hand there. A file left here would diff --git a/test/test_dev.ml b/test/test_dev.ml index 7ee6b241..83387be3 100644 --- a/test/test_dev.ml +++ b/test/test_dev.ml @@ -196,6 +196,23 @@ let () = if not (contains_sub other "flan.abi.x86") then fail "a parked program's other refusals were rewritten too: %S" other +(* A daemon's death is reported by the signal's name. [WSIGNALED] carries + OCaml's own numbering, in which SIGTERM is -11, and a SIGTERM printed as + "signal -11" was once recorded as a segfault. A real process, really + terminated, so the number is whatever [waitpid] hands back. *) +let () = + let p = + Unix.create_process "sleep" [| "sleep"; "30" |] Unix.stdin Unix.stdout + Unix.stderr + in + Unix.kill p Sys.sigterm; + match Unix.waitpid [] p with + | _, Unix.WSIGNALED n when Test_support.signal_name n = "SIGTERM" -> () + | _, Unix.WSIGNALED n -> + fail "a process ended by SIGTERM is reported as %s" + (Test_support.signal_name n) + | _, _ -> fail "a process sent SIGTERM did not end on a signal" + let () = match Sys.command "command -v clang > /dev/null 2>&1 && command -v llc > /dev/null 2>&1" with | 0 -> @@ -5143,8 +5160,8 @@ let () = (match st with | Unix.WEXITED n -> Printf.sprintf "exit %d" n | Unix.WSIGNALED n when n = Sys.sigpipe -> "killed by SIGPIPE" - | Unix.WSIGNALED n -> Printf.sprintf "signal %d" n - | Unix.WSTOPPED n -> Printf.sprintf "stopped on %d" n); + | Unix.WSIGNALED n -> Test_support.signal_name n + | Unix.WSTOPPED n -> "stopped on " ^ Test_support.signal_name n); None | exception Unix.Unix_error _ -> Some (connect rsock) in diff --git a/test/test_emacs.ml b/test/test_emacs.ml index ecee324d..c2b2ebe1 100644 --- a/test/test_emacs.ml +++ b/test/test_emacs.ml @@ -112,9 +112,10 @@ let () = match !status with | Some (Unix.WEXITED n) -> Printf.sprintf "exited with status %d" n | Some (Unix.WSIGNALED n) -> - Printf.sprintf "was killed by signal %d — nothing ran its cleanup, so \ - the socket file is stale rather than gone" n - | Some (Unix.WSTOPPED n) -> Printf.sprintf "stopped on signal %d" n + Printf.sprintf "was killed by %s — nothing ran its cleanup, so \ + the socket file is stale rather than gone" + (Test_support.signal_name n) + | Some (Unix.WSTOPPED n) -> Printf.sprintf "stopped on %s" (Test_support.signal_name n) | None -> "was still running when the client finished, and was terminated" in (* Read before the removal below, because on a failure this is the evidence diff --git a/test/test_repl.ml b/test/test_repl.ml index 9579130f..752b0679 100644 --- a/test/test_repl.ml +++ b/test/test_repl.ml @@ -326,9 +326,11 @@ let () = "the daemon had already been killed by SIGPIPE — a reply written \ into a socket whose reader had gone" | Some (Unix.WSIGNALED n) -> - Printf.printf "the daemon had already been killed by signal %d\n" n + Printf.printf "the daemon had already been killed by %s\n" + (Test_support.signal_name n) | Some (Unix.WSTOPPED n) -> - Printf.printf "the daemon was stopped on signal %d\n" n + Printf.printf "the daemon was stopped on %s\n" + (Test_support.signal_name n) | None -> print_endline "the daemon was still running at the end"); Printf.printf "\n-- the program's own output, last 4k --\n%s\n" prog_out; exit 1 diff --git a/test/test_support.ml b/test/test_support.ml index 41228ca6..a64b73a7 100644 --- a/test/test_support.ml +++ b/test/test_support.ml @@ -152,6 +152,24 @@ let rec connect ?(ms = 5000) path = thing after a failed wait is always the report of it. *) let listen_why = ref "" +(* A signal as [WSIGNALED] carries it, by name. OCaml numbers the signals it + knows negatively and in its own order — [Sys.sigterm] is -11 and + [Sys.sigsegv] is -10 — so printing the number reads as the wrong signal to + anyone who knows the POSIX table: a daemon terminated by SIGTERM was once + recorded here as "a transient signal 11", a segfault that never happened. *) +let signal_name n = + let known = + [ Sys.sigabrt, "SIGABRT"; Sys.sigalrm, "SIGALRM"; Sys.sigfpe, "SIGFPE"; + Sys.sighup, "SIGHUP"; Sys.sigill, "SIGILL"; Sys.sigint, "SIGINT"; + Sys.sigkill, "SIGKILL"; Sys.sigpipe, "SIGPIPE"; Sys.sigquit, "SIGQUIT"; + Sys.sigsegv, "SIGSEGV"; Sys.sigterm, "SIGTERM"; Sys.sigusr1, "SIGUSR1"; + Sys.sigusr2, "SIGUSR2"; Sys.sigchld, "SIGCHLD"; Sys.sigbus, "SIGBUS"; + Sys.sigtrap, "SIGTRAP"; Sys.sigxcpu, "SIGXCPU" ] + in + match List.assoc_opt n known with + | Some s -> s + | None -> Printf.sprintf "signal %d" n + let listening ?(ms = 30000) ~pid path = let died = ref None in ignore @@ -171,10 +189,10 @@ let listening ?(ms = 30000) ~pid path = | Some (Unix.WEXITED n) -> Printf.sprintf "exited with status %d before binding %s" n path | Some (Unix.WSIGNALED n) -> - Printf.sprintf "was killed by signal %d before binding %s" n path + Printf.sprintf "was killed by %s before binding %s" (signal_name n) path (* Unreachable without WUNTRACED, and here only for exhaustiveness. *) | Some (Unix.WSTOPPED n) -> - Printf.sprintf "stopped on signal %d without binding %s" n path + Printf.sprintf "stopped on %s without binding %s" (signal_name n) path | None -> Printf.sprintf "was still running after %ds without binding %s, so it was the \ From 04f5d5a3493465ea9876420a2515ece3fe7b6256 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:22:42 +0700 Subject: [PATCH 05/10] Under Evil, a digit in the break buffer takes the restart, because the buffer's keys take precedence over Evil's --- emacs/flan-cnr.el | 8 ++++++++ emacs/test-flan-cider.el | 41 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/emacs/flan-cnr.el b/emacs/flan-cnr.el index 72b3dbcd..6b2ffb75 100644 --- a/emacs/flan-cnr.el +++ b/emacs/flan-cnr.el @@ -702,6 +702,14 @@ anyone who would rather TAB always moved." map) "Keys in `flan-cnr-mode'.") +;; Evil's normal state binds 0, the other digits and RET above any major +;; mode's map, so under Evil a digit moved point or started a count and took +;; nothing. This map takes precedence over Evil's state maps instead; the keys +;; it does not bind, j and k among them, are still Evil's. +(with-eval-after-load 'evil + (when (fboundp 'evil-make-overriding-map) + (evil-make-overriding-map flan-cnr-mode-map))) + (define-derived-mode flan-cnr-mode special-mode "flan-break" "What a stopped Flan program is offering." (setq buffer-read-only t)) diff --git a/emacs/test-flan-cider.el b/emacs/test-flan-cider.el index a0247452..aacc92d5 100644 --- a/emacs/test-flan-cider.el +++ b/emacs/test-flan-cider.el @@ -1704,6 +1704,47 @@ stopped program, which is the case where it should fire." (file-name-directory load-file-name)) nil t) +;; The break buffer's keys under Evil, pressed through the command loop rather +;; than called. Evil's normal state binds 0 (beginning of line), 1-9 (a count) +;; and RET (next line) above any major mode's map, so without the buffer's map +;; taking precedence a digit moved point and took nothing. Run only where +;; Evil is installed: -Q loads no packages, so it is looked for in the two +;; places a package install puts it, and turned off again afterwards so nothing +;; above or below this runs under it. +(let* ((dirs (append (file-expand-wildcards "~/.config/emacs/elpa/evil-[0-9]*") + (file-expand-wildcards "~/.emacs.d/elpa/evil-[0-9]*") + (file-expand-wildcards "~/.config/emacs/elpa/goto-chg-*") + (file-expand-wildcards "~/.emacs.d/elpa/goto-chg-*"))) + (load-path (append dirs load-path))) + (if (not (require 'evil nil t)) + (message " skip the break buffer under Evil (Evil is not installed)") + (evil-mode 1) + (unwind-protect + (let* ((sent nil) + (flan-cnr-request-function + (lambda (form) (setq sent form) (list :status "ok"))) + (buf (test-flan--cnr + (list :condition "Missing" :restarts '("retry" "skip"))))) + (switch-to-buffer buf) + (evil-initialize-state) + (goto-char (point-min)) + (execute-kbd-macro (kbd "0")) + (test-flan--check "under Evil, 0 takes restart 0" + (equal sent '(:op "restart-at" :index 0 :name "retry"))) + (setq sent nil) + (switch-to-buffer buf) + (execute-kbd-macro (kbd "1")) + (test-flan--check "under Evil, 1 takes restart 1" + (equal sent '(:op "restart-at" :index 1 :name "skip"))) + (setq sent nil) + (switch-to-buffer buf) + (goto-char (point-min)) + (search-forward " 1: ") + (execute-kbd-macro (kbd "RET")) + (test-flan--check "under Evil, RET takes the restart on its line" + (equal sent '(:op "restart-at" :index 1 :name "skip")))) + (evil-mode -1)))) + (message "\n%d checks, %d failures" test-flan--ran test-flan--failures) (kill-emacs (if (> test-flan--failures 0) 1 0)) From 5477c759090739a3ea8ae4611031be9624adbbee Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:24:20 +0700 Subject: [PATCH 06/10] Three of the four under-load and dogfooding entries are settled, and the fourth records what was tried --- TODO.org | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/TODO.org b/TODO.org index fcf9257b..148b42ed 100644 --- a/TODO.org +++ b/TODO.org @@ -1631,16 +1631,23 @@ sanitizer flag to it. Named as the check worth adding next; a day rather than an hour. The x86 backend is not a gap here — that pair is refused by name, because there is no sanitizer pass over hand-written assembly. -** TODO A transient signal 11 on a globals daemon -Seen once, never reproduced, on a daemon whose fixture had just gained a host -global =Vec= and a run-time-new one. A reproduction under load would settle it. +** DONE A transient signal 11 on a globals daemon +CLOSED: [2026-09-25] +Not a segfault. The report was OCaml's signal number, and in OCaml's numbering +-11 is SIGTERM (SIGSEGV is -10). Nothing in the daemon sends itself SIGTERM, so +it was killed from outside. The test binaries now print a signal by name +(=Test_support.signal_name=); a number from =WSIGNALED= is never printed raw. -** TODO test_dev daemons fail to bind under load -Daemons exiting with status 1 or 2 before binding their socket, across most of the -file at once, while the load average is high. No stale socket or leftover daemon -afterwards; green on a quiet machine. Distinct from the registry race and the -stale-park re-run flake, both of which are fixed. A daemon that dies before it -binds died on the compiler's side, before any program it built has run a line. +** DONE test_dev daemons fail to bind under load +CLOSED: [2026-09-25] +/tmp is a tmpfs, and every session left its =flan-dev-= build directory +behind (about 8MB; a test_dev run left 300MB). Several concurrent runs filled +it and the daemons after that died on the link with ENOSPC. A session now +removes its directory when it ends through =close=, and each new session +removes the directories of pids that no longer exist. A live pid's directory is +never touched, so a failed build's IR stays until the next session starts. +The half-write test's abort also goes through =aborted= now, which took the +same run down with an uncaught =Wire.Closed=. ** DONE A test binary that hangs is killed by its own alarm A reader branch that forgets to advance loops for ever and the suite waits as long @@ -1781,13 +1788,20 @@ function of that name. Not reproduced: the file type checks, =flan reload= of the same form builds, and a minimal =defclass= + =get= + =return= program compiles. So it is the session path against an installed program, and what is missing is what that daemon had installed at the time. +Also not reproduced against a live daemon (2026-09-25): an =eval= of the +=defclass=, of a =defn= doing the =get= and =return=, and of both in one form, +and an =eval-expr= of the =get=, all succeed. In a session every installed +function of no arguments returning =()= has the type =(CFn [] ())=, not only +=pause= — a bare =pause= or =tick= asked of the session says so — so the +keyword may not be what resolved. The next report wants the exact form sent. -** TODO A digit does not take the restart RET takes -Pressing =0= left the program stopped; RET on the same line resumed it. Both -end in =flan-cnr-take=, but the digit path (=flan-cnr-take-number=, -=flan-cnr.el:559=) scans from =point-min= for the line whose =flan-cnr-index= -matches and calls =take= inside a =save-excursion=. Not reproduced yet — needs -a non-raylib program stopped under a test daemon. +** DONE A digit does not take the restart RET takes +CLOSED: [2026-09-25] +Evil's normal state binds =0= (beginning of line), =1=-=9= (a count) and RET +above the major mode's map, so the digit never reached =flan-cnr-take-number=. +=flan-cnr-mode-map= is now an Evil overriding map; keys it does not bind stay +Evil's. The other special-mode buffers (inspect, watch, doc, disassembly, +diagnostics, lower) have the same exposure and are not changed. ** TODO Eval in the frame, from the break loop An expression is evaluated at a frame boundary, so it sees globals and not the From cbd910c117cf6e5020f59ae349b42c23ccd9b114 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:36:52 +0700 Subject: [PATCH 07/10] An arena's budget holds when a block grows in place, and a shrink is never over budget --- runtime/flan_rt.c | 5 ++++- test/programs/exhausted.flan | 18 ++++++++++++++++++ test/test_acceptance.ml | 2 +- 3 files changed, 23 insertions(+), 2 deletions(-) diff --git a/runtime/flan_rt.c b/runtime/flan_rt.c index 00a9ddef..f295fa44 100644 --- a/runtime/flan_rt.c +++ b/runtime/flan_rt.c @@ -1185,7 +1185,7 @@ static void *flan_heap_proc(flan_allocator *a, int32_t mode, void *p, * caller passes old_size for exactly this reason, and it is the one * number a wrong answer here would read off the end of. */ void *q; - if (flan_over_budget(a, size - old_size)) return NULL; + if (size > old_size && flan_over_budget(a, size - old_size)) return NULL; q = flan_heap_proc(a, FLAN_ALLOC_ALLOC, NULL, 0, size, align); if (!q) return NULL; if (p && old_size > 0) @@ -1331,6 +1331,9 @@ static void *flan_arena_proc(flan_allocator *a, int32_t mode, void *p, * without copying, which is the common shape. */ if (p && (uint8_t *)p + old_size == ar->base + ar->offset) { int64_t end = (int64_t)((uint8_t *)p - ar->base) + size; + /* The budget is checked here as on every other path; a block grown in + * place is still more live bytes. */ + if (size > old_size && flan_over_budget(a, size - old_size)) return NULL; if (end > ar->cap || end < 0) return NULL; ar->offset = end; if (end > ar->peak) ar->peak = end; diff --git a/test/programs/exhausted.flan b/test/programs/exhausted.flan index 206c99bc..0a1a1e2d 100644 --- a/test/programs/exhausted.flan +++ b/test/programs/exhausted.flan @@ -115,6 +115,24 @@ (println "INSERTIONSORT"))) ; INSERTIONSORT (println failures) ; 1 — failed once, retried once + ;; And an arena, whose budget is checked when a Vec grows its block in + ;; place as well as when it allocates a new one. A Vec that is the only thing + ;; pushing into an arena always grows in place, so without that check the + ;; ceiling would never be met. + (set tight (arena-new 65536)) + (set-alloc-budget tight 64) + (set failures 0) + (handler-bind + [(StorageExhausted [c] + (set failures (+ failures 1)) + (set-alloc-budget tight (* 2 (alloc-budget tight))) + (invoke-restart 'retry))] + (let [v (vec-new i32 tight)] + (dotimes [i 1000] (push v i)) + (println (length v)) ; 1000 + (println (at v 999)))) ; 999 + (println (> failures 0)) ; true + ;; And the restart is not once-per-program: it is established at each ;; allocation, so a later one offers it again. (set-alloc-budget tight 0) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 58a8cf4e..d8f8d00d 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -1663,7 +1663,7 @@ let () = flow an optimiser would otherwise launder. *) let exhausted_out = "64\n0\n126\ntrue\ntrue\n4\ntrue\n0\n8\n7\ntrue\n\ - 13\n73\n84\n90\nINSERTIONSORT\n1\n" + 13\n73\n84\n90\nINSERTIONSORT\n1\n1000\n999\ntrue\n" in outputs "storage exhausted, retried" "programs/exhausted.flan" exhausted_out; outputs ~opt:"-O0" "storage exhausted, retried, -O0" "programs/exhausted.flan" From cf71caefc237f622cfd4a38761347ed256c984b0 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:36:57 +0700 Subject: [PATCH 08/10] spec-memory.md cites the defer_ok field on the line it is declared --- spec-memory.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec-memory.md b/spec-memory.md index 578dcc11..1220f8f3 100644 --- a/spec-memory.md +++ b/spec-memory.md @@ -501,7 +501,7 @@ rejected: - **Odin's `defer delete`** can be written only where its scope is the whole function. `defer` is function-scoped: it is accepted at the top level of a function body and in a `let` that is itself at the top level, to any depth, - and refused in a branch or a loop body (`check.ml:477`, the `defer_ok` field, + and refused in a branch or a loop body (`check.ml:495`, the `defer_ok` field, and the `Ast.Defer` arm at `check.ml:3661`). So `(defer (free v))` for a `let`-bound `v` works when that `let` is at the top level of the body, and runs at function exit rather than at the end of the `let`. There is no From 918edfb0b56783cccdd6cd3bf2ce9883453e9562 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:37:37 +0700 Subject: [PATCH 09/10] plan.org describes containers, pools and classes as they are after the repeal --- TODO.org | 10 +++++++--- plan.org | 38 ++++++++++++++++++++------------------ 2 files changed, 27 insertions(+), 21 deletions(-) diff --git a/TODO.org b/TODO.org index 25ab769a..95a4021a 100644 --- a/TODO.org +++ b/TODO.org @@ -1966,9 +1966,13 @@ CLOSED: [2026-09-25] The five predicates are the checker's five, with =integer?= in place of =copyable?=, and the section no longer names =(Handle $t)= or =pool-new=. -** TODO plan.org's Data model section still describes move-only containers -"Owning containers move rather than copy on assignment" and "one =Vec= field -makes it move-only" predate the repeal, under which everything copies. +** DONE plan.org's Data model section still describes move-only containers +CLOSED: [2026-09-25] +The Data model section says assignment copies a container's header and the +copies alias one buffer. The memory tiers and the classes section name the pool +and generational handles as a library over a =Vec=, and the classes gate that +named =Handle= is replaced by what classes are as built. The milestone record +of what was frozen is left as history. ** DONE plan.org cites the wrong mechanism for jank's relinking bug CLOSED: [2026-09-25] diff --git a/plan.org b/plan.org index 1498b8da..e8b45efd 100644 --- a/plan.org +++ b/plan.org @@ -5,7 +5,7 @@ Two documents are normative and are settled ahead of implementation. Anything in this plan that contradicts them is out of date. - [[file:spec-memory.md][spec-memory.md]] — ownership, the four container types, - copies and moves, assignable places, generics without type classes, function + copies, assignable places, generics without type classes, function values. - [[file:spec-conditions.md][spec-conditions.md]] — the six hard cases of conditions/restarts: what ~signal~ returns, no-handler behaviour, restart @@ -56,14 +56,15 @@ Allocators all the way down; ~malloc~ hidden behind them. | Tier | Strategy | Cost | |-----------+---------------------------------+----------------| | Frame | arena, bulk reset each frame | free | -| Entities | pool + generational handles | free | +| Entities | a pool over a ~Vec~, in a library | free | | Subsystem | region, freed wholesale | free | | Dev/REPL | leaks by design, reset on reload| dev only | - Allocator is part of the calling convention, so a refcounted allocator can be added later without a language change. -- Generational handles instead of pointers for cross-references: a stale - reference is detectable, not undefined behaviour. +- Generational handles instead of pointers for cross-references, written as a + library over a ~Vec~ rather than provided by the language: a stale reference + is detectable, not undefined behaviour. - Symbols and code live in a permanent arena that only grows. ** Why no persistent collections @@ -93,8 +94,8 @@ world. |-------------+-----------------+------------------+------| | ~[n T]~ | inline, n items | copies | no | | ~[T]~ | ptr+len | copies the view | no | - | ~(Vec T)~ | ptr+len+cap | *moves* | yes | - | ~(Map K V)~ | open addressing | *moves* | yes | + | ~(Vec T)~ | ptr+len+cap | copies the header | yes | + | ~(Map K V)~ | open addressing | copies the header | yes | ~Vec~ and ~Map~ are monomorphic on element type and record their allocator. Not a Lua-style array/hash hybrid — that is what makes Lua's layout and performance unpredictable. @@ -105,17 +106,15 @@ world. returns ~(Option V)~; ~put~ is the ~()~-returning upsert. See spec-memory.md for the deferred move-aware operations. - Operations: ~get~, ~put~, ~remove~, ~push~, ~pop~, ~at~, ~length~, ~update~. - Copying is explicit: ~(clone m)~, and owning containers move rather than copy on - assignment. No ~!~ convention — nothing is immutable, so it + An independent copy is explicit: ~(clone m)~. Assignment copies a container's + header, and the two headers alias one buffer. No ~!~ convention — nothing is immutable, so it would carry no information. No ~assoc~; it only existed as the copy-returning form. - ~const~ qualifier on references and slices: compile-time contract that a callee will not mutate. Zero runtime cost. -- Value structs copy on assignment — but only *value* structs. Ownership is - structural: a struct is a value type iff every field is, so one ~Vec~ field - makes it move-only. This is what keeps "copies on assignment" from meaning a - shallow copy that aliases owned storage. Deep copies are always explicit: - ~(clone x)~. Value structs are the snapshot / undo / replay story; they need no - separate type. +- Structs copy on assignment. A struct with a ~Vec~ field copies the header, so + the two copies alias one buffer, as in Odin. Deep copies are always explicit: + ~(clone x)~. Structs without owning fields are the snapshot / undo / replay + story; they need no separate type. - Literals live in read-only memory. - *A global's initialiser may be computed.* A value the linker can write goes into the image and costs nothing to start; anything else is stored at @@ -156,8 +155,8 @@ identity, extensibility, and live schema changes: Classes require a managed allocation strategy and runtime class/shape metadata, but *not necessarily a tracing GC*. The initial likely choices are a world or -session arena, pool allocation behind generational ~(Handle T)~ values, or an -explicitly owned region. A small tracing GC confined to class instances remains +session arena, a library pool behind generational handles, or an explicitly +owned region. A small tracing GC confined to class instances remains an option if cyclic graphs prove burdensome; it never changes ~struct~ layout or the C ABI. @@ -187,8 +186,11 @@ surprising lazy mutation on field access: #+end_src The precise class syntax, inheritance, storage strategy, and migration API are -not frozen. Do not add classes until ordinary ~struct~, ~Handle~, and reload -semantics are working. +not frozen. What is built is narrower than this section: a ~defclass~ instance +is a dyn map with its class in the object header, and a redefined class +migrates each instance lazily at its next touch, so nothing enumerates live +instances (TODO.org, "defclass is a named dyn map with a shape tag" and "A +redefined defclass migrates its instances lazily"). Immutability also serves the optimiser: a value known never to be mutated can be copied into registers and stack-allocated freely. Mutability is what forces heap From 414e1e5ae6e999061e1d9fcb7cd4b6fb63f46e31 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 25 Sep 2026 07:40:33 +0700 Subject: [PATCH 10/10] The temp directory is left to the build-plumbing lane, and under Evil the break buffer takes only the keys it binds The dead-session sweep and the remove-on-close half are dropped: the sweep contradicts the recorded non-goals, and the other lane owns removal on a clean close. The Evil keys are defined per state from the map's own bindings, so special-mode-map's h, SPC, < and - no longer shadow Evil's. --- TODO.org | 20 +++++----- emacs/flan-cnr.el | 18 +++++++-- emacs/test-flan-cider.el | 10 ++++- lib/dev.ml | 82 ++++++---------------------------------- test/test_dev.ml | 39 ------------------- 5 files changed, 44 insertions(+), 125 deletions(-) diff --git a/TODO.org b/TODO.org index 148b42ed..456f723c 100644 --- a/TODO.org +++ b/TODO.org @@ -1640,14 +1640,13 @@ it was killed from outside. The test binaries now print a signal by name ** DONE test_dev daemons fail to bind under load CLOSED: [2026-09-25] -/tmp is a tmpfs, and every session left its =flan-dev-= build directory -behind (about 8MB; a test_dev run left 300MB). Several concurrent runs filled -it and the daemons after that died on the link with ENOSPC. A session now -removes its directory when it ends through =close=, and each new session -removes the directories of pids that no longer exist. A live pid's directory is -never touched, so a failed build's IR stays until the next session starts. -The half-write test's abort also goes through =aborted= now, which took the -same run down with an uncaught =Wire.Closed=. +The failure was a full /tmp, not the load. /tmp is a tmpfs, and every session +left its =flan-dev-= build directory behind (a test_dev run leaves about +300MB); a few concurrent runs filled it and each daemon after that died on its +link with ENOSPC. The fix is the session removing its own directory on a clean +=close=, which the build-plumbing lane owns. The half-write test's abort also +goes through =aborted= now, which took the same run down with an uncaught +=Wire.Closed=. ** DONE A test binary that hangs is killed by its own alarm A reader branch that forgets to advance loops for ever and the suite waits as long @@ -1799,8 +1798,9 @@ keyword may not be what resolved. The next report wants the exact form sent. CLOSED: [2026-09-25] Evil's normal state binds =0= (beginning of line), =1=-=9= (a count) and RET above the major mode's map, so the digit never reached =flan-cnr-take-number=. -=flan-cnr-mode-map= is now an Evil overriding map; keys it does not bind stay -Evil's. The other special-mode buffers (inspect, watch, doc, disassembly, +The keys =flan-cnr-mode-map= itself binds are given to Evil's normal and +motion states in that mode; every other key, including what =special-mode-map= +binds, stays Evil's. The other special-mode buffers (inspect, watch, doc, disassembly, diagnostics, lower) have the same exposure and are not changed. ** TODO Eval in the frame, from the break loop diff --git a/emacs/flan-cnr.el b/emacs/flan-cnr.el index 6b2ffb75..dea14019 100644 --- a/emacs/flan-cnr.el +++ b/emacs/flan-cnr.el @@ -704,11 +704,21 @@ anyone who would rather TAB always moved." ;; Evil's normal state binds 0, the other digits and RET above any major ;; mode's map, so under Evil a digit moved point or started a count and took -;; nothing. This map takes precedence over Evil's state maps instead; the keys -;; it does not bind, j and k among them, are still Evil's. +;; nothing. The keys this file binds are given to Evil's normal and motion +;; states in this mode. Only those keys: `flan-cnr-mode-map' inherits +;; `special-mode-map', and making it an overriding map would carry h, SPC, < +;; and - over from there as well. Every key not listed stays Evil's. (with-eval-after-load 'evil - (when (fboundp 'evil-make-overriding-map) - (evil-make-overriding-map flan-cnr-mode-map))) + (when (fboundp 'evil-define-key*) + ;; Collected first, and without the parent's bindings, because + ;; `evil-define-key*' writes into the map being walked. + (let ((own nil)) + (map-keymap-internal (lambda (key def) + (when (commandp def) (push (cons key def) own))) + flan-cnr-mode-map) + (dolist (b own) + (evil-define-key* '(normal motion) flan-cnr-mode-map + (vector (car b)) (cdr b)))))) (define-derived-mode flan-cnr-mode special-mode "flan-break" "What a stopped Flan program is offering." diff --git a/emacs/test-flan-cider.el b/emacs/test-flan-cider.el index aacc92d5..a13ead5d 100644 --- a/emacs/test-flan-cider.el +++ b/emacs/test-flan-cider.el @@ -1742,7 +1742,15 @@ stopped program, which is the case where it should fire." (search-forward " 1: ") (execute-kbd-macro (kbd "RET")) (test-flan--check "under Evil, RET takes the restart on its line" - (equal sent '(:op "restart-at" :index 1 :name "skip")))) + (equal sent '(:op "restart-at" :index 1 :name "skip"))) + ;; And the keys the buffer does not bind are still Evil's, not the + ;; ones `special-mode-map' would bring with it. + (switch-to-buffer buf) + (dolist (k '(("h" . evil-backward-char) ("SPC" . evil-forward-char) + ("<" . evil-shift-left) + ("-" . evil-previous-line-first-non-blank))) + (test-flan--check (format "under Evil, %s is still Evil's" (car k)) + (eq (key-binding (kbd (car k))) (cdr k))))) (evil-mode -1)))) (message "\n%d checks, %d failures" test-flan--ran test-flan--failures) diff --git a/lib/dev.ml b/lib/dev.ml index ca813039..693575a2 100644 --- a/lib/dev.ml +++ b/lib/dev.ml @@ -4241,72 +4241,6 @@ let accept_loop ?grace t ls = in go () -(* ── The session's directory ───────────────────────────────────────── *) - -(* Every session builds into [$TMPDIR/flan-dev-]: the host executable, - its IR or listing, one module per evaluation. About 8MB before the first - evaluation, and nothing used to remove it, so every daemon that ever ran - left one behind. On a machine whose /tmp is a tmpfs that is memory, and a - test run starts some forty daemons: enough of them at once filled /tmp, and - the daemons after that died before binding, on the link, with "No space - left on device" — the "fail to bind under load" flake. - - Two halves. A session that ends through [close] removes its own directory - on the way out. One that does not — a signal, a program that died where it - stood through [_exit], a failed build — cannot, so the next session to start - removes every [flan-dev-] whose process no longer exists. A pid that - is alive keeps its directory whoever it is now: a reused pid costs one - directory kept too long, never one removed from under a session. - - Only ESRCH counts as gone. EPERM is a live process that belongs to someone - else, and a directory in a shared /tmp that is not ours fails to remove - anyway. *) -let dir_prefix = "flan-dev-" - -let session_dir_of pid = - Filename.concat (Filename.get_temp_dir_name ()) - (Printf.sprintf "%s%d" dir_prefix pid) - -(* Best effort throughout: what cannot be removed stays. The chmod is for a - directory a session left read-only, which test_dev's cannot-write-a-module - case does on purpose. *) -let rec remove_tree path = - match Unix.lstat path with - | { Unix.st_kind = Unix.S_DIR; _ } -> - (try Unix.chmod path 0o700 with Unix.Unix_error _ -> ()); - (match Sys.readdir path with - | names -> Array.iter (fun n -> remove_tree (Filename.concat path n)) names - | exception Sys_error _ -> ()); - (try Unix.rmdir path with Unix.Unix_error _ -> ()) - | _ -> (try Unix.unlink path with Unix.Unix_error _ -> ()) - | exception Unix.Unix_error _ -> () - -let sweep_dead_sessions () = - let tmp = Filename.get_temp_dir_name () in - let own = Unix.getpid () and plen = String.length dir_prefix in - match Sys.readdir tmp with - | exception Sys_error _ -> () - | names -> - Array.iter - (fun n -> - if String.length n > plen && String.sub n 0 plen = dir_prefix then - match int_of_string_opt (String.sub n plen (String.length n - plen)) with - | Some pid when pid > 0 && pid <> own -> - (match Unix.kill pid 0 with - | () -> () - | exception Unix.Unix_error (Unix.ESRCH, _, _) -> - remove_tree (Filename.concat tmp n) - | exception Unix.Unix_error _ -> ()) - | _ -> ()) - names - -(* This process's directory, after clearing out the dead ones. *) -let session_dir () = - sweep_dead_sessions (); - let dir = session_dir_of (Unix.getpid ()) in - (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); - dir - (* [debug] is off by default, which keeps [flan dev] exactly what it was: a -O2 host and -O2 modules. It is opt-in rather than always-on because a debug build is an -O0 build — [llvm.dbg.declare] describes an alloca and mem2reg @@ -4322,7 +4256,11 @@ let two_process ?(debug = false) ?(x86 = true) ~file ~sock () = directory it was relative to. *) let file = try Unix.realpath file with Unix.Unix_error _ -> file in let session, l = Session.create ~debug ~x86 ~file () in - let dir = session_dir () in + let dir = + Filename.concat (Filename.get_temp_dir_name ()) + (Printf.sprintf "flan-dev-%d" (Unix.getpid ())) + in + (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); let exe = Filename.concat dir "program" in (* [keep] so the host's own IR survives the build. It is the text [llc] was actually given, not a second emission of it, which is the difference @@ -4413,8 +4351,7 @@ let two_process ?(debug = false) ?(x86 = true) ~file ~sock () = (try Unix.kill child Sys.sigterm with Unix.Unix_error _ -> ()); (try Unix.close ls with Unix.Unix_error _ -> ()); (try Unix.close rd with Unix.Unix_error _ -> ()); - (try Unix.unlink sock with Unix.Unix_error _ -> ()); - remove_tree dir) + (try Unix.unlink sock with Unix.Unix_error _ -> ())) (fun () -> accept_loop t ls) (* ── One process: the program and the compiler in the same binary ──── *) @@ -5297,7 +5234,6 @@ let merged_serve () = Printf.eprintf "flan dev: %s\n%!" (Printexc.to_string e)); (try Unix.close ls with Unix.Unix_error _ -> ()); (try Unix.unlink sock with Unix.Unix_error _ -> ()); - remove_tree t.dir; (* [close] from the editor ends the session, and so now does an editor that stopped being there; in one process either means the program too — which is what the daemon did by killing its child. [_exit] for the loader-lock @@ -5314,7 +5250,11 @@ let start_merged ?(debug = false) ?(x86 = true) ~file ~sock () = let t0 = Unix.gettimeofday () in let file = try Unix.realpath file with Unix.Unix_error _ -> file in let session, l = Session.create ~debug ~x86 ~file () in - let dir = session_dir () in + let dir = + Filename.concat (Filename.get_temp_dir_name ()) + (Printf.sprintf "flan-dev-%d" (Unix.getpid ())) + in + (try Unix.mkdir dir 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); let exe = Filename.concat dir "program" in (* The host's IR goes straight to its final home rather than being written into the build's working directory and moved: the merged link is spelled diff --git a/test/test_dev.ml b/test/test_dev.ml index 83387be3..fe2e9d6d 100644 --- a/test/test_dev.ml +++ b/test/test_dev.ml @@ -233,33 +233,6 @@ let () = rewrites: a true statement about the backend and no test of the verb. The default itself is checked further down, on a daemon that does not need frames. *) - (* A session builds into [$TMPDIR/flan-dev-], about 8MB before its - first evaluation. Left behind by every daemon, those filled a tmpfs /tmp - under a few concurrent runs of this file, and every daemon after that - died on its link before binding. So: a dead session's directory is - swept by the next session to start, a live one's is not, and a session - that ends through [close] takes its own with it. The dead pid is a - process this test ran and reaped; this test's own pid is the live one. *) - let session_dir p = - Filename.concat (Filename.get_temp_dir_name ()) - (Printf.sprintf "flan-dev-%d" p) - in - let dead = - let p = - Unix.create_process "true" [| "true" |] Unix.stdin Unix.stdout - Unix.stderr - in - ignore (Unix.waitpid [] p); - p - in - let plant p = - let d = session_dir p in - (try Unix.mkdir d 0o700 with Unix.Unix_error (Unix.EEXIST, _, _) -> ()); - Out_channel.with_open_bin (Filename.concat d "program") (fun oc -> - output_string oc "left behind") - in - plant dead; - plant (Unix.getpid ()); let pid = Unix.create_process flan [| flan; "dev"; "programs/dev-loop.flan"; "-s"; sock; "--llvm" |] @@ -270,15 +243,6 @@ let () = if not (listening ~pid sock) then fail "the daemon %s" !listen_why else begin - if Sys.file_exists (session_dir dead) then - fail "a dead session's directory %s outlived the next session's start" - (session_dir dead); - if not (Sys.file_exists (session_dir (Unix.getpid ()))) then - fail "a live process's session directory was swept"; - (try Sys.remove (Filename.concat (session_dir (Unix.getpid ())) "program") - with Sys_error _ -> ()); - (try Unix.rmdir (session_dir (Unix.getpid ())) - with Unix.Unix_error _ -> ()); (* The daemon owns the program's lifetime and kills it on [close], so every step waits for the program to have got there. "ok" from an eval means the module was queued, not that it has been installed. *) @@ -850,9 +814,6 @@ let () = transcript is the claim — after everything the first run printed and before anything the second did. *) ignore (Unix.waitpid [] pid); - if Sys.file_exists (session_dir pid) then - fail "a session that ended through close left %s behind" - (session_dir pid); let text = Buffer.contents output in let wanted = "1\n5\n105\n777\npk\n106\n" in if text <> wanted then