Merge: an Option opens without a match, and a file reads in one call

lib/prelude.ml

@@ -347,6 +347,70 @@ let source = {flan|
         (set m (max m (at s i))))
       (Some m))))
 
+;; ── Opening an Option without writing the match ───────────────────────
+;;
+;; Everything above answers an (Option $t), and until now `match` was the only
+;; thing that could open one. That is the right *primitive* — it is the form
+;; that makes the empty case unforgettable — and it is the wrong thing to write
+;; when the empty case is one word:
+;;
+;;   (match (edn/read src) (Some v) v None edn/Value.Nil)
+;;   (or-else (edn/read src) edn/Value.Nil)
+;;
+;; **Neither takes a {:where}, and that is a decision rather than an
+;; oversight.** A predicate buys an *operation* on the variable — [ordered?]
+;; is what lets sort! write `<` — and these perform no operation on their
+;; payload at all: they move it out of the Option, or they look at the tag and
+;; never touch the payload. That is the one move [ident] in
+;; test/programs/generics.flan makes, which needs nothing declared, so these
+;; instantiate at every type including the ones that own storage.
+;;
+;; Two, and the ones that were declined are worth naming because a reader will
+;; look for them:
+;;
+;;   none?           (not (some? o)) is the whole of it, and this file already
+;;                   refuses a wrapper whose only method is the thing it wraps
+;;                   — see the Builder entry under "Still refused".
+;;   an unwrap that  Refused for the reason file-size below is an Option in the
+;;   signals on None first place: absence is a reply and not a fault, and
+;;                   making it a condition puts a handler search on the
+;;                   ordinary path. Whether an empty Option is an error is the
+;;                   *caller's* question, and the caller has handler-bind if
+;;                   the answer is yes.
+;;   a lazy or-else  Would need a (Fn [] $t) — sort-by!'s shape, available the
+;;                   day something wants it. A macro would get laziness for
+;;                   free and need no generics, and costs more than it buys
+;;                   here: a prelude macro drops every prelude defn that
+;;                   depends on it out of a macro-module build (see the
+;;                   bootstrap hook at the foot of this file), and a macro has
+;;                   no way to report a malformed call.
+
+;; This is Java's `Optional.orElse`, hyphenated: eager, and it answers the
+;; payload's type rather than another Option. Worth saying which, because Rust
+;; spells something else `or_else` — there it takes a closure and answers an
+;; `Option<T>`, so borrowing that name for this behaviour would be the wrong
+;; loan twice over. Java's own lazy sibling is `orElseGet`, which is the one
+;; declined above.
+;;
+;; **At a $t that owns storage the result is a header copy, and the branch not
+;; taken is still the caller's to free.** Since the copyable? repeal every
+;; value copies as its header and the copies alias one buffer (see the section
+;; comment above), so (or-else o d) over a (Vec u8) hands back a second header
+;; onto o's block or onto d's — and the one it did not choose was never
+;; released by anything here. That is the same contract `at` on a slice of Vecs
+;; has; it is written down here because an "or a default" reads like it
+;; consumes the default and it does not.
+(defn or-else [o (Option $t)  d $t] $t
+  (match o (Some v) v None d))
+
+;; Clojure's `some?`, and the `?`-asks convention this file already spells with
+;; ok?, even? and file-exists?. It is what makes a `when` or a `cond` possible
+;; at all — or-else can say "this or that" and cannot say "only if there is
+;; one" — and it is the honest shape for the case where the payload is not
+;; wanted, which a match would still have to bind a name for.
+(defn some? [o (Option $t)] bool
+  (match o (Some _v) true None false))
+
 ;; map! writes back into the slice it was handed, for the same reason sort!
 ;; does — a slice is non-owning, and transforming a thing you already own
 ;; should not allocate. A map that produces a *different* element type is not

test/programs/edn-read.flan

@@ -20,6 +20,13 @@
 ;;;;     element, and a dedup written with `=` would make it two — two Vec
 ;;;;     headers over two blocks are never the same header.
 ;;;;
+;;;;   * (edn/read-file path) is the same read with the buffer owned and freed
+;;;;     inside the call, which is only safe because of the property below it.
+;;;;     Two cases: the real file by path, whose answer has to equal the embed
+;;;;     above byte for byte, and a missing one, where slurp's FileError has to
+;;;;     arrive at a handler *outside* read-file with `use-value` still armed —
+;;;;     the pass-through decision, run rather than asserted.
+;;;;
 ;;;;   * a Value owns its strings. The last case reads a document out of a
 ;;;;     buffer and then overwrites every byte of that buffer in place. A
 ;;;;     reader holding views prints the overwriting bytes; one holding copies
@@ -94,6 +101,55 @@
             _ (println "no name"))
         None (println "malformed")))))
 
+;; The path-taking entry point over the same file the embed above holds, and
+;; the whole reason both of this session's additions exist. What a program
+;; wants to write is one form:
+;;
+;;   (defvar game-data edn/Value
+;;     (with-allocator frame
+;;       (or-else (edn/read-file "game-data.edn") edn/Value.Nil)))
+;;
+;; and it is written as a defn here because the *initialiser* is still refused
+;; — "a global's value must be a compile-time constant — this one is computed"
+;; — which is a separate piece of work on globals and nothing to do with
+;; read-file or or-else. Everything inside the with-allocator is verbatim, so
+;; the day a computed initialiser is allowed, load-game-data and the `set`
+;; below it collapse back into the defvar above and this comment goes with
+;; them.
+;;
+;; The texture path printed here has to be the one show-tileset printed, which
+;; is what says read-file read the file and not merely something.
+(defvar game-data edn/Value)
+
+(defn load-game-data [] edn/Value
+  (or-else (edn/read-file "programs/assets/edn/tileset.edn") edn/Value.Nil))
+
+(defn by-path [] ()
+  (set game-data (load-game-data))
+  (match (field game-data "texture-path")
+    (Text s) (println s)
+    _ (println "no texture path")))
+
+;; A missing file, answered from outside read-file. Nothing in the package
+;; handles FileError, so the condition walks past it to here with both restarts
+;; armed, and `use-value` names a path read-file then slurps instead — the read
+;; resumes as if that file had been asked for all along, which is exactly what
+;; slurp.flan asserts for slurp alone.
+;;
+;; `some?` is the assertion: the answer is a real document, so the restart was
+;; taken rather than the read quietly answering None. Distinguishing the two is
+;; the reason read-file passes the condition through instead of folding a
+;; missing file into the None that means "malformed".
+(defvar saw-file-error i64)
+
+(defn by-missing-path [] ()
+  (handler-bind
+    [(FileError [c]
+       (set saw-file-error (+ saw-file-error 1))
+       (invoke-restart 'use-value "programs/assets/edn/tileset.edn"))]
+    (println (some? (edn/read-file "programs/assets/edn/not-here.edn"))))
+  (println saw-file-error))
+
 (defn main [] i32
   (set frame (arena-new 262144))
   (with-allocator frame
@@ -126,7 +182,11 @@
       ;; And the refusal, which has to be distinguishable from the document
       ;; that is literally nil.
       (match (edn/read (bytes "#{1 2"))  (Some _v) (println "read") None (println "malformed"))
-      (match (edn/read (bytes "nil"))    (Some _v) (println "read") None (println "malformed"))))
+      (match (edn/read (bytes "nil"))    (Some _v) (println "read") None (println "malformed"))
+      (println "")
+
+      (by-path)
+      (by-missing-path)))
   (free-all frame)
   (arena-destroy frame)
   0)

test/programs/generics.flan

@@ -11,7 +11,9 @@
 ;;;; a variable bound inside a slice, a generic calling a generic at its own
 ;;;; variable so that instantiation has to be transitive, the four where
 ;;;; predicates, two variables at once, println deferred to the instantiation,
-;;;; and the collapsed prelude family the whole feature was for.
+;;;; the collapsed prelude family the whole feature was for, and the family
+;;;; over (Option $t) — or-else and some? — which is the one that declares no
+;;;; predicate at all, so a $t that owns storage instantiates it too.
 
 ;; One variable, several types, and (ident 3) and (ident 7) share one copy.
 ;; The identity needs its parameter once, so it needs nothing declared: a type
@@ -73,6 +75,16 @@
     (push v x)
     v))
 
+;; An empty (Option (Vec u8)), which main needs to reach or-else's None branch
+;; at a type that owns storage. It is a function and not a bare None at the
+;; call site because a bare None there is refused — "nothing here says what
+;; None is an Option of" — and a return type is one of the two places the
+;; checker names as somewhere to say it. It is also the shape every real caller
+;; is in: what arrives at or-else came out of something, the way edn/read's
+;; answer does.
+(defn none-vec [] (Option (Vec u8))
+  None)
+
 ;; (zeroed) takes its type from the position it is written in, so a variable
 ;; in that position is answered by the instantiation like any other type.
 (defn zero-of [x $t] $t
@@ -135,6 +147,35 @@
     (match (min-of (slice ns 0 4)) (Some m) (println m) _ (println -1))
     (match (max-of (slice fs 0 3)) (Some m) (println m) _ (println -1.0))
     (match (index-of (slice ns 0 4) 18) (Some i) (println i) _ (println -1))
+
+    ;; or-else and some?, which are the same family over (Option $t) and take
+    ;; no predicate: they move the payload out or read the tag, and neither is
+    ;; an operation the variable has to be declared to support.
+    ;;
+    ;; Both branches at two scalar types, because a default that is returned
+    ;; and a default that is discarded are two different lowerings and only one
+    ;; of them is exercised by a call that happens to be Some.
+    (println (or-else (index-of (slice ns 0 4) 18) -1))   ; the Some branch
+    (println (or-else (index-of (slice ns 0 4) 77) -1))   ; the None branch
+    (println (or-else (max-of (slice fs 0 3)) 0.0))
+    (println (or-else (max-of (slice fs 0 0)) 0.0))
+    (println (some? (index-of (slice ns 0 4) 18)))
+    (println (some? (index-of (slice ns 0 4) 77)))
+    (println (some? (parse-i64 (bytes "12"))))
+
+    ;; And at a $t that owns storage, which is the case the scalars above say
+    ;; nothing about. What comes back is a *header* onto one of the two
+    ;; buffers, so both are still the caller's to free — hence two frees and
+    ;; not one, and the lengths are what say which header each answer holds.
+    (let [full (vec-new u8)
+          empty (vec-new u8)]
+      (push full 65)
+      (push full 66)
+      (println (len (or-else (Some full) empty)))   ; 2, full's header
+      (println (len (or-else (none-vec) empty)))    ; 0, empty's
+      (free full)
+      (free empty))
+
     (println (widen 3 0.0))
     (println (widen 3 (i64 0)))
     (println (zero-of 9))

test/test_acceptance.ml

@@ -619,13 +619,25 @@ let () =
 
        Last, that a malformed document is distinguishable from the document
        that is literally nil, which is why the entry point answers an Option
-       and not a Value. *)
+       and not a Value.
+
+       Then the path-taking entry point, whose two lines are the two halves of
+       what it decided. The texture path repeats the first line of the run,
+       from a file read at run time rather than embedded, which says read-file
+       read *that* file and that freeing its buffer inside the call took none
+       of the document with it. `true` then `1` is the pass-through: slurp's
+       FileError for a missing path walked past read-file to a handler here
+       with `use-value` still armed, the read resumed against the path the
+       handler named, and the answer is a document — not the None that means
+       malformed, which is the collapse read-file refuses to make. *)
     let edn_read_out =
       "./source-assets/Sprout Lands Premium/Objects/Mushrooms, Flowers, \
        Stones.png\n\
        54\ntrue\ntrue\ntrue\nfalse\n\n\
        0\n2\n2\n2\n1\n1\n2\n2\n2\n2\n\n\
-       level-1\nmalformed\nread\n"
+       level-1\nmalformed\nread\n\n\
+       ./source-assets/Sprout Lands Premium/Objects/Mushrooms, Flowers, \
+       Stones.png\ntrue\n1\n"
     in
     outputs "edn/read over the tileset" "programs/edn-read.flan" edn_read_out;
     outputs ~opt:"-O0" "edn/read over the tileset, -O0"
@@ -1898,10 +1910,19 @@ let () =
 
     (* Generics end to end: one written body per family, several emitted, and
        the collapsed prelude running underneath it. Every line of the expected
-       output is an answer a per-type copy used to give. *)
+       output is an answer a per-type copy used to give.
+
+       The nine lines after the first [2.5 0] are or-else and some?, the one
+       family here that declares no predicate. Both branches appear at both
+       scalar types because a default that is returned and one that is
+       discarded are two different lowerings, and the last pair — [2 0] — is
+       the same pair at a $t that owns storage, where each answer is a header
+       onto whichever of the two buffers the branch chose. *)
     let generics_out =
       "3\n4.5\ntrue\n7\n5\n-1\n5\n42\n3\n1\n10\n1\n8\n\
-       3\n4.5\ntext\n1\n2.5\n9\n36\n2\n2.5\n0\n3\n3\n0\n21\n7\n3\n4.5\n"
+       3\n4.5\ntext\n1\n2.5\n9\n36\n2\n2.5\n0\n\
+       0\n-1\n2.5\n0\ntrue\nfalse\ntrue\n2\n0\n\
+       3\n3\n0\n21\n7\n3\n4.5\n"
     in
     outputs "generics" "programs/generics.flan" generics_out;
     outputs ~opt:"-O0" "generics, -O0" "programs/generics.flan" generics_out;

vendor/edn/read.flan

@@ -17,6 +17,12 @@
 ;;;; take one as a parameter if the idiom could not say it — and the idiom
 ;;;; says it, so there is no allocator parameter here and nothing lost.
 ;;;;
+;;;; `read-file` is the one exception and states its own reason at the bottom
+;;;; of this file: the buffer it slurps is named against the heap because its
+;;;; life is strictly inside the call and the caller cannot observe it, so it
+;;;; is not the caller's tier to choose. The document it answers still lands
+;;;; wherever the context says.
+;;;;
 ;;;; The tier has to be a region, and that is enforced rather than documented:
 ;;;; a (Vec Value) whose elements own storage traps at its construction against
 ;;;; any allocator that can free one block. Calling `read` with the heap in the
@@ -233,3 +239,48 @@
         t (next (addr c))
         v (read-value (addr c) t)]
     (if (ok? (addr c)) (Some v) None)))
+
+;; The same, from a path, and the reason it is worth having is the line above
+;; it: **the source buffer is dead the moment `read` returns.** Every string in
+;; the document is a copy in the allocator — that is this file's "A Value owns
+;; its strings" section, and test_acceptance.ml's edn-read case proves it by
+;; overwriting every byte of the buffer after the read and still printing the
+;; string it found. So the buffer has no reader once this returns, which means
+;; it does not have to be the caller's to hold, and a caller writing
+;;
+;;   (defconst raw (embed "game-data.edn"))
+;;   (edn/read raw)
+;;
+;; or a slurp-and-free pair around `read` is keeping a name alive for a value
+;; whose whole life fits inside one call.
+;;
+;; **The heap is named, and this is the one place in the package that names an
+;; allocator.** The file header argues that `read` takes none because the tier
+;; is the caller's choice; that argument does not reach this buffer, because
+;; the caller can never observe it. Left to the context, (with-allocator frame
+;; (edn/read-file p)) would grow the region by the file's size for bytes that
+;; die immediately, and the `free` below would buy nothing back — a bump
+;; allocator has no FLAN_CAN_FREE, so freeing into one is a no-op. Against the
+;; heap the free is real, and the document still lands in whatever tier the
+;; caller chose, because that is where `read`'s own (vec-new) and (map-new) go.
+;;
+;; The `defer` rather than a trailing (free src) is for the transfer path:
+;; `read` allocates, so it can signal StorageExhausted, and a handler that
+;; answers by transferring out would otherwise leave the buffer behind.
+;;
+;; **A FileError passes straight through, and that is the decision, not an
+;; omission.** The return type here is already saying something: `None` means
+;; the document was malformed, which the header above argues at length has to
+;; stay distinguishable from the document that is literally `nil`. Folding "the
+;; file was not there" into that same `None` would collapse the distinction the
+;; Option exists for. And this function has nothing to answer a FileError
+;; *with* — `use-value` wants a path only the caller knows, and whether a
+;; missing file is fatal or is a cue to write a default is the caller's policy
+;; in every program. Nothing here establishes a handler, so slurp's condition
+;; reaches the caller's with both restarts still armed; edn-read.flan runs the
+;; `use-value` half, where the handler names another path, the read happens
+;; against that file, and this function never learns that anything went wrong.
+(defn read-file [path string] (Option Value)
+  (let [src (slurp path (heap-allocator))]
+    (defer (free src))
+    (read (as-slice src))))

vendor/json/json.flan

@@ -105,6 +105,24 @@
 ;;;; for the closer, and it does not track what a comma may follow because that
 ;;;; is a grammar with a stack of its own.
 ;;;;
+;;;; ── There is no json/read-file, and that is not an oversight ────────
+;;;;
+;;;; vendor/edn grew one and this did not, because the thing that made it safe
+;;;; there is the thing this package does not have. `edn/read-file` can slurp a
+;;;; buffer, read it, and free the buffer inside the one call only because
+;;;; `edn/read` answers a Value whose every string is a copy — the source is
+;;;; dead the moment the read returns. This package's whole surface is the
+;;;; cursor, and a Token's `text` is a slice INTO the caller's buffer, which
+;;;; the divergence section at the top of this file argues for at length. A
+;;;; read-file here would hand back a cursor over memory it had just released.
+;;;;
+;;;; So the prerequisite is not two lines, it is a `json/read` answering a
+;;;; self-contained document — and there is no Value type here to answer with.
+;;;; Write that first and read-file follows it for free; until then the caller
+;;;; holds the buffer, which is what the cursor's contract already says. What
+;;;; DOES carry over is the prelude's or-else and some?: string-of, int-of,
+;;;; float-of and bool-of all answer an Option and all take them.
+;;;;
 ;;;; ── Errors ──────────────────────────────────────────────────────────
 ;;;;
 ;;;; On the cursor, not in the return type — edn's argument, unchanged: an