diff --git a/lib/prelude.ml b/lib/prelude.ml index 4eb5d96..ea4c42b 100644 --- a/lib/prelude.ml +++ b/lib/prelude.ml @@ -347,6 +347,70 @@ let source = {flan| (set m (max m (at s i)))) (Some m)))) +;; ── Opening an Option without writing the match ─────────────────────── +;; +;; Everything above answers an (Option $t), and until now `match` was the only +;; thing that could open one. That is the right *primitive* — it is the form +;; that makes the empty case unforgettable — and it is the wrong thing to write +;; when the empty case is one word: +;; +;; (match (edn/read src) (Some v) v None edn/Value.Nil) +;; (or-else (edn/read src) edn/Value.Nil) +;; +;; **Neither takes a {:where}, and that is a decision rather than an +;; oversight.** A predicate buys an *operation* on the variable — [ordered?] +;; is what lets sort! write `<` — and these perform no operation on their +;; payload at all: they move it out of the Option, or they look at the tag and +;; never touch the payload. That is the one move [ident] in +;; test/programs/generics.flan makes, which needs nothing declared, so these +;; instantiate at every type including the ones that own storage. +;; +;; Two, and the ones that were declined are worth naming because a reader will +;; look for them: +;; +;; none? (not (some? o)) is the whole of it, and this file already +;; refuses a wrapper whose only method is the thing it wraps +;; — see the Builder entry under "Still refused". +;; an unwrap that Refused for the reason file-size below is an Option in the +;; signals on None first place: absence is a reply and not a fault, and +;; making it a condition puts a handler search on the +;; ordinary path. Whether an empty Option is an error is the +;; *caller's* question, and the caller has handler-bind if +;; the answer is yes. +;; a lazy or-else Would need a (Fn [] $t) — sort-by!'s shape, available the +;; day something wants it. A macro would get laziness for +;; free and need no generics, and costs more than it buys +;; here: a prelude macro drops every prelude defn that +;; depends on it out of a macro-module build (see the +;; bootstrap hook at the foot of this file), and a macro has +;; no way to report a malformed call. + +;; This is Java's `Optional.orElse`, hyphenated: eager, and it answers the +;; payload's type rather than another Option. Worth saying which, because Rust +;; spells something else `or_else` — there it takes a closure and answers an +;; `Option`, so borrowing that name for this behaviour would be the wrong +;; loan twice over. Java's own lazy sibling is `orElseGet`, which is the one +;; declined above. +;; +;; **At a $t that owns storage the result is a header copy, and the branch not +;; taken is still the caller's to free.** Since the copyable? repeal every +;; value copies as its header and the copies alias one buffer (see the section +;; comment above), so (or-else o d) over a (Vec u8) hands back a second header +;; onto o's block or onto d's — and the one it did not choose was never +;; released by anything here. That is the same contract `at` on a slice of Vecs +;; has; it is written down here because an "or a default" reads like it +;; consumes the default and it does not. +(defn or-else [o (Option $t) d $t] $t + (match o (Some v) v None d)) + +;; Clojure's `some?`, and the `?`-asks convention this file already spells with +;; ok?, even? and file-exists?. It is what makes a `when` or a `cond` possible +;; at all — or-else can say "this or that" and cannot say "only if there is +;; one" — and it is the honest shape for the case where the payload is not +;; wanted, which a match would still have to bind a name for. +(defn some? [o (Option $t)] bool + (match o (Some _v) true None false)) + ;; map! writes back into the slice it was handed, for the same reason sort! ;; does — a slice is non-owning, and transforming a thing you already own ;; should not allocate. A map that produces a *different* element type is not diff --git a/test/programs/edn-read.flan b/test/programs/edn-read.flan index f2b7f4a..358fd0e 100644 --- a/test/programs/edn-read.flan +++ b/test/programs/edn-read.flan @@ -20,6 +20,13 @@ ;;;; element, and a dedup written with `=` would make it two — two Vec ;;;; headers over two blocks are never the same header. ;;;; +;;;; * (edn/read-file path) is the same read with the buffer owned and freed +;;;; inside the call, which is only safe because of the property below it. +;;;; Two cases: the real file by path, whose answer has to equal the embed +;;;; above byte for byte, and a missing one, where slurp's FileError has to +;;;; arrive at a handler *outside* read-file with `use-value` still armed — +;;;; the pass-through decision, run rather than asserted. +;;;; ;;;; * a Value owns its strings. The last case reads a document out of a ;;;; buffer and then overwrites every byte of that buffer in place. A ;;;; reader holding views prints the overwriting bytes; one holding copies @@ -94,6 +101,55 @@ _ (println "no name")) None (println "malformed"))))) +;; The path-taking entry point over the same file the embed above holds, and +;; the whole reason both of this session's additions exist. What a program +;; wants to write is one form: +;; +;; (defvar game-data edn/Value +;; (with-allocator frame +;; (or-else (edn/read-file "game-data.edn") edn/Value.Nil))) +;; +;; and it is written as a defn here because the *initialiser* is still refused +;; — "a global's value must be a compile-time constant — this one is computed" +;; — which is a separate piece of work on globals and nothing to do with +;; read-file or or-else. Everything inside the with-allocator is verbatim, so +;; the day a computed initialiser is allowed, load-game-data and the `set` +;; below it collapse back into the defvar above and this comment goes with +;; them. +;; +;; The texture path printed here has to be the one show-tileset printed, which +;; is what says read-file read the file and not merely something. +(defvar game-data edn/Value) + +(defn load-game-data [] edn/Value + (or-else (edn/read-file "programs/assets/edn/tileset.edn") edn/Value.Nil)) + +(defn by-path [] () + (set game-data (load-game-data)) + (match (field game-data "texture-path") + (Text s) (println s) + _ (println "no texture path"))) + +;; A missing file, answered from outside read-file. Nothing in the package +;; handles FileError, so the condition walks past it to here with both restarts +;; armed, and `use-value` names a path read-file then slurps instead — the read +;; resumes as if that file had been asked for all along, which is exactly what +;; slurp.flan asserts for slurp alone. +;; +;; `some?` is the assertion: the answer is a real document, so the restart was +;; taken rather than the read quietly answering None. Distinguishing the two is +;; the reason read-file passes the condition through instead of folding a +;; missing file into the None that means "malformed". +(defvar saw-file-error i64) + +(defn by-missing-path [] () + (handler-bind + [(FileError [c] + (set saw-file-error (+ saw-file-error 1)) + (invoke-restart 'use-value "programs/assets/edn/tileset.edn"))] + (println (some? (edn/read-file "programs/assets/edn/not-here.edn")))) + (println saw-file-error)) + (defn main [] i32 (set frame (arena-new 262144)) (with-allocator frame @@ -126,7 +182,11 @@ ;; And the refusal, which has to be distinguishable from the document ;; that is literally nil. (match (edn/read (bytes "#{1 2")) (Some _v) (println "read") None (println "malformed")) - (match (edn/read (bytes "nil")) (Some _v) (println "read") None (println "malformed")))) + (match (edn/read (bytes "nil")) (Some _v) (println "read") None (println "malformed")) + (println "") + + (by-path) + (by-missing-path))) (free-all frame) (arena-destroy frame) 0) diff --git a/test/programs/generics.flan b/test/programs/generics.flan index c16c704..7fb2f1c 100644 --- a/test/programs/generics.flan +++ b/test/programs/generics.flan @@ -11,7 +11,9 @@ ;;;; a variable bound inside a slice, a generic calling a generic at its own ;;;; variable so that instantiation has to be transitive, the four where ;;;; predicates, two variables at once, println deferred to the instantiation, -;;;; and the collapsed prelude family the whole feature was for. +;;;; the collapsed prelude family the whole feature was for, and the family +;;;; over (Option $t) — or-else and some? — which is the one that declares no +;;;; predicate at all, so a $t that owns storage instantiates it too. ;; One variable, several types, and (ident 3) and (ident 7) share one copy. ;; The identity needs its parameter once, so it needs nothing declared: a type @@ -73,6 +75,16 @@ (push v x) v)) +;; An empty (Option (Vec u8)), which main needs to reach or-else's None branch +;; at a type that owns storage. It is a function and not a bare None at the +;; call site because a bare None there is refused — "nothing here says what +;; None is an Option of" — and a return type is one of the two places the +;; checker names as somewhere to say it. It is also the shape every real caller +;; is in: what arrives at or-else came out of something, the way edn/read's +;; answer does. +(defn none-vec [] (Option (Vec u8)) + None) + ;; (zeroed) takes its type from the position it is written in, so a variable ;; in that position is answered by the instantiation like any other type. (defn zero-of [x $t] $t @@ -135,6 +147,35 @@ (match (min-of (slice ns 0 4)) (Some m) (println m) _ (println -1)) (match (max-of (slice fs 0 3)) (Some m) (println m) _ (println -1.0)) (match (index-of (slice ns 0 4) 18) (Some i) (println i) _ (println -1)) + + ;; or-else and some?, which are the same family over (Option $t) and take + ;; no predicate: they move the payload out or read the tag, and neither is + ;; an operation the variable has to be declared to support. + ;; + ;; Both branches at two scalar types, because a default that is returned + ;; and a default that is discarded are two different lowerings and only one + ;; of them is exercised by a call that happens to be Some. + (println (or-else (index-of (slice ns 0 4) 18) -1)) ; the Some branch + (println (or-else (index-of (slice ns 0 4) 77) -1)) ; the None branch + (println (or-else (max-of (slice fs 0 3)) 0.0)) + (println (or-else (max-of (slice fs 0 0)) 0.0)) + (println (some? (index-of (slice ns 0 4) 18))) + (println (some? (index-of (slice ns 0 4) 77))) + (println (some? (parse-i64 (bytes "12")))) + + ;; And at a $t that owns storage, which is the case the scalars above say + ;; nothing about. What comes back is a *header* onto one of the two + ;; buffers, so both are still the caller's to free — hence two frees and + ;; not one, and the lengths are what say which header each answer holds. + (let [full (vec-new u8) + empty (vec-new u8)] + (push full 65) + (push full 66) + (println (len (or-else (Some full) empty))) ; 2, full's header + (println (len (or-else (none-vec) empty))) ; 0, empty's + (free full) + (free empty)) + (println (widen 3 0.0)) (println (widen 3 (i64 0))) (println (zero-of 9)) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 4dc28f9..2b1c5e9 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -619,13 +619,25 @@ let () = Last, that a malformed document is distinguishable from the document that is literally nil, which is why the entry point answers an Option - and not a Value. *) + and not a Value. + + Then the path-taking entry point, whose two lines are the two halves of + what it decided. The texture path repeats the first line of the run, + from a file read at run time rather than embedded, which says read-file + read *that* file and that freeing its buffer inside the call took none + of the document with it. `true` then `1` is the pass-through: slurp's + FileError for a missing path walked past read-file to a handler here + with `use-value` still armed, the read resumed against the path the + handler named, and the answer is a document — not the None that means + malformed, which is the collapse read-file refuses to make. *) let edn_read_out = "./source-assets/Sprout Lands Premium/Objects/Mushrooms, Flowers, \ Stones.png\n\ 54\ntrue\ntrue\ntrue\nfalse\n\n\ 0\n2\n2\n2\n1\n1\n2\n2\n2\n2\n\n\ - level-1\nmalformed\nread\n" + level-1\nmalformed\nread\n\n\ + ./source-assets/Sprout Lands Premium/Objects/Mushrooms, Flowers, \ + Stones.png\ntrue\n1\n" in outputs "edn/read over the tileset" "programs/edn-read.flan" edn_read_out; outputs ~opt:"-O0" "edn/read over the tileset, -O0" @@ -1898,10 +1910,19 @@ let () = (* Generics end to end: one written body per family, several emitted, and the collapsed prelude running underneath it. Every line of the expected - output is an answer a per-type copy used to give. *) + output is an answer a per-type copy used to give. + + The nine lines after the first [2.5 0] are or-else and some?, the one + family here that declares no predicate. Both branches appear at both + scalar types because a default that is returned and one that is + discarded are two different lowerings, and the last pair — [2 0] — is + the same pair at a $t that owns storage, where each answer is a header + onto whichever of the two buffers the branch chose. *) let generics_out = "3\n4.5\ntrue\n7\n5\n-1\n5\n42\n3\n1\n10\n1\n8\n\ - 3\n4.5\ntext\n1\n2.5\n9\n36\n2\n2.5\n0\n3\n3\n0\n21\n7\n3\n4.5\n" + 3\n4.5\ntext\n1\n2.5\n9\n36\n2\n2.5\n0\n\ + 0\n-1\n2.5\n0\ntrue\nfalse\ntrue\n2\n0\n\ + 3\n3\n0\n21\n7\n3\n4.5\n" in outputs "generics" "programs/generics.flan" generics_out; outputs ~opt:"-O0" "generics, -O0" "programs/generics.flan" generics_out; diff --git a/vendor/edn/read.flan b/vendor/edn/read.flan index b09bba4..85e5d68 100644 --- a/vendor/edn/read.flan +++ b/vendor/edn/read.flan @@ -17,6 +17,12 @@ ;;;; take one as a parameter if the idiom could not say it — and the idiom ;;;; says it, so there is no allocator parameter here and nothing lost. ;;;; +;;;; `read-file` is the one exception and states its own reason at the bottom +;;;; of this file: the buffer it slurps is named against the heap because its +;;;; life is strictly inside the call and the caller cannot observe it, so it +;;;; is not the caller's tier to choose. The document it answers still lands +;;;; wherever the context says. +;;;; ;;;; The tier has to be a region, and that is enforced rather than documented: ;;;; a (Vec Value) whose elements own storage traps at its construction against ;;;; any allocator that can free one block. Calling `read` with the heap in the @@ -233,3 +239,48 @@ t (next (addr c)) v (read-value (addr c) t)] (if (ok? (addr c)) (Some v) None))) + +;; The same, from a path, and the reason it is worth having is the line above +;; it: **the source buffer is dead the moment `read` returns.** Every string in +;; the document is a copy in the allocator — that is this file's "A Value owns +;; its strings" section, and test_acceptance.ml's edn-read case proves it by +;; overwriting every byte of the buffer after the read and still printing the +;; string it found. So the buffer has no reader once this returns, which means +;; it does not have to be the caller's to hold, and a caller writing +;; +;; (defconst raw (embed "game-data.edn")) +;; (edn/read raw) +;; +;; or a slurp-and-free pair around `read` is keeping a name alive for a value +;; whose whole life fits inside one call. +;; +;; **The heap is named, and this is the one place in the package that names an +;; allocator.** The file header argues that `read` takes none because the tier +;; is the caller's choice; that argument does not reach this buffer, because +;; the caller can never observe it. Left to the context, (with-allocator frame +;; (edn/read-file p)) would grow the region by the file's size for bytes that +;; die immediately, and the `free` below would buy nothing back — a bump +;; allocator has no FLAN_CAN_FREE, so freeing into one is a no-op. Against the +;; heap the free is real, and the document still lands in whatever tier the +;; caller chose, because that is where `read`'s own (vec-new) and (map-new) go. +;; +;; The `defer` rather than a trailing (free src) is for the transfer path: +;; `read` allocates, so it can signal StorageExhausted, and a handler that +;; answers by transferring out would otherwise leave the buffer behind. +;; +;; **A FileError passes straight through, and that is the decision, not an +;; omission.** The return type here is already saying something: `None` means +;; the document was malformed, which the header above argues at length has to +;; stay distinguishable from the document that is literally `nil`. Folding "the +;; file was not there" into that same `None` would collapse the distinction the +;; Option exists for. And this function has nothing to answer a FileError +;; *with* — `use-value` wants a path only the caller knows, and whether a +;; missing file is fatal or is a cue to write a default is the caller's policy +;; in every program. Nothing here establishes a handler, so slurp's condition +;; reaches the caller's with both restarts still armed; edn-read.flan runs the +;; `use-value` half, where the handler names another path, the read happens +;; against that file, and this function never learns that anything went wrong. +(defn read-file [path string] (Option Value) + (let [src (slurp path (heap-allocator))] + (defer (free src)) + (read (as-slice src)))) diff --git a/vendor/json/json.flan b/vendor/json/json.flan index edf141b..31152c1 100644 --- a/vendor/json/json.flan +++ b/vendor/json/json.flan @@ -105,6 +105,24 @@ ;;;; for the closer, and it does not track what a comma may follow because that ;;;; is a grammar with a stack of its own. ;;;; +;;;; ── There is no json/read-file, and that is not an oversight ──────── +;;;; +;;;; vendor/edn grew one and this did not, because the thing that made it safe +;;;; there is the thing this package does not have. `edn/read-file` can slurp a +;;;; buffer, read it, and free the buffer inside the one call only because +;;;; `edn/read` answers a Value whose every string is a copy — the source is +;;;; dead the moment the read returns. This package's whole surface is the +;;;; cursor, and a Token's `text` is a slice INTO the caller's buffer, which +;;;; the divergence section at the top of this file argues for at length. A +;;;; read-file here would hand back a cursor over memory it had just released. +;;;; +;;;; So the prerequisite is not two lines, it is a `json/read` answering a +;;;; self-contained document — and there is no Value type here to answer with. +;;;; Write that first and read-file follows it for free; until then the caller +;;;; holds the buffer, which is what the cursor's contract already says. What +;;;; DOES carry over is the prelude's or-else and some?: string-of, int-of, +;;;; float-of and bool-of all answer an Option and all take them. +;;;; ;;;; ── Errors ────────────────────────────────────────────────────────── ;;;; ;;;; On the cursor, not in the return type — edn's argument, unchanged: an