;;;; The dynamic reader: an EDN document, and no type to read it into. ;;;; ;;;; `edn.flan` answers "what is the next token". This answers "what is in the ;;;; file", for a caller that has no struct to hand — a config file whose keys ;;;; are not known until it is read, a tileset, a save. `(edn/defedn T path)` ;;;; in provide.flan is the other direction: the shape known at compile time, ;;;; read into a struct, at no run-time cost. Together they are the two sides ;;;; of one capability, and this is the dynamic one. ;;;; ;;;; ── What a document reads as ───────────────────────────────────────── ;;;; ;;;; Plain dyn values, the runtime's own: ;;;; ;;;; nil true 42 1.5 nil, bool, int, float ;;;; "text" a dyn text — a copy, owned by the collector ;;;; :key symbol a keyword, interned, so equality is identity ;;;; [1 2] a dyn vec ;;;; {:a 1} a dyn map — keys are whole values, so :a and "a" ;;;; stay two keys the way EDN says they are ;;;; #{1 2} a dyn map from each element to true. There is no ;;;; set kind; a map's put already replaces an equal ;;;; key, so the dedup is the representation's own, and ;;;; (has-key? s x) is the membership test. ;;;; ;;;; This file used to define edn/Value — a tagged union with its own ;;;; structural equality, its own set dedup and its own table compare, written ;;;; before the dyn runtime existed. That was one capability implemented twice ;;;; on the same side, and the duplicity audit (docs/SPIKE-DUPLICITY.md §5) ;;;; retired it: everything value=? and its four helpers did is what ;;;; flan_dyn_eq does, and the (Map Value Value) the typed side refused — ;;;; keyable says no — is exactly what the dyn map serves without being asked. ;;;; ;;;; ── Where the storage comes from ───────────────────────────────────── ;;;; ;;;; The collector's heap, all of it. A dyn value's storage is the dyn ;;;; runtime's — that is what lets the collector find the values inside it — ;;;; so `read` neither takes an allocator nor consults the ambient one, and ;;;; there is no free-all to call and nothing to tear down. The strings are ;;;; copies: boxing a string is flan_dyn_from_bytes, which copies into the ;;;; heap, so the document does not point at the source buffer at all once ;;;; `read` has returned. Overwrite the buffer, free it, read the next file ;;;; into it — the document stands. ;;;; ;;;; ── Malformed input is nil, and the narrowing is stated ────────────── ;;;; ;;;; `read` answers nil for a document that failed to tokenize — and nil is ;;;; also what the document `nil` reads as. The old (Option Value) return kept ;;;; those apart; a dyn nil cannot, and wrapping dyn in an Option today would ;;;; put the document where the collector cannot see it (a dyn inside a typed ;;;; container is unrooted until the per-type descriptors land — the queue's ;;;; item 2). A caller who needs the distinction drives its own cursor and ;;;; asks it afterwards, which is also how the error *position* has always ;;;; been got: ;;;; ;;;; (let [c (edn/cursor src) ;;;; v (edn/read-value (addr c) (edn/next (addr c)))] ;;;; (if (edn/ok? (addr c)) ... (edn/error-pos (addr c)) ...)) ;; ── Copying a token's text ────────────────────────────────────────── ;; Not the dyn reader's own — everything below boxes through the runtime, ;; which copies for itself — but provide.flan's generated readers build typed ;; strings out of token text and this is where that copy has always lived. ;; The (Vec u8) is the copy; the string is a view of it, and the Vec header is ;; dropped here on purpose. Nothing individually owns a block in a region — ;; free-all owns all of them — so keeping the header around to free through ;; would be keeping a handle for an operation that never happens. (defn copy-text [s [u8]] string (let [b (vec-new u8)] (append (addr b) s) (string (slice b)))) ;; ── Reading ───────────────────────────────────────────────────────── ;; One token in hand, and the cursor for whatever that token opens. Public ;; because it is the entry point for a caller who wants the error position — ;; see the header. (defn read-value [c (Ptr Cursor) t Token] dyn (cond (= (.kind t) tok-nil) nil (= (.kind t) tok-bool) (match (bool-of t) (Some v) v None false) (= (.kind t) tok-int) (match (int-of t) (Some v) v None (i64 0)) (= (.kind t) tok-float) (match (float-of t) (Some v) v None 0.0) ;; The box copies the bytes into the collector's heap, which is the "a ;; document owns its strings" rule this file has always had: a view into ;; the source buffer would be garbage with nothing to say so the moment ;; the caller reads the next file into it. (= (.kind t) tok-string) (string (.text t)) (= (.kind t) tok-keyword) (keyword (.text t)) ;; A symbol becomes a keyword. Nothing that reads a document this way ;; tells the two apart, and a case nobody can act on differently is a ;; case that only makes matches longer. (= (.kind t) tok-symbol) (keyword (.text t)) (= (.kind t) tok-vec-open) (let [items (vec-new dyn) u (next c)] (while (and (ok? c) (!= (.kind u) tok-vec-close) (!= (.kind u) tok-eof)) (push items (read-value c u)) (set u (next c))) items) ;; A set ends on tok-map-close, because `}` is the byte that ends it. The ;; dedup is the map's own: put replaces the value of an equal key, so a ;; set with a duplicate in it never exists and #{[0 0] [0 0]} is one ;; element by structure, not by header identity. (= (.kind t) tok-set-open) (let [s {} u (next c)] (while (and (ok? c) (!= (.kind u) tok-map-close) (!= (.kind u) tok-eof)) (put s (read-value c u) true) (set u (next c))) s) ;; A map's key is a whole value, read by the same recursion as anything ;; else — :a and "a" are two keys, [0 0] can key a map, and the old ;; (Map string Value) narrowing that collapsed them is gone with the type ;; that forced it. (= (.kind t) tok-map-open) (let [m {} k (next c)] (while (and (ok? c) (!= (.kind k) tok-map-close) (!= (.kind k) tok-eof)) (let [key (read-value c k) u (next c)] (put m key (read-value c u))) (set k (next c))) m) :else nil)) ;; The whole document, from a byte slice. nil when the input was malformed — ;; the header says what that conflates and what to do when it matters. (defn read [src [u8]] dyn (let [c (cursor src) t (next (addr c)) v (read-value (addr c) t)] (if (ok? (addr c)) v nil))) ;; The same, from a path. The buffer is slurped against the heap, read, and ;; freed on the way out — it can be, because the document copies every byte it ;; keeps. The heap is named rather than left to the context because the caller ;; can never observe this buffer, so its tier was never the caller's to ;; choose; the document itself lands in the collector's heap wherever this is ;; called from. ;; ;; A FileError passes straight through, and that is the decision, not an ;; omission: this function has nothing to answer one with — `use-value` wants ;; a path only the caller knows, and whether a missing file is fatal or a cue ;; to write a default is the caller's policy in every program. Nothing here ;; establishes a handler, so slurp's condition reaches the caller's with both ;; restarts still armed. (defn read-file [path string] dyn (let [src (slurp path (heap-allocator))] (defer (free src)) (read (slice src))))