;;;; The dynamic reader: an EDN document, and no type to read it into. ;;;; ;;;; `edn.fln` answers "what is the next token". This answers "what is in the ;;;; file", for a caller that has no struct to hand — a config file whose keys ;;;; are not known until it is read, a tileset, a save. `edn/defedn(T, path)` ;;;; in provide.fln is the other direction: the shape known at compile time, ;;;; read into a struct, at no run-time cost. Together they are the two sides ;;;; of one capability, and this is the dynamic one. ;;;; ;;;; ── What a document reads as ───────────────────────────────────────── ;;;; ;;;; Plain dyn values, the runtime's own: ;;;; ;;;; nil true 42 1.5 nil, bool, int, float ;;;; "text" a dyn text — a copy, owned by the collector ;;;; :key symbol a keyword, interned, so equality is identity ;;;; [1 2] a dyn vec ;;;; {:a 1} a dyn map — keys are whole values, so :a and "a" ;;;; stay two keys the way EDN says they are ;;;; #{1 2} a dyn map from each element to true. There is no ;;;; set kind; a map's put already replaces an equal ;;;; key, so the dedup is the representation's own, and ;;;; (has-key s x) is the membership test. ;;;; ;;;; This file used to define edn/Value — a tagged union with its own ;;;; structural equality, its own set dedup and its own table compare, written ;;;; before the dyn runtime existed. That was one capability implemented twice ;;;; on the same side, and the duplicity audit (docs/SPIKE-DUPLICITY.md §5) ;;;; retired it: everything value=? and its four helpers did is what ;;;; flan_dyn_eq does, and the (Map Value Value) the typed side refused — ;;;; keyable says no — is exactly what the dyn map serves without being asked. ;;;; ;;;; ── Where the storage comes from ───────────────────────────────────── ;;;; ;;;; The collector's heap, all of it. A dyn value's storage is the dyn ;;;; runtime's — that is what lets the collector find the values inside it — ;;;; so `read` neither takes an allocator nor consults the ambient one, and ;;;; there is no free-all to call and nothing to tear down. The strings are ;;;; copies: boxing a string is flan_dyn_from_bytes, which copies into the ;;;; heap, so the document does not point at the source buffer at all once ;;;; `read` has returned. Overwrite the buffer, free it, read the next file ;;;; into it — the document stands. ;;;; ;;;; ── Malformed input is nil, and the narrowing is stated ────────────── ;;;; ;;;; `read` answers nil for a document that failed to tokenize — and nil is ;;;; also what the document `nil` reads as. The old (Option Value) return kept ;;;; those apart; a dyn nil cannot, and wrapping dyn in an Option today would ;;;; put the document where the collector cannot see it (a dyn inside a typed ;;;; container is unrooted until the per-type descriptors land — the queue's ;;;; item 2). A caller who needs the distinction drives its own cursor and ;;;; asks it afterwards, which is also how the error *position* has always ;;;; been got: ;;;; ;;;; (let [c (edn/cursor src) ;;;; v (edn/read-value (addr c) (edn/next (addr c)))] ;;;; (if (edn/is-ok (addr c)) ... (edn/error-pos (addr c)) ...)) ;; ── Copying a token's text ────────────────────────────────────────── ;; Not the dyn reader's own — everything below boxes through the runtime, ;; which copies for itself — but provide.fln's generated readers build typed ;; strings out of token text and this is where that copy has always lived. ;; The (Vec u8) is the copy; the string is a view of it, and the Vec header is ;; dropped here on purpose. Nothing individually owns a block in a region — ;; free-all owns all of them — so keeping the header around to free through ;; would be keeping a handle for an operation that never happens. fn copy-text(s: [const u8]) -> str let b = vec-new(u8) append(addr(b), s) str(slice(b)) ;; ── Reading ───────────────────────────────────────────────────────── ;; One token in hand, and the cursor for whatever that token opens. Public ;; because it is the entry point for a caller who wants the error position — ;; see the header. fn read-value(c: Ptr(Cursor), t: Token) -> dyn if t.kind == tok-nil nil elif t.kind == tok-bool match bool-of(t) Some(v) -> v None -> false elif t.kind == tok-int match int-of(t) Some(v) -> v None -> i64(0) elif t.kind == tok-float match float-of(t) Some(v) -> v None -> 0.0 ;; The box copies the bytes into the collector's heap, which is the "a ;; document owns its strings" rule this file has always had: a view into ;; the source buffer would be garbage with nothing to say so the moment ;; the caller reads the next file into it. elif t.kind == tok-string str(t.text) elif t.kind == tok-keyword keyword(t.text) ;; A symbol becomes a keyword. Nothing that reads a document this way ;; tells the two apart, and a case nobody can act on differently is a ;; case that only makes matches longer. elif t.kind == tok-symbol keyword(t.text) elif t.kind == tok-vec-open let items = vec-new(dyn) u = next(c) while is-ok(c) and u.kind != tok-vec-close and u.kind != tok-eof push(items, read-value(c, u)) u = next(c) items ;; A set ends on tok-map-close, because `}` is the byte that ends it. The ;; dedup is the map's own: put replaces the value of an equal key, so a ;; set with a duplicate in it never exists and #{[0 0] [0 0]} is one ;; element by structure, not by header identity. elif t.kind == tok-set-open let s = {} u = next(c) while is-ok(c) and u.kind != tok-map-close and u.kind != tok-eof put(s, read-value(c, u), true) u = next(c) s ;; A map's key is a whole value, read by the same recursion as anything ;; else — :a and "a" are two keys, [0 0] can key a map, and the old ;; (Map str Value) narrowing that collapsed them is gone with the type ;; that forced it. elif t.kind == tok-map-open let m = {} k = next(c) while is-ok(c) and k.kind != tok-map-close and k.kind != tok-eof let key = read-value(c, k) u = next(c) put(m, key, read-value(c, u)) k = next(c) m else nil ;; The whole document, from a byte slice. nil when the input was malformed — ;; the header says what that conflates and what to do when it matters. fn read(src: [const u8]) -> dyn let c = cursor(src) t = next(addr(c)) v = read-value(addr(c), t) if is-ok(addr(c)) then v else nil ;; The same, from a path. The buffer is slurped against the heap, read, and ;; freed on the way out — it can be, because the document copies every byte it ;; keeps. The heap is named rather than left to the context because the caller ;; can never observe this buffer, so its tier was never the caller's to ;; choose; the document itself lands in the collector's heap wherever this is ;; called from. ;; ;; A FileError passes straight through, and that is the decision, not an ;; omission: this function has nothing to answer one with — `use-value` wants ;; a path only the caller knows, and whether a missing file is fatal or a cue ;; to write a default is the caller's policy in every program. Nothing here ;; establishes a handler, so slurp's condition reaches the caller's with both ;; restarts still armed. fn read-file(path: str) -> dyn let src = slurp(path, heap-allocator()) defer free(src) read(slice(src))