165 lines
7.8 KiB
Plaintext
165 lines
7.8 KiB
Plaintext
;;;; The dynamic reader: an EDN document, and no type to read it into.
|
|
;;;;
|
|
;;;; `edn.flan` answers "what is the next token". This answers "what is in the
|
|
;;;; file", for a caller that has no struct to hand — a config file whose keys
|
|
;;;; are not known until it is read, a tileset, a save. `(edn/defedn T path)`
|
|
;;;; in provide.flan is the other direction: the shape known at compile time,
|
|
;;;; read into a struct, at no run-time cost. Together they are the two sides
|
|
;;;; of one capability, and this is the dynamic one.
|
|
;;;;
|
|
;;;; ── What a document reads as ─────────────────────────────────────────
|
|
;;;;
|
|
;;;; Plain dyn values, the runtime's own:
|
|
;;;;
|
|
;;;; nil true 42 1.5 nil, bool, int, float
|
|
;;;; "text" a dyn text — a copy, owned by the collector
|
|
;;;; :key symbol a keyword, interned, so equality is identity
|
|
;;;; [1 2] a dyn vec
|
|
;;;; {:a 1} a dyn map — keys are whole values, so :a and "a"
|
|
;;;; stay two keys the way EDN says they are
|
|
;;;; #{1 2} a dyn map from each element to true. There is no
|
|
;;;; set kind; a map's put already replaces an equal
|
|
;;;; key, so the dedup is the representation's own, and
|
|
;;;; (has-key s x) is the membership test.
|
|
;;;;
|
|
;;;; This file used to define edn/Value — a tagged union with its own
|
|
;;;; structural equality, its own set dedup and its own table compare, written
|
|
;;;; before the dyn runtime existed. That was one capability implemented twice
|
|
;;;; on the same side, and the duplicity audit (docs/SPIKE-DUPLICITY.md §5)
|
|
;;;; retired it: everything value=? and its four helpers did is what
|
|
;;;; flan_dyn_eq does, and the (Map Value Value) the typed side refused —
|
|
;;;; keyable says no — is exactly what the dyn map serves without being asked.
|
|
;;;;
|
|
;;;; ── Where the storage comes from ─────────────────────────────────────
|
|
;;;;
|
|
;;;; The collector's heap, all of it. A dyn value's storage is the dyn
|
|
;;;; runtime's — that is what lets the collector find the values inside it —
|
|
;;;; so `read` neither takes an allocator nor consults the ambient one, and
|
|
;;;; there is no free-all to call and nothing to tear down. The strings are
|
|
;;;; copies: boxing a string is flan_dyn_from_bytes, which copies into the
|
|
;;;; heap, so the document does not point at the source buffer at all once
|
|
;;;; `read` has returned. Overwrite the buffer, free it, read the next file
|
|
;;;; into it — the document stands.
|
|
;;;;
|
|
;;;; ── Malformed input is nil, and the narrowing is stated ──────────────
|
|
;;;;
|
|
;;;; `read` answers nil for a document that failed to tokenize — and nil is
|
|
;;;; also what the document `nil` reads as. The old (Option Value) return kept
|
|
;;;; those apart; a dyn nil cannot, and wrapping dyn in an Option today would
|
|
;;;; put the document where the collector cannot see it (a dyn inside a typed
|
|
;;;; container is unrooted until the per-type descriptors land — the queue's
|
|
;;;; item 2). A caller who needs the distinction drives its own cursor and
|
|
;;;; asks it afterwards, which is also how the error *position* has always
|
|
;;;; been got:
|
|
;;;;
|
|
;;;; (let [c (edn/cursor src)
|
|
;;;; v (edn/read-value (addr c) (edn/next (addr c)))]
|
|
;;;; (if (edn/is-ok (addr c)) ... (edn/error-pos (addr c)) ...))
|
|
|
|
;; ── Copying a token's text ──────────────────────────────────────────
|
|
|
|
;; Not the dyn reader's own — everything below boxes through the runtime,
|
|
;; which copies for itself — but provide.flan's generated readers build typed
|
|
;; strings out of token text and this is where that copy has always lived.
|
|
;; The (Vec u8) is the copy; the string is a view of it, and the Vec header is
|
|
;; dropped here on purpose. Nothing individually owns a block in a region —
|
|
;; free-all owns all of them — so keeping the header around to free through
|
|
;; would be keeping a handle for an operation that never happens.
|
|
fn copy-text(s: [const u8]) -> str
|
|
let b = vec-new(u8)
|
|
append(addr(b), s)
|
|
str(slice(b))
|
|
|
|
;; ── Reading ─────────────────────────────────────────────────────────
|
|
|
|
;; One token in hand, and the cursor for whatever that token opens. Public
|
|
;; because it is the entry point for a caller who wants the error position —
|
|
;; see the header.
|
|
fn read-value(c: Ptr(Cursor), t: Token) -> dyn
|
|
if t.kind == tok-nil
|
|
nil
|
|
elif t.kind == tok-bool
|
|
match bool-of(t)
|
|
Some(v) -> v
|
|
None -> false
|
|
elif t.kind == tok-int
|
|
match int-of(t)
|
|
Some(v) -> v
|
|
None -> i64(0)
|
|
elif t.kind == tok-float
|
|
match float-of(t)
|
|
Some(v) -> v
|
|
None -> 0.0
|
|
;; The box copies the bytes into the collector's heap, which is the "a
|
|
;; document owns its strings" rule this file has always had: a view into
|
|
;; the source buffer would be garbage with nothing to say so the moment
|
|
;; the caller reads the next file into it.
|
|
elif t.kind == tok-string
|
|
str(t.text)
|
|
elif t.kind == tok-keyword
|
|
keyword(t.text)
|
|
;; A symbol becomes a keyword. Nothing that reads a document this way
|
|
;; tells the two apart, and a case nobody can act on differently is a
|
|
;; case that only makes matches longer.
|
|
elif t.kind == tok-symbol
|
|
keyword(t.text)
|
|
elif t.kind == tok-vec-open
|
|
let items = vec-new(dyn)
|
|
u = next(c)
|
|
while is-ok(c) and u.kind != tok-vec-close and u.kind != tok-eof
|
|
push(items, read-value(c, u))
|
|
u = next(c)
|
|
items
|
|
;; A set ends on tok-map-close, because `}` is the byte that ends it. The
|
|
;; dedup is the map's own: put replaces the value of an equal key, so a
|
|
;; set with a duplicate in it never exists and #{[0 0] [0 0]} is one
|
|
;; element by structure, not by header identity.
|
|
elif t.kind == tok-set-open
|
|
let s = {}
|
|
u = next(c)
|
|
while is-ok(c) and u.kind != tok-map-close and u.kind != tok-eof
|
|
put(s, read-value(c, u), true)
|
|
u = next(c)
|
|
s
|
|
;; A map's key is a whole value, read by the same recursion as anything
|
|
;; else — :a and "a" are two keys, [0 0] can key a map, and the old
|
|
;; (Map str Value) narrowing that collapsed them is gone with the type
|
|
;; that forced it.
|
|
elif t.kind == tok-map-open
|
|
let m = {}
|
|
k = next(c)
|
|
while is-ok(c) and k.kind != tok-map-close and k.kind != tok-eof
|
|
let key = read-value(c, k)
|
|
u = next(c)
|
|
put(m, key, read-value(c, u))
|
|
k = next(c)
|
|
m
|
|
else
|
|
nil
|
|
|
|
;; The whole document, from a byte slice. nil when the input was malformed —
|
|
;; the header says what that conflates and what to do when it matters.
|
|
fn read(src: [const u8]) -> dyn
|
|
let c = cursor(src)
|
|
t = next(addr(c))
|
|
v = read-value(addr(c), t)
|
|
if is-ok(addr(c)) then v else nil
|
|
|
|
;; The same, from a path. The buffer is slurped against the heap, read, and
|
|
;; freed on the way out — it can be, because the document copies every byte it
|
|
;; keeps. The heap is named rather than left to the context because the caller
|
|
;; can never observe this buffer, so its tier was never the caller's to
|
|
;; choose; the document itself lands in the collector's heap wherever this is
|
|
;; called from.
|
|
;;
|
|
;; A FileError passes straight through, and that is the decision, not an
|
|
;; omission: this function has nothing to answer one with — `use-value` wants
|
|
;; a path only the caller knows, and whether a missing file is fatal or a cue
|
|
;; to write a default is the caller's policy in every program. Nothing here
|
|
;; establishes a handler, so slurp's condition reaches the caller's with both
|
|
;; restarts still armed.
|
|
fn read-file(path: str) -> dyn
|
|
let src = slurp(path, heap-allocator())
|
|
defer free(src)
|
|
read(slice(src))
|