Joseph Ferano
cc2cfa175b
The tokenizer refused #{} because "it needs a hash set to even
represent" — which is a claim about a reader, and a tokenizer represents
nothing. #{ now pushes } on the same balance stack { does, there is one
new token kind and no new closer, and err-set is gone rather than kept
with a message it no longer earns. skip-value needed nothing: it is
written against the depth and not against the kinds.
The dynamic reader moves out of test/programs/arena-edn.flan and into
vendor/edn/read.flan as (edn/read bytes), answering an (Option Value)
against whichever allocator the caller bound. Two decisions are written
down where they are made:
* a set is a Value.Set holding a deduplicated (Vec Value), because
(Map Value bool) does not typecheck — keyable refuses a key holding
a Vec or a Map — and restricting elements to keyable Values would
refuse #{[0 0] [1 0]}, which is the file this was built for. Insert
is O(n) against a structural value=?, so building the tileset's 54
pairs is 1458 comparisons, once.
* a Value copies every string into the allocator where a Token stays
a view. A view handed back out of the function that owns the buffer
is a dangling pointer, and free-all would not even take it. Odin's
json parser clones for the same reason.
An imported defdata was a refusal in load.ml — "not implemented yet
(milestone 4)" — and it had to go first. It is the type's name plus the
Type. half of a constructor symbol, which arrives as a Var node when the
case has no fields and a Struct node when it has; a match pattern needed
nothing, because a case resolves against the scrutinee's type and was
never a top-level name. programs/pkg-data.flan is that on its own.
programs/edn-read.flan reads assets/edn/tileset.edn, which is the
editor's real output: :texture-path and a :selected-cells of 54 integer
pairs, with no type declared for any of it. It also overwrites the
source buffer in place after reading and prints the document back, which
is the copy contract asserted rather than described.
128 lines
5.2 KiB
Plaintext
128 lines
5.2 KiB
Plaintext
;;;; An EDN document read into a dynamic value, against an arena.
|
|
;;;;
|
|
;;;; This is the other half of programs/edn.flan. That one reads a document
|
|
;;;; whose shape is known into a struct, by hand, which is what the compiler's
|
|
;;;; (read-edn Enemy bytes) will emit. This one is what a reader handed *no*
|
|
;;;; target type has to answer with: a data type naming itself through a
|
|
;;;; (Vec Value) and a (Map string Value), holding whatever was in the file.
|
|
;;;;
|
|
;;;; ── The allocator story, which is the point of the program ───────────
|
|
;;;;
|
|
;;;; edn/read takes no allocator and names none. It does not need to:
|
|
;;;; spec-memory.md puts the allocator in the calling convention, so every
|
|
;;;; (vec-new) and (map-new) inside it takes the *context*, and the caller
|
|
;;;; chooses the tier with (with-allocator ...) around the call. An explicit
|
|
;;;; allocator at a construction site overrides that, which is how a reader
|
|
;;;; would take one as a parameter if it wanted to — but the existing idiom
|
|
;;;; already does the job, so there is no new machinery here and none needed.
|
|
;;;;
|
|
;;;; The tier has to be a region and not the heap, and that is enforced rather
|
|
;;;; than documented: a (Vec Value) whose elements own storage traps at its
|
|
;;;; construction against any allocator that can free one block. See
|
|
;;;; programs/arena-region.flan.
|
|
;;;;
|
|
;;;; ── What the region buys, said plainly ───────────────────────────────
|
|
;;;;
|
|
;;;; The document below is four levels deep and every level allocates. There is
|
|
;;;; no teardown anywhere in this file: no drop, no destructor, no recursive
|
|
;;;; free, not even a (free) call. One (free-all frame) at the bottom of main
|
|
;;;; releases every Vec block, every Map block and every entry in them, because
|
|
;;;; they all came out of the same region.
|
|
;;;;
|
|
;;;; Odin's core:encoding/json ships a hand-written recursive destroy_value in
|
|
;;;; the *library* for the heap case, and names parsing against temp_allocator
|
|
;;;; and calling free_all as the idiomatic alternative. This is that
|
|
;;;; alternative, and it needs nothing from the language that was not already
|
|
;;;; there.
|
|
;;;;
|
|
;;;; ── The reader moved, and what is left here ──────────────────────────
|
|
;;;;
|
|
;;;; read-value used to be written out in this file. It is vendor/edn's now, as
|
|
;;;; (edn/read bytes), and what this program keeps is the half that was always
|
|
;;;; the demonstration: the walk back over a document nobody declared a type
|
|
;;;; for, and the one free-all that ends it. The strings are the package's own
|
|
;;;; copies in the region now rather than views into `doc`, which is why there
|
|
;;;; is nothing left here saying which parts of the value outlive the release.
|
|
|
|
(import edn "vendor:edn")
|
|
|
|
(defvar frame Allocator)
|
|
|
|
(defn count-leaves [v edn/Value] i32
|
|
(match v
|
|
(List items)
|
|
(let [n 0]
|
|
(dotimes [i (len items)]
|
|
(set n (+ n (count-leaves (at items i)))))
|
|
n)
|
|
(Set items)
|
|
(let [n 0]
|
|
(dotimes [i (len items)]
|
|
(set n (+ n (count-leaves (at items i)))))
|
|
n)
|
|
;; map-next! fills an out-parameter with a copy of the value's bytes,
|
|
;; which for a Value holding a container is a second header over the same
|
|
;; block. In a region that is an alias and not a second owner — nothing
|
|
;; here owns anything, the arena does — so walking a map is the ordinary
|
|
;; iteration and needs no accessor of its own.
|
|
(Table entries)
|
|
(let [n 0
|
|
cur (i64 0)
|
|
k ""
|
|
v edn/Value.Nil]
|
|
(while (map-next! entries (addr cur) (addr k) (addr v))
|
|
(set n (+ n (count-leaves v))))
|
|
n)
|
|
_ 1))
|
|
|
|
(defn sum-ints [v edn/Value] i64
|
|
(match v
|
|
(Int n) n
|
|
(List items)
|
|
(let [t (i64 0)]
|
|
(dotimes [i (len items)]
|
|
(set t (+ t (sum-ints (at items i)))))
|
|
t)
|
|
(Table entries)
|
|
(match (get entries "xs") (Some x) (sum-ints x) None (i64 0))
|
|
_ (i64 0)))
|
|
|
|
(defn describe [v edn/Value] string
|
|
(match v
|
|
Nil "nil" (Bool _b) "bool" (Int _n) "int" (Float _x) "float"
|
|
(Text _s) "string" (Key _s) "keyword" (List _i) "vector"
|
|
(Set _i) "set" (Table _e) "map"))
|
|
|
|
(defconst doc
|
|
"{:name \"level-1\"
|
|
:xs [1 2 3]
|
|
:spawns [{:kind :grunt :at [10 20]}
|
|
{:kind :boss :at [30 40]}]
|
|
:gravity 9.8
|
|
:looping true}")
|
|
|
|
(defn main [] i32
|
|
(set frame (arena-new 65536))
|
|
(with-allocator frame
|
|
;; None is the malformed document, and it cannot happen for a literal that
|
|
;; is right here — but reading it back out of the Option is what makes the
|
|
;; refusal visible at the call site instead of arriving as a Nil that looks
|
|
;; like data.
|
|
(match (edn/read (bytes doc))
|
|
(Some v)
|
|
(do
|
|
(println (describe v))
|
|
(println (count-leaves v))
|
|
(println (sum-ints v))
|
|
(match v
|
|
(Table entries)
|
|
(match (get entries "name")
|
|
(Some n) (println (describe n))
|
|
None (println "missing"))
|
|
_ (println "not a map")))
|
|
None (println "malformed")))
|
|
;; The whole document, in one operation and with no per-element teardown.
|
|
(free-all frame)
|
|
(arena-destroy frame)
|
|
0)
|