Joseph Ferano
cc2cfa175b
The tokenizer refused #{} because "it needs a hash set to even
represent" — which is a claim about a reader, and a tokenizer represents
nothing. #{ now pushes } on the same balance stack { does, there is one
new token kind and no new closer, and err-set is gone rather than kept
with a message it no longer earns. skip-value needed nothing: it is
written against the depth and not against the kinds.
The dynamic reader moves out of test/programs/arena-edn.flan and into
vendor/edn/read.flan as (edn/read bytes), answering an (Option Value)
against whichever allocator the caller bound. Two decisions are written
down where they are made:
* a set is a Value.Set holding a deduplicated (Vec Value), because
(Map Value bool) does not typecheck — keyable refuses a key holding
a Vec or a Map — and restricting elements to keyable Values would
refuse #{[0 0] [1 0]}, which is the file this was built for. Insert
is O(n) against a structural value=?, so building the tileset's 54
pairs is 1458 comparisons, once.
* a Value copies every string into the allocator where a Token stays
a view. A view handed back out of the function that owns the buffer
is a dangling pointer, and free-all would not even take it. Odin's
json parser clones for the same reason.
An imported defdata was a refusal in load.ml — "not implemented yet
(milestone 4)" — and it had to go first. It is the type's name plus the
Type. half of a constructor symbol, which arrives as a Var node when the
case has no fields and a Struct node when it has; a match pattern needed
nothing, because a case resolves against the scrutinee's type and was
never a top-level name. programs/pkg-data.flan is that on its own.
programs/edn-read.flan reads assets/edn/tileset.edn, which is the
editor's real output: :texture-path and a :selected-cells of 54 integer
pairs, with no type declared for any of it. It also overwrites the
source buffer in place after reading and prints the document back, which
is the copy contract asserted rather than described.
126 lines
5.2 KiB
Plaintext
126 lines
5.2 KiB
Plaintext
;;;; (edn/read bytes) over the file it was built for, plus the two properties
|
|
;;;; that are only claims until something runs them.
|
|
;;;;
|
|
;;;; assets/edn/tileset.edn is the real thing, copied out of the editor that
|
|
;;;; writes it: a map of :texture-path to a string and :selected-cells to a set
|
|
;;;; of 54 integer pairs. Nothing here declares a type for it — the point.
|
|
;;;;
|
|
;;;; It is in a subdirectory of assets/ rather than in it, because programs/
|
|
;;;; embed.flan holds (embed-dir "assets") in a [3 EmbedFile] and a fourth file
|
|
;;;; beside the other three is a type error in a program that has nothing to do
|
|
;;;; with this one. embed-dir does not descend, which is what makes a
|
|
;;;; subdirectory the answer rather than a second assets directory.
|
|
;;;; The hand-written struct reader in programs/edn.flan is the other route and
|
|
;;;; needs a defstruct per file; this one needs nothing and reads a file whose
|
|
;;;; keys it has never heard of.
|
|
;;;;
|
|
;;;; The two properties:
|
|
;;;;
|
|
;;;; * a set holds each value once, by *structure*. #{[0 0] [0 0]} is one
|
|
;;;; element, and a dedup written with `=` would make it two — two Vec
|
|
;;;; headers over two blocks are never the same header.
|
|
;;;;
|
|
;;;; * a Value owns its strings. The last case reads a document out of a
|
|
;;;; buffer and then overwrites every byte of that buffer in place. A
|
|
;;;; reader holding views prints the overwriting bytes; one holding copies
|
|
;;;; prints what was in the file. The buffer is written rather than freed
|
|
;;;; because a freed buffer is a read of released memory, which can pass by
|
|
;;;; luck; overwriting it cannot.
|
|
|
|
(import edn "vendor:edn")
|
|
|
|
(defvar frame Allocator)
|
|
|
|
;; The document, read at compile time. An embed is bytes in the binary, so
|
|
;; there is no file open here and no path to get wrong at run time.
|
|
(defconst tileset (embed "assets/edn/tileset.edn"))
|
|
|
|
;; [a b] as a Value, so a membership test can be written against a pair this
|
|
;; program made rather than one it found. Building a Value from outside the
|
|
;; package is the same two forms as building one inside it.
|
|
(defn pair [a i64 b i64] edn/Value
|
|
(let [items (vec-new edn/Value)]
|
|
(push items (edn/Value.Int {.n a}))
|
|
(push items (edn/Value.Int {.n b}))
|
|
(edn/Value.List {.items items})))
|
|
|
|
(defn field [v edn/Value k string] edn/Value
|
|
(match v
|
|
(Table entries) (match (get entries k) (Some x) x None edn/Value.Nil)
|
|
_ edn/Value.Nil))
|
|
|
|
(defn show-tileset [] ()
|
|
(match (edn/read tileset)
|
|
(Some doc)
|
|
(do
|
|
(match (field doc "texture-path")
|
|
(Text s) (println s)
|
|
_ (println "no texture path"))
|
|
(match (field doc "selected-cells")
|
|
(Set cells)
|
|
(do
|
|
(println (len cells))
|
|
;; Two cells that are in the file and one that is not. A reader
|
|
;; that flattened the pairs into 108 integers would still have
|
|
;; the right count of *something*, and would miss both of these.
|
|
(println (edn/member? cells (pair (i64 4) (i64 3))))
|
|
(println (edn/member? cells (pair (i64 0) (i64 0))))
|
|
(println (edn/member? cells (pair (i64 3) (i64 4))))
|
|
(println (edn/member? cells (pair (i64 9) (i64 9)))))
|
|
_ (println "no cells")))
|
|
None (println "malformed")))
|
|
|
|
;; The size of a set after the dedup, which is the whole of what the dedup can
|
|
;; be asked for.
|
|
(defn set-size [src string] ()
|
|
(match (edn/read (bytes src))
|
|
(Some v) (match v (Set items) (println (len items)) _ (println "not a set"))
|
|
None (println "malformed")))
|
|
|
|
;; A document in a buffer this program owns and can write to. (bytes "literal")
|
|
;; is not that — a literal is constant data behind a writable-looking slice —
|
|
;; so the source is built with append! and the write goes through as-slice.
|
|
(defn survives-its-buffer [] ()
|
|
(let [buf (vec-new u8)]
|
|
(append! (addr buf) (bytes "{:name \"level-1\" :xs [1 2]}"))
|
|
(let [src (as-slice buf)
|
|
v (edn/read src)]
|
|
(dotimes [i (len src)]
|
|
(set (at src i) \x))
|
|
(match v
|
|
(Some doc)
|
|
(match (field doc "name")
|
|
(Text s) (println s)
|
|
_ (println "no name"))
|
|
None (println "malformed")))))
|
|
|
|
(defn main [] i32
|
|
(set frame (arena-new 262144))
|
|
(with-allocator frame
|
|
(do
|
|
(show-tileset)
|
|
(println "")
|
|
|
|
;; Dedup, on each kind of element a set can hold. The nested pair is the
|
|
;; one a structural compare is needed for; the nested *set* is the one
|
|
;; that also needs the compare to ignore order, or #{1 2} and #{2 1}
|
|
;; would be two elements.
|
|
(set-size "#{}")
|
|
(set-size "#{1 1 2}")
|
|
(set-size "#{[0 0] [0 0] [0 1]}")
|
|
(set-size "#{\"a\" \"a\" :a :a}")
|
|
(set-size "#{#{1 2} #{2 1}}")
|
|
(set-size "#{{:a 1} {:a 1} {:a 2}}")
|
|
(set-size "#{1 1.0}") ; an int and a float are two values
|
|
(set-size "#{true false true}")
|
|
(println "")
|
|
|
|
(survives-its-buffer)
|
|
;; And the refusal, which has to be distinguishable from the document
|
|
;; that is literally nil.
|
|
(match (edn/read (bytes "#{1 2")) (Some _v) (println "read") None (println "malformed"))
|
|
(match (edn/read (bytes "nil")) (Some _v) (println "read") None (println "malformed"))))
|
|
(free-all frame)
|
|
(arena-destroy frame)
|
|
0)
|