The tokenizer refused #{} because "it needs a hash set to even
represent" — which is a claim about a reader, and a tokenizer represents
nothing. #{ now pushes } on the same balance stack { does, there is one
new token kind and no new closer, and err-set is gone rather than kept
with a message it no longer earns. skip-value needed nothing: it is
written against the depth and not against the kinds.
The dynamic reader moves out of test/programs/arena-edn.flan and into
vendor/edn/read.flan as (edn/read bytes), answering an (Option Value)
against whichever allocator the caller bound. Two decisions are written
down where they are made:
* a set is a Value.Set holding a deduplicated (Vec Value), because
(Map Value bool) does not typecheck — keyable refuses a key holding
a Vec or a Map — and restricting elements to keyable Values would
refuse #{[0 0] [1 0]}, which is the file this was built for. Insert
is O(n) against a structural value=?, so building the tileset's 54
pairs is 1458 comparisons, once.
* a Value copies every string into the allocator where a Token stays
a view. A view handed back out of the function that owns the buffer
is a dangling pointer, and free-all would not even take it. Odin's
json parser clones for the same reason.
An imported defdata was a refusal in load.ml — "not implemented yet
(milestone 4)" — and it had to go first. It is the type's name plus the
Type. half of a constructor symbol, which arrives as a Var node when the
case has no fields and a Struct node when it has; a match pattern needed
nothing, because a case resolves against the scrutinee's type and was
never a top-level name. programs/pkg-data.flan is that on its own.
programs/edn-read.flan reads assets/edn/tileset.edn, which is the
editor's real output: :texture-path and a :selected-cells of 54 integer
pairs, with no type declared for any of it. It also overwrites the
source buffer in place after reading and prints the document back, which
is the copy contract asserted rather than described.
Decision 1. Odin's #load and #load_directory are the model, spelled as
ordinary named calls — an s-expression language already has a head
position and does not need Odin's `#`. (embed "p") is a [u8], (embed "p"
string) is a string, and (embed-dir "d") is a [n EmbedFile] sorted by
name.
Two spellings rather than one that changes type with its context. Odin
threads a type_hint everywhere and can afford it; with structural
equality and no implicit widening, the same text meaning two types here
would be a wart. The path is a literal and resolves relative to the file
the form is written in, both of which are Odin's rules and for Odin's
reasons: the bytes must be in hand before any value exists, and a
package's assets must not depend on where flan was invoked from.
The bytes reach the program as a [Str] node typed [u8], not as a [Bytes]
prim over a string. [Bytes] is identity — emit.ml lowers String and
Slice _ to the same %slice — and wrapping the literal in a prim would
make the node non-constant, so an (embed-dir) bound with defconst could
not be an LLVM constant. Both string emitters take the bytes and ignore
the node's type, so it is the same constant either way and one a global
can hold. emit.ml's escape is byte-exact, so a PNG survives the .ll.
The directory lookup is a linear scan in the prelude over a slice of
EmbedFile. A directory embed is tens of entries out of cache-warm
.rodata, and a compile-time perfect hash would be a build-time map with
its own failure modes that nothing has asked for. Sorted because readdir
order is filesystem-dependent and an unsorted embed would make two
builds of identical sources emit different .ll.
The slice points into .rodata, so a store through it segfaults at -O0
and is deleted at -O2 — the same measured trap the prelude's ASCII-case
note describes for (bytes "Hi"). Inherited, not widened; clone into a
Vec for a mutable copy.
