Joseph Ferano
aeeb6de59d
The prelude's declare surface was five f32 functions, and the five were there because somebody needed each one. Everything else a caller wanted was written as a declare at the top of their own file -- the identical libm call with none of the caveats written down. So the rest of libm is here: tan, the three inverses, the three logarithms, exp, fmod, hypot, cbrt, fabs, and an f64 face for every one of them including the five that already existed. A declare is a line, a symbol already on the link, and nothing in either backend, which is why this was cheap enough to do completely rather than one function at a time. The f64 half is not decoration. f32 is what a position is; f64 is what a measurement is -- the clock, parse-f64, format-f64, any sum over more than a few thousand terms -- and having only the f32 face forced a cast down and back at each of those boundaries, which is where the precision went. The paragraph the sqrt note draws for itself is now drawn once for the family: IEEE-754 specifies sqrt, fabs, floor, ceil, round and fmod as exact or correctly rounded, so those agree bit for bit across glibc, musl and wasi-libc; it requires nothing of the rest, so the sand-grid rule covers all of them unchanged. floor, ceil and round are Flan at f32 and libm at f64, and that is not an inconsistency: the f32 bodies work because every f32 with a fraction fits in an i32, and at f64 that trick is gone. abs-i32 and abs-i64 are Flan, one per width because min and max are builtins and no generic covers the numeric types. pi and tau at both widths, written out rather than derived so the compiler rounds each literal once. programs/math3.flan covers it at values that are exact in binary, so nothing pins one libm's last bit. The -O0 case is the one that matters: at -O2 LLVM folds a call over two literals and leaves no symbol to resolve, which is how a missing -lm hid the first time.
1858 lines
92 KiB
OCaml
1858 lines
92 KiB
OCaml
(** The milestone-2 prelude, written in Flan.
|
||
|
||
Printing is deliberately *not* a primitive (plan.org, Milestone-2
|
||
primitives): [write-stdout] is the one output primitive and everything
|
||
above it is Flan. That is what keeps a second backend cheap — a primitive
|
||
is the only thing implemented twice.
|
||
|
||
It lives here as a string rather than as a file because there is no package
|
||
loader yet; at milestone 3 it becomes an ordinary [core:] package and this
|
||
module goes away. The acceptance programs may call anything defined here.
|
||
|
||
No printing function is here at all any more. [print] and [println] are
|
||
the whole printing surface, and neither is a function: both are
|
||
compiler-provided and structural, a walk over the concrete type at the
|
||
call site (check.ml, and the walk itself in render.ml). That is plan.org's
|
||
Milestone 5 item, and it needed none of the rest of milestone 5 -- there
|
||
is nothing to dispatch on at run time and no user-supplied printer to
|
||
choose between, so no type variables are involved. The earlier note here
|
||
said a single [println] had to wait for generics; it did not.
|
||
|
||
The per-type family that used to live here -- [print-str], [print-i64],
|
||
[print-f64], [print-bytes], [print-line], [newline] -- is gone, and [print]
|
||
is strictly the better call for every one of them. [print] is the same walk
|
||
as [println] without the trailing newline, so it covers the no-newline case
|
||
that was the family's remaining excuse (see [show] in
|
||
test/programs/slices.flan). And because this language has no implicit
|
||
widening, [(print-i64 x)] forced an explicit [(i64 x)] at every site;
|
||
[(print x)] takes the value as it is. That is not only shorter: the cast
|
||
through the signed printer turned a [u64] above 2^63 into a negative
|
||
number, where [print] routes it through [flan_u64_to_bytes] and prints what
|
||
it actually holds. *)
|
||
|
||
let source = {flan|
|
||
;; The condition every allocating operation signals when the allocator cannot
|
||
;; satisfy a request — spec-memory.md, "Allocation failure". It is here rather
|
||
;; than built by the checker because it is an ordinary value struct and the
|
||
;; checker already knows how to build one of those; nothing about it is
|
||
;; special except who signals it.
|
||
;;
|
||
;; Fixed numeric fields and no rendered message, because formatting would
|
||
;; allocate and this is the one path that must not. :allocator is the
|
||
;; allocator's address, which is its identity — the same thing the epoch hangs
|
||
;; off — so a handler can tell which region ran out. Rendering happens in the
|
||
;; handler or the break loop, where a working allocator is known.
|
||
(defstruct StorageExhausted [bytes i64 align i64 allocator i64])
|
||
|
||
;; What an out-of-range index signals. Same shape as StorageExhausted and for
|
||
;; the same reasons: fixed numeric fields, no rendered message, nothing that
|
||
;; allocates — the condition is built on the failing frame's stack and the
|
||
;; formatting is the handler's or the break loop's job, where a working
|
||
;; allocator is known.
|
||
;;
|
||
;; It is signalled with `error`, from the runtime rather than from Flan:
|
||
;; flan_bounds_error, flan_slice_error and flan_slice_promise_error in
|
||
;; runtime/flan_rt.c — the last of those is (slice-from-ptr p n), whose
|
||
;; message is about the caller's promise because there is no container to
|
||
;; report, and which fills these fields with (0, n, 0): the condition it
|
||
;; violated, 0 <= n, written as a range. Between them they are where every
|
||
;; bounds check branches. **The three fields there are a C struct that
|
||
;; has to agree with this one field for field**, the same hand-kept agreement
|
||
;; flan_name_id keeps with Check.type_id.
|
||
;;
|
||
;; `low` and `high` are the same index for an (at xs i), and the two ends of
|
||
;; the range for a (slice xs lo hi). One condition type rather than two,
|
||
;; because a handler that wants to survive a bad index should not have to
|
||
;; write two clauses to cover the two ways of writing one.
|
||
;;
|
||
;; **Nothing establishes a restart at the failing site**, which is the
|
||
;; difference from StorageExhausted and from FileError. Those offer `retry`
|
||
;; because their attempt is repeatable: free something, or supply another path,
|
||
;; and the same operation succeeds the second time. Nothing a handler can do
|
||
;; makes index 51 valid for a length-50 array, so there is no attempt to
|
||
;; re-run. `use-value` for the index is the near miss and is not built: it
|
||
;; would put an alloca and a restart frame on every indexing operation, and
|
||
;; what it buys is a *different element*, silently.
|
||
;;
|
||
;; The restarts that matter are the ones the program already established — a
|
||
;; frame loop's `continue`, sand.flan's shape — and they are on the restart
|
||
;; stack and reachable from a handler or from the break loop without anything
|
||
;; being pushed here. That is plan.org's "restarts go at the resync point,
|
||
;; once", with allocation and file failure as the named exceptions and this on
|
||
;; the default side of the rule.
|
||
(defstruct BoundsError [low i64 high i64 length i64])
|
||
|
||
;; What an arithmetic operation with no answer signals. Three situations, and
|
||
;; until now none of them had a defined behaviour: a divide or remainder by
|
||
;; zero, which was a raw SIGFPE with no message and no location; the one
|
||
;; division that overflows, (/ most-negative -1), whose true quotient is one
|
||
;; past the top of the type and which `idiv` also makes a SIGFPE; and a float
|
||
;; to integer cast whose value does not fit, where x86 produces a fixed
|
||
;; "integer indefinite" and LLVM calls the whole thing undefined and may fold
|
||
;; it to anything.
|
||
;;
|
||
;; They signal, for the reason spelled out over BoundsError and for one more
|
||
;; that is specific to these: a SIGFPE cannot be caught and resumed, so the
|
||
;; only way to get a message naming the file and the line is to test *before*
|
||
;; the instruction. Once that branch is being paid for, making it a condition
|
||
;; rather than a die costs nothing further, and a program that genuinely does
|
||
;; not care installs a handler once at startup and never thinks about it again.
|
||
;;
|
||
;; Same shape as the two above, and for the same reasons: fixed numeric
|
||
;; fields, no rendered message, nothing that allocates. It is signalled from
|
||
;; the runtime — flan_arith_error in runtime/flan_rt.c — so **these three
|
||
;; fields are a C struct that has to agree with this one field for field**,
|
||
;; the same hand-kept agreement flan_bounds_cond keeps with BoundsError.
|
||
;;
|
||
;; `op` is a small integer and not a keyword, exactly as FileError's `op` is,
|
||
;; because the field is filled in from C and a keyword is not a thing that
|
||
;; exists there. The codes:
|
||
;;
|
||
;; 0 (/ a 0) 1 (% a 0)
|
||
;; 2 (/ min -1) 3 (% min -1)
|
||
;; 4 a float to integer cast whose value does not fit
|
||
;;
|
||
;; `lhs` and `rhs` are the two operands for codes 0 through 3 and the
|
||
;; destination type's representable range for code 4 — the violated condition
|
||
;; written as a range, which is what flan_slice_promise_error already does
|
||
;; with BoundsError's fields. Two meanings over two fields rather than two
|
||
;; condition types, so that a handler writes one clause and not five. The
|
||
;; value that did not fit is not carried, because it is a float and these
|
||
;; fields are not; what the handler needs in order to say something useful is
|
||
;; the range it missed.
|
||
;;
|
||
;; The fields are the low 64 bits of whatever they hold. A u64 operand above
|
||
;; 2^63 therefore reads back negative, which is the same reinterpretation
|
||
;; every i64 field in every condition here makes and is not worth a fourth
|
||
;; field to fix.
|
||
;;
|
||
;; **No restart is established at the failing operation**, which is
|
||
;; BoundsError's decision and not StorageExhausted's. The sketch this started
|
||
;; from asked for `use-value`, and the implementation is what ruled it out: a
|
||
;; restart frame is allocated by the restart-case that offers it, on its own
|
||
;; stack, and a transfer carries that frame's address (runtime/flan_rt.c, the
|
||
;; restart stack). The runtime cannot hold one on a program's behalf, so
|
||
;; `use-value` here would mean an alloca and a restart frame emitted at every
|
||
;; division in every checked build — the identical cost refused for indexing
|
||
;; a few lines above, buying a silently different answer. What answers a
|
||
;; division by zero is the restart the program already established, a frame
|
||
;; loop's `continue`, which is reachable from a handler without anything being
|
||
;; pushed here.
|
||
(defstruct ArithError [op i32 lhs i64 rhs i64])
|
||
|
||
;; A breakpoint. (pause) stops the program where it stands and hands it to the
|
||
;; break loop, with the whole stack under it readable — C-c C-b lists the
|
||
;; frames, TAB opens one, and taking `continue` resumes at the call.
|
||
;;
|
||
;; It is spelled `pause` and not `break` because `break` is reserved for
|
||
;; leaving a loop (parse.ml refuses it by name, with the milestone), and a
|
||
;; breakpoint and a loop exit in the same word would be the worst kind of
|
||
;; collision: both are legal in the same place and mean opposite things.
|
||
;;
|
||
;; Nothing in the compiler knows about this. It is `error` under a
|
||
;; `restart-case`, which is exactly what a breakpoint is in a language that
|
||
;; already has conditions: the break loop is entered because nothing handled
|
||
;; the condition, and `continue` is an ordinary restart whose body is empty, so
|
||
;; taking it returns here and the caller carries on. A handler-bind above it
|
||
;; can therefore also intercept a Pause and decline to stop, which is the
|
||
;; behaviour a release build wants and gets for free.
|
||
(defstruct Pause [])
|
||
|
||
(defn pause [] ()
|
||
(restart-case (error (Pause {}))
|
||
(continue [] (do))))
|
||
|
||
;; A seeded PRNG in Flan rather than libc's, because a grid hash is only a
|
||
;; regression test if the sequence is byte-identical on native and wasm32
|
||
;; (plan.org, RNG is ours). PCG-XSH-RR 32: one u64 LCG step per draw, folded
|
||
;; down to 32 bits by an xorshift and rotated by the state's top five bits.
|
||
(defvar rand-state u64 6364136223846793005)
|
||
|
||
(defn rand-seed [seed u64] ()
|
||
(set rand-state (+ (* seed 6364136223846793005) 1442695040888963407)))
|
||
|
||
(defn rand-u32 [] u32
|
||
(let [s rand-state]
|
||
(set rand-state (+ (* s 6364136223846793005) 1442695040888963407))
|
||
;; The rotate is masked to 5 bits: a 32-bit shift by 32 is poison in LLVM,
|
||
;; and r = 0 is the case that would ask for it.
|
||
(let [x (u32 (>> (bit-xor (>> s 18) s) 27))
|
||
r (u32 (>> s 59))]
|
||
(bit-or (>> x r) (<< x (bit-and (- 32 r) 31))))))
|
||
|
||
;; In [0, 1). The divisor is 2^32 exactly, so the result never reaches 1.0.
|
||
(defn rand-f32 [] f32
|
||
(/ (f32 (rand-u32)) 4294967296.0))
|
||
|
||
;; ── Slice algorithms, all in place ────────────────────────────────────
|
||
;;
|
||
;; One family per element type, because there are no generics: each of these
|
||
;; is a *copy* per element type, and the set below is i32 (what indices, ids
|
||
;; and tile values are), f32 (what positions, velocities and weights are) and
|
||
;; [u8] (what a field coming out of `split` is).
|
||
;;
|
||
;; A slice is ptr+len and non-owning, so these mutate the storage they were
|
||
;; handed: sorting (slice grid 4 9) sorts those five elements of grid and
|
||
;; leaves the rest alone. That was originally forced — there was no allocator
|
||
;; to return a new sequence from — and it stays the right shape now that there
|
||
;; is one, because sorting a thing you already own should not allocate. The
|
||
;; allocating tier is further down, and a caller sorts a Vec by sorting
|
||
;; (as-slice v).
|
||
;;
|
||
;; **map, filter, reduce and a sort taking a comparator are here now**, in a
|
||
;; section of their own after the f32 family. They were blocked on *function
|
||
;; values* and not on generics, which is why they arrived without generics:
|
||
;; a (Fn [T ...] R) is an ordinary parameter type. What they are still one
|
||
;; copy per element type for *is* generics — sum-i32 and sum-f32 are the same
|
||
;; shape and the same argument — so the set is the same i32 and f32 the rest of
|
||
;; this family covers.
|
||
|
||
;; ── One family, over one type variable ────────────────────────────────
|
||
;;
|
||
;; What used to be a copy per element type. A [$t] binds a type variable in
|
||
;; the signature and every call site instantiates the body at the types it
|
||
;; passes, so [(sort! xs)] over a [i32] and over a [f32] are two emitted
|
||
;; bodies from one written one.
|
||
;;
|
||
;; **Two things in the signatures are not decoration.**
|
||
;;
|
||
;; [{:where (ordered? $t)}] is what lets the body write [<] at all. A type
|
||
;; variable supports only what it is declared to support — an unconstrained
|
||
;; one is refused at the *definition*, not at some later call site — and
|
||
;; [ordered?] is the predicate that admits [<], [<=], [>], [>=], [min] and
|
||
;; [max]. It admits [=] and [copyable?] too: every type the language orders is
|
||
;; a number or an enum, so it is equatable and it is not move-only.
|
||
;;
|
||
;; [{:where (copyable? $t)}] is the opt-out from the other default. A type
|
||
;; variable is **move-only** until it says otherwise, because move is the
|
||
;; stricter rule and assuming it can only refuse a valid program rather than
|
||
;; admit a broken one: [reduce]'s accumulator is read into [f] and then
|
||
;; assigned again, which is correct at [i32] and a double move at [(Vec i32)],
|
||
;; and the checker cannot tell which until it substitutes.
|
||
;;
|
||
;; **Two of these ten are forced and the rest are convention, and the
|
||
;; difference is worth knowing.** [filter] and [reduce] do not check without
|
||
;; [copyable?]: the first returns a [(Vec $t)], and a Vec of an owning element
|
||
;; is refused, and the second holds its accumulator in a local and reads it
|
||
;; twice. [swap!], [reverse!], [map!] and [sort-by!] check *without* it,
|
||
;; because the move analysis tracks locals and parameters and does not track a
|
||
;; read out of a slice — so [(let [t (at s i)] ... (set (at s j) t))] is not
|
||
;; seen as a move even when the element owns storage. They declare it anyway,
|
||
;; and should: at [[(Vec i32)]] those bodies would duplicate a header. It is
|
||
;; the one place move-by-default is not conservative, and until element-level
|
||
;; moves are tracked, a [copyable?] on a body that moves elements between
|
||
;; slots is a convention the reader has to keep rather than a fact the checker
|
||
;; enforces.
|
||
;;
|
||
;; **What did not collapse, and why it should not.** [sum-i32] and [sum-f32]
|
||
;; widen their element into [i64] and [f64]; "the wider type $t accumulates
|
||
;; into" is a type-level function, which is a constraint system of a different
|
||
;; kind, and a generic [sum] that took its accumulator and its [+] would just
|
||
;; be [reduce]. [append-i64!] and [append-f64!] are two different primitives.
|
||
;; [sort-bytes!] needs [bytes<?] rather than [<] — a [[u8]] is not [ordered?]
|
||
;; and cannot be — so it is [sort-by!] with the comparison written in, and it
|
||
;; keeps its name because the stability contract in its comment is worth
|
||
;; keeping attached to something.
|
||
|
||
(defn swap! [s [$t] i i32 j i32] ()
|
||
{:where (copyable? $t)}
|
||
(let [t (at s i)]
|
||
(set (at s i) (at s j))
|
||
(set (at s j) t)))
|
||
|
||
(defn reverse! [s [$t]] ()
|
||
{:where (copyable? $t)}
|
||
(let [i 0
|
||
j (- (len s) 1)]
|
||
(while (< i j)
|
||
(swap! s i j)
|
||
(set i (+ i 1))
|
||
(set j (- j 1)))))
|
||
|
||
;; Insertion sort: in place, no recursion, no auxiliary array — quicksort
|
||
;; would want a stack and mergesort a buffer, and neither exists. Ascending,
|
||
;; and stable, though with no payload type to carry that is not yet
|
||
;; observable.
|
||
;;
|
||
;; One caveat that only arises at f32: **a NaN in the input makes the order
|
||
;; undefined.** Every comparison against a NaN is false, so the insertion loop
|
||
;; never moves one and never moves anything past one; what comes out is sorted
|
||
;; within each run between NaNs and not sorted across them. That is what C's
|
||
;; qsort with a naive comparator does too, and the only fix is not to have
|
||
;; NaNs in the array — there is no ordering of the reals a NaN sits anywhere
|
||
;; in.
|
||
(defn sort! [s [$t]] ()
|
||
{:where (ordered? $t)}
|
||
(let [i 1]
|
||
(while (< i (len s))
|
||
(let [j i]
|
||
;; `and` short-circuits, which is load-bearing: at j = 0 the left test
|
||
;; fails and (at s -1) is never evaluated, so this does not trap.
|
||
(while (and (> j 0) (> (at s (- j 1)) (at s j)))
|
||
(swap! s (- j 1) j)
|
||
(set j (- j 1))))
|
||
(set i (+ i 1)))))
|
||
|
||
;; The same insertion sort, with the one comparison it had written in replaced
|
||
;; by the one it is told. before? answers "does a come before b", so passing
|
||
;; (fn [a b] (< a b)) is ascending and reversing it is descending — and a
|
||
;; caller wanting a key rather than an order writes the comparison.
|
||
;;
|
||
;; It is stable exactly as sort! is: the loop stops the moment before? says
|
||
;; no, so equal elements never swap past each other. A before? that is not a
|
||
;; strict weak ordering — one answering true for both (a b) and (b a) — is the
|
||
;; caller's mistake and shows up as an order, not as a loop: the inner while
|
||
;; is bounded by j reaching 0 whatever the comparison says.
|
||
;;
|
||
;; This one needs no [ordered?]: the comparison it cannot have is the
|
||
;; comparison it is given. It is the shape every generic had to take before
|
||
;; predicates existed, and it stays because passing a comparison is a real
|
||
;; thing to want and not only a workaround.
|
||
(defn sort-by! [s [$t] before? (Fn [$t $t] bool)] ()
|
||
{:where (copyable? $t)}
|
||
(let [i 1]
|
||
(while (< i (len s))
|
||
(let [j i]
|
||
(while (and (> j 0) (before? (at s j) (at s (- j 1))))
|
||
(swap! s (- j 1) j)
|
||
(set j (- j 1))))
|
||
(set i (+ i 1)))))
|
||
|
||
;; The first index holding x. None rather than -1, because Option is what the
|
||
;; language has and a sentinel index is the bug this avoids.
|
||
(defn index-of [s [$t] x $t] (Option i32)
|
||
{:where (equal? $t)}
|
||
(dotimes [i (len s)]
|
||
(when (= (at s i) x)
|
||
(return (Some i))))
|
||
None)
|
||
|
||
;; None for an empty slice: there is no least i32 that is also an honest
|
||
;; answer, and returning one would be a value the caller cannot tell from a
|
||
;; real element. A NaN in the input is not special-cased and propagates the
|
||
;; way it does through the builtins — the comparison fails, so the running
|
||
;; value simply does not change.
|
||
;;
|
||
;; Named min-of rather than min because [min] and [max] are builtins over two
|
||
;; or more numbers, and a defn cannot shadow a builtin: nothing shadows [+]
|
||
;; either. These reduce a slice, which is a different operation with a
|
||
;; different arity, so the different name is honest rather than a workaround.
|
||
(defn min-of [s [$t]] (Option $t)
|
||
{:where (ordered? $t)}
|
||
(if (= (len s) 0)
|
||
None
|
||
(let [m (at s 0)]
|
||
(dotimes [i (len s)]
|
||
(set m (min m (at s i))))
|
||
(Some m))))
|
||
|
||
(defn max-of [s [$t]] (Option $t)
|
||
{:where (ordered? $t)}
|
||
(if (= (len s) 0)
|
||
None
|
||
(let [m (at s 0)]
|
||
(dotimes [i (len s)]
|
||
(set m (max m (at s i))))
|
||
(Some m))))
|
||
|
||
;; map! writes back into the slice it was handed, for the same reason sort!
|
||
;; does — a slice is non-owning, and transforming a thing you already own
|
||
;; should not allocate. A map that produces a *different* element type is not
|
||
;; here: it is two type variables and a second signature, and nothing has
|
||
;; wanted it.
|
||
(defn map! [s [$t] f (Fn [$t] $t)] ()
|
||
{:where (copyable? $t)}
|
||
(dotimes [i (len s)]
|
||
(set (at s i) (f (at s i)))))
|
||
|
||
;; The general fold, of which sum-i32 is the special case with the + written
|
||
;; in. The accumulator comes first in the step, which is the order that reads
|
||
;; as (f acc x) and the order Odin's slice.reduce uses.
|
||
(defn reduce [s [$t] init $t f (Fn [$t $t] $t)] $t
|
||
{:where (copyable? $t)}
|
||
(let [acc init]
|
||
(dotimes [i (len s)]
|
||
(set acc (f acc (at s i))))
|
||
acc))
|
||
|
||
;; A new Vec holding the elements the predicate kept, in the order they were
|
||
;; in. Owned by the caller: (free v), or let a (free-all a) take the region.
|
||
;;
|
||
;; This is the one that proves the containers and the generics compose. It
|
||
;; allocates — (vec-new t), push, returns (Vec t) — and the type-erased Vec
|
||
;; runtime needed no change at all, because SizeOf and AlignOf are computed at
|
||
;; the instantiation site, where the element type is concrete.
|
||
(defn filter [s [$t] keep? (Fn [$t] bool)] (Vec $t)
|
||
{:where (copyable? $t)}
|
||
(let [v (vec-new t)]
|
||
(dotimes [i (len s)]
|
||
(when (keep? (at s i))
|
||
(push v (at s i))))
|
||
v))
|
||
|
||
;; ── The per-type layer that stays ─────────────────────────────────────
|
||
;;
|
||
;; sum is the one shape a type variable cannot express, and it is worth being
|
||
;; precise about why rather than leaving two near-identical functions looking
|
||
;; like an oversight. Each of these *widens*: sum-i32 accumulates in i64 and
|
||
;; sum-f32 in f64, with an explicit cast per element, because there is no
|
||
;; implicit widening anywhere in the language and summing a screenful into the
|
||
;; element's own type is how a total silently wraps or absorbs. "The wider
|
||
;; type $t accumulates into" is a function from types to types — an associated
|
||
;; type, or a constraint system of a kind {:where} is not — and a generic sum
|
||
;; that took its accumulator and its + as parameters would be reduce, which is
|
||
;; above.
|
||
|
||
;; Accumulates in i64 and each element is widened explicitly — there is no
|
||
;; implicit widening anywhere in the language, and summing a screenful of i32
|
||
;; into an i32 is how a total silently wraps.
|
||
(defn sum-i32 [s [i32]] i64
|
||
(let [t (i64 0)]
|
||
(dotimes [i (len s)]
|
||
(set t (+ t (i64 (at s i)))))
|
||
t))
|
||
|
||
;; Accumulates in f64 and widens each element explicitly, which is sum-i32's
|
||
;; argument in its floating form and a stronger one: summing a screenful of
|
||
;; f32 in f32 does not wrap, it *absorbs* — once the running total is large
|
||
;; enough, adding a small element rounds to no change at all, and the answer
|
||
;; is silently short rather than obviously wrong. An f64 accumulator has 29
|
||
;; more bits of mantissa and pushes that failure out of reach of any array a
|
||
;; game holds.
|
||
(defn sum-f32 [s [f32]] f64
|
||
(let [t 0.0]
|
||
(dotimes [i (len s)]
|
||
(set t (+ t (f64 (at s i)))))
|
||
t))
|
||
|
||
;; ── Bytes ─────────────────────────────────────────────────────────────
|
||
;;
|
||
;; Over [u8] and not over string, so (bytes s) is what a caller writes and one
|
||
;; copy of each serves strings and byte slices both — which is as close to a
|
||
;; generic as a language without them gets. Nothing here allocates: every
|
||
;; result is a bool, an index, or a number.
|
||
|
||
(defn bytes=? [a [u8] b [u8]] bool
|
||
(if (!= (len a) (len b))
|
||
false
|
||
(do
|
||
(dotimes [i (len a)]
|
||
(when (!= (at a i) (at b i))
|
||
(return false)))
|
||
true)))
|
||
|
||
;; The length test comes first and `and` short-circuits, so the slice is only
|
||
;; built once it is known to be in bounds — otherwise a prefix longer than the
|
||
;; string would trap rather than answer false.
|
||
(defn starts-with? [s [u8] p [u8]] bool
|
||
(and (<= (len p) (len s))
|
||
(bytes=? (slice s 0 (len p)) p)))
|
||
|
||
(defn ends-with? [s [u8] p [u8]] bool
|
||
(and (<= (len p) (len s))
|
||
(bytes=? (slice s (- (len s) (len p)) (len s)) p)))
|
||
|
||
;; The whole slice is an integer, or it is None. bytes->i64 is strtoll, which
|
||
;; answers 0 for "" and for "abc" and stops at the first junk byte in "12x" —
|
||
;; three wrong answers a caller cannot tell from a real 12. This is also the
|
||
;; one that has to be Flan rather than the primitive: strtoll is locale- and
|
||
;; libc-dependent, and a parser in the language gives the same answer on
|
||
;; wasm32 as on native for the same reason rand-f32 does.
|
||
;; Overflow wraps, as all arithmetic here does; it is not reported.
|
||
(defn parse-i64 [s [u8]] (Option i64)
|
||
(let [i 0
|
||
n (i64 0)
|
||
neg false]
|
||
(when (= (len s) 0)
|
||
(return None))
|
||
(when (or (= (at s 0) \-) (= (at s 0) \+))
|
||
(set neg (= (at s 0) \-))
|
||
(set i 1))
|
||
(when (= i (len s))
|
||
(return None)) ; a lone sign is not a number
|
||
(while (< i (len s))
|
||
(let [b (at s i)]
|
||
(when (or (< b \0) (> b \9))
|
||
(return None))
|
||
(set n (+ (* n 10) (i64 (- b \0)))))
|
||
(set i (+ i 1)))
|
||
(if neg (Some (- 0 n)) (Some n))))
|
||
|
||
;; ── Numbers ───────────────────────────────────────────────────────────
|
||
;;
|
||
;; Only the ones that encode a decision. abs is (max x (- 0 x)); a wrapper over
|
||
;; that is a function emitted into every program to save a caller nothing. The
|
||
;; one honest caveat on that abs: at the least representable integer it
|
||
;; answers itself, because the negation wraps. That is what every two's-complement abs does, a
|
||
;; function here would do it too, and the only fix is not to hand it that
|
||
;; value — so it is written down rather than wrapped.
|
||
;;
|
||
;; The float abs is the same one-liner, (max x (- 0.0 x)), and it is not
|
||
;; wrapped for the same reason — but the caveat above does not carry over, so
|
||
;; it is not inherited by silence. f32 negation is exact at every value, there
|
||
;; is no least representable float that negates to itself, and the two edge
|
||
;; inputs both come out right: -0.0 answers +0.0 (the max picks the subtracted
|
||
;; side, since neither zero is greater than the other), and a NaN answers a
|
||
;; NaN (every comparison fails, so the same max picks the subtracted side,
|
||
;; which is still a NaN). There is nothing left for a function to fix.
|
||
|
||
;; clamp is a macro and not a function, and the reason is the objection above
|
||
;; turned around rather than dropped. min and max are builtins that work at
|
||
;; every numeric type; a clamp *function* cannot, because there are no
|
||
;; generics, so it would be one copy per type — a clamp-i32, a clamp-f32, a
|
||
;; clamp-i64 — each emitted into every program to save a caller eleven
|
||
;; characters. A macro is type-agnostic for free and emits nothing at all: what
|
||
;; the program contains after expansion is the (min hi (max lo x)) the caller
|
||
;; would have written.
|
||
;;
|
||
;; Each of x, lo and hi appears exactly once in the expansion, so nothing here
|
||
;; is evaluated twice and an argument with a side effect behaves as it reads.
|
||
;;
|
||
;; lo above hi is not checked, and the answer there is hi — the outer min wins.
|
||
;; That is the same rule Odin's clamp follows and there is nowhere better to
|
||
;; put a complaint: a macro has no error facility (see `unless` at the foot of
|
||
;; this file), so a diagnostic would have to be a run-time one, in the one
|
||
;; construct whose whole point is that it costs nothing at run time.
|
||
(defmacro clamp [args]
|
||
(if (!= (len args) 3)
|
||
`(clamp-takes-a-value-a-low-and-a-high)
|
||
`(min ~(at args 2) (max ~(at args 1) ~(at args 0)))))
|
||
|
||
;; Zero for zero, and zero for NaN — neither is positive nor negative, so
|
||
;; neither comparison fires. A caller that needs to know which it got should
|
||
;; be testing for NaN, not reading a sign.
|
||
(defn sign-f32 [x f32] f32
|
||
(cond
|
||
(> x 0.0) 1.0
|
||
(< x 0.0) -1.0
|
||
:else 0.0))
|
||
|
||
;; Written as the weighted sum and not as a + t*(b - a): the second form does
|
||
;; not return b exactly at t = 1.0 once rounding is involved, and a position
|
||
;; that does not arrive is the bug an interpolation gets reported for.
|
||
(defn lerp [a f32 b f32 t f32] f32
|
||
(+ (* (- 1.0 t) a) (* t b)))
|
||
|
||
;; ── More of the RNG ───────────────────────────────────────────────────
|
||
;;
|
||
;; Both draw exactly one rand-u32, so the sequence a program consumes is the
|
||
;; same one; neither touches the generator.
|
||
|
||
;; [lo, hi). An empty or reversed range answers lo — a defined value rather
|
||
;; than a remainder by zero, which is immediate undefined behaviour and not a
|
||
;; wrong number. The span must fit in i32, since hi - lo is computed there.
|
||
;; One draw, and therefore modulo bias: the low (2^32 % span) values of the
|
||
;; range come up very slightly more often. Rejection sampling would remove it
|
||
;; and would consume an unpredictable number of draws, which is the one thing
|
||
;; this generator exists not to do.
|
||
(defn rand-i32-range [lo i32 hi i32] i32
|
||
(if (<= hi lo)
|
||
lo
|
||
(+ lo (i32 (% (rand-u32) (u32 (- hi lo)))))))
|
||
|
||
;; [lo, hi), because rand-f32 never reaches 1.0.
|
||
(defn rand-f32-range [lo f32 hi f32] f32
|
||
(+ lo (* (rand-f32) (- hi lo))))
|
||
|
||
;; ── Rounding, and the one thing that is not Flan ──────────────────────
|
||
;;
|
||
;; All three answer an f32 and take the f32 path, because that is what a
|
||
;; position, a tile coordinate and a velocity are here. f64 versions wait for
|
||
;; a program that wants them, for the same reason the f32 slice algorithms do.
|
||
;;
|
||
;; The cast to i32 truncates toward zero, which is the only rounding mode the
|
||
;; language has, so each of these is that cast plus the correction the mode
|
||
;; does not make. Three inputs would make the cast itself undefined and each
|
||
;; is named before it happens: NaN (which fails every comparison, so it is
|
||
;; tested for by (not (= x x)) and nothing else), and the two infinities,
|
||
;; which are caught by the magnitude test. Above 2^23 an f32 has no fractional
|
||
;; bits left at all, so returning x there is not an approximation — it is the
|
||
;; answer — and it doubles as the guard that keeps the cast inside i32.
|
||
|
||
;; Zero is returned as itself rather than through the cast, which would turn
|
||
;; -0.0 into +0.0. That is one line for a value most callers never look at,
|
||
;; and it is here because floorf is specified to return it: a sign of zero is
|
||
;; how a caller recovers which side a position approached from once the
|
||
;; magnitude has already been rounded away.
|
||
(defn floor-f32 [x f32] f32
|
||
(if (or (not (= x x))
|
||
(>= x 8388608.0)
|
||
(<= x -8388608.0)
|
||
(= x 0.0))
|
||
x
|
||
(let [t (f32 (i32 x))]
|
||
(if (> t x) (- t 1.0) t))))
|
||
|
||
;; One deviation from C's ceilf, written down rather than branched around:
|
||
;; between -1.0 and 0.0 this answers +0.0 where IEEE asks for -0.0, because
|
||
;; the outer (- 0.0 …) is a subtraction and not a negation. Nothing here reads
|
||
;; the sign of a zero; a caller that does should test the input instead.
|
||
(defn ceil-f32 [x f32] f32
|
||
(- 0.0 (floor-f32 (- 0.0 x))))
|
||
|
||
;; Half away from zero, which is C's round and not the even-tie rule: -2.5
|
||
;; goes to -3. Written as floor of the *magnitude* and mirrored, because
|
||
;; (floor-f32 (+ x 0.5)) is wrong twice over — it is half-*up* rather than
|
||
;; half-away for negatives, and at the largest f32 below 0.5 the addition
|
||
;; itself rounds to 1.0 and answers 1 for a number under a half.
|
||
(defn round-f32 [x f32] f32
|
||
(let [m (if (< x 0.0) (- 0.0 x) x)
|
||
f (floor-f32 m)
|
||
r (if (>= (- m f) 0.5) (+ f 1.0) f)]
|
||
(if (< x 0.0) (- 0.0 r) r)))
|
||
|
||
;; sqrt is the one function in this file that is not Flan, and it is a
|
||
;; `declare` rather than a body for a reason that is not laziness. Every other
|
||
;; number here is reachable from the four operations and a cast; a square root
|
||
;; is not. Newton's method needs a starting guess, a good one comes from
|
||
;; reinterpreting the exponent bits, and the language has no bit-cast between
|
||
;; f32 and u32 — only value-preserving casts. Without it the iteration needs a
|
||
;; scaling loop to normalise, converges slowly from a poor guess, and produces
|
||
;; a result that is *close*, which is exactly what a standard library must not
|
||
;; hand back. IEEE-754 makes sqrt correctly rounded, so libm's answer is the
|
||
;; same bit pattern on native and on wasm32 — the byte-identical property that
|
||
;; keeps rand-u32 in Flan is, for this one, an argument for going out to C.
|
||
;;
|
||
;; The cost is one `declare` line in every module, which LLVM drops where it
|
||
;; is unused, and one -lm on every link, which build.ml now passes. That flag
|
||
;; is not optional and not obvious: at -O2 LLVM folds most sqrtf calls into
|
||
;; the hardware instruction and nothing is left to resolve, so this appears to
|
||
;; link without it and then fails at -O0, where the call survives.
|
||
;;
|
||
;; The better fix belongs to the compiler and not here: llvm.sqrt.f32 as a
|
||
;; builtin in check.ml and emit.ml is one instruction with no symbol at all.
|
||
(declare sqrt-f32 [x f32] f32 "sqrtf")
|
||
|
||
;; sin and cos go out to libm too, and the argument is *not* the one above —
|
||
;; it is weaker, and which way it is weaker is the thing to know before
|
||
;; calling them. IEEE-754 requires sqrt to be correctly rounded, which is why
|
||
;; sqrtf's answer is the same bit pattern wherever it runs. It requires
|
||
;; nothing of the kind for sinf and cosf: each implementation is free to be a
|
||
;; fraction of an ulp off in its own direction, and glibc, musl and wasi-libc
|
||
;; do differ. So these two are the one place in this file where native and
|
||
;; wasm32 may not agree bit for bit, and a program whose output is hashed
|
||
;; across targets — the sand grid of plan.org's "RNG is ours", which is why
|
||
;; rand-u32 above is written in Flan and not called out of libc — must not
|
||
;; route that hash through a sine.
|
||
;;
|
||
;; They are here anyway, because the alternative on offer today is worse: a
|
||
;; caller that wants an angle writes the same two `declare` lines at the top
|
||
;; of its own file (examples/core-input-gestures-testbed.flan did, before
|
||
;; this), which is the identical libm call with the identical caveat and
|
||
;; nobody's name on it. One copy with the caveat written down beats a copy per
|
||
;; file with none.
|
||
;;
|
||
;; The fix, if a program ever does need trig that agrees across targets, is a
|
||
;; body rather than a declare: Cody-Waite reduction onto [-pi/4, pi/4] and a
|
||
;; minimax polynomial, which is reachable from the four operations and
|
||
;; floor-f32 and would therefore be exactly as reproducible as rand-u32. That
|
||
;; is a numerics job with its own accuracy budget, and it waits for a program
|
||
;; that needs it.
|
||
(declare sin-f32 [x f32] f32 "sinf")
|
||
(declare cos-f32 [x f32] f32 "cosf")
|
||
|
||
;; atan2 and pow inherit the paragraph above in full, and not the sqrt one.
|
||
;; IEEE-754 requires nothing of atan2f or powf either, so these are the third
|
||
;; and fourth places in this file where native and wasm32 may disagree in the
|
||
;; last bit, and the sand-grid rule stands unchanged: a hash compared across
|
||
;; targets must not be routed through any of the four.
|
||
;;
|
||
;; They are here for the reason the trig pair is. Without them a program that
|
||
;; wants a heading or a falloff curve writes the identical two declare lines at
|
||
;; the top of its own file, which is the same libm call with the same caveat
|
||
;; and nobody's name on it.
|
||
;;
|
||
;; atan2's y comes first, as it does in C, and the order is the answer rather
|
||
;; than a convention: knowing the quadrant of (y, x) is the whole of what it
|
||
;; has over (atan (/ y x)), and it is recovered from the two signs. It is
|
||
;; defined at x = 0, where the division is not.
|
||
;;
|
||
;; One caveat of pow-f32's own, because it is the one that gets reported as a
|
||
;; bug: it is not exact at integer exponents. powf goes through a logarithm,
|
||
;; so (pow-f32 10.0 2.0) is 100.0 or the float next to it depending on the
|
||
;; libm, and an index computed by casting that to i32 is off by one on the
|
||
;; wrong side. A small integer power is a multiplication, and should be
|
||
;; written as one.
|
||
(declare atan2-f32 [y f32 x f32] f32 "atan2f")
|
||
(declare pow-f32 [x f32 y f32] f32 "powf")
|
||
|
||
;; ── The rest of libm, and both widths ─────────────────────────────────
|
||
;;
|
||
;; The five above were the whole of it for a long time, and the reason they
|
||
;; were is the reason the rest are here now: every one of these is a line, a
|
||
;; symbol that is already on the link, and nothing in the compiler. A program
|
||
;; that wanted a logarithm wrote the `declare` at the top of its own file —
|
||
;; which is the identical call with none of the caveats written down and
|
||
;; nobody's name on it.
|
||
;;
|
||
;; **The f64 half is not decoration.** f32 is what a position and a colour
|
||
;; are, and f64 is what a *measurement* is: the clock below is nanoseconds in
|
||
;; an i64 and seconds in an f64, parse-f64 and format-f64 are both f64, and a
|
||
;; sum over more than a few thousand f32 terms has already lost the low bits
|
||
;; the answer was about. Having only the f32 face forced a cast down and back
|
||
;; at every one of those boundaries, and a cast down is where the precision
|
||
;; went.
|
||
;;
|
||
;; The split below is the one the sqrt paragraph draws, applied to the whole
|
||
;; family, and it is the only thing here worth knowing before calling:
|
||
;;
|
||
;; **Exact on every target.** IEEE-754 specifies these as exact operations
|
||
;; or as correctly rounded, so the answer is the same bit pattern under
|
||
;; glibc, musl and wasi-libc, and a hash taken across targets may be routed
|
||
;; through them. sqrt, fabs, floor, ceil, round, fmod.
|
||
;;
|
||
;; **Not.** IEEE-754 requires nothing of these and the three libms do
|
||
;; differ in the last bit. The sand-grid rule from the sin/cos paragraph
|
||
;; above covers all of them without change: a value compared across targets
|
||
;; must not have been through one. Everything else here.
|
||
|
||
(declare sqrt-f64 [x f64] f64 "sqrt")
|
||
|
||
;; Magnitude, and the f32/f64 pair is libm's because fabs is a sign-bit clear
|
||
;; that the compiler folds into one instruction — cheaper than the branch a
|
||
;; Flan body would be, and right for -0.0 and for NaN, which a (< x 0.0) test
|
||
;; is not: -0.0 is not less than zero, so the branch returns it unchanged and
|
||
;; hands back a negative zero from a function named abs.
|
||
(declare abs-f32 [x f32] f32 "fabsf")
|
||
(declare abs-f64 [x f64] f64 "fabs")
|
||
|
||
;; The f64 faces of the three rounding functions floor-f32, ceil-f32 and
|
||
;; round-f32 are libm's rather than Flan's, and that is not an inconsistency.
|
||
;; Those three are Flan because of a cast: (i32 x) is the whole of floor-f32's
|
||
;; body, and it works precisely because every f32 with a fractional part fits
|
||
;; in an i32. At f64 it does not — the exact range runs to 2^53 and i64's cast
|
||
;; would have to carry its own guard — so the trick that made them free is not
|
||
;; available and the libm call is both shorter and exact.
|
||
(declare floor-f64 [x f64] f64 "floor")
|
||
(declare ceil-f64 [x f64] f64 "ceil")
|
||
(declare round-f64 [x f64] f64 "round")
|
||
|
||
;; Remainder, and it is C's fmod and not a modulo: the sign follows the
|
||
;; *dividend*, so (fmod-f32 -1.0 3.0) is -1.0 and not 2.0. An angle wrapped
|
||
;; into [0, tau) therefore needs the add-and-fmod-again that every wrap
|
||
;; function has, and this is the line where that is written down rather than
|
||
;; discovered. It is exact — the result is the true remainder, representable
|
||
;; by construction — so it belongs to the first group above.
|
||
(declare fmod-f32 [x f32 y f32] f32 "fmodf")
|
||
(declare fmod-f64 [x f64 y f64] f64 "fmod")
|
||
|
||
;; The trigonometric family, in full and at both widths. tan is separate from
|
||
;; (/ (sin-f32 x) (cos-f32 x)) for the reason atan2 is separate from a
|
||
;; division: near pi/2 the quotient is a ratio of two small errors and tanf
|
||
;; is not.
|
||
(declare tan-f32 [x f32] f32 "tanf")
|
||
(declare sin-f64 [x f64] f64 "sin")
|
||
(declare cos-f64 [x f64] f64 "cos")
|
||
(declare tan-f64 [x f64] f64 "tan")
|
||
|
||
;; The inverses. asin and acos answer NaN outside [-1, 1] rather than
|
||
;; clamping, which is what catches a dot product that drifted to 1.0000001
|
||
;; through rounding — clamp it at the call site, on purpose, and the drift is
|
||
;; visible instead of silently becoming an angle of zero.
|
||
(declare asin-f32 [x f32] f32 "asinf")
|
||
(declare acos-f32 [x f32] f32 "acosf")
|
||
(declare atan-f32 [x f32] f32 "atanf")
|
||
(declare asin-f64 [x f64] f64 "asin")
|
||
(declare acos-f64 [x f64] f64 "acos")
|
||
(declare atan-f64 [x f64] f64 "atan")
|
||
(declare atan2-f64 [y f64 x f64] f64 "atan2")
|
||
|
||
;; Logarithms and the exponential. log is the natural one, as in C and unlike
|
||
;; the spreadsheet convention — log2 and log10 are the other two and are named
|
||
;; for their bases, so nothing here is ambiguous. log2 is not (/ (log x)
|
||
;; (log 2.0)): it is exact at every power of two, which is the whole reason a
|
||
;; bit-width or an octave is computed with it.
|
||
;;
|
||
;; All four answer -inf at zero and NaN below it rather than signalling. A
|
||
;; condition per logarithm would cost a handler search on a path whose callers
|
||
;; are loops over samples, and NaN is the value that propagates to wherever
|
||
;; the caller does check.
|
||
(declare log-f32 [x f32] f32 "logf")
|
||
(declare log2-f32 [x f32] f32 "log2f")
|
||
(declare log10-f32 [x f32] f32 "log10f")
|
||
(declare exp-f32 [x f32] f32 "expf")
|
||
(declare log-f64 [x f64] f64 "log")
|
||
(declare log2-f64 [x f64] f64 "log2")
|
||
(declare log10-f64 [x f64] f64 "log10")
|
||
(declare exp-f64 [x f64] f64 "exp")
|
||
(declare pow-f64 [x f64 y f64] f64 "pow")
|
||
|
||
;; hypot over (sqrt-f32 (+ (* x x) (* y y))) because the obvious form
|
||
;; overflows on inputs the answer does not: the square of an f32 above ~1.8e19
|
||
;; is infinity, so a distance between two far-apart points comes back inf when
|
||
;; the distance itself is perfectly representable. hypotf scales first. It
|
||
;; costs more than the naive form and is worth it exactly when the naive form
|
||
;; is wrong.
|
||
(declare hypot-f32 [x f32 y f32] f32 "hypotf")
|
||
(declare hypot-f64 [x f64 y f64] f64 "hypot")
|
||
|
||
;; Cube root, and it is here because (pow-f32 x 0.33333334) is not it: pow
|
||
;; goes through a logarithm, which is undefined for a negative base, so the
|
||
;; obvious spelling answers NaN for every negative number where cbrt answers
|
||
;; the negative root.
|
||
(declare cbrt-f32 [x f32] f32 "cbrtf")
|
||
(declare cbrt-f64 [x f64] f64 "cbrt")
|
||
|
||
;; Integer magnitude, one per width because there are no generics over the
|
||
;; numeric types and min and max are builtins rather than functions, so a
|
||
;; single abs is not expressible today.
|
||
;;
|
||
;; The most negative value of each width has no positive counterpart, and this
|
||
;; does not special-case it: the subtraction is the same subtraction written
|
||
;; anywhere else and meets whatever the build's overflow rule is. Saturating
|
||
;; to the maximum would be a wrong answer returned quietly, which is the one
|
||
;; thing this file does not do.
|
||
(defn abs-i32 [x i32] i32
|
||
(if (< x 0) (- 0 x) x))
|
||
|
||
(defn abs-i64 [x i64] i64
|
||
(if (< x 0) (- 0 x) x))
|
||
|
||
;; pi and tau at both widths, because a defconst has a type and a cast between
|
||
;; them is where digits go missing. tau is 2pi and is written out rather than
|
||
;; multiplied, so the f32 one is the nearest f32 to tau and not twice the
|
||
;; nearest f32 to pi — which is the same number here and is not guaranteed to
|
||
;; be for the derived form in general.
|
||
;;
|
||
;; Both are given to more digits than either width holds. That is deliberate:
|
||
;; the literal is rounded once, by the compiler, to the nearest value of the
|
||
;; declared type, which is the best available answer and is the same answer on
|
||
;; both targets.
|
||
(defconst pi-f32 f32 3.14159265358979323846)
|
||
(defconst pi-f64 f64 3.14159265358979323846)
|
||
(defconst tau-f32 f32 6.28318530717958647692)
|
||
(defconst tau-f64 f64 6.28318530717958647692)
|
||
|
||
;; ── Byte classes ──────────────────────────────────────────────────────
|
||
;;
|
||
;; ASCII only, and deliberately: a byte is a byte here, there is no code point
|
||
;; type, and a UTF-8 continuation byte is not a digit under any locale. Both
|
||
;; exist because something below needs them — parse-f64 the first, trim the
|
||
;; second — and both are what a caller writing a tokenizer reaches for anyway.
|
||
|
||
(defn digit? [b u8] bool
|
||
(and (>= b \0) (<= b \9)))
|
||
|
||
(defn space? [b u8] bool
|
||
(or (= b \space) (= b \tab) (= b \newline) (= b \return)))
|
||
|
||
;; ── More of the bytes family ──────────────────────────────────────────
|
||
|
||
;; Substring search, first occurrence. The length test is first and returns
|
||
;; before the loop, so a needle longer than the haystack answers None rather
|
||
;; than building a slice that runs off the end. An empty needle is Some 0,
|
||
;; which is the answer that makes (index-of-bytes s p) agree with
|
||
;; (starts-with? s p) on every p.
|
||
;;
|
||
;; Naive, O(n·m), and that is the deliberate choice: Boyer–Moore wants a skip
|
||
;; table, which is an array sized by the needle, which is an allocation.
|
||
(defn index-of-bytes [s [u8] p [u8]] (Option i32)
|
||
(when (> (len p) (len s))
|
||
(return None))
|
||
(let [last (- (len s) (len p))
|
||
i 0]
|
||
(while (<= i last)
|
||
(when (bytes=? (slice s i (+ i (len p))) p)
|
||
(return (Some i)))
|
||
(set i (+ i 1))))
|
||
None)
|
||
|
||
;; Returns a slice *of the input*, which is the whole reason trim can exist
|
||
;; without an allocator: there is no new storage, only a narrower view of the
|
||
;; caller's. It follows that the result dies with its owner, and that trimming
|
||
;; does not modify anything.
|
||
;;
|
||
;; The two loops both test (< lo hi), so an all-whitespace input walks lo up
|
||
;; to hi and stops there, and the result is the empty slice. Without that test
|
||
;; lo would pass hi and (slice s lo hi) would be a reversed range, which traps.
|
||
(defn trim [s [u8]] [u8]
|
||
(let [lo 0
|
||
hi (len s)]
|
||
(while (and (< lo hi) (space? (at s lo)))
|
||
(set lo (+ lo 1)))
|
||
(while (and (< lo hi) (space? (at s (- hi 1))))
|
||
(set hi (- hi 1)))
|
||
(slice s lo hi)))
|
||
|
||
;; The grammar is Flan's and the rounding is libc's, which is a split and not
|
||
;; a dodge. parse-i64 is entirely Flan because strtoll's *answers* are wrong
|
||
;; for a caller — 0 for "", 0 for "abc", 12 for "12x" — and reproducing
|
||
;; correct-to-the-last-bit decimal-to-binary conversion is a different problem
|
||
;; from rejecting junk. So this validates the whole slice first, and only a
|
||
;; slice that is entirely a number is handed to bytes->f64; every string this
|
||
;; returns Some for is one strtod converts exactly, correctly rounded, and
|
||
;; identically everywhere, because that much IEEE-754 requires.
|
||
;;
|
||
;; The locale worry that keeps parse-i64 in Flan does apply to strtod's
|
||
;; decimal point — and is moot here because nothing in the runtime calls
|
||
;; setlocale, so the program stays in the C locale for its whole life. If that
|
||
;; ever stops being true this function is the thing that breaks.
|
||
;;
|
||
;; Accepts [+-]? digits [. digits] [eE [+-] digits], needing at least one
|
||
;; mantissa digit; refuses "", ".", "1e", "nan", "0x10", " 1" and "1 ". The
|
||
;; 511 cap is flan_bytes_to_f64's buffer: past it the shim truncates, and a
|
||
;; validator that said yes to 600 digits would be approving a different
|
||
;; number than the one strtod reads.
|
||
(defn parse-f64 [s [u8]] (Option f64)
|
||
(let [i 0
|
||
digits 0]
|
||
(when (or (= (len s) 0) (> (len s) 511))
|
||
(return None))
|
||
(when (or (= (at s 0) \-) (= (at s 0) \+))
|
||
(set i 1))
|
||
(while (and (< i (len s)) (digit? (at s i)))
|
||
(set i (+ i 1))
|
||
(set digits (+ digits 1)))
|
||
(when (and (< i (len s)) (= (at s i) \.))
|
||
(set i (+ i 1))
|
||
(while (and (< i (len s)) (digit? (at s i)))
|
||
(set i (+ i 1))
|
||
(set digits (+ digits 1))))
|
||
(when (= digits 0)
|
||
(return None)) ; "." and "+" and "e5" are not numbers
|
||
(when (and (< i (len s)) (or (= (at s i) \e) (= (at s i) \E)))
|
||
(set i (+ i 1))
|
||
(when (and (< i (len s)) (or (= (at s i) \-) (= (at s i) \+)))
|
||
(set i (+ i 1)))
|
||
(let [e 0]
|
||
(while (and (< i (len s)) (digit? (at s i)))
|
||
(set i (+ i 1))
|
||
(set e (+ e 1)))
|
||
(when (= e 0)
|
||
(return None)))) ; a lone exponent marker
|
||
;; Trailing junk is the case strtod is silent about, so the position has
|
||
;; to land exactly on the end.
|
||
(if (= i (len s)) (Some (bytes->f64 s)) None)))
|
||
|
||
;; ── UTF-8 ─────────────────────────────────────────────────────────────
|
||
;;
|
||
;; Ported from Odin's core/unicode/utf8/utf8.odin, which is the one corner of
|
||
;; a string library that is allocation-free by construction: decoding is
|
||
;; classification, and every answer it gives is a number. Everything else in
|
||
;; Odin's core/strings takes `allocator := context.allocator`, which is why
|
||
;; this corner came first and the rest waited; most of that rest is ported now
|
||
;; and lives in the building section below. core/fmt is still absent, and the
|
||
;; reason it stays absent is not allocation — see the refusal list at the foot
|
||
;; of this file.
|
||
;;
|
||
;; Odin's 256-entry accept_sizes table becomes a cond over the lead byte here.
|
||
;; The table is the cache-friendly form and the cond is the one you can check
|
||
;; by reading, and nothing in a game decodes UTF-8 in a hot loop — DrawText
|
||
;; hands the bytes straight to raylib.
|
||
;;
|
||
;; The four rules that table encodes, and which a hand-written decoder gets
|
||
;; wrong one at a time:
|
||
;;
|
||
;; 0x80..0xc1 never a lead byte. 0x80..0xbf are continuation bytes, and
|
||
;; 0xc0 and 0xc1 could only ever begin an *overlong* two-byte
|
||
;; spelling of an ASCII character — the encoding that lets
|
||
;; "\xc0\xaf" smuggle a "/" past a check for one.
|
||
;; 0xe0 second byte 0xa0..0xbf and not 0x80..0xbf; the low half is
|
||
;; the overlong three-byte range.
|
||
;; 0xed second byte 0x80..0x9f. The high half is U+D800..U+DFFF,
|
||
;; the UTF-16 surrogates, which are not scalar values.
|
||
;; 0xf0, 0xf4 second byte 0x90..0xbf and 0x80..0x8f: overlong below,
|
||
;; and past U+10FFFF above. 0xf5..0xff lead nothing at all.
|
||
;;
|
||
;; A rune is an i32 and not a type of its own. That is Odin's answer too —
|
||
;; its `rune` is a four-byte integer distinguished only by a flag on the
|
||
;; basic-type row (src/types.cpp, the Basic_rune entry) — so nothing in the
|
||
;; checker has to learn a new type for any of this.
|
||
|
||
;; One deliberate divergence from Odin, and it is the parse-i64 argument over
|
||
;; again. Odin's decode_rune answers RUNE_ERROR — U+FFFD — for malformed
|
||
;; bytes, and U+FFFD is a perfectly real code point that a well-formed string
|
||
;; may contain, so a caller cannot tell a decoded replacement character from a
|
||
;; failure to decode. This carries `ok` instead, and leaves `code` 0 when it
|
||
;; is false.
|
||
;;
|
||
;; `width` is 1 on a malformed byte and 0 only for an empty input. That is
|
||
;; Odin's rule and it is load-bearing rather than cosmetic: every loop below
|
||
;; advances by `width`, so a 0 there on a bad byte is an infinite loop, not a
|
||
;; wrong number.
|
||
(defstruct Rune [code i32 width i32 ok bool])
|
||
|
||
(defn rune-start? [b u8] bool
|
||
(!= (bit-and b 0xc0) 0x80))
|
||
|
||
(defn decode-rune [s [u8]] Rune
|
||
(when (= (len s) 0)
|
||
(return (Rune {.code 0 .width 0 .ok false})))
|
||
(let [b0 (at s 0)]
|
||
(when (< b0 0x80)
|
||
(return (Rune {.code (i32 b0) .width 1 .ok true})))
|
||
;; size 0 means "this byte cannot lead"; lo/hi are the *second* byte's
|
||
;; accepted range, which is the only place the overlong and surrogate
|
||
;; rules live. Bytes three and four are always 0x80..0xbf.
|
||
(let [size 0
|
||
lo (u8 0x80)
|
||
hi (u8 0xbf)]
|
||
(cond
|
||
(< b0 0xc2) (set size 0)
|
||
(<= b0 0xdf) (set size 2)
|
||
(= b0 0xe0) (do (set size 3) (set lo (u8 0xa0)))
|
||
(<= b0 0xec) (set size 3)
|
||
(= b0 0xed) (do (set size 3) (set hi (u8 0x9f)))
|
||
(<= b0 0xef) (set size 3)
|
||
(= b0 0xf0) (do (set size 4) (set lo (u8 0x90)))
|
||
(<= b0 0xf3) (set size 4)
|
||
(= b0 0xf4) (do (set size 4) (set hi (u8 0x8f)))
|
||
:else (set size 0))
|
||
(when (= size 0)
|
||
(return (Rune {.code 0 .width 1 .ok false})))
|
||
;; A sequence cut off by the end of the slice. Width 1, so a caller
|
||
;; scanning a buffer boundary makes progress instead of stalling.
|
||
(when (> size (len s))
|
||
(return (Rune {.code 0 .width 1 .ok false})))
|
||
(let [b1 (at s 1)]
|
||
(when (or (< b1 lo) (> b1 hi))
|
||
(return (Rune {.code 0 .width 1 .ok false})))
|
||
(when (= size 2)
|
||
(return (Rune {.code (bit-or (<< (i32 (bit-and b0 0x1f)) 6)
|
||
(i32 (bit-and b1 0x3f)))
|
||
.width 2 .ok true})))
|
||
(let [b2 (at s 2)]
|
||
(when (or (< b2 0x80) (> b2 0xbf))
|
||
(return (Rune {.code 0 .width 1 .ok false})))
|
||
(when (= size 3)
|
||
(return (Rune {.code (bit-or (bit-or (<< (i32 (bit-and b0 0x0f)) 12)
|
||
(<< (i32 (bit-and b1 0x3f)) 6))
|
||
(i32 (bit-and b2 0x3f)))
|
||
.width 3 .ok true})))
|
||
(let [b3 (at s 3)]
|
||
(when (or (< b3 0x80) (> b3 0xbf))
|
||
(return (Rune {.code 0 .width 1 .ok false})))
|
||
(Rune {.code (bit-or (bit-or (<< (i32 (bit-and b0 0x07)) 18)
|
||
(bit-or (<< (i32 (bit-and b1 0x3f)) 12)
|
||
(<< (i32 (bit-and b2 0x3f)) 6)))
|
||
(i32 (bit-and b3 0x3f)))
|
||
.width 4 .ok true})))))))
|
||
|
||
;; Decode at a byte offset. None when the offset is not on a rune boundary or
|
||
;; the bytes there are malformed, which is stricter than Odin's rune_at — that
|
||
;; one hands back RUNE_ERROR and the caller carries on with a wrong character.
|
||
(defn rune-at [s [u8] i i32] (Option i32)
|
||
(if (or (< i 0) (>= i (len s)))
|
||
None
|
||
(let [r (decode-rune (slice s i (len s)))]
|
||
(if (.ok r) (Some (.code r)) None))))
|
||
|
||
;; Counted through decode-rune rather than through a second walk of its own.
|
||
;; Odin keeps a separate rune_count_in_bytes that re-implements the size
|
||
;; table; two copies of that classification is two places for the surrogate
|
||
;; rule to be right in only one of them.
|
||
;;
|
||
;; A malformed byte counts as one, which is what a replacement-character
|
||
;; renderer would draw, so this agrees with what the screen shows.
|
||
(defn rune-count [s [u8]] i32
|
||
(let [i 0
|
||
n 0]
|
||
(while (< i (len s))
|
||
(let [r (decode-rune (slice s i (len s)))]
|
||
(set i (+ i (.width r)))
|
||
(set n (+ n 1))))
|
||
n))
|
||
|
||
(defn valid-utf8? [s [u8]] bool
|
||
(let [i 0]
|
||
(while (< i (len s))
|
||
(let [r (decode-rune (slice s i (len s)))]
|
||
(when (not (.ok r))
|
||
(return false))
|
||
(set i (+ i (.width r)))))
|
||
true))
|
||
|
||
;; How many bytes this code point encodes to, or None if it is not a scalar
|
||
;; value. Odin's rune_size answers -1 for the refusals; a sentinel index is
|
||
;; exactly what index-of avoids above, so this is an Option like the rest
|
||
;; of the file.
|
||
(defn rune-size [code i32] (Option i32)
|
||
(cond
|
||
(< code 0) None
|
||
(<= code 0x7f) (Some 1)
|
||
(<= code 0x7ff) (Some 2)
|
||
(and (>= code 0xd800) (<= code 0xdfff)) None
|
||
(<= code 0xffff) (Some 3)
|
||
(<= code 0x10ffff) (Some 4)
|
||
:else None))
|
||
|
||
;; Encoding is the one operation here whose result is not a slice of its
|
||
;; input, because the bytes it makes existed nowhere before. With no allocator
|
||
;; the only shape left is Odin's own allocation-free one — strings.Builder
|
||
;; built by builder_from_bytes over a caller's backing array (builder.odin,
|
||
;; builder_from_bytes: "Uses Nil Allocator - Does NOT allocate") — reduced to
|
||
;; its essential case: write into a buffer the caller owns, and say how much
|
||
;; was written.
|
||
;;
|
||
;; None rather than a partial write when the buffer is short, and None rather
|
||
;; than Odin's silent substitution of U+FFFD for an invalid rune. Odin's
|
||
;; encode_rune rewrites a surrogate or an out-of-range value to the
|
||
;; replacement character and reports success; the caller then finds three
|
||
;; bytes of U+FFFD in its buffer and no indication that it asked for something
|
||
;; else. Nothing is written at all when this answers None.
|
||
(defn encode-rune! [dst [u8] code i32] (Option i32)
|
||
(match (rune-size code)
|
||
None None
|
||
(Some w)
|
||
(if (> w (len dst))
|
||
None
|
||
(do
|
||
(cond
|
||
(= w 1)
|
||
(set (at dst 0) (u8 code))
|
||
(= w 2)
|
||
(do (set (at dst 0) (u8 (bit-or 0xc0 (>> code 6))))
|
||
(set (at dst 1) (u8 (bit-or 0x80 (bit-and code 0x3f)))))
|
||
(= w 3)
|
||
(do (set (at dst 0) (u8 (bit-or 0xe0 (>> code 12))))
|
||
(set (at dst 1) (u8 (bit-or 0x80 (bit-and (>> code 6) 0x3f))))
|
||
(set (at dst 2) (u8 (bit-or 0x80 (bit-and code 0x3f)))))
|
||
:else
|
||
(do (set (at dst 0) (u8 (bit-or 0xf0 (>> code 18))))
|
||
(set (at dst 1) (u8 (bit-or 0x80 (bit-and (>> code 12) 0x3f))))
|
||
(set (at dst 2) (u8 (bit-or 0x80 (bit-and (>> code 6) 0x3f))))
|
||
(set (at dst 3) (u8 (bit-or 0x80 (bit-and code 0x3f))))))
|
||
(Some w)))))
|
||
|
||
;; ── Splitting ─────────────────────────────────────────────────────────
|
||
;;
|
||
;; The iterator, which owns nothing. `split` returning a sequence of fields has
|
||
;; to allocate that sequence, and it does — it is in the building section below
|
||
;; — but this stays the right call whenever you do not want to own the result:
|
||
;; it is Odin's split_by_byte_iterator (strings.odin), a cursor holding the
|
||
;; rest of the input and handing back one field at a time. Every field is a
|
||
;; slice *of the caller's bytes*; nothing is copied, nothing is owned, and
|
||
;; there is no free to remember. `split` is built on exactly this.
|
||
;;
|
||
;; One divergence, and it is a wart of Odin's rather than a decision. Odin's
|
||
;; iterator stops on an empty final field, so "a,b," iterates a and b and the
|
||
;; trailing empty field is lost — while Odin's own allocating strings.split
|
||
;; returns ["a", "b", ""] for the same input. The two disagree. This follows
|
||
;; split: n separators always yield n+1 fields, an empty input yields one
|
||
;; empty field, and `rest` is exhausted only after the last one is taken. That
|
||
;; is the rule you can state without exceptions, and the one a caller counting
|
||
;; comma-separated columns needs.
|
||
(defstruct Split [rest [u8] sep u8 more bool])
|
||
|
||
(defn split-on-byte [s [u8] sep u8] Split
|
||
(Split {.rest s .sep sep .more true}))
|
||
|
||
(defn split-next! [it (Ptr Split)] (Option [u8])
|
||
(when (not (.more it))
|
||
(return None))
|
||
(match (index-of (.rest it) (.sep it))
|
||
(Some i)
|
||
(let [field (slice (.rest it) 0 i)]
|
||
(set (.rest it) (slice (.rest it) (+ i 1) (len (.rest it))))
|
||
(Some field))
|
||
None
|
||
(let [field (.rest it)]
|
||
(set (.more it) false)
|
||
(set (.rest it) (slice (.rest it) (len (.rest it)) (len (.rest it))))
|
||
(Some field))))
|
||
|
||
;; ── ASCII case ────────────────────────────────────────────────────────
|
||
;;
|
||
;; Byte in, byte out, and *not* a function over a slice. Odin's to_lower and
|
||
;; to_upper both allocate a new string (core/strings/conversion.odin) and so do
|
||
;; the ones in the building section below; these are the forms that allocate
|
||
;; nothing, and they stay the right call when a copy is not wanted — folding a
|
||
;; comparison over two inputs beats lowering both and comparing. What is *not*
|
||
;; on offer is the third shape, lowering a [u8] in place, and it is worth
|
||
;; saying why rather than shipping it. A string
|
||
;; literal is emitted `private unnamed_addr constant` (emit.ml), so (bytes
|
||
;; "Hello") is a [u8] pointing straight into read-only memory. An in-place
|
||
;; lower-ascii! type checks against that slice, and what happens next depends
|
||
;; on the optimiser — which is the worst of the available answers. Measured,
|
||
;; with (set (at (bytes "Hi") 0) \h):
|
||
;;
|
||
;; -O0 the store is emitted against the constant and the program takes
|
||
;; SIGSEGV.
|
||
;; -O2 LLVM deletes the store as undefined behaviour and the program
|
||
;; carries on and prints "Hi".
|
||
;;
|
||
;; So the same source either dies or silently does nothing depending on a
|
||
;; flag, and the -O2 half is the quiet-wrongness class this file keeps
|
||
;; refusing elsewhere. Given a byte function instead, a caller that really
|
||
;; does own its buffer writes the two-line loop itself over storage it can
|
||
;; see the declaration of.
|
||
;;
|
||
;; ASCII only, and only the 26 letters: case outside ASCII is not a byte
|
||
;; operation at all — it is per-code-point, it is not length-preserving (ß
|
||
;; upcases to SS), and it is locale-dependent (Turkish dotless ı). A byte
|
||
;; table that pretended otherwise would be wrong in the quiet way.
|
||
(defn lower-ascii [b u8] u8
|
||
(if (and (>= b \A) (<= b \Z)) (+ b 32) b))
|
||
|
||
(defn upper-ascii [b u8] u8
|
||
(if (and (>= b \a) (<= b \z)) (- b 32) b))
|
||
|
||
;; Case-insensitive comparison as a fold over both inputs, which is the useful
|
||
;; half of to_lower and needs no storage at all: comparing two lowered copies
|
||
;; is what a caller wanted, and this is that answer without either copy.
|
||
(defn bytes-ci=? [a [u8] b [u8]] bool
|
||
(if (!= (len a) (len b))
|
||
false
|
||
(do
|
||
(dotimes [i (len a)]
|
||
(when (!= (lower-ascii (at a i)) (lower-ascii (at b i)))
|
||
(return false)))
|
||
true)))
|
||
|
||
;; ── Ordering byte slices, and sorting them ────────────────────────────
|
||
;;
|
||
;; The third element type the slice family covers, and the one a caller of
|
||
;; `split` actually has: a [[u8]] of fields, wanting to come out in order.
|
||
;;
|
||
;; The order is bytewise-lexicographic — memcmp's, and the one every sane
|
||
;; sorted format uses. It is explicitly *not* alphabetical and not a collation:
|
||
;; "Zebra" sorts before "apple" because 'Z' is 90 and 'a' is 97, and a
|
||
;; non-ASCII byte sorts by its UTF-8 encoding, which for code points happens to
|
||
;; agree with code-point order and for anything a human would call alphabetical
|
||
;; does not. A locale-aware comparison is not a byte operation at all, for the
|
||
;; same reasons the ASCII-case note above gives.
|
||
;;
|
||
;; The comparison is over u8 and therefore unsigned, which is the bug a version
|
||
;; written over a signed byte type has: 0x80 would compare *below* 0x00 and
|
||
;; every multi-byte character would sort before every ASCII one.
|
||
;;
|
||
;; A prefix sorts before what extends it — "ab" before "abc" — which falls out
|
||
;; of running to the shorter length and then comparing lengths, and is the case
|
||
;; a loop written to (len a) alone reads off the end for.
|
||
(defn bytes<? [a [u8] b [u8]] bool
|
||
(let [n (min (len a) (len b))]
|
||
(dotimes [i n]
|
||
(when (!= (at a i) (at b i))
|
||
(return (< (at a i) (at b i)))))
|
||
(< (len a) (len b))))
|
||
|
||
;; sort-by! with the comparison written in, over the same in-place contract:
|
||
;; the *slices* move, never the bytes they point at, so this sorts a [[u8]] of
|
||
;; fields borrowed from one buffer without touching the buffer. Stable, and
|
||
;; here that is observable — two equal fields are two distinct slices of
|
||
;; different parts of the input, and a caller can see which one came first.
|
||
;;
|
||
;; It keeps a name of its own rather than collapsing into sort!, and the
|
||
;; reason is the point of the predicates: a [u8] is not ordered? and cannot
|
||
;; be, because < is defined on machine numbers and comparing two slices
|
||
;; lexicographically is a loop and not an instruction. bytes<? is that loop.
|
||
;; So this is the shape a generic takes when the operation it needs is not a
|
||
;; primitive: pass it in.
|
||
(defn sort-bytes! [s [[u8]]] ()
|
||
(sort-by! s (fn [a b] (bytes<? a b))))
|
||
|
||
;; ── Building bytes, which is the tier that needed an allocator ────────
|
||
;;
|
||
;; Everything above this line is slice-based and allocation-free, because when
|
||
;; it was written there was nothing to allocate from. Everything below it
|
||
;; *returns new storage*, which is the whole difference, and there are three
|
||
;; rules that hold for all of it.
|
||
;;
|
||
;; **The result is owned and the caller frees it.** Each of these hands back a
|
||
;; (Vec u8) or a (Vec [u8]), which is move-only: it goes with the call that
|
||
;; takes it, and nothing is released at scope exit — not at the end of a let,
|
||
;; not at the end of a function (spec-memory.md, "When storage is released").
|
||
;; A caller writes (free v) or lets a (free-all a) take the whole region.
|
||
;;
|
||
;; **The allocator is the context's, and `with-allocator` is the override.**
|
||
;; spec-memory.md makes allocation use the current implicit allocator and
|
||
;; forbids falling back to a hidden global one. (vec-new) and (map-new) take an
|
||
;; optional trailing allocator because the checker builds them; a Flan defn has
|
||
;; fixed arity and cannot, so the choice here was an allocator parameter on
|
||
;; every one of these signatures or none. None: a caller wanting a frame arena
|
||
;; writes (with-allocator a (join parts sep)) and the Vec records the arena, so
|
||
;; the free and the clone never need it named again.
|
||
;;
|
||
;; **No Result, anywhere.** Running out of storage signals StorageExhausted
|
||
;; under a `retry` restart and no allocating operation returns an error
|
||
;; (spec-memory.md, "Allocation failure"), so these signatures say what they
|
||
;; produce and nothing about how they might fail.
|
||
|
||
;; The builder. It is not a type: strings.Builder in Odin is a struct wrapping
|
||
;; a [dynamic]u8, and here the (Vec u8) *is* that, with push already on it — a
|
||
;; wrapper would be a move-only struct owning a Vec whose only method is the
|
||
;; one the Vec already has. What was actually missing is appending a run of
|
||
;; bytes rather than one, and that is this.
|
||
;;
|
||
;; It takes a (Ptr (Vec u8)) and not a (Vec u8), and the difference is not
|
||
;; style: a Vec parameter *moves*, so (append! b s) taking one by value would
|
||
;; consume the caller's builder on the first call and refuse the second.
|
||
(defn append! [b (Ptr (Vec u8)) s [u8]] ()
|
||
(dotimes [i (len s)]
|
||
(push (deref b) (at s i))))
|
||
|
||
;; The two number appends, and they are the reason this shape is worth having
|
||
;; rather than a formatter that answers a slice. i64->bytes and f64->bytes
|
||
;; render into one shared static buffer in the runtime, so two of their results
|
||
;; cannot be held at once and (concat [(i64->bytes a) (i64->bytes b)]) is two
|
||
;; views of the same bytes — the second call overwrote the first. These copy
|
||
;; out of that buffer before returning, so the hazard ends at the call: a
|
||
;; builder can hold as many numbers as it likes.
|
||
(defn append-i64! [b (Ptr (Vec u8)) n i64] ()
|
||
(append! b (i64->bytes n)))
|
||
|
||
(defn append-f64! [b (Ptr (Vec u8)) x f64] ()
|
||
(append! b (f64->bytes x)))
|
||
|
||
;; concat and join. Both take a slice of slices, which is the shape a caller
|
||
;; already has: an array literal of them, [(bytes "a") (bytes b)], slices to a
|
||
;; [[u8]] and copies nothing.
|
||
;;
|
||
;; join with an empty separator is concat, and concat is here anyway because
|
||
;; the empty (bytes "") a caller would have to write is the kind of argument
|
||
;; that reads like a mistake at the call site.
|
||
(defn concat [parts [[u8]]] (Vec u8)
|
||
(let [b (vec-new u8)]
|
||
(dotimes [i (len parts)]
|
||
(append! (addr b) (at parts i)))
|
||
b))
|
||
|
||
;; n parts yield n-1 separators, and the empty slice of parts yields the empty
|
||
;; result rather than a leading separator — which is the off-by-one a join
|
||
;; written as "append part then separator, then chop the tail" gets wrong on
|
||
;; exactly that input, because there is no tail to chop.
|
||
(defn join [parts [[u8]] sep [u8]] (Vec u8)
|
||
(let [b (vec-new u8)]
|
||
(dotimes [i (len parts)]
|
||
(when (> i 0)
|
||
(append! (addr b) sep))
|
||
(append! (addr b) (at parts i)))
|
||
b))
|
||
|
||
(defn repeat-bytes [s [u8] n i32] (Vec u8)
|
||
(let [b (vec-new u8)]
|
||
(dotimes [i n]
|
||
(append! (addr b) s))
|
||
b))
|
||
|
||
;; The allocating halves of the ASCII case pair. The note above lower-ascii
|
||
;; explains why lowering a [u8] *in place* is a trap — a string literal is
|
||
;; emitted into .rodata, so the store either segfaults at -O0 or is deleted at
|
||
;; -O2 — and this is the shape that has no such hole: the bytes it writes are
|
||
;; its own.
|
||
(defn to-lower [s [u8]] (Vec u8)
|
||
(let [b (vec-new u8)]
|
||
(dotimes [i (len s)]
|
||
(push b (lower-ascii (at s i))))
|
||
b))
|
||
|
||
(defn to-upper [s [u8]] (Vec u8)
|
||
(let [b (vec-new u8)]
|
||
(dotimes [i (len s)]
|
||
(push b (upper-ascii (at s i))))
|
||
b))
|
||
|
||
;; Every non-overlapping occurrence, left to right, which is the rule that
|
||
;; makes (replace-bytes (bytes "aaa") (bytes "aa") (bytes "b")) answer "ba" and
|
||
;; not "bb" or "b".
|
||
;;
|
||
;; An empty `from` matches nothing and the result is a copy of the input. The
|
||
;; alternative reading — that it matches at every position — is what turns this
|
||
;; into an infinite loop, and Odin's replace guards the same case for the same
|
||
;; reason.
|
||
;;
|
||
;; The guard is an `if` and not an early `(return b)`, which is not a style
|
||
;; choice: returning a Vec *moves* it, and the move analysis is a dead set over
|
||
;; the whole function, so a `return b` on one branch kills the binding for the
|
||
;; `b` at the foot of the other. One exit, one move.
|
||
(defn replace-bytes [s [u8] from [u8] to [u8]] (Vec u8)
|
||
(let [b (vec-new u8)
|
||
i 0]
|
||
(if (= (len from) 0)
|
||
(append! (addr b) s)
|
||
(while (< i (len s))
|
||
(match (index-of-bytes (slice s i (len s)) from)
|
||
(Some k)
|
||
(do
|
||
(append! (addr b) (slice s i (+ i k)))
|
||
(append! (addr b) to)
|
||
(set i (+ i k (len from))))
|
||
None
|
||
(do
|
||
(append! (addr b) (slice s i (len s)))
|
||
(set i (len s))))))
|
||
b))
|
||
|
||
;; A (Vec [u8]) cannot be written at a let, and this one-line function is where
|
||
;; the type is said instead. (vec-new) takes its element type as a *bare
|
||
;; symbol* — check.ml's vec_new_elem resolves one name and nothing else — so
|
||
;; (vec-new [u8]) is not accepted, and a let has no type annotation to say it
|
||
;; the other way. A return type does say it. That is a compiler gap rather than
|
||
;; a language decision, and it is written down in NEXT.md.
|
||
(defn slices-new [] (Vec [u8]) (vec-new))
|
||
|
||
;; split, which the file used to refuse by name. The fields are slices *of the
|
||
;; input* and not copies, so nothing here owns bytes and the result dies with
|
||
;; whatever `s` pointed at — a (Vec (Vec u8)) is the shape that would own them
|
||
;; and it is refused outright, because a Vec's elements are copied and released
|
||
;; bytewise and an owner cannot survive that.
|
||
;;
|
||
;; The rule is split-on-byte's, unchanged and worth restating: n separators
|
||
;; always yield n+1 fields, so the empty input yields one empty field and a
|
||
;; trailing separator yields a trailing empty one. That is Odin's allocating
|
||
;; strings.split and not Odin's iterator, which disagree with each other.
|
||
(defn split [s [u8] sep u8] (Vec [u8])
|
||
(let [v (slices-new)
|
||
it (split-on-byte s sep)
|
||
going true]
|
||
(while going
|
||
(match (split-next! (addr it))
|
||
(Some f) (push v f)
|
||
None (set going false)))
|
||
v))
|
||
|
||
;; ── A number with a precision ─────────────────────────────────────────
|
||
;;
|
||
;; The one formatting job the runtime cannot do. f64->bytes is snprintf "%g",
|
||
;; which is six significant digits and switches to exponent notation on its
|
||
;; own: a frame time of 0.0166667 is what a caller wanted two decimals of, and
|
||
;; 1.23457e+06 is what a score looks like once it passes a million. There is no
|
||
;; precision to pass it, and there cannot be — it renders into one shared
|
||
;; static buffer in the runtime, which is the same reason two of its results
|
||
;; cannot be held at once.
|
||
;;
|
||
;; This returns a Vec, so neither problem is inherited. It uses i64->bytes
|
||
;; twice and the two calls are strictly sequential — the integer part is copied
|
||
;; into the Vec before the fraction is rendered — which is the discipline the
|
||
;; shared buffer requires and the one append-i64! exists to make automatic.
|
||
;;
|
||
;; Half away from zero, the same rule round-f32 follows, applied at the last
|
||
;; digit kept. That is not bit-for-bit printf: printf rounds the *binary* value
|
||
;; to nearest-even at the decimal digit, and this rounds the decimal expansion
|
||
;; half-up, so a value sitting exactly on a half — 0.999995 at five places —
|
||
;; comes out 1.00000 here and may come out 0.99999 there. Choosing the rule the
|
||
;; rest of this file already uses beats matching a libc whose answer is not the
|
||
;; same on every target anyway.
|
||
;;
|
||
;; Precision is clamped to 0..9 rather than refused. 10^9 is the largest power
|
||
;; of ten that leaves room in the f64 product below, and a precision argument
|
||
;; is almost always a literal, so a refusal would be a run-time condition for a
|
||
;; mistake visible in the source.
|
||
;;
|
||
;; It is the `clamp` macro two hundred lines up, and this is the call that
|
||
;; proves a prelude function may call a prelude macro — which it could not
|
||
;; until macro.ml grew its bootstrap reduction. The cycle it breaks: a macro
|
||
;; module is compiled *from* the prelude, so a prelude function calling a macro
|
||
;; would have to be compiled into the very module that expands it. For that one
|
||
;; build the prelude drops every defn that reaches a macro, this one included.
|
||
;; A prelude *macro* may still not call a macro, and says so by name.
|
||
;;
|
||
;; Three inputs do not have decimal expansions and are named before the cast
|
||
;; that would be undefined on them: NaN, which fails every comparison and is
|
||
;; therefore tested with (not (= x x)) and nothing else, and the two
|
||
;; infinities, which are the values satisfying (= x (* x 2.0)) away from zero.
|
||
;; A magnitude past 9e18 has no fractional bits left at all and would not fit
|
||
;; in the i64 the integer part is carried in, so it falls back to f64->bytes —
|
||
;; which is the honest answer there rather than an approximation of one.
|
||
;;
|
||
;; -0.0 prints as "0.00": the sign test is (< x 0.0), which -0.0 fails. A
|
||
;; caller that needs the sign of a zero should not be reading it out of text.
|
||
(defn format-f64 [x f64 prec i32] (Vec u8)
|
||
(let [b (vec-new u8)
|
||
p (clamp prec 0 9)]
|
||
(cond
|
||
(not (= x x))
|
||
(append! (addr b) (bytes "nan"))
|
||
|
||
(and (= x (* x 2.0)) (!= x 0.0))
|
||
(append! (addr b) (bytes (if (< x 0.0) "-inf" "inf")))
|
||
|
||
:else
|
||
(let [neg (< x 0.0)
|
||
m (if neg (- 0.0 x) x)]
|
||
(if (>= m 9.0e18)
|
||
(append! (addr b) (f64->bytes x))
|
||
(let [scale (i64 1)]
|
||
(dotimes [i p]
|
||
(set scale (* scale 10)))
|
||
;; The split is exact: (i64 m) truncates toward zero and m is
|
||
;; non-negative here, and the subtraction of an integer from the
|
||
;; float it came from is exact at every magnitude an f64 can hold.
|
||
;; Only the scaling below rounds, and it rounds a value already
|
||
;; under 1.
|
||
(let [ip (i64 m)
|
||
fr (i64 (+ (* (- m (f64 ip)) (f64 scale)) 0.5))]
|
||
;; The carry, which is the bug this shape is otherwise written
|
||
;; with: 0.999995 at five places scales to exactly 100000, which
|
||
;; is not a fraction at all — it is the next integer, and without
|
||
;; this line it prints as "0.100000".
|
||
(when (>= fr scale)
|
||
(set fr 0)
|
||
(set ip (+ ip 1)))
|
||
;; The sign goes on separately, because the integer part is a
|
||
;; magnitude: -0.5 at one place has an integer part of 0, and
|
||
;; i64->bytes of 0 has no sign to carry.
|
||
(when neg
|
||
(push b \-))
|
||
(append-i64! (addr b) ip)
|
||
(when (> p 0)
|
||
(push b \.)
|
||
;; Left-padded with zeros to exactly p digits. fr is under
|
||
;; scale by the carry above, so it never needs more, and
|
||
;; without the padding 1.005 at three places prints "1.5".
|
||
(let [d (i64->bytes fr)]
|
||
(dotimes [i (- p (len d))]
|
||
(push b \0))
|
||
(append! (addr b) d))))))))
|
||
b))
|
||
|
||
;; ── Still refused, and what the reason is now ─────────────────────────
|
||
;;
|
||
;; This list used to be one sentence long — every entry needed to produce bytes
|
||
;; that did not exist in its input, and there was no allocator. That sentence
|
||
;; stopped being true when `Vec` landed, and most of the list has moved up into
|
||
;; the building section above: join, concat, split, to-lower, to-upper, repeat
|
||
;; and replace are all written now, and `string-from-bytes` turned out to be
|
||
;; the `string` builtin all along — (string (as-slice v)) is the round trip,
|
||
;; and the layouts being identical is exactly why it is free.
|
||
;;
|
||
;; What is left is refused for four *different* reasons, which is why they are
|
||
;; named separately rather than under one heading.
|
||
;;
|
||
;; pad, center Nothing. These are three lines each over repeat-bytes
|
||
;; and concat, and they are absent only because no
|
||
;; caller has asked. Write them when one does.
|
||
;; format, sprintf A format *string* — Odin's fmt.aprintf family. It
|
||
;; needs variadic arguments of mixed type, which is a
|
||
;; function-value and generics question, not an
|
||
;; allocation one. format-f64 above is the piece of it
|
||
;; that was actually wanted, and `print`/`println` are
|
||
;; already the structural walk over any one value.
|
||
;; map that changes the Generics, and only that. map!, filter, reduce and
|
||
;; element type sort-by! landed the day function values did — see
|
||
;; "The ones that take a function" above — at i32 and
|
||
;; f32, the two element types the rest of that family
|
||
;; covers. A map from [i32] to [f32] is the one shape
|
||
;; that did not come with them, because it is one copy
|
||
;; per *ordered pair* of types rather than per type,
|
||
;; which is where a per-type family stops being honest.
|
||
;; map-keys, map-values Generics — and the reason changed, which is the
|
||
;; point of naming them separately. It used to be the
|
||
;; missing Map iterator; `map-next!` is that iterator
|
||
;; and walking a map is expressible now. What a defn
|
||
;; still cannot say is (defn map-keys [m {K V}] (Vec K)):
|
||
;; a prelude function has to name its types, and there
|
||
;; is no K. The loop is three lines at the call site,
|
||
;; where K is known, and that is where it stays until
|
||
;; there are generics.
|
||
;;
|
||
;; Builder Not refused — declined. strings.Builder in Odin
|
||
;; wraps a [dynamic]u8; here the (Vec u8) *is* that and
|
||
;; already has push, so the struct would be a move-only
|
||
;; wrapper whose only method is the one it wraps. What
|
||
;; was missing was appending a run of bytes, and
|
||
;; `append!` above is that.
|
||
;; ── Files: embedding, slurp and barf ──────────────────────────────────
|
||
;;
|
||
;; One entry per file in an (embed-dir "...") — Odin's Load_Directory_File
|
||
;; (base/runtime/core.odin), which is the same two fields for the same reason:
|
||
;; a directory embed is only useful if you can find one file in it by the name
|
||
;; it had on disk.
|
||
;;
|
||
;; `data` points into the program's own .rodata, exactly as a string literal
|
||
;; does, so an embed costs nothing at run time and nothing at startup. It is
|
||
;; also read-only, and the same trap the ASCII-case note above measures applies
|
||
;; here: a store through it either segfaults at -O0 or is deleted at -O2. To
|
||
;; get a mutable copy, clone the bytes into a Vec.
|
||
(defstruct EmbedFile [name string data [u8]])
|
||
|
||
;; A linear scan, deliberately. A directory embed is tens of entries, the scan
|
||
;; is over names already in cache-warm .rodata, and the alternative — a
|
||
;; compile-time perfect hash — is a build-time map with its own failure modes
|
||
;; that nothing here has asked for. If a program ever embeds thousands of
|
||
;; files, sort-and-bisect is the next step and it does not change this type.
|
||
;;
|
||
;; It takes a slice rather than the array (embed-dir) answers, because an array
|
||
;; length is part of its type and there are no generics: write
|
||
;; (embed-find (slice assets 0 (len assets)) "brush.png").
|
||
(defn embed-find [files [EmbedFile] name string] (Option [u8])
|
||
(dotimes [i (len files)]
|
||
(when (bytes=? (bytes (.name (at files i))) (bytes name))
|
||
(return (Some (.data (at files i))))))
|
||
None)
|
||
|
||
;; The condition slurp and barf signal — spec-conditions.md, and the same shape
|
||
;; StorageExhausted has: a value struct on the signalling frame's stack, fixed
|
||
;; fields, no rendered message. `path` is the path that failed, which is a
|
||
;; string literal or a string the handler itself supplied, so naming it costs
|
||
;; no allocation either.
|
||
;;
|
||
;; One type rather than a family, because conditions have no hierarchy today
|
||
;; (spec-conditions.md §1) and a family would need one handler clause per
|
||
;; member to say "any file error". The parent link NEXT.md decides on is the
|
||
;; answer to that, and it is not built; when it is, these reasons can become
|
||
;; types without any call site changing.
|
||
(defstruct FileError [path string op i32 reason i32])
|
||
|
||
(defconst file-op-read i32 0)
|
||
(defconst file-op-write i32 1)
|
||
|
||
(defconst file-missing i32 1)
|
||
(defconst file-denied i32 2)
|
||
(defconst file-io i32 3)
|
||
;; What `barf` signals on the web target, every time. Decision 2: writing is
|
||
;; desktop-only, and it signals rather than refusing at build time (Flan has no
|
||
;; conditional compilation, so isolating code to desktop is not expressible) or
|
||
;; silently doing nothing (which is how a save file disappears with nothing
|
||
;; said).
|
||
(defconst file-unsupported i32 4)
|
||
|
||
;; ── Form: what a macro takes and what it answers ──────────────────────
|
||
;;
|
||
;; The reader's output, mirrored on the Flan side, because a macro is a
|
||
;; function [Form] -> Form and there is no interpreter: running one means
|
||
;; compiling it and dlopening it into the compiler. So the compiler and the
|
||
;; loaded macro have to agree on the *layout* of a Form, not merely on its
|
||
;; shape. lib/form.ml is the other half of this declaration and the two are
|
||
;; edited together.
|
||
;;
|
||
;; It mirrors Form.value and not Form.t: there is no `loc` field. A macro
|
||
;; cannot invent a source location and should not carry one, so the compiler
|
||
;; stamps the *call site's* location onto every node of what a macro returns.
|
||
;; That is the structural version of "keep the source location of the call
|
||
;; site attached to what a macro produces", and it is what the queued
|
||
;; structured-error work will read.
|
||
;;
|
||
;; Case order is the tag order (docs/BUILT.md, data types), so this list is a layout
|
||
;; contract with lib/expand.ml's marshaller and may not be reordered.
|
||
(defdata Form
|
||
[(Sym [s string])
|
||
(Kw [s string])
|
||
(Int [i i64])
|
||
(Float [x f64])
|
||
(Str [s string])
|
||
(Byte [b i32])
|
||
(List [xs [Form]])
|
||
(Vec [xs [Form]])
|
||
(Map [xs [Form]])])
|
||
|
||
;; The list-building surface quasiquote desugars into. Three functions and no
|
||
;; more: `form-nil` starts one, `form-cons` puts a form on the front, and
|
||
;; `form-append` is what ~@ splices with. Everything else — a vector literal,
|
||
;; a length, an index — is already the language's.
|
||
;;
|
||
;; Each allocates a fresh (Vec Form) and hands back a borrow of it that
|
||
;; outlives the call. That is a leak, on purpose: a macro runs inside the
|
||
;; compiler, its result is read after it returns, and the whole expansion is
|
||
;; bounded by the size of the program being compiled. `drop` is what would
|
||
;; change this, and it does not exist.
|
||
(defn form-nil [] [Form]
|
||
(let [v (vec-new Form)]
|
||
(as-slice v)))
|
||
|
||
(defn form-cons [x Form rest [Form]] [Form]
|
||
(let [v (vec-new Form)]
|
||
(push v x)
|
||
(dotimes [i (len rest)]
|
||
(push v (at rest i)))
|
||
(as-slice v)))
|
||
|
||
(defn form-append [a [Form] b [Form]] [Form]
|
||
(let [v (vec-new Form)]
|
||
(dotimes [i (len a)]
|
||
(push v (at a i)))
|
||
(dotimes [i (len b)]
|
||
(push v (at b i)))
|
||
(as-slice v)))
|
||
|
||
;; The rest of a macro's arguments, which is what a variadic body is: a macro
|
||
;; takes one parameter, the slice of the forms at its call site.
|
||
(defn form-rest [xs [Form] from i32] [Form]
|
||
(let [v (vec-new Form)
|
||
i from]
|
||
(while (< i (len xs))
|
||
(push v (at xs i))
|
||
(set i (+ i 1)))
|
||
(as-slice v)))
|
||
|
||
;; A name no reader can produce. `~` is a delimiter now (it opens an unquote),
|
||
;; so no symbol coming out of read_all can contain one, and a gensym therefore
|
||
;; cannot collide with a name someone wrote. Non-hygienic expansion with an
|
||
;; explicit gensym is the settled decision (plan.org, open decision 2); this is
|
||
;; the escape hatch that makes it liveable.
|
||
;;
|
||
;; The counter lives in the loaded module rather than in the compiler, which is
|
||
;; the one place this departs from NEXT.md's sketch. A module is dlopened once
|
||
;; per compiler process and every macro in a program shares it, so the counter
|
||
;; is process-wide in practice; a second module would restart it, and the day
|
||
;; there is one, the fix is to seed this from the module's index.
|
||
(defvar gensym-n i64 0)
|
||
|
||
(defn gensym [] Form
|
||
(set gensym-n (+ gensym-n 1))
|
||
(let [v (vec-new u8)]
|
||
(push v 126) ; ~
|
||
(push v 103) ; g
|
||
(let [d (i64->bytes gensym-n)]
|
||
(dotimes [i (len d)]
|
||
(push v (at d i))))
|
||
(Form.Sym {.s (string (as-slice v))})))
|
||
|
||
;; ── The first special form to stop being one ──────────────────────────
|
||
;;
|
||
;; plan.org milestone 5 says when, unless, until, cond and dotimes are special
|
||
;; forms only until macros land. This is the one that moved, and it is here to
|
||
;; show that the move is possible and cheap, not because it was the most
|
||
;; valuable of the five: it is the one no other part of the prelude uses, so
|
||
;; moving it cannot make the prelude depend on the expander that compiles it.
|
||
;;
|
||
;; The expansion is exactly what parse.ml built by hand until now -- an if over
|
||
;; (not test) with the body in a do -- so every test written against the
|
||
;; special form is a test of this, unchanged.
|
||
;;
|
||
;; The one thing the compiler could say and this cannot is a reason. A macro
|
||
;; has no error facility: it runs inside the compiler and anything it signals
|
||
;; aborts the compile with no location. So a malformed (unless) answers a name
|
||
;; nothing defines, and the report is "unknown name unless-takes-a-test-and-a-
|
||
;; body" at the call site, which is the right place and the wrong sentence.
|
||
;; That is the next thing a macro needs and it is written down in NEXT.md.
|
||
(defmacro unless [args]
|
||
(if (< (len args) 2)
|
||
`(unless-takes-a-test-and-a-body)
|
||
`(if (not ~(at args 0)) (do ~@(form-rest args 1)))))
|
||
|
||
;; ── into: a fused transformation, and not a transducer ────────────────
|
||
;;
|
||
;; (into xs (vec-new i32) (map double) (filter even?))
|
||
;;
|
||
;; Source, destination, then any number of transforms. It reads as a sentence
|
||
;; — take this, put it there, doing these — and the variadic tail has to trail
|
||
;; anyway, which is the mechanical reason the transforms cannot sit in the
|
||
;; middle.
|
||
;;
|
||
;; **A macro, not transducers.** Transducers compose at run time: function
|
||
;; values, closures, an allocation, and a chain of indirect calls per element.
|
||
;; Rust has no transducers either — it has iterators, which fuse at compile
|
||
;; time through monomorphisation, and that needs generics. A macro reaches the
|
||
;; same place with neither. (map double) expands to (double x) written straight
|
||
;; into the loop body, so the function name is *syntax* and never a value:
|
||
;; there is no intermediate collection at any step, no closure, no generics and
|
||
;; nothing to inline. What it gives up is building a transformation at run time
|
||
;; and passing it around, which is transducers' actual selling point and is
|
||
;; close to useless in a game.
|
||
;;
|
||
;; **The destination is in the form on purpose.** Every collecting operation
|
||
;; here allocates from an explicit allocator, which is a frozen rule in
|
||
;; spec-memory.md. A ->> chain would hide where the result goes; naming the
|
||
;; destination means this macro knows its type, emits the right loop, and the
|
||
;; rule is honoured by construction. `(vec-new i32 a)` names an allocator here
|
||
;; as it does anywhere else, because the destination form is written out
|
||
;; untouched.
|
||
;;
|
||
;; **The destination is a Vec**, because push is what fills it. A Map
|
||
;; destination is refused by push, which says "push takes a (Vec T)" and names
|
||
;; the real problem; there is no second lowering for it and no reason to invent
|
||
;; one before something wants it.
|
||
;;
|
||
;; **Reductions do not share this form**, and that was the open question. (into
|
||
;; xs 0 (map cost) (sum)) reads oddly because zero is not a collection, and the
|
||
;; oddness is the tell: the whole reason the destination sits in the form is
|
||
;; that it is the allocation, and a seed is not one. Keeping `into` to
|
||
;; collections means the destination is always honest about what it is. A
|
||
;; reducing macro of the same shape is a separate form when something wants it.
|
||
|
||
;; The chain, built from the inside out: the innermost form is the push, and
|
||
;; each transform wraps whatever the transforms after it produced. Walked in
|
||
;; reverse for that reason, which is what the loop's two names are.
|
||
;;
|
||
;; One element name throughout, shadowed by each (map f) stage. A let binding's
|
||
;; value is checked before the name is bound, so (let [x (f x)] ...) reads the
|
||
;; outer x and binds the inner one — that is the language's rule and not an
|
||
;; accident of the compiler; the debug-info suffix `x~2` exists precisely so a
|
||
;; debugger does not lie about which is which. The name is a gensym, so it
|
||
;; cannot collide with anything at the call site.
|
||
;;
|
||
;; A transform that is neither map nor filter expands to a call to a name
|
||
;; nothing defines, which is how a macro reports anything at all: it has no
|
||
;; error facility, so the report is "unknown name" at the call site — the right
|
||
;; place and the wrong sentence. The bad transform is passed along so that at
|
||
;; least it is named.
|
||
(defn into-wrap [ts [Form] dst Form x Form] Form
|
||
(loop [k (len ts) body `(push ~dst ~x)]
|
||
(if (= k 0)
|
||
body
|
||
(let [t (at ts (- k 1))
|
||
items (form-items t)]
|
||
(if (!= (len items) 2)
|
||
`(into-transform-is-map-or-filter-of-one-function ~t)
|
||
(let [head (at items 0)
|
||
f (at items 1)]
|
||
(cond
|
||
(form-sym? head "map") (recur (- k 1) `(let [~x (~f ~x)] ~body))
|
||
(form-sym? head "filter") (recur (- k 1) `(when (~f ~x) ~body))
|
||
:else `(into-transform-is-map-or-filter ~t))))))))
|
||
|
||
;; The items of a list form, and the empty slice for anything else — a
|
||
;; non-list transform falls into the arity complaint above rather than needing
|
||
;; a case of its own.
|
||
(defn form-items [f Form] [Form]
|
||
(match f
|
||
(Form.List xs) xs
|
||
_ (form-nil)))
|
||
|
||
(defn form-sym? [f Form name string] bool
|
||
(match f
|
||
(Form.Sym s) (bytes=? (bytes s) (bytes name))
|
||
_ false))
|
||
|
||
(defn form-is-sym? [f Form] bool
|
||
(match f
|
||
(Form.Sym s) true
|
||
_ false))
|
||
|
||
(defn form-pair [a Form b Form] [Form]
|
||
(form-cons a (form-cons b (form-nil))))
|
||
|
||
;; A source that is already a name is used as it is, and a source that is
|
||
;; anything else is bound to one. Both halves matter.
|
||
;;
|
||
;; Binding it is what a source that is a *call* needs: (len s) and (at s i)
|
||
;; have to be the same s, and without the binding the call would be made twice
|
||
;; per element.
|
||
;;
|
||
;; Not binding a name is what everything else needs. A (Vec T) is move-only, so
|
||
;; (let [s v] ...) would hand v's ownership to the macro's own binding and the
|
||
;; caller would find v dead after an (into v ...) that only read it — and a
|
||
;; fixed array would be *copied* into the binding, once per into. Neither is
|
||
;; what was written. len and at borrow, so used directly the source is only
|
||
;; read. A source that is a call and produces a Vec is still consumed, which is
|
||
;; right: nobody else is holding it.
|
||
(defmacro into [args]
|
||
(if (< (len args) 2)
|
||
`(into-takes-a-source-a-destination-and-transforms)
|
||
(let [from (at args 0)
|
||
named? (form-is-sym? from)
|
||
src (if named? from (gensym))
|
||
bind (if named? (form-nil) (form-pair src from))
|
||
dst (gensym)
|
||
x (gensym)
|
||
i (gensym)]
|
||
`(let [~dst ~(at args 1) ~@bind]
|
||
(dotimes [~i (len ~src)]
|
||
(let [~x (at ~src ~i)]
|
||
~(into-wrap (form-rest args 2) dst x)))
|
||
~dst))))
|
||
|
||
|flan}
|
||
|
||
let file = "<prelude>"
|
||
|
||
(* The bootstrap hook, and the whole of why a prelude function may now call a
|
||
prelude macro.
|
||
|
||
[Check.program] prepends this file to every program, and a macro module is
|
||
built by running [Check.program] over the prelude — so a prelude function
|
||
that calls a macro cannot be compiled *into the very module that would
|
||
expand it*. That is a cycle, not an ordering mistake, and it is broken by
|
||
making the prelude smaller for exactly the one build that cannot afford it:
|
||
while a macro module is being built, [Macro] installs a reduction here that
|
||
drops every [defn] depending, directly or transitively, on a macro. A
|
||
*macro* that lands in that set is refused by name — see [Macro.reduce].
|
||
|
||
A ref rather than a parameter because the reader is [Check.program], which
|
||
cannot be told, and because [Macro] sits above it and cannot be depended on
|
||
from here. *)
|
||
let bootstrap : (Form.t list -> Form.t list) ref = ref (fun fs -> fs)
|
||
|
||
let forms () = !bootstrap (Reader.read_all ~file source)
|