Two things a type provider needs and neither of which a macro could do.
A package macro could not call its own package's functions. Load already
renamed the body so that (next c) reads (edn/next c) — the intent was written
down — and the module was then compiled from the prelude and the defmacros
alone, so the call arrived at the checker as "the call edn/next into an
imported package". The declarations now travel beside the macros in
Parse.imported_decls, trimmed in Macro.compile to what the macro bodies
actually reach. raylib's five with-* are pure quasiquote, so nothing of raylib
is reachable and its module is the one it always was — which matters, because
raylib's declarations are declares against a library a macro module has no
linker argument for.
And a macro had no way to resolve a path. (embed "assets/x.edn") resolves
against the directory of the source file the form is written in; a macro knows
the path it was handed and not what it is relative to, because a Form carries
no location. So the compiler pokes the call site's directory into two C
symbols before every expansion and (macro-slurp "...") joins the two. C data
and not a Flan global: the module is emitted with hidden visibility and only
the flan.macro.* thunks stay exported.
None rather than a condition, which is why this is not slurp: a condition
signalled inside an expansion goes through the module's own copy of the
runtime, and that is the failure Build.macro_module's hidden note measured.
The !-means-mutates convention distinguished nothing — there is no
immutable counterpart to contrast with — so every mutating name drops
the mark: sort, sort-by, sort-bytes, swap, reverse, append, append-i64,
append-f64, encode-rune, split-next, map-remove, map-next, and the test
helpers beside them. Two could not simply shed it: map! is map-in-place,
because map is the into transform's word and means the non-mutating
thing; put! is put-at, because put is the Map builtin. The ?-means-asks
convention stays. Dated records keep the old spellings; watch.clj's
reset-spies! and the other Clojure names are not ours to rename.
Four corrections to the prose and one to the test, none to the design.
The provenance line said "the name is Rust's or_else, the behaviour is
Rust's unwrap_or", which conflates two functions that differ in both
eagerness and return type — Rust's or_else takes a closure and answers
another Option. Java's Optional.orElse is the exact match, and its lazy
sibling orElseGet is the one already declined a paragraph above.
The helper reaching or-else's None branch at an owning type asserted a
refusal nobody had run. Compiled, it is "nothing here says what None is
an Option of — annotate the function's return type or the binding", so
the comment quotes that and the helper is a return type and nothing
else: its other branch was never called, in a program whose header says
every line is a claim.
read-file's comment claimed both restarts arrive unchanged and the test
runs use-value. Narrowed to the mechanism (nothing here establishes a
handler) plus the half that is actually executed.
And edn-read.flan now says what becomes of its defn wrapper when the
computed-initialiser work lands, since that is the only thing keeping
the motivating line from being written as the defvar.
Two things the motivating line wanted and could not have.
or-else and some? are the first prelude family over (Option $t), and the
first that declares no {:where} at all: they move the payload out or read
the tag, and neither is an operation a type variable has to be admitted
to. So they instantiate at every type, including the ones that own
storage — where the answer is a header onto one of the two buffers and
the branch not taken is still the caller's to free, which the comment
says because "or a default" reads like it consumes the default.
none? is declined as (not (some? o)), and an unwrap that signals on None
is declined for the reason file-size is an Option at all: absence is a
reply and not a fault, and whether an empty one is an error is the
caller's question.
edn/read-file is worth having for one fact the package already argued:
every string in a Value is a copy, so the source buffer is dead the
moment read returns and nothing outside the call can be holding it. It
slurps against the heap by name — the one allocator this package names,
because the buffer's life is inside the call and is not the caller's
tier to choose — defers the free for the transfer path, and passes
slurp's FileError straight through with both restarts armed. Folding a
missing file into None would collapse the very distinction the Option
exists for.
There is no json/read-file and json.flan now says why: a Token's text is
a slice into the caller's buffer, so the prerequisite is a json/read
answering a self-contained document, and there is no Value type there to
answer with.
The defvar initialiser in the motivating line is still refused as
computed, so edn-read.flan writes it as a defn and says so; everything
inside the with-allocator is verbatim.
Five more: file-exists?, file-size, delete-file, rename-file and
make-directory. The interesting thing is not the list, it is the line drawn
through it.
file-exists? and file-size answer a value -- a bool and an (Option i64) -- and
are prelude functions over one declare that the compiler knows nothing about.
Absence is the reply to those two questions and not a fault, so a condition
would make the ordinary case pay for a handler search, and there is no restart
a handler could take that would turn "it is not there" into a different
answer.
delete-file, rename-file and make-directory answer () and signal FileError,
and they are check.ml builtins for the one thing a declare cannot do: they go
through file_guard, so each failure arrives under retry and use-value. Those
are restarts a handler really can take -- make the parent directory and retry,
or supply another path -- which is exactly the case a bool return throws away.
op continues the prelude's numbering as 2, 3 and 4.
One C function behind the two questions rather than two, because they are one
question: stat answers whether the path resolves and how big it is in the same
breath. It is stat and not flan_file_size's fopen-plus-ftell, which is shaped
by slurp being about to read the file and is wrong as a general size -- fopen
on a directory succeeds on Linux and ftell then answers a number that is not a
file size. The two coexist and answer different questions.
rename holds the source in the guard's path slot, so a use-value renames a
different file to the same destination. Both readings are plausible until
somebody says which, so check.ml says which.
The errno mapping is not extended. Its three buckets are what a handler can
act on; EEXIST and ENOTEMPTY land in io with everything else, and that is
honest until conditions have a hierarchy to hang a fourth reason off.
All three carry barf's decision 2 unchanged: they change the filesystem, so on
the web they signal rather than succeeding quietly into a filesystem the page
throws away.
Not here, and not half-parsed either: a directory listing, which needs an
allocating builtin and a Vec of owned strings, and streaming IO. Neither has
a name to trip over.
programs/files.flan makes and removes its own tree and takes both restarts on
operations that write. The runtime additions continue the block at the end of
flan_rt.c.
Nothing in it could. A game got a clock from raylib and a program without a
window had none at all, so "how long did that take" was unanswerable in the
half of daily use that is a tool rather than a game.
Two clocks, because the mistake a single one invites is using it for the other
job. monotonic-ns measures: it never goes backwards, nothing adjusts it, and
its zero is arbitrary, so it is meaningless alone and correct as a difference.
unix-ns dates: nanoseconds since 1970, which is what goes in a save file, and
which jumps in either direction when somebody sets the system clock. The names
are picked so that reaching for the wrong one reads wrong.
This is Odin's shape, from core/time/time.odin and core/time/time_linux.odin:
Tick against Time, both an i64 of nanoseconds, over MONOTONIC and REALTIME,
with the seconds-valued face derived rather than a second syscall. Three C
functions here and six Flan names over them, which is the rule flan_rt.c's own
header states -- a primitive is the only thing implemented twice.
The monotonic origin is the first read of the clock in the process, not boot,
and that is the one decision worth arguing. CLOCK_MONOTONIC counts from boot,
so on a machine up a hundred days the raw value is past 2^53 nanoseconds and
monotonic-seconds would lose sub-microsecond resolution depending on the
machine's uptime rather than on anything the program did. Latched to first
read it stays integer-exact for a hundred days of process life, and it also
matches what a game already has: raylib's GetTime is seconds since
InitWindow, so the two numbers now mix without a conversion at every site.
sleep-ns loops on EINTR, because otherwise a signal cuts the wait short and a
frame loop wobbles for reasons nothing in the program explains. It is
documented as at-least and not as a frame limiter; the shape that actually
paces a loop is a deadline recomputed from monotonic-ns each turn, and the
comment says so where somebody will read it.
getenv answers an (Option [u8]) viewing the process environment, which needs
no allocator and no free and is safe precisely because nothing in this
language can call setenv or spawn a process. The absent case rides in the
length rather than in the pointer: there is no null test to write, since a
(Ptr T) here always addresses something, so flan_getenv answers -1 and a
pointer at a valid empty string and the Flan side tests arithmetic.
The runtime additions are a single block at the end of flan_rt.c, with
<time.h> inside it for the reason <errno.h> sits beside the file section.
programs/time.flan asserts invariants and never a reading -- t2 >= t1, a sleep
that did not return early, a date after 2020 and before 2100 -- because the
same file is in the corpus @x86 builds twice and diffs, so a timestamp would
fail a correct compiler on its second run.
The prelude's declare surface was five f32 functions, and the five were there
because somebody needed each one. Everything else a caller wanted was written
as a declare at the top of their own file -- the identical libm call with none
of the caveats written down.
So the rest of libm is here: tan, the three inverses, the three logarithms,
exp, fmod, hypot, cbrt, fabs, and an f64 face for every one of them including
the five that already existed. A declare is a line, a symbol already on the
link, and nothing in either backend, which is why this was cheap enough to do
completely rather than one function at a time.
The f64 half is not decoration. f32 is what a position is; f64 is what a
measurement is -- the clock, parse-f64, format-f64, any sum over more than a
few thousand terms -- and having only the f32 face forced a cast down and back
at each of those boundaries, which is where the precision went.
The paragraph the sqrt note draws for itself is now drawn once for the family:
IEEE-754 specifies sqrt, fabs, floor, ceil, round and fmod as exact or
correctly rounded, so those agree bit for bit across glibc, musl and
wasi-libc; it requires nothing of the rest, so the sand-grid rule covers all
of them unchanged. floor, ceil and round are Flan at f32 and libm at f64, and
that is not an inconsistency: the f32 bodies work because every f32 with a
fraction fits in an i32, and at f64 that trick is gone.
abs-i32 and abs-i64 are Flan, one per width because min and max are builtins
and no generic covers the numeric types. pi and tau at both widths, written
out rather than derived so the compiler rounds each literal once.
programs/math3.flan covers it at values that are exact in binary, so nothing
pins one libm's last bit. The -O0 case is the one that matters: at -O2 LLVM
folds a call over two literals and leaves no symbol to resolve, which is how a
missing -lm hid the first time.
README documented four subcommands of eleven. The seven missing ones are
there now, with import-c and generate-c given a worked example each -- they
are the most valuable thing here that nothing documented at all.
An environment table, checked against the getenv sites rather than against a
list: thirteen variables, each with where it is read, plus the llc/clang
version coupling that breaks C-c C-c while flan build keeps working. The
FLAN_DEV_* set that flan dev hands itself across its own exec is named as
internal rather than left looking settable.
DISCUSS.md's survey of what the x86 backend had no plan for still listed the
whole condition family. x86.ml:1587-1615 lowers all of it and the survey is
104/104; the row is struck through and corrected in place, because other
files cite that table by position.
prelude.ml promised a core: package at milestone 3. Milestone 3 came and went
and the package did not, so the docstring states the limit instead of
promising a way out of it. The loader could carry one -- what is missing is
the decision about what core: means for a program that imports nothing.
Flan's tagged sum has been spelled defunion since it landed, which was
accurate right up until the language wanted C's untagged union as well.
Both cannot be called the same thing, and the tagged one is the one with
an alternative name that says what it is: a case, its fields, and a tag
that steers which case is live is a data type, not a union.
So the form is defdata everywhere -- the parser, the AST, the checker,
both backends, the prelude's Form, the editor's font-locking and imenu,
the docs and every .flan file in the tree. The internal vocabulary moves
with it: Tast.union is Tast.data, uname is dname, the tables the checker
and the emitter keep are datas. Leaving them would have inverted the
words permanently, with surface defunion meaning one thing and
env.unions meaning the other, which is exactly the kind of drift the
comments in those files exist to prevent. What did not move is case,
variant and vfields: a tagged sum still has cases, and it still has one
live at a time.
defunion is not kept as an alias. An alias would compile the day the
untagged form lands and mean the opposite of what it used to -- the same
silent misparse that made defn's return type mandatory, and worse,
because the reader would have no reason to look. The old spelling is a
named refusal instead, parse/defunion-renamed, which says what it is now
called and that the name is reserved for something else. It fires on the
head alone, so (defunion U [A B]) -- which would otherwise have parsed
cleanly as one field A of type B -- is refused with the rest.
Three arithmetic situations had no defined behaviour and the two backends
disagreed about all three: a divide or remainder by zero, which was a raw
SIGFPE with no message and no location; (/ min -1), whose quotient is one past
the top of the type; and a float to integer cast whose value does not fit,
which LLVM called undefined and would fold to anything.
They now signal ArithError with `error`, exactly as a bad index signals
BoundsError, and die with a sentence naming the file, the line and the operands
only if nothing answered. The guards ride the same --checks flag as the bounds
check and are elided with it.
No restart is established at the failing operation. The sketch this started
from asked for use-value, and the implementation ruled it out: a restart frame
is allocated by the restart-case that offers it, on its own stack, so the
runtime cannot hold one on a program's behalf and use-value here would mean an
alloca and a restart frame at every division in every checked build. That is
the cost already refused for indexing, buying a silently different answer.
The x86 backend is unchanged and is the next commit.
Two loose ends from NEXT.md.
slice-from-ptr's run-time refusal borrowed @flan_slice_error and reported a
range and a length the caller never wrote. It has flan_slice_promise_error
now: signals BoundsError, walks the handlers, offers the break loop, falls
through to a message and a status like the two beside it. The sentence names
what was promised and what was passed, and a second line says what is not
checked. The condition fields stay (0, n, 0) — the violated condition as a
range, and not (0, n, n), which reads as in bounds.
And a session now holds the buffer's own defmacros: seeded in Session.create
from the same read that produced decls, and added by Session.eval so a
defmacro typed at the editor joins the set the way a defn does. Not a re-read
of the file, which would put unsaved-versus-saved skew inside expansion. The
commit stays below the checker. Macro.program dedupes the ambient set against
the forms being parsed, left-wins, because unqualified names can now collide.
A map keyed by a type variable cannot be put into inside a generic body:
the hash and the equality are concrete symbols chosen from the concrete
key type, and there is none until the copy exists. The refusal now says
that, and says what hashable? does buy - taking and returning a
(Map $t V) - rather than leaving the reader to infer it.
Closing the hole means adding the map operations to the list of forms
the abstract pass defers to instantiation. That list is print and
println and nothing else, and every member is a place where a refusal
moves from the definition to a call site, which is what the abstract
pass exists to prevent. Two is short enough to hold in your head.
Also written down: four of the prelude's copyable? declarations are
convention rather than checker-enforced. The move analysis tracks
locals, not reads out of a slice, so swap! and friends check without it
- and would still duplicate a header at [(Vec i32)].
swap!, reverse!, sort!, sort-by!, index-of, min-of, max-of, map!,
reduce and filter, each written once over $t. Every call site in the
corpus moves with them.
min-of and max-of are not min and max because min and max are builtins
over two or more numbers and nothing shadows a builtin. These reduce a
slice, which is a different operation at a different arity.
sort-bytes! did not collapse into sort!, and the reason is the point of
the predicates: a [u8] is not ordered? and cannot be, because < is an
instruction and comparing two slices lexicographically is a loop. It is
sort-by! with bytes<? written in, one line, keeping its name and its
stability note. sum-i32/sum-f32 and append-i64!/append-f64! stay for the
reasons the spike gave.
Not what the notes predicted: none of the ten collapses on a signature
change alone. filter and reduce need copyable? because the checker
demands it - reduce's accumulator at (Vec i32) is a double move - and
the rest declare it because a slice of owning elements would have them
duplicating headers.
(into xs (vec-new i32) (map double) (filter even?)). The function name is
syntax and never a value, so (map double) is (double x) written into the
loop body: no intermediate collection, no closure, no generics, nothing
to inline. Transducers would compose at run time and Rust's iterators
need monomorphisation; a macro needs neither. into.flan counts the pulls,
which is the assertion a unit test cannot make.
The destination is in the form because the destination is the allocation,
and that is what makes spec-memory.md's explicit-allocator rule true by
construction rather than by convention. Which also settles the open
question: reductions do not share the form. A seed is not an allocation,
so (into xs 0 (map cost) (sum)) would be a second form wearing the same
spelling, and the destination would stop being honest about what it is.
A source that is already a name is used as it is, not bound. A (Vec T) is
move-only, so binding it would take the caller's ownership for something
that only reads; a fixed array would be copied once per into. A source
that is anything else is still bound once, which is what a call needs.
The wart is that an owning temporary there leaks, because the binding has
a name the caller cannot reach — a call in that position should borrow,
and drop is what would close it.
All four of the prelude's macro limits bit and none blocked anything. The
three refusals are names nothing defines, which is the only error
facility a macro has. into-wrap is a defn using only special forms, so
Macro.reduce does not drop it, and it is the first thing in the prelude
written as a loop/recur.
The slot after a defn's parameters is unconditionally a type. Parse.decl no
longer takes a set of type names, and is_type_form, qualified_type, types_in,
declared_types and prelude_types are gone with the pre-pass that fed them.
What they were for: (Option f64) and (Some 1) are the same s-expression, so the
parser decided which it had by looking the head up in a set of the file's own
type names. Sound -- one top-level namespace means a name cannot be both a type
and a value -- and brittle, because the set had to be complete. It was wrong
twice in one day, the second time parsing (defn f [] (Rune {.code 65}) (bar))
as a function returning a Rune with a one-form body, silently, in every file in
the language.
Two things fall out. A type the parser could not have known -- a struct
declared further down the file, rl/Vector2 behind an unresolved alias, a
prelude type -- never needed recognising, only placing. And a mistyped type is
a mistyped type: (defn f [] f65 0.0) reaches the resolver's near-miss check and
says did you mean f64, where it used to be read as the first form of the body
and reported as an unknown name.
Unit is written (). The old spelling is refused with a message naming the new
one, the rule the colon-to-dot change followed. Internally it is still
Tname "Unit" and Types.Unit, so the resolver, the shim and the emitter did not
change; Cimport still builds Tname "Unit" for C's void without going through
the parser. Types.to_string prints () though -- that printer prints what a
person would write for every other type it knows, [i32], {K V}, (Ptr T), and
Unit was the odd one out once the source spelling moved.
Dropping prelude_types removes one of the two reasons Macro.reduce may only
drop defns: the memoised set a bootstrap build could have poisoned is gone, so
the remaining reason is the plain one.
The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
The refusal list was rewritten and the prologues that pointed at it were not,
so prelude.ml claimed in three places that what it now contains is impossible:
the splitting header said `split` is refused at the foot of the file, forty
lines above `split`; the ASCII-case header said Odin's allocating to_lower is
not available here, next to the one that was written; and the UTF-8 header
said the rest of core/strings is refused rather than ported.
Each keeps its point rather than losing it. The iterator is still the shape
that owns nothing and still the right call when there is no result to own;
lower-ascii and bytes-ci=? are still the right calls when a copy is not
wanted, since folding a comparison over two inputs beats lowering both. What
changed is the reason, which used to be the absence of an allocator and is now
a choice between two shapes that both exist.
And strings.flan told the reader the opposite of what it did -- "not freed",
on the line above the free. vec.flan already had the right framing: the free
is written, it keeps the block because an arena cannot release one, and that
is the difference the capability set exists to state.
The list at the foot of prelude.ml was one sentence -- every entry needed to
produce bytes that did not exist in its input, and there was no allocator --
and that sentence has been false since Vec landed. Seven entries move up into
the code, and string-from-bytes turns out to have been the `string` builtin
all along: (string (as-slice v)) is the round trip, free precisely because
the layouts are identical.
What is left is refused for four different reasons and is written that way
now: pad and center for nothing at all except that no caller has asked;
format and sprintf for variadics of mixed type; map, filter, reduce and
sort-by for function values; map-keys and map-values for a map iterator that
does not exist in the runtime.
NEXT.md's queued section is struck and carries the four findings, each with
the change it wants named -- flan_map_next plus one builtin for the iterator;
milestone 5's function values for the higher-order three; vec_new_elem taking
a type expression rather than a bare name, which is what forces slices-new to
exist; and an array literal with no way to say it is [f32], which is what
forces every float in algorithms.flan to be cast. BUILT.md gets the section.
sort-i32! was the only sort in the language. sort-f32! and sort-bytes! are
the other two, and they are copies rather than an abstraction for a reason
worth naming precisely: map, filter, reduce and a sort taking a comparator
are not blocked on generics, they are blocked on *function values*. Types.Fn
exists and check.ml refuses it with "a function type is not implemented yet --
milestone 5", and there is nothing else in the language to pass. Generics on
top of that is what would make them one copy instead of one per element type.
The f32 family carries one caveat the i32 family cannot have: a NaN makes the
order undefined, because every comparison against one is false, so the
insertion loop never moves it and never moves anything past it. sum-f32
accumulates in f64 for a stronger version of sum-i32's argument -- an f32
total does not wrap, it absorbs, and the answer comes out silently short.
The test prints the difference rather than the total, because %g hides it.
sort-bytes! is the one a caller of split actually wants, and its ordering is
memcmp's: bytewise, unsigned, prefix first. Not alphabetical -- "Zebra" sorts
before "apple" -- and the note says so, for the same reason the ASCII-case
note refuses a locale. The slices move and the bytes never do, so it sorts
fields borrowed out of a string literal, which an in-place byte sort could
not.
f64->bytes is snprintf "%g": six significant digits, exponent notation of its
own accord, and no precision to pass it. A frame time of 1/60 comes back as
0.0166667 and a score past a million as 1.23457e+06. format-f64 returns a Vec
instead, so it inherits neither that nor the shared static scratch buffer --
and it is the reason append-i64! exists, because it renders the integer part
and the fraction through that one buffer in strict sequence.
Half away from zero at the last digit kept, which is round-f32's rule and not
printf's. 0.125 at two places is 0.13 here and 0.12 there; matching printf
would mean pinning a particular libc's nearest-even on the binary value, and
that answer is not the same on every target anyway.
The three cases that ship broken are each one line and each tested: the
carry, where the rounded fraction equals the scale and is the next integer
(0.999995 at five places prints "0.100000" without it); the zero padding,
without which 1.005 at three places prints "1.5"; and the sign, which belongs
to the number rather than to its integer part, since -0.5 has an integer part
of 0 and 0 carries no sign.
The clamp on the precision is spelled (min 9 (max 0 prec)) and not with the
clamp macro, and the reason is a finding: the prelude is never
macro-expanded. macro.ml's pass runs over the file being compiled, and the
prelude arrives at the checker through Check.program's own prepend, so a
prelude function calling a prelude macro resolves the macro's underlying
defn -- the one that takes a [Form] -- and reports an arity error.
Eight functions that the file used to refuse by name, and the refusal was
always one sentence -- there is no allocator -- which stopped being true when
Vec landed. Three rules hold across all of them and are written at the head
of the section: the result is owned and the caller frees it, the allocator is
the context's, and no signature carries a Result because no allocating
operation returns an error.
The builder is not a type. Odin's strings.Builder wraps a [dynamic]u8; here
the (Vec u8) already is that and already has push, so a wrapper would be a
move-only struct whose only method is the one it wraps. What was missing is
appending a run of bytes, and append! is that -- taking a (Ptr (Vec u8)),
because a Vec parameter moves and a by-value builder would be consumed by its
first append.
append-i64! and append-f64! are the argument for the whole shape. The
runtime renders numbers into one shared static buffer, so two of its results
cannot be held at once; these copy out before returning, so a builder holds as
many numbers as it likes. strings.flan puts two integers and a float on one
line to show it.
split returns a (Vec [u8]) and not a (Vec (Vec u8)): the fields borrow the
input, and the owning shape is refused outright because a Vec copies and
releases its elements bytewise. Constructing it needed a one-line slices-new,
because (vec-new) takes its element type as a bare symbol and [u8] is not
one -- a compiler gap, noted rather than worked around in silence.
replace-bytes guards its empty needle with an if and not an early return: a
returned Vec is a move, the dead set spans the function, and a return on one
branch would kill the binding on the other.
The two declares inherit the sin/cos caveat in full and not the sqrt one:
IEEE-754 requires nothing of atan2f or powf either, so they are the third and
fourth places in the prelude where native and wasm32 may differ in the last
bit. Every case in math2.flan is therefore a value that is exact in binary --
a quadrant boundary, a power of two, a perfect square -- rather than one that
would pin a particular libm and then fail on wasi.
clamp is the interesting one. The prelude already argued against wrapping
(min hi (max lo x)) in a function, and that argument gets stronger rather
than weaker: min and max are builtins at every numeric type and there are no
generics, so a clamp *function* is one copy per type. A macro is
type-agnostic for free and emits nothing at all. The test calls the same
three words at i32, i64, u8 and f32 to show it, and counts evaluations to
show that each argument appears once -- the shape that names x twice reads
identically and calls it twice.
lo above hi answers hi and is not checked. A macro has no error facility, so
the only diagnostic available would be a run-time one, in the construct whose
whole point is that it costs nothing at run time.
plan.org milestone 5 says when, unless, until, cond and dotimes are special
forms only until macros land. This is the first one to stop being one, and
running test/programs/macro-unless.flan means the compiler built a shared
object, dlopened it into itself and called a Flan function to find out what
(unless c a b) means.
unless is the one that moved because it is the one nothing else needs: zero
uses in the prelude, so moving it cannot make the prelude depend on the
expander that compiles it. Its coverage is sand.flan, seven calls, compiled
through Session in test_session -- which is the in-process path and the reason
lib/dune now passes -linkall. Say plainly what that coverage is not: nothing
in test/programs used unless before today, so macro-unless.flan is a test
written after the feature. The corpus that was written before it is sand.flan
and web/examples/control.flan, and both compile unchanged.
lib/macro.ml is the half of expansion that has to compile something. Expand is
the image format and the quasiquote desugaring and depends on nothing above
Form; this needs Check, Build and Emit, so it sits above the parser it feeds
and arrives through Parse.expander.
What it does, in order:
- Collects every defmacro from the prelude and from the file. Not from an
imported package: Load learns a package's imports by parsing it, so
collecting from one means a second import resolver over Forms, and that is a
bigger thing than this.
- Builds them in rounds, because a macro's body may call a macro and a body
with an unexpanded call in it will not compile at all -- the call is a name
nothing defines. Round 0 takes every macro that names no macro still
waiting; round 1 expands the rest against round 0's module. A round that
takes nothing while macros remain is a ring and is named. macros.flan has
the round-1 case and macro-cycle.flan has the ring, and the distinction
between them is the one thing here that is easy to get wrong: a call inside
a quasiquote is *not* a compile-order dependency. It is part of what the
macro answers, and the answer is expanded again after it returns. The first
macro-cycle.flan written for this commit quasiquoted, and it was not a cycle
at all -- it hit the fuel instead, correctly.
- Walks bottom up, so a macro never sees a call to another macro in what it is
handed, and re-expands what comes back, so a macro that expands into a call
to itself keeps going. That loop is bounded at 200 and says which macro ran
out: macro-spin.flan.
- Skips all of it when the file names no macro, which is nearly every file.
Otherwise every build in the suite would pay a clang driver to answer a
question nobody asked. When it does build, the module is cached under the
object cache and keyed by the prelude's source plus the file's defmacros, so
a second process pays a dlopen.
lib/dune passes -linkall, which is the one line in another lane's file. The
module installs itself into Parse.expander at initialisation and nothing
references it, so without -linkall the linker drops it from every executable
that does not name the module -- bin/main.exe among them -- and a program
calling a macro fails with an unknown name. The alternative was an install
call at every entry point, including ones in files this lane must not touch.
The one thing a macro cannot do that parse.ml could is give a reason. A macro
runs inside the compiler and anything it signals aborts the compile with no
location, so a malformed (unless) answers a name nothing defines and the
report is "unknown name unless-takes-a-test-and-a-body" at the call site --
right place, wrong sentence. NEXT.md says so.
test_flan.ml's "unless -> if(not)" assertion is gone, because it asserted a
desugaring in a file that no longer does one. Nothing else in the suite
changed.
Running a macro means compiling it and loading it into the compiler, and the
step that reads as small in NEXT.md is not: OCaml has no dlopen for ELF, and
lib/dune had no foreign_stubs. So the boundary is built first and the expander
not at all. lib/dynload_stubs.c is the whole of it — dlopen, dlsym, a
four-argument call into a macro thunk, and a peek/poke family, because OCaml
cannot address the raw memory a Form image has to be laid out in.
Nothing aggregate crosses to C. The unions lane verified a union's memory
layout against clang, which is a different claim from LLVM's convention for an
aggregate passed or returned by value in hand-written IR, so Emit.macro_thunk
wraps every macro in void(ptr,i64,ptr,ptr): the slice is built and the result
stored on the LLVM side, and the compiler's side is four pointers.
Build.macro_module links the runtime in rather than declaring it external, so
the module has no undefined symbols and the compiler's own link needs no
-rdynamic. That is the difference from Build.shared, whose host is a running
Flan program.
defunion Form and the list-building surface quasiquote will desugar into are in
the prelude. Form mirrors Form.value and not Form.t: no loc field, so the
compiler stamps the call site's location onto everything a macro returns.
The compiler builds. dune test was not run, and Form's layout is asserted
nowhere — NEXT.md's new handoff section says what the three numbers are, what
the next two commits should be, and the four decisions this made that the
design did not settle.
The script is in tools/ rather than thrown away, because two lanes are
writing Flan in the old spelling right now and their files need the same
pass at merge.
It works on forms, not on text: a keyword becomes a dot only where it sits
in a field-label position inside a brace, so an enum member in value
position, a map key inside an EDN string and a type-position {K V} are all
left alone. :keys keeps its colon -- it names no field.
Decision 1. Odin's #load and #load_directory are the model, spelled as
ordinary named calls — an s-expression language already has a head
position and does not need Odin's `#`. (embed "p") is a [u8], (embed "p"
string) is a string, and (embed-dir "d") is a [n EmbedFile] sorted by
name.
Two spellings rather than one that changes type with its context. Odin
threads a type_hint everywhere and can afford it; with structural
equality and no implicit widening, the same text meaning two types here
would be a wart. The path is a literal and resolves relative to the file
the form is written in, both of which are Odin's rules and for Odin's
reasons: the bytes must be in hand before any value exists, and a
package's assets must not depend on where flan was invoked from.
The bytes reach the program as a [Str] node typed [u8], not as a [Bytes]
prim over a string. [Bytes] is identity — emit.ml lowers String and
Slice _ to the same %slice — and wrapping the literal in a prim would
make the node non-constant, so an (embed-dir) bound with defconst could
not be an LLVM constant. Both string emitters take the bytes and ignore
the node's type, so it is the same constant either way and one a global
can hold. emit.ml's escape is byte-exact, so a PNG survives the .ll.
The directory lookup is a linear scan in the prelude over a slice of
EmbedFile. A directory embed is tens of entries out of cache-warm
.rodata, and a compile-time perfect hash would be a build-time map with
its own failure modes that nothing has asked for. Sorted because readdir
order is filesystem-dependent and an unsorted embed would make two
builds of identical sources emit different .ll.
The slice points into .rodata, so a store through it segfaults at -O0
and is deleted at -O2 — the same measured trap the prelude's ASCII-case
note describes for (bytes "Hi"). Inherited, not widened; clone into a
Vec for a mutable copy.
Two element types, one runtime, and the element type appears nowhere below
the call site: size_of and align_of are produced where the concrete type is
known, which without generics is simply the concrete call site. That is
Odin's arrangement and it is what spec-memory.md specifies. `at` and `len`
were already the names for a fixed array and a slice, so a Vec extends them
rather than adding a parallel pair — the asymmetry `nth` was removed for —
and the value form and the place form go through one helper so they cannot
drift apart.
StorageExhausted lands with step 2 rather than after it, because the
signatures depend on it: `push` and `reserve` are Unit, `clone` is the
container, and nothing grows a Result. It is built out of nodes that already
existed — a while, a restart-case and an error — so the backend learned
nothing about allocation. The restart is established at the failing
allocation, which spec-memory.md names as the exception to "restarts go at
the resync point, once", and the element a push was given is bound to a slot
before the loop so a retry re-attempts the allocation and not the expression.
Move-only is a dead set on the checker context, and it is flow-sensitive at
an `if`: both arms start from the same set and the union survives the join,
so `(if c (free v) (free v))` is legal and a one-armed free still kills the
binding. The case a dead set cannot answer is a move inside a loop — merged
once at the end of the body it counts one move, not two — so that is a rule,
refused with its reason.
Four decisions the spec did not settle:
The Vec header is six words in every build, not four in release. A layout
that changes with a build flag can disagree across the reload boundary
silently: a redefinition module is built by llc and ld against a host built
separately, and nothing makes the two agree on a struct size. The 32-byte
release layout is deferred on that.
A zeroed Vec has a null allocator, and the first operation needing storage
adopts the context allocator. Odin's behaviour. The alternative was refusing a
Vec-typed struct field until drop lands; shipping the null was a null deref on
the first push.
A Vec's length and index are i32, like every other length here. Widening
indices is one change across all the containers, not a Vec question.
`let` has no type annotation, so a local Vec has nowhere to say what it holds
and the element type is written at the call: `(vec-new i32)`. This is not the
explicit instantiation syntax the generics section rules out — nothing here is
generic and the name resolves as an ordinary type. Where the context says, it
may be left out.
The allocator grew a budget: a ceiling on live bytes, 0 for none. The retry
restart is only answerable by a handler that can make the *same* request
succeed, and for a fixed backing store the handler that works is the one that
raises the ceiling — releasing the region a container lives in invalidates
the container, which is what the epoch check catches. The spec's "grows the
arena and then invokes retry" needed something to grow.
The generation word is bumped on every reallocation and read by nothing. The
stale-slice trap it is for needs a slice that can carry the Vec's identity,
and a slice is ptr+len. Said plainly rather than implied by the word's
presence.
The gestures testbed declared sinf and cosf at the top of its own file,
which is a copy in every file that wants an angle. The reason sqrt is a
declare does not transplant: IEEE-754 makes sqrt correctly rounded and
requires nothing of the kind for sinf, so these two are the one place in
the prelude where native and wasm32 may disagree bit for bit. That is
written down beside them, along with what the fix would be if a program
ever needs trig that agrees across targets.
Float abs stays unwrapped for the reason integer abs is -- it is
(max x (- 0.0 x)) over two builtins. The integer caveat does not carry
over and the note says so: -0.0 answers +0.0 and a NaN answers a NaN,
both checked.
print-str, print-i64, print-f64, print-bytes, print-line and newline leave
the prelude. print and println are the whole printing surface now, and print
is the better call at every one of the sites that used them: it is the same
structural walk without the newline, so the no-newline case the family was
kept for is covered, and it takes the value as it is. The old print-i64
forced an explicit (i64 x) at every call site, because this language widens
nothing implicitly; that cast is gone from 127 places.
Dropping it moves one answer. hash-grid returns u64, and the cast through
the signed printer showed sand-headless's hash as -2851001042534928384.
print routes a u64 through flan_u64_to_bytes, so it now prints
15595743031174623232 — the same 64 bits, read as the unsigned number they
are. The pinned expectation follows the correction.
test-flan-dev.el and test_session.ml both reached for print-line as "a name
the prelude has"; they reach for rand-seed instead.
session.ml already had this: a compile-time walk over a Tast type that
emits the calls to print a value of it, handling every concrete type the
language has. It was dev-build-only and went to flan_dev_emit, and
prelude.ml justified the per-type print-* functions by saying a real
println had to wait for milestone 5 and generics. It did not. plan.org
specifies println as compiler-provided and per concrete type, which is
not overloading: there is nothing to dispatch on at run time and no
user-supplied printer to choose between, so no type variables appear.
The walk moves to render.ml, parameterised on an emitter and a slot
allocator. The emitter is five functions rather than five extern names
because the two sides are not both extern calls -- the REPL's are, and
stdout's compose a conversion with a write. The slot allocator differs
too: the REPL builds a thunk's frame, println takes slots from the
enclosing function being checked, once per call site.
Two runtime shims, both only reachable from the walk. flan_u64_to_bytes,
because routing u64 through the signed printer makes 0xFFFF...F read as
-1, which is the one way println could disagree with the REPL about a
value both can hold. flan_escape_bytes, so a string nested in a printed
structure is quoted and escaped -- same table as flan_dev_emit_str, noted
in both, because the REPL and println must not disagree about what a
struct looks like.
A string at top level prints raw and nested prints quoted. Not a conflict:
(println "hello") has to print hello, and a struct's string field has to
be distinguishable from the punctuation around it. The split is top-level
vs nested, so it lives in check.ml and not in the walk.
Found on the way: a field of an Option had no gep in emit.ml, so the
walk's Option arm had never run -- the REPL would have failed on one too.
Option is { i8, T } with no declared name, so its layout is now spelled
out. Nothing in the surface language reaches a field of an Option; the
printer does, to read the tag without unwrapping a None.
The print-* functions stay. They print without a newline, which println
cannot express -- slices.flan's show prints elements separated by spaces
-- and they are raw where print is structural.
println.flan covers every arm at -O0 and -O2: the u64, the raw/quoted
split, both Option arms, the depth and span caps, and the slice arm's
loop twice over plus once inside a dotimes, which is where per-call-site
slot allocation would show if it were per-iteration.
encode-rune! says nothing is written when it answers None, and every None case
in the table passed that claim without testing it: an encoder that lays the
lead byte down and only then notices the buffer is short returns None exactly
as a correct one does. So a known byte goes into scratch, a refused encoding
is asked for, and the byte is read back. Storing before the length test turns
the line from 65 -1 65 -1 65 into 65 -1 0 -1 0.
The read-only claim beside lower-ascii was reasoned from the emitted linkage
rather than observed, and observing it was worse than the guess. With
(set (at (bytes "Hi") 0) \h): at -O0 the store is emitted against the constant
and the program takes SIGSEGV; at -O2 LLVM deletes it as undefined behaviour
and the program prints "Hi" and exits 0. The same source either dies or
silently does nothing depending on a flag. The comment now says that instead
of predicting a segfault.
Odin's core/strings and all of core/fmt take an allocator; core/unicode/utf8
does not, because decoding is classification and every answer is a number.
That line is where the port stops, and the refusals at the foot of the file
say so by name rather than leaving a caller to find out.
The accept_sizes table becomes a cond over the lead byte. Its four awkward
rows are the ones a hand-written decoder gets wrong one at a time, so they are
written out: 0xc0/0xc1 lead nothing, 0xe0 and 0xf0 have a raised second-byte
floor against overlongs, 0xed has a lowered ceiling against the surrogates.
Two divergences from Odin, both the parse-i64 argument again. A malformed
sequence carries ok:false instead of decoding to U+FFFD, which is a real code
point a caller cannot tell from a failure; and encode-rune! answers None
rather than silently substituting U+FFFD for a rune it was not given. Width
stays 1 on a bad byte, which is Odin's rule and load-bearing: every loop here
advances by it, and a 0 would hang rather than answer wrong.
split cannot return a sequence it would have to own, so the cursor is what
survives. It follows the allocating strings.split rather than Odin's own
iterator, which drops a trailing empty field and disagrees with it.
Case conversion is byte-wise and not in place: a literal is emitted into
read-only memory, so lowering (bytes "Hi") would type check and segfault.
floor, ceil and round over f32, which is what a position and a tile coordinate
are here. The only rounding mode available is the cast's truncation toward
zero, so each of these is that cast plus the correction the mode does not
make, and the content is which inputs make the cast itself undefined. NaN
fails every comparison, so it needs its own (not (= x x)) and nothing else
finds it; the infinities fall out of the magnitude test; and above 2^23 an f32
has no fractional bits left, which makes returning the input there the exact
answer and also the guard that keeps the cast inside i32.
round is half away from zero, written as floor of the magnitude and mirrored.
The obvious (floor-f32 (+ x 0.5)) is wrong twice: half-up rather than
half-away, so -2.5 comes out -2, and at the largest f32 below 0.5 the addition
alone rounds to 1.0 and answers 1 for a number under a half. Both are in the
table, which is why every case there is a negative or a half.
sqrt is the decision in this commit and it goes out to libm, which is a change
to the release link and so is said out loud. Every other number in the prelude
is reachable from the four operations and a cast; a square root is not.
Newton's method needs a starting guess, the good guess comes from
reinterpreting the exponent bits, and the language has only value-preserving
casts - no bit-cast between f32 and u32. Without one the iteration needs a
scaling loop to normalise and still produces a result that is merely close,
which is the one thing a standard library must not hand back. IEEE-754 makes
sqrt correctly rounded, so libm's answer is the same bit pattern on native and
on wasm32; for this function the byte-identical argument points at C rather
than away from it.
The cost is -lm on every link, and its placement matters. It goes after the
objects, not in the leading flags, because --as-needed drops a library named
before the object that wants it. Worse, at -O2 LLVM folds most sqrtf calls
into the hardware instruction and the symbol never has to resolve - so this
looked linked before the flag existed and failed only at -O0, which is exactly
why the table runs both. Untested against --target=wasm32: wasi-libc ships
libm.a as a stub because the symbols live in libc, so it should be inert
there, but nothing here exercises it.
The better fix is not in this lane. llvm.sqrt.f32 as a builtin in check.ml and
emit.ml is one instruction, no symbol and no flag, and it belongs to whoever
owns the compiler.
Finishing the text family the previous lane started. All three are over [u8]
and none of them allocates, which is what decides their shapes.
trim answers a slice of its input. That is the only shape available without an
allocator, and it is also the better one: there is no new storage, only a
narrower view of the caller's, so the result dies with its owner and trimming
modifies nothing. Both loops test (< lo hi), because an all-whitespace input
otherwise walks lo past hi and (slice s lo hi) traps on a reversed range - the
same trap the bounds table already asserts on. That input is in the case list.
index-of-bytes is naive and stays naive. Boyer-Moore wants a skip table sized
by the needle, which is an array, which is an allocation. The empty needle
answers Some 0 so that index-of-bytes and starts-with? agree on every needle,
and the length test returns before the loop so a needle longer than the
haystack cannot build a window off the end.
parse-f64 splits the work where the two halves actually differ: the grammar is
Flan's and the rounding is libc's. parse-i64 is entirely Flan because strtoll's
answers are wrong for a caller - 0 for "", 0 for "abc", 12 for "12x" - and not
because decimal-to-binary conversion is suspect. Reimplementing correctly
rounded conversion is a different and much larger problem than rejecting junk,
and IEEE-754 already guarantees strtod gives the same bits everywhere. So this
validates the whole slice and only a slice that is entirely a number reaches
bytes->f64. Every refusal in the table - "", "abc", "1x", ".", "1e", " 1",
"1 ", "0x10", "nan" - is a plausible number out of strtod.
Two caveats, both written into the source rather than discovered later. The
locale worry that keeps parse-i64 in Flan does apply to strtod's decimal point,
and is moot only because nothing in the runtime calls setlocale; if that stops
being true this is what breaks. And the length is capped at 511 because
flan_bytes_to_f64 truncates there - a validator that approved 600 digits would
be approving a different number than the one strtod reads.
digit? and space? exist because parse-f64 and trim need them, and calc-me loses
its own byte-identical digit?. One top-level namespace makes the second
definition an error rather than a shadow, which is the rule doing its job: two
copies that later drift apart is exactly what it prevents.
Finishing the 2D lane's unfinished work: the collision family was written and
had no tests when the session ended. It is the best material a headless table
gets, since every one of these is pure and needs no GL context.
Two plausible tests in a row turned out to check nothing, and that is the part
worth keeping. A struct round trip is symmetric and passes for any field order -
the texture lane found that one. The second is subtler: no axis-aligned geometry
can pin Vector2's fields, because exchanging x and y is a reflection that is
applied on the way in and undone on the way out. Swapping the shim's own typedef
leaves every collision case passing. Distances never even see it.
What does pin Vector2 is the rotated camera, because a rotation is not
axis-aligned and does not commute with the reflection. That case is load-bearing
and the comment now says so, because the collision cases look like they cover
the same ground and do not.
What the new cases do pin is Rectangle, completely: swapping width and height
turns three of the four predicates the wrong way. Verified by doing it.
collision-lines answers (Option Vector2) rather than a bool and an
out-parameter, because raylib leaves the out-parameter untouched when the
segments do not meet and a caller who forgets reads whatever was there.
bytes->i64 is strtoll behind a primitive, and strtoll returns 0 for "", for
"abc", for a lone "-", and for the "12" in "12x". None of those is
distinguishable from a real 0, so any program that parses input it did not
write is already wrong and has no way to find out. parse-i64 takes the whole
slice or refuses it and says so with None. It is also the version that answers
the same on wasm32: strtoll is libc's and locale-sensitive, which is the same
argument that put the PRNG in the prelude rather than leaving it to rand().
The byte predicates are over [u8] rather than over string on purpose. (bytes s)
is one call at the call site, and in exchange one copy of each function serves
strings and byte slices both — which is as near a generic as this gets. Each
tests its length before it slices, and `and` short-circuits, so a prefix longer
than the subject answers false instead of tripping the slice bounds check.
sign-f32 and lerp are the only two numeric helpers here, because they are the
only two that decide something. clamp is (min hi (max lo x)) and abs is
(max x (- 0 x)) over builtins that already exist — a prelude wrapper is a
function emitted into every program to save a caller nothing. sign-f32 answers
0.0 for NaN, which is a choice and is written down. lerp is the weighted sum
and not a + t*(b - a): the latter does not land on b exactly at t = 1.0, and a
position that never quite arrives is what interpolation gets bug reports for.
floor, ceil and round are deliberately absent. (f32 (i32 x)) is fptosi, which
is poison out of range, and shipping that as a documented limitation is the
same class of bug NEXT.md already records twice under Sharp edges. Correct
lowering is llvm.floor.f32 in emit.ml, which is not this lane. sqrt is absent
for a different reason: it is an extern to libm, and what libm means on wasm32
is a decision the FFI owns, not the prelude.
rand-i32-range answers lo for an empty or reversed range rather than dividing
by zero, which is immediate undefined behaviour and not merely a wrong number.
Both range functions draw exactly one rand-u32 and neither changes it, so the
sand hash still pins the generator; the new test pins the derivations off a
fixed seed, which nothing else would have caught.
A sequence library normally returns new sequences. There is no allocator, so
every one of these mutates the storage it was handed and a slice is the handle
that makes that useful: (slice grid 4 9) is ptr+len into grid, so sorting it
sorts those five elements and leaves the rest of grid alone. The test asserts
exactly that — it sorts a subslice and prints the whole owning array — because
it is the property that would die silently the day a slice parameter started
being copied rather than passed by value, and -O2's mem2reg would hide it.
Insertion sort rather than anything faster. Quicksort wants a stack and
mergesort wants a buffer, and neither exists; insertion sort needs a swap and
two indices. It is also the only one of the three whose inner loop is short
enough to read, which matters more than the asymptotics on the slice sizes a
frame loop actually sorts. The `and` guarding it short-circuits, and that is
load-bearing: at j = 0 the left test fails and (at s -1) is never evaluated,
so the bounds check never fires.
Over [i32] and nothing else. There are no generics, so a second element type
is a second copy of all seven functions emitted into every program that links
the prelude, and i32 is the type indices, ids and tile values already have.
An f32 set waits for a program that wants one.
min-i32 and max-i32 return (Option i32) rather than a sentinel because there
is no i32 that means "the slice was empty" and is not also a possible element.
sum-i32 accumulates in i64 and widens each element explicitly — there is no
implicit widening anywhere, and an i32 total over a screenful of i32 is how a
sum wraps without anyone noticing.
