The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
print-str, print-i64, print-f64, print-bytes, print-line and newline leave
the prelude. print and println are the whole printing surface now, and print
is the better call at every one of the sites that used them: it is the same
structural walk without the newline, so the no-newline case the family was
kept for is covered, and it takes the value as it is. The old print-i64
forced an explicit (i64 x) at every call site, because this language widens
nothing implicitly; that cast is gone from 127 places.
Dropping it moves one answer. hash-grid returns u64, and the cast through
the signed printer showed sand-headless's hash as -2851001042534928384.
print routes a u64 through flan_u64_to_bytes, so it now prints
15595743031174623232 — the same 64 bits, read as the unsigned number they
are. The pinned expectation follows the correction.
test-flan-dev.el and test_session.ml both reached for print-line as "a name
the prelude has"; they reach for rand-seed instead.
encode-rune! says nothing is written when it answers None, and every None case
in the table passed that claim without testing it: an encoder that lays the
lead byte down and only then notices the buffer is short returns None exactly
as a correct one does. So a known byte goes into scratch, a refused encoding
is asked for, and the byte is read back. Storing before the length test turns
the line from 65 -1 65 -1 65 into 65 -1 0 -1 0.
The read-only claim beside lower-ascii was reasoned from the emitted linkage
rather than observed, and observing it was worse than the guess. With
(set (at (bytes "Hi") 0) \h): at -O0 the store is emitted against the constant
and the program takes SIGSEGV; at -O2 LLVM deletes it as undefined behaviour
and the program prints "Hi" and exits 0. The same source either dies or
silently does nothing depending on a flag. The comment now says that instead
of predicting a segfault.
Relaxing 0xf0's second-byte floor from 0x90 to 0x80 left the whole suite
green: every other row of the table had a case pinning it and that one did
not, so f0 80 80 af decoded happily as "/". The same smuggled slash the
two- and three-byte cases exist to catch, missed in the fourth width.
Seven mutations verified red after this: the lead-byte floor at 0xc2 and the
second-byte bounds on 0xe0, 0xed, 0xf0 and 0xf4, the truncated-sequence
width, and the split cursor dropping its trailing empty field. An eighth,
lower-ascii written as a bit-xor, is red on the bytes either side of the
letters — which is why those are in the table and the letters alone are not.
A decoder that only masks and shifts gets every well-formed character right,
so a corpus of real text passes it. What separates it from a correct one is
the second group here: an overlong two- and three-byte "/", a surrogate, a
code point past U+10FFFF, a lead byte that leads nothing, a lone continuation
byte, and a character truncated by the end of its slice. Each isolates one row
of the accept_sizes table, and each must answer width 1 so a scan advances.
The invalid sequences are byte arrays because no valid string contains them
and the reader has no \xNN escape to spell them with.
Encoding is checked by round trip. An encoder and a decoder wrong in the same
direction agree with each other, and expected bytes would not catch that.
The emoji line caught a use-after-return while this was being written: a
(defn whole [a [4 u8]] [u8] (slice a 0 4)) helper returns a slice into the
copy a [n T] parameter makes in the callee's frame. The compiler accepts it in
silence. The comment stays where the helper was.
