The blind spots the mutation pass named are covered

Says what closed each one and how, and that two of the four named
buffers now have evidence rather than a reading. The count of nineteen
stands as the old one: the mutation pass has not been re-run.
This commit is contained in:
Joseph Ferano 2026-09-12 10:54:53 +07:00
parent 9b80fc2084
commit 6f67b7114f

51
NEXT.md
View File

@ -82,11 +82,13 @@ it *would* have written.
which is a compiler feature of the same shape the bounds checks already have, or they belong to the checker. Not
decided. `test_sanitize` pins the current answer with a control that must *not* report, so a future clang changing
this is a test failure rather than a discovery.
2. **Four named buffers got no evidence at all.** The 4K result cap, the dev registry overflow guard,
`SNAP_MAX`/`SNAP_NAMES` and `condition_name[128]` are on the daemon and agent paths, which need a socket and are not
in the corpus. Their guards were read and are correct; that is reading, not testing. `escaped[ESCAPE_MAX]` is the one
that *is* covered, because `println.flan` drives a 1100-character string through it on purpose — 1019 bytes out
against a worst case of 1021 into 1024. `scratch[SCRATCH]` never sees more than 20 characters of 64.
2. **Two of four named buffers still have no evidence.** `SNAP_MAX`/`SNAP_NAMES` and `condition_name[128]` are on the
daemon and agent paths, which need a socket and are not in the corpus; their guards were read and are correct, and
that is reading, not testing. The other two — the 4K result cap and the dev registry overflow guard — are now run
for real by `test/dev_limits.c`, a C main beside `reload_host.c`, one process per limit because the name table
never shrinks and the overflow case aborts. `escaped[ESCAPE_MAX]` was already covered, because `println.flan`
drives a 1100-character string through it on purpose — 1019 bytes out against a worst case of 1021 into 1024.
`scratch[SCRATCH]` never sees more than 20 characters of 64.
3. **Valgrind over the headless corpus, not done.** ASan does not see uninitialised reads, which is where `zeroed` and
struct padding live. MSan is out: it needs every dependency instrumented and raylib settles that.
@ -407,17 +409,36 @@ plan.org's single line on it (831) names a `for` the language does not have and
### Test blind spots, from a mutation pass
Sixty mutations, nineteen left the whole suite green. The severe cluster is closed (`cleanup.flan`,
`signedness.flan`); these are not:
Sixty mutations, nineteen left the whole suite green. The severe cluster was closed first (`cleanup.flan`,
`signedness.flan`); the rest are closed now. Every one below was re-planted, watched leave the suite green, and then
watched fail against the new test before the mutation was reverted — a test nobody saw fail is not evidence.
- `Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies — each confirmed to prune a
function a valid program calls, so the build fails to link.
- `flan_dev_global`'s size-change guard — the layout-drift check, with no test that retypes a global across a reload.
- A local shadowing an imported name is qualified anyway.
- The 4K result cap and the registry overflow guard have **no coverage at all**, rather than a missing assertion.
- The reader accepts an unknown string escape; `+5` stops being a number.
- And a warning: a reader mutation makes the suite **hang** rather than fail. A green run is not the only outcome to
plan for in CI.
- **`Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies.** `programs/reach-walk.flan`
calls three functions from three places that are each the only route to them. The failure is not a wrong answer:
the function is not emitted and the program stops linking, so the case catches the build exception rather than
comparing output. The `addr` case goes through a `deref` place deliberately, so the index case cannot stand in for
it.
- **`flan_dev_global`'s size-change guard.** `programs/reload-v5.flan` is v4 with `extra` as an `i32`, loaded on top
of v3 in a host run of its own, because what it does is abort. The message is asserted next to the exit status: a
process that died for another reason is not this guard firing.
- **A local shadowing an imported name.** `programs/shadow-pkg.flan` binds locals over its own constant and var;
`pkg-shadow.flan` prints four numbers that separate the expression renamer from the place renamer. Nothing refuses
a renamer that qualifies through a binding — it reads the top-level name instead and runs — so only the number says
so.
- **The 4K result cap and the registry overflow guard.** `test/dev_limits.c`, a second C main beside `reload_host.c`,
drives them directly: neither has a Flan spelling and no corpus program reaches either. One process per mode — the
name table never shrinks and the overflow case aborts.
- **The reader's unknown string escape, and `+5`.** Rows in the reader table, with the escapes it *does* know
asserted on their decoded bytes rather than through `Form.to_string`, which escapes them again and would compare
the source with itself.
- **The hang.** A reader branch that forgets to advance loops for ever, and `dune test` waits as long as it is left
to; in CI that is a job the runner kills with nothing named. `test/watchdog.ml` arms an alarm on every test binary
— generous, because an alarm that fires on a slow machine is a flake and a flake is how a watchdog gets deleted —
and a five-second one around every read in `test_flan`. The first read that does not return wedges the rest, so a
looping reader costs five seconds and names the row instead of never finishing.
What is still open here: the mutation pass has not been re-run since, so the count of nineteen is the old one. The
sanitized sweep (`@sanitize`) is under the same watchdog but has never been observed to fire it.
### Asked for by the editor lanes