The blind spots the mutation pass named are covered
Says what closed each one and how, and that two of the four named buffers now have evidence rather than a reading. The count of nineteen stands as the old one: the mutation pass has not been re-run.
This commit is contained in:
parent
9b80fc2084
commit
6f67b7114f
51
NEXT.md
51
NEXT.md
@ -82,11 +82,13 @@ it *would* have written.
|
||||
which is a compiler feature of the same shape the bounds checks already have, or they belong to the checker. Not
|
||||
decided. `test_sanitize` pins the current answer with a control that must *not* report, so a future clang changing
|
||||
this is a test failure rather than a discovery.
|
||||
2. **Four named buffers got no evidence at all.** The 4K result cap, the dev registry overflow guard,
|
||||
`SNAP_MAX`/`SNAP_NAMES` and `condition_name[128]` are on the daemon and agent paths, which need a socket and are not
|
||||
in the corpus. Their guards were read and are correct; that is reading, not testing. `escaped[ESCAPE_MAX]` is the one
|
||||
that *is* covered, because `println.flan` drives a 1100-character string through it on purpose — 1019 bytes out
|
||||
against a worst case of 1021 into 1024. `scratch[SCRATCH]` never sees more than 20 characters of 64.
|
||||
2. **Two of four named buffers still have no evidence.** `SNAP_MAX`/`SNAP_NAMES` and `condition_name[128]` are on the
|
||||
daemon and agent paths, which need a socket and are not in the corpus; their guards were read and are correct, and
|
||||
that is reading, not testing. The other two — the 4K result cap and the dev registry overflow guard — are now run
|
||||
for real by `test/dev_limits.c`, a C main beside `reload_host.c`, one process per limit because the name table
|
||||
never shrinks and the overflow case aborts. `escaped[ESCAPE_MAX]` was already covered, because `println.flan`
|
||||
drives a 1100-character string through it on purpose — 1019 bytes out against a worst case of 1021 into 1024.
|
||||
`scratch[SCRATCH]` never sees more than 20 characters of 64.
|
||||
3. **Valgrind over the headless corpus, not done.** ASan does not see uninitialised reads, which is where `zeroed` and
|
||||
struct padding live. MSan is out: it needs every dependency instrumented and raylib settles that.
|
||||
|
||||
@ -407,17 +409,36 @@ plan.org's single line on it (831) names a `for` the language does not have and
|
||||
|
||||
### Test blind spots, from a mutation pass
|
||||
|
||||
Sixty mutations, nineteen left the whole suite green. The severe cluster is closed (`cleanup.flan`,
|
||||
`signedness.flan`); these are not:
|
||||
Sixty mutations, nineteen left the whole suite green. The severe cluster was closed first (`cleanup.flan`,
|
||||
`signedness.flan`); the rest are closed now. Every one below was re-planted, watched leave the suite green, and then
|
||||
watched fail against the new test before the mutation was reverted — a test nobody saw fail is not evidence.
|
||||
|
||||
- `Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies — each confirmed to prune a
|
||||
function a valid program calls, so the build fails to link.
|
||||
- `flan_dev_global`'s size-change guard — the layout-drift check, with no test that retypes a global across a reload.
|
||||
- A local shadowing an imported name is qualified anyway.
|
||||
- The 4K result cap and the registry overflow guard have **no coverage at all**, rather than a missing assertion.
|
||||
- The reader accepts an unknown string escape; `+5` stops being a number.
|
||||
- And a warning: a reader mutation makes the suite **hang** rather than fail. A green run is not the only outcome to
|
||||
plan for in CI.
|
||||
- **`Reach`'s walk of index expressions, `addr` places and `restart-case` clause bodies.** `programs/reach-walk.flan`
|
||||
calls three functions from three places that are each the only route to them. The failure is not a wrong answer:
|
||||
the function is not emitted and the program stops linking, so the case catches the build exception rather than
|
||||
comparing output. The `addr` case goes through a `deref` place deliberately, so the index case cannot stand in for
|
||||
it.
|
||||
- **`flan_dev_global`'s size-change guard.** `programs/reload-v5.flan` is v4 with `extra` as an `i32`, loaded on top
|
||||
of v3 in a host run of its own, because what it does is abort. The message is asserted next to the exit status: a
|
||||
process that died for another reason is not this guard firing.
|
||||
- **A local shadowing an imported name.** `programs/shadow-pkg.flan` binds locals over its own constant and var;
|
||||
`pkg-shadow.flan` prints four numbers that separate the expression renamer from the place renamer. Nothing refuses
|
||||
a renamer that qualifies through a binding — it reads the top-level name instead and runs — so only the number says
|
||||
so.
|
||||
- **The 4K result cap and the registry overflow guard.** `test/dev_limits.c`, a second C main beside `reload_host.c`,
|
||||
drives them directly: neither has a Flan spelling and no corpus program reaches either. One process per mode — the
|
||||
name table never shrinks and the overflow case aborts.
|
||||
- **The reader's unknown string escape, and `+5`.** Rows in the reader table, with the escapes it *does* know
|
||||
asserted on their decoded bytes rather than through `Form.to_string`, which escapes them again and would compare
|
||||
the source with itself.
|
||||
- **The hang.** A reader branch that forgets to advance loops for ever, and `dune test` waits as long as it is left
|
||||
to; in CI that is a job the runner kills with nothing named. `test/watchdog.ml` arms an alarm on every test binary
|
||||
— generous, because an alarm that fires on a slow machine is a flake and a flake is how a watchdog gets deleted —
|
||||
and a five-second one around every read in `test_flan`. The first read that does not return wedges the rest, so a
|
||||
looping reader costs five seconds and names the row instead of never finishing.
|
||||
|
||||
What is still open here: the mutation pass has not been re-run since, so the count of nineteen is the old one. The
|
||||
sanitized sweep (`@sanitize`) is under the same watchdog but has never been observed to fire it.
|
||||
|
||||
### Asked for by the editor lanes
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user