flan/test/dune
Joseph Ferano 92fae67fb3 The collector's Flan-side roots go under the sanitizers
The sweep's note said there was no Flan program that reached flan_dyn.c, so
the only sanitized run over the collector was dyn_ops.c's -- which pushes
its roots by hand. That note stopped being true when the dyn programs
landed, and it stayed in the file. The gap it left is the one that matters
for the backend lane just committed: a root the *emitter* forgot is a live
object swept, and no amount of C testing can see a mistake the compiler
made. dyn-vec and dyn-defer are in the corpus now -- the second for the
roots that come off on a transfer's path out rather than a return's -- and
so is p13-dyn-collect, which is the only program anywhere that allocates
past flan_dyn.c's one-megabyte floor and therefore the only one under which
a mark and a sweep actually run. Everything else in that list agrees with
ASan by never collecting at all.

p13 lives in spike/x86 because that is the lane that wrote it, so the
alias's deps grew a glob for that directory; it is in this sweep for what it
does and not for where it sits.

What this cannot cover, and the compiler says so itself when asked: there is
no sanitizer pass over hand-written assembly, so --x86 --sanitize is refused
by name. ASan sees the x86 lane's roots only from the collector's side of
the call, never as frame slots. p13 through --x86 under the @x86 sweep is
what stands in for it, and it is a weaker check honestly labelled rather
than a stronger one assumed.

--force @sanitize: clean, and non-empty, which the previous run was not --
an alias satisfied from cache prints nothing and reads exactly like a pass.
dune test --force still green.
2026-09-19 16:01:26 +07:00

431 lines
20 KiB
Plaintext

(tests
(names test_flan test_acceptance test_reload test_agent test_session test_dev test_emacs test_repl test_cider test_dyn)
; Explicit because test_sanitize lives in this directory and is not one of
; these: two stanzas in one directory have to say which modules are whose.
; watchdog is every binary's clock: a hanging test reports nothing, so each
; of these arms an alarm that turns "for ever" into a failing run.
(modules test_flan test_acceptance test_reload test_agent test_session
test_dev test_emacs test_repl test_cider test_dyn watchdog)
(libraries flan unix)
; The acceptance programs are part of the test corpus: if the reader, the
; parser or the checker regresses on them we want to know here, not at the CLI.
(deps
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
; The brush sheet. sand.flan no longer embeds it — the front-end was cut back
; to what lisp/sand.lisp and sand.jank have — so nothing in the corpus reads
; it today. Kept as a dependency because it is still in the workspace and an
; embed is read by the *checker*, relative to the file the form is written
; in, which is the rule any program picking it up again would meet.
(file %{workspace_root}/brush.png)
; The raylib bindings, because sand.flan and the FFI case import them and an
; import reads the directory at build time. sand.flan itself is above: the
; headless case imports it as a single-file package.
(glob_files %{workspace_root}/vendor/raylib/*)
; The dev agent package: its Flan declarations and the C that implements them.
(glob_files %{workspace_root}/vendor/agent/*)
; The EDN package — the tokenizer and the dynamic reader over it — which
; programs/edn.flan, arena-edn.flan and edn-read.flan import; and the JSON
; tokenizer, which programs/json.flan does.
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/vendor/json/*)
; The ported raylib examples. Only one of them has a headless acceptance
; case, but it imports its example as a package and that example imports
; examples/digits.flan, so the directory has to be here whole.
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
; The package tree the multi-level cases import: pkg-diamond reaches shape
; through area and draw, and pkg-cycle reaches a ring. Each directory is a
; package, so each comes whole — a glob per directory rather than one over
; programs/pkgs/*, because dune's glob does not descend.
(glob_files programs/pkgs/shape/*)
(glob_files programs/pkgs/area/*)
(glob_files programs/pkgs/draw/*)
(glob_files programs/pkgs/ring-a/*)
(glob_files programs/pkgs/ring-b/*)
(glob_files programs/pkgs/ring-c/*)
; The package that exports a data type, which pkg-data.flan imports.
(glob_files programs/pkgs/tree/*)
; The packages that declare macros: one whose macros a program calls
; qualified, and the two whose macros do not terminate — a ring, and one
; that never settles. Each is its own directory, so each needs its own glob.
(glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*)
; The synthetic C header the importer's table reads. Committed rather than
; reached for on the machine: the raylib case needs raylib installed, at the
; right version, with a variable set, so it skips everywhere and covers
; nothing. This one does not move.
(glob_files headers/*.h)
; The files programs/embed.flan bakes in. An embed reads them at *compile*
; time, so they are a dependency of the checker run and not of the program.
(glob_files programs/assets/*)
; And the tileset programs/edn-read.flan bakes in, which is under assets/ and
; not in it: embed.flan holds (embed-dir "assets") in a [3 EmbedFile], so a
; fourth file beside those three is a type error in an unrelated program.
; embed-dir does not descend and neither does a glob, so this is its own line.
(glob_files programs/assets/edn/*)
; The reload primitive's host: a C main that dlopens what Build.shared made.
(file reload_host.c)
; A shared object that is not a redefinition module, for the agent's refusal
; path. Its destructor is what proves the handle was closed rather than lost.
(file noinstall.c)
; The other C main: flan_dev.c's two fixed limits, which no Flan program
; reaches, driven directly.
(file dev_limits.c)
; And the third: the dynamic-value runtime, which has no Flan spelling yet
; at all. Its host program is under programs/ and is picked up by the glob
; above; the header dyn_ops.c includes is the compiler's, dropped into the
; build directory beside each translation unit, so it is not a dependency
; here.
(file dyn_ops.c)
; test_dev runs the compiler itself: flan dev launches and owns a program.
(file %{workspace_root}/bin/main.exe)
; The Emacs client, which test_emacs drives against a real daemon.
(glob_files %{workspace_root}/emacs/*.el)
; The checker's own source, which test_flan reads: there is no reflecting
; over a match, so the only way to assert that Check.builtins still lists
; every builtin arm is to read the arms.
(file %{workspace_root}/lib/check.ml)
; The WASI host the wasm32 case runs its module under, when no wasmtime or
; wasmer is installed.
(file wasm-run.mjs)))
; The web target, in its own stanza rather than in the table above because it
; is the one case whose toolchain is a separate install: emscripten, and a
; raylib archive built by vendor/raylib/build-web.sh. It probes for both and
; skips with the reason, so it is green on a machine that has neither.
(test
(name test_web)
(modules test_web)
(libraries flan unix)
(deps
(glob_files programs/*.flan)
(glob_files programs/assets/*)
(glob_files programs/assets/edn/*)
; The raylib bindings and the ported example the raylib case builds. The
; example imports examples/digits.flan, so the directory comes whole.
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/examples/*)
; sand.flan for the browser, with the sheet it embeds and the dev agent it
; imports — the agent's directory has to be whole, because the file that
; makes a web build possible is the one Build selects out of it.
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/agent/*)
; flan run --target=web is refused by the CLI, so the CLI has to be here.
(file %{workspace_root}/bin/main.exe)))
; The corpus a second time under ASan and UBSan. Its own alias and not part of
; `dune test`: a sanitized build is a statically linked 1.8MB binary that takes
; tens of seconds to produce, so the sweep is minutes against the existing
; suite's seconds, and a test nobody will wait for is a test nobody runs.
;
; dune build --root . @sanitize
; An executable plus a rule rather than a (test ...): a test stanza attaches
; to the @runtest alias and offers no way to be attached to another one, which
; is the whole point here.
(executable
(name test_sanitize)
(modules test_sanitize watchdog)
(libraries flan unix))
(rule
(alias sanitize)
(deps
test_sanitize.exe
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/vendor/json/*)
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
(glob_files programs/assets/*)
(glob_files programs/assets/edn/*)
; p13-dyn-collect.flan, which lives with the x86 probes because that is the
; lane that wrote it, and is in this sweep because of what it does rather
; than where it is: it is the only program anywhere that allocates past
; flan_dyn.c's one-megabyte floor, so it is the only one under which a mark
; and a sweep actually run. Every other Flan program here agrees with ASan
; by never collecting at all.
(glob_files %{workspace_root}/spike/x86/*.flan)
; The dyn runtime's C main, which is the one thing in this sweep that is not
; a Flan program. It is also the translation unit here that frees the most,
; which is what makes it worth a sanitized run at all. See [dyn_sweep].
(file dyn_ops.c))
(action (run ./test_sanitize.exe)))
; The corpus a third time, under Valgrind's memcheck. Its own alias for the
; same reason @sanitize has one, only more so: memcheck runs the program on a
; synthetic CPU, so the corpus is tens of minutes rather than seconds.
;
; dune build --root . @valgrind
;
; Why a third sweep when @sanitize exists: ASan answers "is this address
; mine", and cannot answer "were these bytes ever written". That second
; question is MSan's, MSan needs every dependency instrumented and raylib
; settles it, and memcheck answers both while needing no instrumentation at
; all. NEXT.md asked for exactly this.
(executable
(name test_valgrind)
(modules test_valgrind watchdog)
(libraries flan unix str))
(rule
(alias valgrind)
(deps
test_valgrind.exe
; The suppression file, which is all reasons and no suppressions; its own
; header says why that is the finding rather than an oversight.
(file valgrind.supp)
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/vendor/json/*)
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
(glob_files programs/assets/*)
(glob_files programs/assets/edn/*)
; The package tree the multi-level cases import, as in the test stanza
; above: a glob per directory, because dune's glob does not descend.
(glob_files programs/pkgs/shape/*)
(glob_files programs/pkgs/area/*)
(glob_files programs/pkgs/draw/*)
(glob_files programs/pkgs/ring-a/*)
(glob_files programs/pkgs/ring-b/*)
(glob_files programs/pkgs/ring-c/*)
(glob_files programs/pkgs/tree/*)
; And the macro-declaring packages, for the same reason.
(glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*))
(action (run ./test_valgrind.exe)))
; The corpus a fourth time, through the hand-written x86-64 backend, compared
; against LLVM on what each program prints and what it exits with. Its own
; alias for the same reason the two above have one -- it builds every program
; twice and runs both, which is a couple of minutes against `dune test`'s
; seconds -- but the reason it exists at all is different. @sanitize and
; @valgrind ask whether the runtime is sound. This one asks whether the
; second backend still lowers the language: it refuses by name rather than
; miscompiling, so when another lane adds a primitive the backend says so
; loudly, and nothing was listening. Two such refusals sat in the tree for a
; month. Now they fail a build somebody can run.
;
; dune build --root . @x86
;
; A rule with no executable beside it, unlike @sanitize and @valgrind: the
; check already exists as spike/x86/survey.sh, which is what every handoff
; quotes its counts from, and a second implementation in OCaml would be a
; second thing to drift. SURVEY_STRICT=1 turns its report into an exit
; status. FLAN is passed because the script otherwise runs `dune build` on
; the compiler, and a dune inside a dune action waits on a lock it cannot
; get; main.exe is in the deps instead. SURVEY_QUIET keeps the skip
; breakdown out of a passing build's log.
(rule
(alias x86)
(deps
(file %{workspace_root}/spike/x86/survey.sh)
(glob_files %{workspace_root}/spike/x86/*.flan)
; The js spike's programs are plain flan programs and the sweep reads them
; now, so they have to be in the build tree the sweep runs from -- otherwise
; the glob matches nothing, and nothing is exactly what it was reporting
; while p1-int-semantics.flan sat there with a wrong shift in it.
(glob_files %{workspace_root}/spike/js/*.flan)
(file %{workspace_root}/bin/main.exe)
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/vendor/json/*)
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
(glob_files programs/assets/*)
(glob_files programs/assets/edn/*)
; A glob per package directory, because dune's glob does not descend.
(glob_files programs/pkgs/shape/*)
(glob_files programs/pkgs/area/*)
(glob_files programs/pkgs/draw/*)
(glob_files programs/pkgs/ring-a/*)
(glob_files programs/pkgs/ring-b/*)
(glob_files programs/pkgs/ring-c/*)
(glob_files programs/pkgs/tree/*)
(glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*))
(action
(setenv SURVEY_STRICT 1
(setenv SURVEY_QUIET 1
(setenv FLAN %{workspace_root}/bin/main.exe
(run bash %{workspace_root}/spike/x86/survey.sh))))))
; The reference page, checked against the compiler that is supposed to have
; produced everything on it. Two scripts, one alias, because they are halves of
; the same claim: web/examples/check.sh runs each program the page shows and
; diffs it against the .out recorded beside it, and web/examples/quotes.sh
; re-derives every *other* block -- the usage text, the refusal messages, the
; LLVM excerpt, the keybindings -- and looks for it in web/index.html. A
; paraphrase reads exactly like a quotation, which is the whole reason the
; second one exists.
;
; dune build --root . @page
;
; Opt-in rather than part of `dune test`, and for a reason that is NOT the one
; @sanitize, @valgrind and @x86 give. Those three are minutes; this is eleven
; seconds, five of which is breakdemo's deliberate sleep. Slowness is measured
; and is not the argument here. The argument is what a failure
; means: these two fail when a *document* has gone stale, not when the compiler
; has regressed, and a suite that goes red because prose drifted teaches the
; person running it to skim past red. `dune test` should mean "the language
; broke". This should mean "the page is lying". Keeping them apart is what lets
; both stay worth reading. See @checks below, which is how they get run.
;
; A rule rather than a (test ...) for the reason the @x86 comment gives: a test
; stanza attaches to @runtest and offers no way to be attached to anything else.
; FLAN is passed for the same reason too -- the scripts otherwise run `dune
; build` themselves, and a dune inside a dune action waits on a lock it cannot
; get. Everything either script reads has to be in the deps, because the action
; runs in _build and a glob that came up empty would let check.sh loop over no
; programs and exit 0 -- an alias that passes because it checked nothing.
(rule
(alias page)
(deps
(glob_files %{workspace_root}/web/examples/*)
; pkg.flan imports "geom" as a package, and dune's glob does not descend.
(glob_files %{workspace_root}/web/examples/geom/*)
(file %{workspace_root}/web/index.html)
(file %{workspace_root}/bin/main.exe)
; quotes.sh re-derives its needles from these: calc-me for the arithmetic
; answer, the corpus for the sand hash and the renderer's field spelling,
; conditions.org and flan-mode.el for the lines quoted verbatim.
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/conditions.org)
(glob_files %{workspace_root}/emacs/*.el)
(glob_files programs/*.flan)
; breakdemo.flan is built --dev, so it imports the agent; raylib is here for
; the binding line quotes.sh greps out of it.
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/raylib/*))
(action
(progn
(setenv FLAN %{workspace_root}/bin/main.exe
(run sh %{workspace_root}/web/examples/check.sh))
(setenv FLAN %{workspace_root}/bin/main.exe
(run sh %{workspace_root}/web/examples/quotes.sh)))))
; The indirection cell, driven from outside the language. spike/x86/cells.sh
; preloads a shared object whose constructor stores a different body into
; flan.cell.twice with dlsym, and checks that a --dev build notices and a
; release build does not -- 22 22 against 42 42, both backends, four builds.
; Its own alias and not part of @x86 because it is a different question:
; survey.sh asks whether the backend agrees with LLVM about what a program
; prints, and no program can answer this one, because a dev build starts with
; every cell already pointing at the body this build compiled.
;
; dune build --root . @cells
;
; Wired here for the reason everything on this page is wired: it was a real
; pass/fail check that nothing ran, which is how a check becomes decoration.
(rule
(alias cells)
(deps
(file %{workspace_root}/spike/x86/cells.sh)
(file %{workspace_root}/spike/x86/cell-override.c)
(glob_files %{workspace_root}/spike/x86/*.flan)
(file %{workspace_root}/bin/main.exe))
(action
(setenv FLAN %{workspace_root}/bin/main.exe
(run bash %{workspace_root}/spike/x86/cells.sh))))
; Everything that checks something and is not `dune test`, in one word.
;
; dune build --root . @checks
;
; This exists because the opt-in aliases had the same disease as the scripts
; they replaced. @x86 was added after two backend refusals sat unnoticed for a
; month, and an alias nobody types is no better than a script nobody types --
; it only looks more responsible. Five names to remember is four too many, and
; the cost of remembering them is paid every time, by whoever is least likely
; to know they exist.
;
; What is in it is the whole design. @page, @x86 and @cells: a couple of
; minutes between them, they need nothing installed that a build does not
; already need, and they are exactly the two failures this repository has
; actually had plus the one that was found beside them. @sanitize and @valgrind
; are deliberately out. They are tens of minutes and they want ASan and
; Valgrind present, and folding them in would make @checks the thing you do not
; have time for -- which is the disease, not the cure.
;
; What it buys is that deciding to check and checking everything are now the
; same act, so the gap between "somebody wondered" and "everything was
; verified" is one command instead of five.
;
; It used to say here that nothing runs this, because there was no CI. There is
; now: .github/workflows/checks.yml runs `dune build`, `dune test --force` and
; this alias on every push. That does not make the local run redundant, and the
; workflow says why in as many words — an Ubuntu runner has no raylib, no
; emscripten, no wasi sysroot and no lldb, so every one of those cases takes
; the skip path it already had and the tick is green over less than this alias
; covers here. The habit written down in README.md still carries the rest: a
; lane's handoff quotes this alias, the way the x86 handoffs already quote
; survey.sh.
(alias
(name checks)
(deps
(alias page)
(alias x86)
(alias js)
(alias cells)))
; The JS dialect's sweep, opt-in in the same way and for the same reason:
;
; dune build --root . @js
;
; Shaped after @x86 above, deps and all, because it walks the same corpus.
; Two differences. It needs node, which the script probes for and reports on
; rather than failing over -- this is opt-in, and a machine without node
; should say so and stop, not report zero of everything. And SURVEY_STRICT
; means something narrower here: a refusal is a *decision* in a dialect, not
; a gap, so only a DIFFER and a CRASH fail the alias. See the header of
; spike/js/survey.sh.
(rule
(alias js)
(deps
(file %{workspace_root}/spike/js/survey.sh)
(glob_files %{workspace_root}/spike/js/*.flan)
(file %{workspace_root}/bin/main.exe)
(file %{workspace_root}/calc-me.flan)
(file %{workspace_root}/sand.flan)
(file %{workspace_root}/brush.png)
(glob_files %{workspace_root}/vendor/raylib/*)
(glob_files %{workspace_root}/vendor/agent/*)
(glob_files %{workspace_root}/vendor/edn/*)
(glob_files %{workspace_root}/vendor/json/*)
(glob_files %{workspace_root}/examples/*)
(glob_files programs/*.flan)
(glob_files programs/assets/*)
(glob_files programs/assets/edn/*)
(glob_files programs/pkgs/shape/*)
(glob_files programs/pkgs/area/*)
(glob_files programs/pkgs/draw/*)
(glob_files programs/pkgs/ring-a/*)
(glob_files programs/pkgs/ring-b/*)
(glob_files programs/pkgs/ring-c/*)
(glob_files programs/pkgs/tree/*)
(glob_files programs/pkgs/mac/*)
(glob_files programs/pkgs/macring/*)
(glob_files programs/pkgs/macspin/*))
(action
(setenv SURVEY_STRICT 1
(setenv SURVEY_QUIET 1
(setenv FLAN %{workspace_root}/bin/main.exe
(run bash %{workspace_root}/spike/js/survey.sh))))))