; The corpus: every file a sweep over test/programs can reach, in one place. ; Each stanza below that walks the programs depends on this alias rather than ; listing the files, so a package or an asset directory added under programs/ ; is in every sweep without a line anywhere else. source_tree rather than a ; glob per directory, because glob_files does not descend and every package is ; a directory. (alias (name corpus) (deps (source_tree programs) ; The acceptance programs are part of the test corpus: if the reader, the ; parser or the checker regresses on them we want to know here, not at the ; CLI. (file %{workspace_root}/calc-me.flan) (file %{workspace_root}/sand.fln) ; The brush sheet. sand.fln no longer embeds it — the front-end was cut back ; to what lisp/sand.lisp and sand.jank have — so nothing in the corpus reads ; it today. Kept as a dependency because it is still in the workspace and an ; embed is read by the *checker*, relative to the file the form is written ; in, which is the rule any program picking it up again would meet. (file %{workspace_root}/brush.png) ; The raylib bindings, because sand.fln and the FFI case import them and an ; import reads the directory at build time. sand.fln itself is above: the ; headless case imports it as a single-file package. (glob_files %{workspace_root}/vendor/raylib/*) ; rlgl's matrix stack, which an example imports beside raylib. (glob_files %{workspace_root}/vendor/rlgl/*) ; The dev agent package: its Flan declarations and the C that implements them. (glob_files %{workspace_root}/vendor/agent/*) ; The EDN package — the tokenizer and the dynamic reader over it — which ; programs/edn.flan and edn-read.flan import; and the JSON ; tokenizer, which programs/json.flan does. (glob_files %{workspace_root}/vendor/edn/*) (glob_files %{workspace_root}/vendor/json/*) ; The ported raylib examples. Only one of them has a headless acceptance ; case, but it imports its example as a package and that example imports ; examples/digits.flan, so the directory has to be here whole. (glob_files %{workspace_root}/examples/*))) (tests (names test_flan test_acceptance test_reload test_agent test_session test_dev test_emacs test_repl test_cider test_dyn) ; Explicit because test_sanitize lives in this directory and is not one of ; these: two stanzas in one directory have to say which modules are whose. ; watchdog is every binary's clock: a hanging test reports nothing, so each ; of these arms an alarm that turns "for ever" into a failing run. test_support ; is the other shared module: the failure counter, the report tail, the poll, ; the daemon wait and the front half of a compile, which the binaries that ; wanted them had each been carrying their own copy of. (modules test_flan test_acceptance test_reload test_agent test_session test_dev test_emacs test_repl test_cider test_dyn watchdog test_support own_tmp) (libraries flan unix) (deps (alias corpus) ; The synthetic C header the importer's table reads. Committed rather than ; reached for on the machine: the raylib case needs raylib installed, at the ; right version, with a variable set, so it skips everywhere and covers ; nothing. This one does not move. (glob_files headers/*.h) ; The reload primitive's host: a C main that dlopens what Build.shared made. (file reload_host.c) ; A shared object that is not a redefinition module, for the agent's refusal ; path. Its destructor is what proves the handle was closed rather than lost. (file noinstall.c) ; The other C main: flan_dev.c's two fixed limits, which no Flan program ; reaches, driven directly. (file dev_limits.c) ; The break loop driven from inside the stopped thread, through the agent's ; poll hook. (file agent_hooks.c) ; And the third: the dynamic-value runtime, which has no Flan spelling yet ; at all. Its host program is under programs/ and comes in with the ; corpus; the header dyn_ops.c includes is the compiler's, dropped into the ; build directory beside each translation unit, so it is not a dependency ; here. (file dyn_ops.c) ; test_dev runs the compiler itself: flan dev launches and owns a program. (file %{workspace_root}/bin/main.exe) ; The Emacs client, which test_emacs drives against a real daemon. (glob_files %{workspace_root}/emacs/*.el) ; The checker's own source, which test_flan reads: there is no reflecting ; over a match, so the only way to assert that Check.builtins still lists ; every builtin arm is to read the arms. (file %{workspace_root}/lib/check.ml) ; The WASI host the wasm32 case runs its module under, when no wasmtime or ; wasmer is installed. (file wasm-run.mjs))) ; The web target, in its own stanza rather than in the table above because it ; is the one case whose toolchain is a separate install: emscripten, and a ; raylib archive built by vendor/raylib/build-web.sh. It probes for both and ; skips with the reason, so it is green on a machine that has neither. (test (name test_web) (modules test_web test_support own_tmp) (libraries flan unix) (deps (alias corpus) ; flan run --target=web is refused by the CLI, so the CLI has to be here. (file %{workspace_root}/bin/main.exe))) ; The indented syntax: its two hand-converted programs, the paren -> indented ; -> paren round trip over every .flan the build tree holds, and the import ; programs in test/syntax/mixed. Its own stanza for its deps: the round trip ; walks every directory that has a .flan in it, which is more than the corpus ; alias carries, and the other binaries have no use for the rest. (test (name test_syntax) (modules test_syntax test_support watchdog own_tmp) (libraries flan unix) (deps (alias corpus) (source_tree syntax) (file %{workspace_root}/conditions-play.flan) (glob_files %{workspace_root}/web/examples/*.flan) (glob_files %{workspace_root}/web/examples/geom/*.flan))) ; The corpus a second time under ASan and UBSan. Its own alias and not part of ; `dune test`: a sanitized build is a statically linked 1.8MB binary that takes ; tens of seconds to produce, so the sweep is minutes against the existing ; suite's seconds, and a test nobody will wait for is a test nobody runs. ; ; dune build --root . @sanitize ; An executable plus a rule rather than a (test ...): a test stanza attaches ; to the @runtest alias and offers no way to be attached to another one, which ; is the whole point here. (executable (name test_sanitize) (modules test_sanitize watchdog test_support own_tmp) (libraries flan unix)) (rule (alias sanitize) (deps (alias corpus) test_sanitize.exe ; ASAN_OPTIONS=detect_leaks=1:detect_stack_use_after_return=1 asks the leak ; question and keeps the frame check; see test_sanitize.ml. (env_var ASAN_OPTIONS) ; The dyn runtime's C main, which is the one thing in this sweep that is not ; a Flan program: flan_dyn.c has no Flan spelling yet. It is also the one ; translation unit here that frees the most, which is what makes it worth a ; sanitized run at all. See [dyn_sweep]. (file dyn_ops.c) ; [dev_session] drives a real flan dev --sanitize. (file %{workspace_root}/bin/main.exe)) (action (run ./test_sanitize.exe))) ; The corpus a third time, under Valgrind's memcheck. Its own alias for the ; same reason @sanitize has one, only more so: memcheck runs the program on a ; synthetic CPU, so the corpus is tens of minutes rather than seconds. ; ; dune build --root . @valgrind ; ; Why a third sweep when @sanitize exists: ASan answers "is this address ; mine", and cannot answer "were these bytes ever written". That second ; question is MSan's, MSan needs every dependency instrumented and raylib ; settles it, and memcheck answers both while needing no instrumentation at ; all. TODO.org, "ASan does not see an uninitialised read; valgrind does", ; asked for exactly this. (executable (name test_valgrind) (modules test_valgrind watchdog test_support own_tmp) (libraries flan unix str)) (rule (alias valgrind) (deps (alias corpus) test_valgrind.exe ; The suppression file, which is all reasons and no suppressions; its own ; header says why that is the finding rather than an oversight. (file valgrind.supp) ; FLAN_LEAKS=1 turns memcheck's leak check on; see test_valgrind.ml. (env_var FLAN_LEAKS)) (action (run ./test_valgrind.exe))) ; The corpus a fourth time, through the hand-written x86-64 backend, compared ; against LLVM on what each program prints and what it exits with. Its own ; alias for the same reason the two above have one -- it builds every program ; twice and runs both, which is a couple of minutes against `dune test`'s ; seconds -- but the reason it exists at all is different. @sanitize and ; @valgrind ask whether the runtime is sound. This one asks whether the ; second backend still lowers the language: it refuses by name rather than ; miscompiling, so when another lane adds a primitive the backend says so ; loudly, and nothing was listening. Two such refusals sat in the tree for a ; month. Now they fail a build somebody can run. The LLVM side is built at ; -O0, the level this backend corresponds to. ; ; dune build --root . @x86 ; ; A rule with no executable beside it, unlike @sanitize and @valgrind: the ; check already exists as survey-x86.sh, which is what every handoff ; quotes its counts from, and a second implementation in OCaml would be a ; second thing to drift. SURVEY_STRICT=1 turns its report into an exit ; status. FLAN is passed because the script otherwise runs `dune build` on ; the compiler, and a dune inside a dune action waits on a lock it cannot ; get; main.exe is in the deps instead. SURVEY_QUIET keeps the skip ; breakdown out of a passing build's log. (rule (alias x86) (deps (alias corpus) (file survey-x86.sh) (file %{workspace_root}/bin/main.exe)) (action (setenv SURVEY_STRICT 1 (setenv SURVEY_QUIET 1 (setenv FLAN %{workspace_root}/bin/main.exe (run bash survey-x86.sh)))))) ; The reference page, checked against the compiler that is supposed to have ; produced everything on it. Two scripts, one alias, because they are halves of ; the same claim: web/examples/check.sh runs each program the page shows and ; diffs it against the .out recorded beside it, and web/examples/quotes.sh ; re-derives every *other* block -- the usage text, the refusal messages, the ; LLVM excerpt, the keybindings -- and looks for it in web/index.html. A ; paraphrase reads exactly like a quotation, which is the whole reason the ; second one exists. ; ; dune build --root . @page ; ; Opt-in rather than part of `dune test`, and for a reason that is NOT the one ; @sanitize, @valgrind and @x86 give. Those three are minutes; this is eleven ; seconds, five of which is breakdemo's deliberate sleep. Slowness is measured ; and is not the argument here. The argument is what a failure ; means: these two fail when a *document* has gone stale, not when the compiler ; has regressed, and a suite that goes red because prose drifted teaches the ; person running it to skim past red. `dune test` should mean "the language ; broke". This should mean "the page is lying". Keeping them apart is what lets ; both stay worth reading. See @checks below, which is how they get run. ; ; A rule rather than a (test ...) for the reason the @x86 comment gives: a test ; stanza attaches to @runtest and offers no way to be attached to anything else. ; FLAN is passed for the same reason too -- the scripts otherwise run `dune ; build` themselves, and a dune inside a dune action waits on a lock it cannot ; get. Everything either script reads has to be in the deps, because the action ; runs in _build and a glob that came up empty would let check.sh loop over no ; programs and exit 0 -- an alias that passes because it checked nothing. (rule (alias page) (deps (glob_files %{workspace_root}/web/examples/*) ; pkg.flan imports "geom" as a package, and dune's glob does not descend. (glob_files %{workspace_root}/web/examples/geom/*) (file %{workspace_root}/web/index.html) (file %{workspace_root}/bin/main.exe) ; quotes.sh re-derives its needles from these: calc-me for the arithmetic ; answer, the corpus for the sand hash and the renderer's field spelling, ; conditions.org and flan-mode.el for the lines quoted verbatim. (file %{workspace_root}/calc-me.flan) (file %{workspace_root}/conditions.org) (glob_files %{workspace_root}/emacs/*.el) (source_tree programs) ; And sand.fln itself, which the hash comes from: quotes.sh runs ; test/programs/sand-headless.flan and that imports ../../sand.fln as a ; single-file package. Missing until now, the same hole @sanitize had — the ; alias was green only because `dune test` had already put the file in ; _build, and in CI it sits inside @checks where the same thing hides it. (file %{workspace_root}/sand.fln) ; And the package sand.fln imports, for the same reason: without it the ; headless program cannot resolve vendor:edn in _build and the sand-hash ; needle comes up empty, which quotes.sh reports as "whatever this quotes ; has moved" rather than as a missing dependency. (glob_files %{workspace_root}/vendor/edn/*) ; breakdemo.flan is built --dev, so it imports the agent; raylib is here for ; the binding line quotes.sh greps out of it. (glob_files %{workspace_root}/vendor/agent/*) (glob_files %{workspace_root}/vendor/raylib/*)) (action (progn (setenv FLAN %{workspace_root}/bin/main.exe (run sh %{workspace_root}/web/examples/check.sh)) (setenv FLAN %{workspace_root}/bin/main.exe (run sh %{workspace_root}/web/examples/quotes.sh))))) ; The indirection cell, driven from outside the language. cells.sh ; preloads a shared object whose constructor stores a different body into ; flan.cell.twice with dlsym, and checks that a --dev build notices and a ; release build does not -- 22 22 against 42 42, both backends, four builds. ; Its own alias and not part of @x86 because it is a different question: ; survey.sh asks whether the backend agrees with LLVM about what a program ; prints, and no program can answer this one, because a dev build starts with ; every cell already pointing at the body this build compiled. ; ; dune build --root . @cells ; ; Wired here for the reason everything on this page is wired: it was a real ; pass/fail check that nothing ran, which is how a check becomes decoration. (rule (alias cells) (deps (file cells.sh) (file cell-override.c) (file programs/x86-p8-cell.flan) (file %{workspace_root}/bin/main.exe)) (action (setenv FLAN %{workspace_root}/bin/main.exe (run bash cells.sh)))) ; Everything that checks something and is not `dune test`, in one word. ; ; dune build --root . @checks ; ; This exists because the opt-in aliases had the same disease as the scripts ; they replaced. @x86 was added after two backend refusals sat unnoticed for a ; month, and an alias nobody types is no better than a script nobody types -- ; it only looks more responsible. Five names to remember is four too many, and ; the cost of remembering them is paid every time, by whoever is least likely ; to know they exist. ; ; What is in it is the whole design. @page, @x86 and @cells: a couple of ; minutes between them, they need nothing installed that a build does not ; already need, and they are exactly the two failures this repository has ; actually had plus the one that was found beside them. @sanitize and @valgrind ; are deliberately out. They are tens of minutes and they want ASan and ; Valgrind present, and folding them in would make @checks the thing you do not ; have time for -- which is the disease, not the cure. ; ; What it buys is that deciding to check and checking everything are now the ; same act, so the gap between "somebody wondered" and "everything was ; verified" is one command instead of five. ; ; It used to say here that nothing runs this, because there was no CI. There is ; now: .github/workflows/checks.yml runs `dune build`, `dune test --force` and ; this alias on every push. That does not make the local run redundant, and the ; workflow says why in as many words — an Ubuntu runner has no raylib, no ; emscripten, no wasi sysroot and no lldb, so every one of those cases takes ; the skip path it already had and the tick is green over less than this alias ; covers here. The habit written down in README.md still carries the rest: a ; lane's handoff quotes this alias, the way the x86 handoffs already quote ; survey.sh. (alias (name checks) (deps (alias page) (alias x86) (alias js) (alias cells))) ; The JS dialect's sweep, opt-in in the same way and for the same reason: ; ; dune build --root . @js ; ; Shaped after @x86 above, deps and all, because it walks the same corpus. ; Two differences. It needs node, which the script probes for and reports on ; rather than failing over -- this is opt-in, and a machine without node ; should say so and stop, not report zero of everything. And SURVEY_STRICT ; means something narrower here: a refusal is a *decision* in a dialect, not ; a gap, so only a DIFFER and a CRASH fail the alias. See the header of ; survey-js.sh. (rule (alias js) (deps (alias corpus) (file survey-js.sh) (file %{workspace_root}/bin/main.exe)) (action (setenv SURVEY_STRICT 1 (setenv SURVEY_QUIET 1 (setenv FLAN %{workspace_root}/bin/main.exe (run bash survey-js.sh))))))