Joseph Ferano
16498a4e60
A GitHub Actions workflow on push: dune build, dune test --force, dune build @checks. @x86 parity is not under dune test, so the routine suite never protected it; both of this repository's silent failures would have been caught by one person typing one command, and the problem was never the command. The suite step keeps its log and greps it for Fatal error, because a suite that passes while leaving an unhandled exception on stderr is one that is telling you something and being ignored. FLAN_LLC is pinned to the llc matching clang's version rather than left to PATH order: the live loop goes llc + ld -shared + dlopen and never calls the clang driver, so a mismatch breaks every reload test while flan build keeps working, which is a bad failure to debug from a log. What an Ubuntu runner cannot cover -- raylib by exact Fedora soname, emscripten, a wasi sysroot, lldb -- is written in the workflow with the skip path each one already takes, so the tick does not read as more than it is. README.md and test/dune both said there was no CI; both now say what there is and what it misses.
341 lines
16 KiB
Plaintext
341 lines
16 KiB
Plaintext
(tests
|
|
(names test_flan test_acceptance test_reload test_agent test_session test_dev test_emacs test_repl test_cider)
|
|
; Explicit because test_sanitize lives in this directory and is not one of
|
|
; these: two stanzas in one directory have to say which modules are whose.
|
|
; watchdog is every binary's clock: a hanging test reports nothing, so each
|
|
; of these arms an alarm that turns "for ever" into a failing run.
|
|
(modules test_flan test_acceptance test_reload test_agent test_session
|
|
test_dev test_emacs test_repl test_cider watchdog)
|
|
(libraries flan unix)
|
|
; The acceptance programs are part of the test corpus: if the reader, the
|
|
; parser or the checker regresses on them we want to know here, not at the CLI.
|
|
(deps
|
|
(file %{workspace_root}/calc-me.flan)
|
|
(file %{workspace_root}/sand.flan)
|
|
; The brush sheet. sand.flan no longer embeds it — the front-end was cut back
|
|
; to what lisp/sand.lisp and sand.jank have — so nothing in the corpus reads
|
|
; it today. Kept as a dependency because it is still in the workspace and an
|
|
; embed is read by the *checker*, relative to the file the form is written
|
|
; in, which is the rule any program picking it up again would meet.
|
|
(file %{workspace_root}/brush.png)
|
|
; The raylib bindings, because sand.flan and the FFI case import them and an
|
|
; import reads the directory at build time. sand.flan itself is above: the
|
|
; headless case imports it as a single-file package.
|
|
(glob_files %{workspace_root}/vendor/raylib/*)
|
|
; The dev agent package: its Flan declarations and the C that implements them.
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
; The EDN tokenizer, which programs/edn.flan imports.
|
|
(glob_files %{workspace_root}/vendor/edn/*)
|
|
; The ported raylib examples. Only one of them has a headless acceptance
|
|
; case, but it imports its example as a package and that example imports
|
|
; examples/digits.flan, so the directory has to be here whole.
|
|
(glob_files %{workspace_root}/examples/*)
|
|
(glob_files programs/*.flan)
|
|
; The package tree the multi-level cases import: pkg-diamond reaches shape
|
|
; through area and draw, and pkg-cycle reaches a ring. Each directory is a
|
|
; package, so each comes whole — a glob per directory rather than one over
|
|
; programs/pkgs/*, because dune's glob does not descend.
|
|
(glob_files programs/pkgs/shape/*)
|
|
(glob_files programs/pkgs/area/*)
|
|
(glob_files programs/pkgs/draw/*)
|
|
(glob_files programs/pkgs/ring-a/*)
|
|
(glob_files programs/pkgs/ring-b/*)
|
|
(glob_files programs/pkgs/ring-c/*)
|
|
; The packages that declare macros: one whose macros a program calls
|
|
; qualified, and the two whose macros do not terminate — a ring, and one
|
|
; that never settles. Each is its own directory, so each needs its own glob.
|
|
(glob_files programs/pkgs/mac/*)
|
|
(glob_files programs/pkgs/macring/*)
|
|
(glob_files programs/pkgs/macspin/*)
|
|
; The synthetic C header the importer's table reads. Committed rather than
|
|
; reached for on the machine: the raylib case needs raylib installed, at the
|
|
; right version, with a variable set, so it skips everywhere and covers
|
|
; nothing. This one does not move.
|
|
(glob_files headers/*.h)
|
|
; The files programs/embed.flan bakes in. An embed reads them at *compile*
|
|
; time, so they are a dependency of the checker run and not of the program.
|
|
(glob_files programs/assets/*)
|
|
; The reload primitive's host: a C main that dlopens what Build.shared made.
|
|
(file reload_host.c)
|
|
; A shared object that is not a redefinition module, for the agent's refusal
|
|
; path. Its destructor is what proves the handle was closed rather than lost.
|
|
(file noinstall.c)
|
|
; The other C main: flan_dev.c's two fixed limits, which no Flan program
|
|
; reaches, driven directly.
|
|
(file dev_limits.c)
|
|
; test_dev runs the compiler itself: flan dev launches and owns a program.
|
|
(file %{workspace_root}/bin/main.exe)
|
|
; The Emacs client, which test_emacs drives against a real daemon.
|
|
(glob_files %{workspace_root}/emacs/*.el)
|
|
; The WASI host the wasm32 case runs its module under, when no wasmtime or
|
|
; wasmer is installed.
|
|
(file wasm-run.mjs)))
|
|
|
|
; The web target, in its own stanza rather than in the table above because it
|
|
; is the one case whose toolchain is a separate install: emscripten, and a
|
|
; raylib archive built by vendor/raylib/build-web.sh. It probes for both and
|
|
; skips with the reason, so it is green on a machine that has neither.
|
|
(test
|
|
(name test_web)
|
|
(modules test_web)
|
|
(libraries flan unix)
|
|
(deps
|
|
(glob_files programs/*.flan)
|
|
(glob_files programs/assets/*)
|
|
; The raylib bindings and the ported example the raylib case builds. The
|
|
; example imports examples/digits.flan, so the directory comes whole.
|
|
(glob_files %{workspace_root}/vendor/raylib/*)
|
|
(glob_files %{workspace_root}/examples/*)
|
|
; sand.flan for the browser, with the sheet it embeds and the dev agent it
|
|
; imports — the agent's directory has to be whole, because the file that
|
|
; makes a web build possible is the one Build selects out of it.
|
|
(file %{workspace_root}/sand.flan)
|
|
(file %{workspace_root}/brush.png)
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
; flan run --target=web is refused by the CLI, so the CLI has to be here.
|
|
(file %{workspace_root}/bin/main.exe)))
|
|
|
|
; The corpus a second time under ASan and UBSan. Its own alias and not part of
|
|
; `dune test`: a sanitized build is a statically linked 1.8MB binary that takes
|
|
; tens of seconds to produce, so the sweep is minutes against the existing
|
|
; suite's seconds, and a test nobody will wait for is a test nobody runs.
|
|
;
|
|
; dune build --root . @sanitize
|
|
; An executable plus a rule rather than a (test ...): a test stanza attaches
|
|
; to the @runtest alias and offers no way to be attached to another one, which
|
|
; is the whole point here.
|
|
(executable
|
|
(name test_sanitize)
|
|
(modules test_sanitize watchdog)
|
|
(libraries flan unix))
|
|
|
|
(rule
|
|
(alias sanitize)
|
|
(deps
|
|
test_sanitize.exe
|
|
(file %{workspace_root}/calc-me.flan)
|
|
(file %{workspace_root}/sand.flan)
|
|
(file %{workspace_root}/brush.png)
|
|
(glob_files %{workspace_root}/vendor/raylib/*)
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
(glob_files %{workspace_root}/vendor/edn/*)
|
|
(glob_files %{workspace_root}/examples/*)
|
|
(glob_files programs/*.flan)
|
|
(glob_files programs/assets/*))
|
|
(action (run ./test_sanitize.exe)))
|
|
|
|
; The corpus a third time, under Valgrind's memcheck. Its own alias for the
|
|
; same reason @sanitize has one, only more so: memcheck runs the program on a
|
|
; synthetic CPU, so the corpus is tens of minutes rather than seconds.
|
|
;
|
|
; dune build --root . @valgrind
|
|
;
|
|
; Why a third sweep when @sanitize exists: ASan answers "is this address
|
|
; mine", and cannot answer "were these bytes ever written". That second
|
|
; question is MSan's, MSan needs every dependency instrumented and raylib
|
|
; settles it, and memcheck answers both while needing no instrumentation at
|
|
; all. NEXT.md asked for exactly this.
|
|
(executable
|
|
(name test_valgrind)
|
|
(modules test_valgrind watchdog)
|
|
(libraries flan unix str))
|
|
|
|
(rule
|
|
(alias valgrind)
|
|
(deps
|
|
test_valgrind.exe
|
|
; The suppression file, which is all reasons and no suppressions; its own
|
|
; header says why that is the finding rather than an oversight.
|
|
(file valgrind.supp)
|
|
(file %{workspace_root}/calc-me.flan)
|
|
(file %{workspace_root}/sand.flan)
|
|
(file %{workspace_root}/brush.png)
|
|
(glob_files %{workspace_root}/vendor/raylib/*)
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
(glob_files %{workspace_root}/vendor/edn/*)
|
|
(glob_files %{workspace_root}/examples/*)
|
|
(glob_files programs/*.flan)
|
|
(glob_files programs/assets/*)
|
|
; The package tree the multi-level cases import, as in the test stanza
|
|
; above: a glob per directory, because dune's glob does not descend.
|
|
(glob_files programs/pkgs/shape/*)
|
|
(glob_files programs/pkgs/area/*)
|
|
(glob_files programs/pkgs/draw/*)
|
|
(glob_files programs/pkgs/ring-a/*)
|
|
(glob_files programs/pkgs/ring-b/*)
|
|
(glob_files programs/pkgs/ring-c/*)
|
|
; And the macro-declaring packages, for the same reason.
|
|
(glob_files programs/pkgs/mac/*)
|
|
(glob_files programs/pkgs/macring/*)
|
|
(glob_files programs/pkgs/macspin/*))
|
|
(action (run ./test_valgrind.exe)))
|
|
|
|
; The corpus a fourth time, through the hand-written x86-64 backend, compared
|
|
; against LLVM on what each program prints and what it exits with. Its own
|
|
; alias for the same reason the two above have one -- it builds every program
|
|
; twice and runs both, which is a couple of minutes against `dune test`'s
|
|
; seconds -- but the reason it exists at all is different. @sanitize and
|
|
; @valgrind ask whether the runtime is sound. This one asks whether the
|
|
; second backend still lowers the language: it refuses by name rather than
|
|
; miscompiling, so when another lane adds a primitive the backend says so
|
|
; loudly, and nothing was listening. Two such refusals sat in the tree for a
|
|
; month. Now they fail a build somebody can run.
|
|
;
|
|
; dune build --root . @x86
|
|
;
|
|
; A rule with no executable beside it, unlike @sanitize and @valgrind: the
|
|
; check already exists as spike/x86/survey.sh, which is what every handoff
|
|
; quotes its counts from, and a second implementation in OCaml would be a
|
|
; second thing to drift. SURVEY_STRICT=1 turns its report into an exit
|
|
; status. FLAN is passed because the script otherwise runs `dune build` on
|
|
; the compiler, and a dune inside a dune action waits on a lock it cannot
|
|
; get; main.exe is in the deps instead. SURVEY_QUIET keeps the skip
|
|
; breakdown out of a passing build's log.
|
|
(rule
|
|
(alias x86)
|
|
(deps
|
|
(file %{workspace_root}/spike/x86/survey.sh)
|
|
(glob_files %{workspace_root}/spike/x86/*.flan)
|
|
(file %{workspace_root}/bin/main.exe)
|
|
(file %{workspace_root}/calc-me.flan)
|
|
(file %{workspace_root}/sand.flan)
|
|
(file %{workspace_root}/brush.png)
|
|
(glob_files %{workspace_root}/vendor/raylib/*)
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
(glob_files %{workspace_root}/vendor/edn/*)
|
|
(glob_files %{workspace_root}/examples/*)
|
|
(glob_files programs/*.flan)
|
|
(glob_files programs/assets/*)
|
|
; A glob per package directory, because dune's glob does not descend.
|
|
(glob_files programs/pkgs/shape/*)
|
|
(glob_files programs/pkgs/area/*)
|
|
(glob_files programs/pkgs/draw/*)
|
|
(glob_files programs/pkgs/ring-a/*)
|
|
(glob_files programs/pkgs/ring-b/*)
|
|
(glob_files programs/pkgs/ring-c/*)
|
|
(glob_files programs/pkgs/mac/*)
|
|
(glob_files programs/pkgs/macring/*)
|
|
(glob_files programs/pkgs/macspin/*))
|
|
(action
|
|
(setenv SURVEY_STRICT 1
|
|
(setenv SURVEY_QUIET 1
|
|
(setenv FLAN %{workspace_root}/bin/main.exe
|
|
(run bash %{workspace_root}/spike/x86/survey.sh))))))
|
|
|
|
; The reference page, checked against the compiler that is supposed to have
|
|
; produced everything on it. Two scripts, one alias, because they are halves of
|
|
; the same claim: web/examples/check.sh runs each program the page shows and
|
|
; diffs it against the .out recorded beside it, and web/examples/quotes.sh
|
|
; re-derives every *other* block -- the usage text, the refusal messages, the
|
|
; LLVM excerpt, the keybindings -- and looks for it in web/index.html. A
|
|
; paraphrase reads exactly like a quotation, which is the whole reason the
|
|
; second one exists.
|
|
;
|
|
; dune build --root . @page
|
|
;
|
|
; Opt-in rather than part of `dune test`, and for a reason that is NOT the one
|
|
; @sanitize, @valgrind and @x86 give. Those three are minutes; this is eleven
|
|
; seconds, five of which is breakdemo's deliberate sleep. Slowness is measured
|
|
; and is not the argument here. The argument is what a failure
|
|
; means: these two fail when a *document* has gone stale, not when the compiler
|
|
; has regressed, and a suite that goes red because prose drifted teaches the
|
|
; person running it to skim past red. `dune test` should mean "the language
|
|
; broke". This should mean "the page is lying". Keeping them apart is what lets
|
|
; both stay worth reading. See @checks below, which is how they get run.
|
|
;
|
|
; A rule rather than a (test ...) for the reason the @x86 comment gives: a test
|
|
; stanza attaches to @runtest and offers no way to be attached to anything else.
|
|
; FLAN is passed for the same reason too -- the scripts otherwise run `dune
|
|
; build` themselves, and a dune inside a dune action waits on a lock it cannot
|
|
; get. Everything either script reads has to be in the deps, because the action
|
|
; runs in _build and a glob that came up empty would let check.sh loop over no
|
|
; programs and exit 0 -- an alias that passes because it checked nothing.
|
|
(rule
|
|
(alias page)
|
|
(deps
|
|
(glob_files %{workspace_root}/web/examples/*)
|
|
; pkg.flan imports "geom" as a package, and dune's glob does not descend.
|
|
(glob_files %{workspace_root}/web/examples/geom/*)
|
|
(file %{workspace_root}/web/index.html)
|
|
(file %{workspace_root}/bin/main.exe)
|
|
; quotes.sh re-derives its needles from these: calc-me for the arithmetic
|
|
; answer, the corpus for the sand hash and the renderer's field spelling,
|
|
; conditions.org and flan-mode.el for the lines quoted verbatim.
|
|
(file %{workspace_root}/calc-me.flan)
|
|
(file %{workspace_root}/conditions.org)
|
|
(glob_files %{workspace_root}/emacs/*.el)
|
|
(glob_files programs/*.flan)
|
|
; breakdemo.flan is built --dev, so it imports the agent; raylib is here for
|
|
; the binding line quotes.sh greps out of it.
|
|
(glob_files %{workspace_root}/vendor/agent/*)
|
|
(glob_files %{workspace_root}/vendor/raylib/*))
|
|
(action
|
|
(progn
|
|
(setenv FLAN %{workspace_root}/bin/main.exe
|
|
(run sh %{workspace_root}/web/examples/check.sh))
|
|
(setenv FLAN %{workspace_root}/bin/main.exe
|
|
(run sh %{workspace_root}/web/examples/quotes.sh)))))
|
|
|
|
; The indirection cell, driven from outside the language. spike/x86/cells.sh
|
|
; preloads a shared object whose constructor stores a different body into
|
|
; flan.cell.twice with dlsym, and checks that a --dev build notices and a
|
|
; release build does not -- 22 22 against 42 42, both backends, four builds.
|
|
; Its own alias and not part of @x86 because it is a different question:
|
|
; survey.sh asks whether the backend agrees with LLVM about what a program
|
|
; prints, and no program can answer this one, because a dev build starts with
|
|
; every cell already pointing at the body this build compiled.
|
|
;
|
|
; dune build --root . @cells
|
|
;
|
|
; Wired here for the reason everything on this page is wired: it was a real
|
|
; pass/fail check that nothing ran, which is how a check becomes decoration.
|
|
(rule
|
|
(alias cells)
|
|
(deps
|
|
(file %{workspace_root}/spike/x86/cells.sh)
|
|
(file %{workspace_root}/spike/x86/cell-override.c)
|
|
(glob_files %{workspace_root}/spike/x86/*.flan)
|
|
(file %{workspace_root}/bin/main.exe))
|
|
(action
|
|
(setenv FLAN %{workspace_root}/bin/main.exe
|
|
(run bash %{workspace_root}/spike/x86/cells.sh))))
|
|
|
|
; Everything that checks something and is not `dune test`, in one word.
|
|
;
|
|
; dune build --root . @checks
|
|
;
|
|
; This exists because the opt-in aliases had the same disease as the scripts
|
|
; they replaced. @x86 was added after two backend refusals sat unnoticed for a
|
|
; month, and an alias nobody types is no better than a script nobody types --
|
|
; it only looks more responsible. Five names to remember is four too many, and
|
|
; the cost of remembering them is paid every time, by whoever is least likely
|
|
; to know they exist.
|
|
;
|
|
; What is in it is the whole design. @page, @x86 and @cells: a couple of
|
|
; minutes between them, they need nothing installed that a build does not
|
|
; already need, and they are exactly the two failures this repository has
|
|
; actually had plus the one that was found beside them. @sanitize and @valgrind
|
|
; are deliberately out. They are tens of minutes and they want ASan and
|
|
; Valgrind present, and folding them in would make @checks the thing you do not
|
|
; have time for -- which is the disease, not the cure.
|
|
;
|
|
; What it buys is that deciding to check and checking everything are now the
|
|
; same act, so the gap between "somebody wondered" and "everything was
|
|
; verified" is one command instead of five.
|
|
;
|
|
; It used to say here that nothing runs this, because there was no CI. There is
|
|
; now: .github/workflows/checks.yml runs `dune build`, `dune test --force` and
|
|
; this alias on every push. That does not make the local run redundant, and the
|
|
; workflow says why in as many words — an Ubuntu runner has no raylib, no
|
|
; emscripten, no wasi sysroot and no lldb, so every one of those cases takes
|
|
; the skip path it already had and the tick is green over less than this alias
|
|
; covers here. The habit written down in README.md still carries the rest: a
|
|
; lane's handoff quotes this alias, the way the x86 handoffs already quote
|
|
; survey.sh.
|
|
(alias
|
|
(name checks)
|
|
(deps
|
|
(alias page)
|
|
(alias x86)
|
|
(alias cells)))
|