diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml new file mode 100644 index 0000000..f8292f5 --- /dev/null +++ b/.github/workflows/checks.yml @@ -0,0 +1,148 @@ +# The thing README.md and test/dune both said did not exist. +# +# `dune build @checks` on every push, which is the whole job. Both of this +# repository's silent failures — two x86 refusals that sat for a month, a page +# of examples that stopped compiling for two days — would have been caught by +# one person typing one command, and the argument for CI here is not process: +# it is that nobody typed it. @x86 parity in particular is *not* under +# `dune test`, so the routine suite does not protect it. +# +# `dune test` stays what it is: seconds, run constantly, unchanged by this +# file. The workflow adds a caller, not an alias. +# +# ── What this does not cover, and why ──────────────────────────────────── +# +# Stated plainly, because a green tick that quietly checks less than it looks +# like it does is worse than no tick. +# +# raylib. vendor/raylib/link names `-l:libraylib.so.550` by exact soname, +# which is a Fedora spelling; Ubuntu ships neither that soname nor 5.5. +# So every raylib program is skipped here, on three separate paths that +# all already existed: the acceptance cases probe with `ldconfig -p | +# grep libraylib` and print a skip line; survey.sh records a program that +# will not link as `does-not-compile`, which SURVEY_STRICT deliberately +# does not fail on; and web/examples/check.sh skips shimdemo.flan by name. +# Nothing had to be weakened to make this pass — but the graphics half of +# the corpus is unchecked here and is checked only on a machine with +# raylib installed. +# +# emscripten and the web target. test_web probes for emcc and a raylib +# archive built by vendor/raylib/build-web.sh and skips with the reason. +# Installing an emsdk per run is minutes for a target NEXT.md has +# deprioritised, so it is left out on purpose. +# +# wasm32-wasi. Needs a wasi-libc sysroot and a builtins archive that no +# Ubuntu package supplies. The acceptance case probes by building and +# running the smallest program and skips with what failed, so this is a +# gap in coverage and not a failure. +# +# lldb. The debug-information cases want lldb on PATH and say so when it is +# absent. The DWARF is still emitted and still checked for +# self-consistency; what is skipped is the half that says a person can +# actually sit in a debugger. +# +# @sanitize and @valgrind. Out of @checks by design — tens of minutes each +# — and out of here for the same reason. They are the thing to run before +# a release, by hand, not the thing to run on every push. +# +# One platform. ubuntu-latest only. macOS and Windows portability is +# recorded as real and not current (aligned_alloc, MSG_NOSIGNAL, +# stdatomic), and a matrix that goes red for known reasons teaches people +# to ignore red. + +name: checks + +on: + push: + pull_request: + +# A superseded run is wasted minutes and a confusing status on the commit it +# no longer describes. +concurrency: + group: checks-${{ github.ref }} + cancel-in-progress: true + +jobs: + checks: + runs-on: ubuntu-latest + timeout-minutes: 45 + + steps: + - uses: actions/checkout@v4 + + # clang compiles the IR and the runtime's C; llc and ld are the live + # loop, which never calls the clang driver; as is the --x86 path; emacs + # is what test_emacs and test_cider drive the client with, and they skip + # silently without it, which would be four suites' worth of coverage + # disappearing without a word. + - name: Install the toolchain + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends \ + clang llvm lld binutils emacs-nox + + # llc has to come from the same LLVM release as clang. The live loop + # goes llc + ld -shared + dlopen rather than through the clang driver, + # so a mismatch breaks `C-c C-c` and every reload test while `flan + # build` keeps working perfectly — which is a confusing shape of failure + # to debug from a CI log. Ubuntu installs llc under a version suffix and + # may or may not provide the bare name, so the version is read off clang + # and FLAN_LLC is pinned to match it rather than left to PATH order. + - name: Point FLAN_LLC at the llc that matches clang + run: | + set -eu + clang --version + major=$(clang --version | sed -n 's/.*version \([0-9][0-9]*\).*/\1/p' | head -1) + if command -v "llc-$major" > /dev/null; then + llc="llc-$major" + elif command -v llc > /dev/null \ + && llc --version | grep -q "LLVM version $major"; then + llc=llc + else + echo "no llc at LLVM $major; clang is $(clang --version | head -1)" >&2 + echo "what is on PATH: $(ls /usr/bin/llc* 2>/dev/null || echo none)" >&2 + exit 1 + fi + echo "FLAN_LLC=$llc" >> "$GITHUB_ENV" + "$llc" --version | head -3 + + - uses: ocaml/setup-ocaml@v3 + with: + ocaml-compiler: "5.2" + # There is no (package ...) stanza yet — the install story is Tier 2 + # of docs/REVIEW-production-readiness.md and undecided — so there is + # nothing to pin and nothing to resolve dependencies from. The + # library depends on unix and on nothing else. + opam-pin: false + dune-cache: true + + - run: opam install -y dune + + - name: Build + run: opam exec -- dune build + + # --force because the point of a CI run is to run the tests, not to + # discover that dune's cache already knew the answer. + # + # The log is kept and then read, because a suite that passes while + # leaving an unhandled exception on stderr is a suite telling you + # something and being ignored. That happened here — three suites left a + # message-less `Fatal error: exception Flan.Loc.Error(_)` while + # reporting zero failures — and it was found by a person reading a log, + # which is exactly the kind of finding that should not need a person. + - name: The suite + run: | + set -eu + set -o pipefail + opam exec -- dune test --force 2>&1 | tee suite.log + if grep -n 'Fatal error' suite.log; then + echo "the suite passed but left an unhandled exception above" >&2 + exit 1 + fi + + # The part `dune test` does not cover: the reference page's examples + # still print what the page says, the hand-written x86 backend still + # agrees with LLVM over the whole corpus, and a --dev build still calls + # through its indirection cells. + - name: Everything else that can fail + run: opam exec -- dune build @checks diff --git a/README.md b/README.md index 46ffb41..365c1fd 100644 --- a/README.md +++ b/README.md @@ -268,13 +268,19 @@ hand-written x86 backend still agrees with LLVM, and a `--dev` build still calls through its indirection cells. The two are separate on purpose. A suite that goes red because prose drifted -teaches you to skim past red. But `@checks` only helps if it is run, and nothing -runs it for you — there is no CI here. The convention that has to carry it is -that a lane's handoff quotes `@checks`, the way the x86 handoffs already quote -the survey's counts. Both of this repository's silent failures — two backend -refusals that sat for a month, a page of examples that stopped compiling for two -days — were found by accident, and neither would have survived one person -typing one command. +teaches you to skim past red. Both of this repository's silent failures — two +backend refusals that sat for a month, a page of examples that stopped +compiling for two days — were found by accident, and neither would have +survived one person typing one command. + +`.github/workflows/checks.yml` is now that person: it runs `dune build`, +`dune test --force` and `dune build @checks` on every push. What it cannot run +is written down in the workflow itself rather than left to be discovered — +raylib, emscripten, wasm32-wasi and lldb are all absent from an Ubuntu runner, +and every one of them was already a self-skip, so the tick is green over less +than it is on a machine with those installed. The habit that covers the rest is +still worth keeping: a lane's handoff quotes `@checks`, the way the x86 +handoffs already quote the survey's counts. ## Project map diff --git a/test/dune b/test/dune index a4d558b..08bb45d 100644 --- a/test/dune +++ b/test/dune @@ -319,13 +319,19 @@ ; Valgrind present, and folding them in would make @checks the thing you do not ; have time for -- which is the disease, not the cure. ; -; It is honest to say what this does not do. It does not run itself. Nothing -; here runs itself, because there is no CI, and a commit hook that costs two -; minutes gets switched off in a week. What it buys is that deciding to check -; and checking everything are now the same act, so the gap between "somebody -; wondered" and "everything was verified" is one command instead of five. The -; habit that closes the rest of the gap is written down in README.md: a lane's -; handoff quotes this alias, the way the x86 handoffs already quote survey.sh. +; What it buys is that deciding to check and checking everything are now the +; same act, so the gap between "somebody wondered" and "everything was +; verified" is one command instead of five. +; +; It used to say here that nothing runs this, because there was no CI. There is +; now: .github/workflows/checks.yml runs `dune build`, `dune test --force` and +; this alias on every push. That does not make the local run redundant, and the +; workflow says why in as many words — an Ubuntu runner has no raylib, no +; emscripten, no wasi sysroot and no lldb, so every one of those cases takes +; the skip path it already had and the tick is green over less than this alias +; covers here. The habit written down in README.md still carries the rest: a +; lane's handoff quotes this alias, the way the x86 handoffs already quote +; survey.sh. (alias (name checks) (deps