Joseph Ferano
16498a4e60
A GitHub Actions workflow on push: dune build, dune test --force, dune build @checks. @x86 parity is not under dune test, so the routine suite never protected it; both of this repository's silent failures would have been caught by one person typing one command, and the problem was never the command. The suite step keeps its log and greps it for Fatal error, because a suite that passes while leaving an unhandled exception on stderr is one that is telling you something and being ignored. FLAN_LLC is pinned to the llc matching clang's version rather than left to PATH order: the live loop goes llc + ld -shared + dlopen and never calls the clang driver, so a mismatch breaks every reload test while flan build keeps working, which is a bad failure to debug from a log. What an Ubuntu runner cannot cover -- raylib by exact Fedora soname, emscripten, a wasi sysroot, lldb -- is written in the workflow with the skip path each one already takes, so the tick does not read as more than it is. README.md and test/dune both said there was no CI; both now say what there is and what it misses.
298 lines
12 KiB
Markdown
298 lines
12 KiB
Markdown
<div align="center">
|
||
|
||
<img src="assets/flan-logo.svg" alt="Flan" width="300">
|
||
|
||
# Flan
|
||
|
||
**A statically typed Lisp for native games and interactive development.**
|
||
|
||
</div>
|
||
|
||
Flan is an experimental, ahead-of-time compiled Lisp for programs that need
|
||
predictable memory use and a fast edit–run loop. It combines S-expressions,
|
||
static types, explicit ownership, and a development session that can replace a
|
||
function in a running program without resetting its state.
|
||
|
||
It is being built around games, but the interesting part is broader: a compiled
|
||
language where the running program remains available for inspection,
|
||
experimentation, and small changes.
|
||
|
||
In practical terms: you get parentheses, a debugger that would like to have a
|
||
conversation, and no garbage collector quietly choosing the dramatic moment to
|
||
join your frame loop.
|
||
|
||
## What it has
|
||
|
||
- Native compilation through LLVM, plus an in-progress direct x86-64 backend.
|
||
- C-like data layout: structs, fixed arrays, pointers, slices, and explicit
|
||
allocation. There is no garbage collector.
|
||
- Owned `Vec` and `Map` containers, plus checked moves and borrowing-oriented
|
||
slice operations.
|
||
- Generics, algebraic unions, enums, macros, packages, `defer`, and a C FFI.
|
||
- Conditions and restarts for recoverable failures and interactive debugging.
|
||
- A raylib package and a collection of ported raylib examples.
|
||
- Native, WASI, and web build targets. The cross targets are useful but less
|
||
complete than the native development workflow.
|
||
|
||
The project is exploratory software, not a stable language release. Some
|
||
features are deliberately refused while their semantics are still undecided;
|
||
the compiler aims to say why rather than quietly accepting a partial version.
|
||
It has opinions, but at least they arrive as error messages.
|
||
|
||
## Quick start
|
||
|
||
Building requires a current OCaml/Dune toolchain, LLVM/Clang, and the native C
|
||
toolchain. Raylib is only needed for programs that use the bundled graphics
|
||
package.
|
||
|
||
```sh
|
||
dune build
|
||
dune exec ./bin/main.exe -- run web/examples/hello.flan
|
||
```
|
||
|
||
To build a standalone native executable:
|
||
|
||
```sh
|
||
dune exec ./bin/main.exe -- build web/examples/hello.flan -o hello
|
||
./hello
|
||
```
|
||
|
||
The falling-sand demo uses raylib:
|
||
|
||
```sh
|
||
dune exec ./bin/main.exe -- run sand.flan
|
||
```
|
||
|
||
Once you are iterating regularly, put the built executable on your `PATH` if
|
||
you want to use the shorter `flan` commands shown below.
|
||
|
||
## The live development loop
|
||
|
||
Start a long-lived development session:
|
||
|
||
```sh
|
||
flan dev sand.flan
|
||
```
|
||
|
||
The program runs normally and publishes a local socket beside the source file.
|
||
The bundled Emacs mode can attach to it, evaluate expressions in the live
|
||
process, inspect a stopped program, and recompile a top-level function from
|
||
the buffer. A body change takes effect on the next call; changing a function's
|
||
signature is intentionally rejected. The program keeps its state, which is
|
||
especially nice when you have finally arranged the sand into something almost
|
||
worth saving.
|
||
|
||
To set up the mode:
|
||
|
||
```elisp
|
||
(add-to-list 'load-path "~/path/to/flan/emacs")
|
||
(require 'flan-mode)
|
||
```
|
||
|
||
Then use `M-x flan-dev` to start and attach, or `C-c C-z` to attach to a
|
||
session started in a terminal. The editor workflow is documented in
|
||
[emacs/MANUAL.md](emacs/MANUAL.md).
|
||
|
||
## A small example
|
||
|
||
```lisp
|
||
(defstruct AssetMissing [id i32])
|
||
|
||
(defn load-asset [id i32] i32
|
||
(signal (AssetMissing {.id id}))
|
||
100)
|
||
|
||
(defn asset-or-placeholder [id i32] i32
|
||
(restart-case (load-asset id)
|
||
(use-placeholder [] -1)))
|
||
|
||
(defn main [] ()
|
||
(handler-bind [(AssetMissing [_] (invoke-restart 'use-placeholder))]
|
||
(println (asset-or-placeholder 7))))
|
||
```
|
||
|
||
Here a missing asset signals a typed condition. The handler chooses a restart,
|
||
so execution continues with a placeholder instead of requiring error values to
|
||
be threaded through every caller. See
|
||
[web/examples/restart.flan](web/examples/restart.flan) for a runnable version.
|
||
|
||
## Commands
|
||
|
||
Eleven of them, and the four anyone starts with:
|
||
|
||
```text
|
||
flan check <file.flan> type-check a program
|
||
flan run <file.flan> [flags] [-- args...] build and run it
|
||
flan build <file.flan> [-o out] [flags] build a native executable
|
||
flan dev <file.flan> [-s socket] start a live development session
|
||
```
|
||
|
||
Useful build options include `-O0`…`-O3`, `--debug`, `--sanitize`,
|
||
`--no-bounds-checks`, `--x86`, and `--target=wasm32-wasi|web`. `run` takes the
|
||
same build flags and keeps them: everything after `--` goes to the program, and
|
||
a dash-argument before it that `run` does not offer is refused rather than
|
||
guessed at. `run` is native-only; cross-built output should be run with an
|
||
appropriate WASI runtime or browser. A `.wasm` file is not a tiny native
|
||
executable in a trench coat.
|
||
|
||
The other seven. Four print a stage of the pipeline, which is how you find out
|
||
what the compiler thinks it was given:
|
||
|
||
```text
|
||
flan read <file.flan>... the forms the reader produced
|
||
flan parse <file.flan>... one line per declaration
|
||
flan shim <file.flan>... the C a (declare-c ...) generated
|
||
flan emit <file.flan> [--x86] [--dev] [--debug] [--no-bounds-checks]
|
||
LLVM IR, or x86-64 assembly under --x86
|
||
```
|
||
|
||
One builds a redefinition module the way `flan dev` does, for scripting the
|
||
loop without an editor:
|
||
|
||
```text
|
||
flan reload <program.flan> <forms.flan> [-o out.so] [--debug] [--x86]
|
||
```
|
||
|
||
And two are the C binding tools, which are the most useful thing here that
|
||
nothing else documents.
|
||
|
||
### `flan import-c` — read a header, print the bindings it implies
|
||
|
||
It builds nothing and writes nothing. It reads a C header (through clang's own
|
||
parser), prints the `declare-c` forms it would generate, and then prints every
|
||
function it *refused* and the reason — which is the half that earns its keep,
|
||
because "this binding is missing" and "this binding cannot exist" are different
|
||
problems:
|
||
|
||
```text
|
||
$ flan import-c test/headers/sample.h
|
||
(declare-c set-seed [seed u32] "set_seed")
|
||
(declare-c add-ints [a i32 b i32] i32 "add_ints")
|
||
(declare-c name-length [text string] i32 "name_length")
|
||
...
|
||
;; 8 imported, 12 refused, of 21 functions in test/headers/sample.h
|
||
;; refused name-of: name_of returns char *, and a string only crosses as a
|
||
;; parameter — a C function that returns one returns something Flan has no
|
||
;; owner for
|
||
;; refused printf-like: printf_like is variadic, and a wrapper cannot forward
|
||
;; an argument list it does not know the shape of
|
||
;; refused file-time: file_time long has a width that differs between this
|
||
;; project's own targets (64 bits on x86-64, 32 on wasm32), so no single
|
||
;; Flan type is right for it
|
||
```
|
||
|
||
Hand the package's `.flan` files along with the header and it diffs against
|
||
them too: a `defstruct` whose layout no longer matches the header's record is
|
||
named, which is the failure that otherwise shows up as a wrong pixel.
|
||
|
||
```text
|
||
$ flan import-c vendor/raylib/raylib-5.5.h vendor/raylib/*.flan
|
||
```
|
||
|
||
Anything after the header that is not a `.flan` file is passed to clang, so
|
||
`-I` and `-D` work as they do anywhere else.
|
||
|
||
### `flan generate-c` — write those bindings into the package
|
||
|
||
The same machinery, committing its answer. It takes a package *directory*, not
|
||
a header: the header comes from the package's own `headers` file, at the
|
||
version its `link` file names, because which version may be read is a property
|
||
of the package and not of the command line. It reads every `.flan` in the
|
||
directory except the one it writes, so hand-written declarations keep winning,
|
||
and it writes `generated.flan`.
|
||
|
||
```text
|
||
$ flan generate-c vendor/raylib
|
||
;; refused get-clipboard-text: GetClipboardText returns char *, and a string
|
||
;; only crosses as a parameter — ...
|
||
;; refused load-shader: LoadShader Shader is a struct the package does not
|
||
;; describe — add a defstruct for it, or keep a hand-written declare-c
|
||
```
|
||
|
||
It exits non-zero and writes nothing if the package and the header disagree.
|
||
That is the point of it: the committed file is the one thing in the package
|
||
with no second opinion, so the moment of writing it is the last moment at which
|
||
the installed library can contradict it.
|
||
|
||
## Environment
|
||
|
||
Thirteen variables the toolchain reads, none of which has to be set on a
|
||
machine with clang, LLVM and binutils on `PATH`. Each exists for a machine
|
||
where the thing is somewhere else, or is the wrong one.
|
||
|
||
| Variable | What it replaces | Read at |
|
||
|---|---|---|
|
||
| `FLAN_CLANG` | `clang`, which compiles the IR and the runtime's C | `lib/build.ml:14` |
|
||
| `FLAN_LLC` | `llc`, used only by the live loop | `lib/build.ml:886` |
|
||
| `FLAN_LD` | `ld`, used only by the live loop | `lib/build.ml:887` |
|
||
| `FLAN_AS` | `as`, used only by `--x86` | `lib/build.ml:978` |
|
||
| `FLAN_OBJDUMP` | `objdump`, used only by the disassembly verb | `lib/dev.ml:2018` |
|
||
| `FLAN_OCAMLFIND` | `ocamlfind`, which a merged `flan dev` needs **at run time** | `lib/dev.ml:2898` |
|
||
| `FLAN_EMCC` | `emcc`, for `--target=web` | `lib/build.ml:23` |
|
||
| `FLAN_LIBDIR` | where `flan.cmxa` and `flan.a` are, if not beside the binary | `lib/dev.ml:3187` |
|
||
| `FLAN_CACHE_DIR` | the object cache, default `$XDG_CACHE_HOME/flan/objcache` | `lib/build.ml:84` |
|
||
| `FLAN_WASM_SYSROOT` | the wasi-libc sysroot, default `/usr/wasm32-wasi` | `lib/build.ml:239` |
|
||
| `FLAN_WASM_BUILTINS` | `libclang_rt.builtins-wasm32.a`, which is not in clang's resource directory on Fedora | `lib/build.ml:272` |
|
||
| `FLAN_WEB_SHELL` | the HTML shell a `--target=web` build wraps the module in | `lib/build.ml:458` |
|
||
| `FLAN_DEV_LEAKS` | set (to anything) to have a `--dev` build print what it still held at exit | `runtime/flan_dev.c:1084` |
|
||
|
||
`${FLAN_RAYLIB_WEB}` is not read by the compiler: it is expanded inside
|
||
`vendor/raylib/link`, which is where a package writes a linker argument that
|
||
has to differ per target. Any `${NAME}` in a `link` file expands from the
|
||
environment and an unset one is refused by name.
|
||
|
||
**`llc` and `ld` have to match `clang`.** The live loop does not call the clang
|
||
driver at all — it goes `llc` + `ld -shared` + `dlopen`, which is what makes
|
||
`C-c C-c` cost milliseconds. So an `llc` from a different LLVM release than
|
||
`clang` breaks the dev loop while `flan build` keeps working perfectly, which
|
||
is a confusing shape of failure to meet without warning.
|
||
|
||
Variables beginning `FLAN_DEV_` other than `FLAN_DEV_LEAKS`, plus
|
||
`FLAN_AGENT_SOCKET` and `FLAN_COMPILER_STAMP`, are internal: `flan dev` sets
|
||
them across its own `exec` to hand the merged binary what it needs. Setting
|
||
them by hand is not supported.
|
||
|
||
## Checking it
|
||
|
||
```text
|
||
dune test the suite. Seconds. Run it constantly.
|
||
dune build @checks everything else that can fail. A couple of minutes.
|
||
dune build @sanitize the corpus under ASan and UBSan.
|
||
dune build @valgrind the corpus under memcheck. Tens of minutes.
|
||
```
|
||
|
||
`dune test` means "the language still works". `@checks` — which is `@page`,
|
||
`@x86` and `@cells` — means "and everything written down about it is still
|
||
true": the reference page's examples still print what the page says, the
|
||
hand-written x86 backend still agrees with LLVM, and a `--dev` build still calls
|
||
through its indirection cells.
|
||
|
||
The two are separate on purpose. A suite that goes red because prose drifted
|
||
teaches you to skim past red. Both of this repository's silent failures — two
|
||
backend refusals that sat for a month, a page of examples that stopped
|
||
compiling for two days — were found by accident, and neither would have
|
||
survived one person typing one command.
|
||
|
||
`.github/workflows/checks.yml` is now that person: it runs `dune build`,
|
||
`dune test --force` and `dune build @checks` on every push. What it cannot run
|
||
is written down in the workflow itself rather than left to be discovered —
|
||
raylib, emscripten, wasm32-wasi and lldb are all absent from an Ubuntu runner,
|
||
and every one of them was already a self-skip, so the tick is green over less
|
||
than it is on a machine with those installed. The habit that covers the rest is
|
||
still worth keeping: a lane's handoff quotes `@checks`, the way the x86
|
||
handoffs already quote the survey's counts.
|
||
|
||
## Project map
|
||
|
||
- [web/index.html](web/index.html) — language reference and fuller examples.
|
||
- [spec-memory.md](spec-memory.md) — ownership, containers, and generics.
|
||
- [spec-conditions.md](spec-conditions.md) — conditions, handlers, and restarts.
|
||
- [emacs/MANUAL.md](emacs/MANUAL.md) — the interactive editor workflow.
|
||
- [docs/BUILT.md](docs/BUILT.md) — implementation rationale.
|
||
- [NEXT.md](NEXT.md) — current work and known limits.
|
||
|
||
## License
|
||
|
||
Flan is released under the [MIT License](LICENSE). Third-party material under
|
||
`vendor/` is distributed under its own licenses.
|