From dbf5748c560542a45d3c8b7ac4f490d48181071e Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Fri, 11 Sep 2026 20:26:11 +0700 Subject: [PATCH] Assert the reasons, and say what a permutation proves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every refusal is by name with the reason, so the tests assert on the reasons and weakening one to a bare "cannot" breaks them: a slice, an Option, a union, a fixed array, a map, a returned string, a callback, an unknown type, a struct field C cannot hold, and two Flan names for one C symbol. The rest is text about text, which is the honest scope: what a wrapper does is settled by clang, and what is worth checking in OCaml is the shape of what clang is handed. Two cases assert the typedef's field order against a defstruct and against the same defstruct permuted, because only the pair rules out a generator that sorts — and sorting is exactly the mutation the raylib cases cannot see, since every raylib struct is fields of one size and a rename changes no offset. What the raylib cases do see is a permuted defstruct, and that was run: Rectangle width/height, Vector2 x/y, Image width/height, Image with data moved last, Texture2D id/format, Color r/a and Camera2D offset/target all go red. Texture2D width/mipmaps stays green, which is what NEXT.md already says headless cannot pin — the one green is the control, not a gap. --- NEXT.md | 96 +++++++++++++++++++++++---- test/test_acceptance.ml | 142 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 224 insertions(+), 14 deletions(-) diff --git a/NEXT.md b/NEXT.md index da43f24..fafe3e1 100644 --- a/NEXT.md +++ b/NEXT.md @@ -143,11 +143,12 @@ reader ✅ → parse ✅ → load ✅ → check ✅ → emit ✅ → clang ✅ | `lib/build.ml` | `.ll` + the shim + the packages' C → clang → executable | | `runtime/flan_rt.c` | the host ABI: argv, stdout, exit, 4 conversions | | `runtime/flan_dev.c` | **dev only: the by-name registry a run-time-new name needs** | -| `vendor/raylib/` | **the raylib package: `raylib.flan`, `shim.c`, `link`** | +| `lib/shim.ml` | **`declare-c` -> the generated C that flattens a struct crossing** | +| `vendor/raylib/` | **the raylib package: `raylib.flan` and `link`, and no C at all** | | `vendor/agent/` | **the dev agent: a socket, a loader thread, install at a frame boundary** | | `emacs/` | **`flan-mode.el`, `flan-dev.el`, `flan-repl.el`: the editor half of the dev loop** | | `sand-sim/` | **the falling-sand simulation, with no raylib in it** | -| `bin/main.ml` | `flan read \| parse \| check \| emit \| build \| run \| reload \| dev` | +| `bin/main.ml` | `flan read \| parse \| check \| emit \| shim \| build \| run \| reload \| dev` | | `test/test_flan.ml` | reader, parser and checker | | `test/test_acceptance.ml` | expression/result pairs + whole programs + the traps | | `test/test_reload.ml` | **the reload primitive: recompile one function, load it, call it** | @@ -226,7 +227,7 @@ Putting that in `emit.ml` is three classifiers to write and then keep correct forever, and a mistake shows up as `(.y m)` returning garbage rather than as a link error. -So `vendor/raylib/shim.c` has one wrapper per binding, each one flattening the +So the boundary has one wrapper per binding, each one flattening the aggregates: a struct returns through an out-pointer, a struct argument is passed by pointer, a Flan string crosses as ptr+len and the shim NUL-terminates a copy. clang classifies all of it, per target, for free. `check.ml` enforces @@ -234,11 +235,76 @@ the rule — an aggregate in a `declare` signature is rejected with the reason so the boundary cannot quietly acquire one. This is plan.org's "one narrow host ABI, implemented twice", and `flan_rt.c` is the same pattern. -The price is a hand-written wrapper per raylib call. They are one-liners and -mechanical enough to generate if that ever becomes the bottleneck. +### The wrappers are generated now — `declare-c`, `lib/shim.ml` -`raylib.flan` declares each `-raw` entry point and wraps it in an ordinary Flan -function just below, so the surface sand.flan sees is `(rl/get-mouse-position)` +The price above was a hand-written wrapper per raylib call, and the prediction +that they were mechanical enough to generate "if that ever becomes the +bottleneck" came true at 84 of them. `vendor/raylib/shim.c` is gone; the +directory holds `raylib.flan` and `link` and no C at all. + +One binding is now one line: + +``` +(declare-c draw-texture [t Texture2D x i32 y i32 tint Color] "DrawTexture") +``` + +`declare-c` names raylib's own function in raylib's own signature, and the +compiler emits, into a C file compiled like any other: the typedefs for the +structs involved, made from the Flan `defstruct`s; the `extern` prototype in +the function's true signature; the wrapper that flattens it; and the flattened +`declare` the Flan side calls, with an ordinary Flan `defn` above it when the +signature has a struct in it. `flan shim ` prints the whole file. + +**It is a second form and not a change to `declare`, for one reason worth +remembering:** `(declare start-raw [path string] i32 "flan_agent_start")` in +`vendor/agent` means the symbol takes ptr+len, and `(declare-c init-window [w +i32 h i32 title string] "InitWindow")` means it takes a NUL-terminated `char +*`. Same shape, opposite claims, so no structural rule can separate them. +`declare` is untouched, and `sqrtf` and the agent still work unedited. + +**What the generator guarantees, and what it trusts.** Guaranteed: the C +typedef and the Flan struct come from the same `defstruct`, so they cannot +disagree — permute the `defstruct` and the typedef permutes with it, which is +exactly what makes the permutation runs below meaningful. And clang +type-checks the wrapper against the generated prototype. Trusted: that the +`defstruct` matches the library's real struct, and that the `declare-c` +signature is the function's real signature — no header is read, deliberately, +so nothing can check either. A `_Static_assert` on `sizeof`/`offsetof` was +considered and rejected as circular: both sides would come from the same field +list. + +**One thing got sharper and should be said plainly:** the prototype is now +generated *from the declaration*, so a scalar's width carries ABI weight it did +not before. `f64` where raylib says `float` used to be narrowed by clang at the +hand-written call site; now it emits `double` and raylib reads garbage. All 84 +migrated prototypes were diffed against the deleted `shim.c`'s — which was the +ground truth for the true signatures — and agree. + +**Strings.** The hand-written wrappers sized the NUL-copy per call site: 256 +for a window title, `PATH_MAX` for a path, 512 for drawn text, truncating past +it. A generator has no call site to look at, so it must not be the thing +deciding a string is too long: 256 bytes on the stack, the heap past that, +freed after the call. The only truncation left is on malloc failure, where the +alternative is handing C a null pointer. + +**Two bindings keep a hand-written wrapper, and both wrappers are Flan, not C.** +`collision-point-poly?` takes a slice and `collision-lines` answers with an +`Option`; neither is raylib's own signature. A slice parameter in a `declare-c` +is refused by name, because a slice's length crosses as i64 and the type of the +C count parameter beside the pointer is not recoverable from `[T]` — so that +one declares `(Ptr Vector2)` with an explicit `count i32` and the Flan wrapper +passes `(addr (at points 0))` and `(len points)`. Every other refusal — an +Option, a union, a fixed array, a map, a returned string, a callback, an +unknown type, an unrepresentable struct field, two Flan names for one C symbol +— is by name with the reason, and the acceptance table asserts on the reasons. + +**Known edge, not fixed:** a REPL redefinition that introduces a *new* +`declare-c` cannot work. `Build.shared` is llc + `ld -shared` and compiles no +C, so the wrapper would not exist in the running process. Editing the body of a +function that calls an existing binding is unaffected. + +`raylib.flan` carries the nice signature and the compiler writes the rest, so +the surface sand.flan sees is `(rl/get-mouse-position)` returning a `Vector2`. Verified end to end, headless: `GetColor(0x11223344)` comes back as `17 34 51 68`, four separate bytes — a `Color` is *not* the little-endian reading of the packed integer, so an identity would have passed a @@ -255,8 +321,9 @@ The bindings are 29 calls: window (`init-window`, `close-window`, texture and rectangle intersection (`set-shapes-texture`, `get-shapes-texture`, `get-shapes-texture-rectangle`, `get-collision-rec`), plus the `Key`, `MouseButton` and `TraceLogLevel` enums and the `Vector2`, -`Color`, `Texture2D` and `Rectangle` structs. Adding one is three lines: a -`declare`, an `extern` prototype, and a one-line wrapper. +`Color`, `Texture2D` and `Rectangle` structs. Adding one was three lines — a +`declare`, an `extern` prototype and a one-line wrapper — and is now one +`declare-c`. The texture calls are the first ones with no headless test, because loading one needs a GL context. What the acceptance case does instead is pin the two @@ -269,8 +336,9 @@ comes back permuted the same way and the case passes. `width`, `height` and `mipmaps` are therefore checked only by looking at `sand.flan` running, which draws the brush sprite four ways for that reason. -No raylib headers are needed: `shim.c` declares the prototypes it uses, so the -build depends on the shared library being linkable and not on `raylib-devel`. +No raylib headers are needed: the generated C declares the prototypes it uses, +so the build depends on the shared library being linkable and not on +`raylib-devel`. `vendor/raylib/link` carries `-l:libraylib.so.550` because Fedora ships the runtime library without the `.so` symlink. @@ -286,8 +354,8 @@ turned out to check nothing. - **Axis-aligned geometry cannot pin `Vector2`.** Exchanging `x` and `y` is a reflection, applied to the inputs on the way in and undone on the way out, so the printed answer is unchanged. Every collision predicate, and every - distance, passes with the fields swapped — verified by swapping the shim's - own typedef. Distances are worse: the reflection does not even reach them. + distance, passes with the fields swapped — verified by swapping the + `defstruct`, which is what the typedef is now made from. Distances are worse: the reflection does not even reach them. - **What does pin `Vector2` is the rotated camera**, because a 90-degree rotation is not axis-aligned and therefore does not commute with the reflection. That case is load-bearing and must not be deleted on the grounds @@ -1470,7 +1538,7 @@ object is written to a temporary name and `rename`d into place, so two concurrent builds cannot see a half-written one. Measured: calc-me 160ms → 110ms; sand ~720ms → ~700ms, since sand's time is -mostly linking libraylib and its `shim.c` was never the cost. The cache is +mostly linking libraylib and its C was never the cost. The cache is keyed by content, so it never needs invalidating by hand — `rm -rf` on the directory is only ever a disk-space decision. diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 300a7e7..cb3f018 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -542,6 +542,148 @@ let () = wasm_case "calc-me, wasm32" "../calc-me.flan" ~arg:"1 + 2 * (3 - 0.5) / 2" "3.5\n")); + + (* ── declare-c: the generated FFI shim (lib/shim.ml) ──────────────── + The raylib package is the proof that the generator is real — 84 + hand-written wrappers replaced by 84 one-line declarations, with the + two raylib cases above unchanged — and the permutation runs below are + the proof that the generated C typedefs actually follow the Flan + `defstruct`s rather than merely looking as if they do. + + Everything here is text, not a link: what a wrapper does is settled by + clang, and what is worth asserting in OCaml is the shape of what clang + is handed and the refusals, each by name and reason. *) + let shim_of src = + let decls = Parse.program (Reader.read_all ~file:"" src) in + match (Check.program decls).Tast.cshim with + | Some c -> c + | None -> "" + in + let shim_case name src needles = + match shim_of src with + | c -> + List.iter + (fun n -> + if not (contains c n) then begin + incr failures; + Printf.printf "FAIL %s\n wanted in the generated C: %S\n" + name n + end) + needles + | exception Loc.Error (_, m) -> + incr failures; + Printf.printf "FAIL %s\n refused: %s\n" name m + in + (* A refusal is by name and carries the reason; the tests assert on the + reason, so weakening one to a bare "cannot" breaks them. *) + let shim_refuses name src fragment = + match shim_of src with + | _ -> + incr failures; + Printf.printf "FAIL %s: accepted, and it should not have been\n" name + | exception Loc.Error (_, m) -> + if not (contains m fragment) then begin + incr failures; + Printf.printf "FAIL %s\n reason: %S\n wanted to contain: %S\n" + name m fragment + end + in + let v2 = "(defstruct Vector2 [x f32 y f32])\n" in + let img = + "(defstruct Image [data (Ptr u8) width i32 height i32])\n" + in + + (* A struct argument goes by pointer and a struct return through an + out-pointer, and the prototype says what C really takes. *) + shim_case "declare-c: a struct crosses by pointer, both ways" + (v2 ^ "(declare-c mid [a Vector2 b Vector2] Vector2 \"Mid\")") + [ "extern flan_ty_Vector2"; "*out = Mid(*a0, *a1);"; + "const flan_ty_Vector2"; "*out)" ]; + + (* The typedef is made from the defstruct and nothing else, so its field + order is the defstruct's — which is what makes permuting a defstruct a + real test rather than a rewording. Both orders asserted, because only + the pair rules out a generator that sorts. *) + shim_case "declare-c: the C typedef follows the defstruct's field order" + (v2 ^ "(declare-c f [v Vector2] \"F\")") + [ " float x;\n float y;\n" ]; + shim_case "declare-c: and permuting the defstruct permutes the typedef" + ("(defstruct Vector2 [y f32 x f32])\n(declare-c f [v Vector2] \"F\")") + [ " float y;\n float x;\n" ]; + + (* One type mapper for fields and parameters alike: a bool is C's bool and + never an int, and a pointer field keeps its element type. *) + shim_case "declare-c: field and parameter types come from one mapper" + (img + ^ "(defstruct S [flag bool n u64])\n\ + (declare-c g [s S i (Ptr Image) b bool] u64 \"G\")") + [ " bool flag;\n uint64_t n;\n"; " uint8_t *data;\n"; + "uint64_t G(flan_ty_S"; "bool a2" ]; + + (* A struct held by value pulls its own typedef in, and the definitions are + ordered so the inner one is complete first. *) + shim_case "declare-c: a nested struct is defined before it is used" + (v2 ^ "(defstruct Camera2D [offset Vector2 zoom f32])\n\ + (declare-c h [c Camera2D] \"H\")") + [ "struct flan_ty_Vector2"; " flan_ty_Vector2" ]; + + (* A string is ptr+len on the Flan side and a NUL-terminated copy on C's. + The buffer is sized here and not per call site, because a generator has + no call site to look at: 256 on the stack, the heap past that, and the + copy is freed after the call rather than before the return value is + computed. *) + shim_case "declare-c: a string is copied, NUL-terminated and freed" + "(declare-c open-it [path string] bool \"OpenIt\")" + [ "char a0_b[256];"; "flan_shim_cstr(a0_p, a0_n, a0_b, sizeof a0_b)"; + "bool r = OpenIt(a0);"; "flan_shim_cstr_free(a0, a0_b);"; + " return r;\n" ]; + shim_case "declare-c: two strings get two buffers" + "(declare-c both [a string b string] \"Both\")" + [ "char a0_b[256];"; "char a1_b[256];"; + "flan_shim_cstr_free(a0, a0_b);"; "flan_shim_cstr_free(a1, a1_b);" ]; + + (* [declare] is untouched by any of this: its signature still IS the C + signature, which is what vendor/agent's flan_agent_start and the + prelude's sqrtf depend on. A program with no declare-c generates no C + at all. *) + if shim_of "(declare start [path string] i32 \"flan_agent_start\")" <> "" + then begin + incr failures; + print_endline "FAIL declare (not declare-c) generated a shim" + end; + + shim_refuses "declare-c: a slice parameter, by name and reason" + (v2 ^ "(declare-c poly [pts [Vector2]] bool \"Poly\")") + "the count parameter the C function actually takes"; + shim_refuses "declare-c: an Option" + (v2 ^ "(declare-c maybe [] (Option Vector2) \"Maybe\")") + "which is a Flan shape and not a C one"; + shim_refuses "declare-c: a union" + ("(defunion Shape [(Circle [r f32]) (Square [s f32])])\n\ + (declare-c area [s Shape] f32 \"Area\")") + "a union, and a Flan union has no C layout"; + shim_refuses "declare-c: a fixed array" + "(declare-c takes [xs [4 f32]] \"Takes\")" + "which C passes as a pointer and Flan as a value"; + shim_refuses "declare-c: a map" + "(declare-c takes [m {string i32}] \"Takes\")" + "which has no C representation"; + shim_refuses "declare-c: a returned string" + "(declare-c name [] string \"Name\")" + "a string only crosses as a parameter"; + shim_refuses "declare-c: a callback" + "(declare-c each [f (Fn [i32] Unit)] \"Each\")" + "a C callback is not implemented"; + shim_refuses "declare-c: an unknown type" + "(declare-c f [x Nope] \"F\")" + "which is not a type this shim generator knows"; + shim_refuses "declare-c: a field C cannot hold" + "(defstruct S [xs [i32]])\n(declare-c f [s S] \"F\")" + "field xs of S is a slice"; + shim_refuses "declare-c: two Flan names for one C function" + "(declare-c a [] \"Same\")\n(declare-c b [] \"Same\")" + "one declare-c per C function"; + if !failures = 0 then print_endline "acceptance: all tests passed" else begin Printf.printf "\n%d failure(s)\n" !failures;