Where the generated half is written down, in all four places it belongs
web/index.html had no section on the FFI's generated half at all; it has one now, with the command, the config, and the reason committing the output is what makes the no-header property honest rather than a caveat. BUILT.md gets why the 172 stay, which is the part that is easy to get wrong: 136 of them are exactly what the rule produces and the rest are expressible as overrides, so the superset argument is sound and still leads somewhere bad — deleting them reduces the signature check to a tautology. DISCUSS.md 6a and 6b are answered rather than left open, and 6b's own point about enums turns out to be live in the tree: key-down? keeps its Key parameter because it is hand-written, and the generated key-up? beside it takes an i32.
This commit is contained in:
parent
ecfe180d2e
commit
65fd49f0c6
89
BUILT.md
89
BUILT.md
@ -162,6 +162,8 @@ described is refused with that reason, so `vendor/raylib` describing thirteen
|
||||
structs is what makes the import thirteen structs wide, and describing a
|
||||
fourteenth widens it. Of raylib 5.5's 581 functions, 256 import, 153 are
|
||||
refused, and 172 are left alone because the package already binds them by hand.
|
||||
(253 and 156 once `bindings` excludes raylib's three allocator entry points —
|
||||
see the next section.)
|
||||
|
||||
Not generating `defstruct`s is what makes the check possible at all. Generate
|
||||
them and the header becomes the authority on layout, and comparing the
|
||||
@ -355,6 +357,93 @@ header cache and the same build is 0.92s against 0.83s. So the header read is
|
||||
compiling a shim with 428 wrappers in it, which is the object cache's business
|
||||
and already warm after one build.
|
||||
|
||||
### The bindings are committed now — `generated.flan`, `bindings`, `flan generate-c`
|
||||
|
||||
The section above reads the header at build time, behind an opt-in
|
||||
`?${FLAN_RAYLIB_H}` in `headers`, because a build should need libraylib
|
||||
linkable and not raylib-devel installed. That opt-in was doing two jobs and
|
||||
only one of them was defensible: it decided whether a *check* ran, which is
|
||||
fine to make optional, and it decided whether a package had 172 bindings or
|
||||
428, which is not. `DrawTexturePro` being reachable only by exporting an
|
||||
environment variable is the shape of that second job, and it blocked a real
|
||||
game — see PORTING.md.
|
||||
|
||||
**Generate once, commit the result, regenerate when raylib moves.**
|
||||
`flan generate-c vendor/raylib` reads the header named by `headers`, writes
|
||||
`vendor/raylib/generated.flan`, and that file is checked in. No header is
|
||||
needed by anybody: every build gets all 425 declarations, they are greppable,
|
||||
and they show up in a diff when the library moves. **Caching is not the
|
||||
argument** — the dump is already cached on disk and in memory, so a build that
|
||||
reads a header pays for it once either way. The argument is the dependency and
|
||||
the diff.
|
||||
|
||||
**What it costs is the check, so regeneration runs it and the check gates the
|
||||
write.** A file on disk has no second opinion, so nothing compares the bindings
|
||||
against the library on an ordinary build any more. The one function that writes
|
||||
`generated.flan` therefore compares first — every `defstruct` against the
|
||||
header's record, every hand-written `declare-c` against the header's signature
|
||||
— and writes nothing when they disagree. It is not possible to regenerate
|
||||
without comparing, because there is no other way to write the file. Pointed at
|
||||
the 5.1-dev header on this machine while the package is written for 5.5, it
|
||||
reports the same ten real differences the diff above found and writes nothing.
|
||||
|
||||
**The 172 hand-written lines stay, and the reason is not caution.** It was
|
||||
tempting to delete them: 136 of the 172 are exactly what the kebab rule would
|
||||
have produced, and the other 36 could be spelled as name overrides, so the
|
||||
generated set really is a superset. The argument against is the one this
|
||||
section is about. Everything the generator emits agrees with the header *by
|
||||
construction* — the declaration and the prototype come from one dump — so
|
||||
diffing generated output against the header it came from is a tautology, and
|
||||
replacing the hand-written set would quietly reduce the signature half of the
|
||||
check to nothing. The hand-written lines were transcribed from raylib's
|
||||
documentation by a person; they are the only declarations in the package a
|
||||
header can actually contradict. All ten of the 5.1-dev differences came from
|
||||
them. They are not a parallel set to maintain — they are the second opinion,
|
||||
and the check is what maintains them.
|
||||
|
||||
The opt-in did not go away, it stopped deciding anything important. With
|
||||
`FLAN_RAYLIB_H` set, an ordinary build still reads the header, and since every
|
||||
C symbol is now bound — by hand or by generation — the importer generates
|
||||
nothing and the read is purely the check. It is a *better* check than before:
|
||||
425 declarations rather than 172, because `generated.flan` is a package file
|
||||
like any other and is checked like one.
|
||||
|
||||
**`bindings`, beside `headers`, is what survives regeneration.** A committed
|
||||
generated file cannot be hand-corrected — the next run overwrites it and the
|
||||
edit is destroyed without anybody being told, which is the worst shape an edit
|
||||
can have — so the corrections have to live somewhere regeneration *reads*. Two
|
||||
directives, which are the two things the header cannot decide: `exclude <symbol
|
||||
or pattern>` and `name <symbol> <flan-name>`. A postprocessing transform pass
|
||||
was considered and rejected: a second program to understand, run over text the
|
||||
generator had already committed to.
|
||||
|
||||
Both are applied *while* the declarations are made, which is not a detail. The
|
||||
kebab rule is consulted in exactly one place, so collision groups are computed
|
||||
on the name a function will really take — which means renaming one of two
|
||||
colliding symbols dissolves the collision instead of leaving both refused, and
|
||||
`Spin2D`/`spin2d` gains a way out that is not a hand-written line. An excluded
|
||||
symbol still reports that it was excluded rather than going quiet: "there is no
|
||||
such binding" and "the package decided against this binding" are different
|
||||
answers.
|
||||
|
||||
What is actually in raylib's: `exclude Mem*`, because raylib exports
|
||||
malloc/realloc/free under its own names and binding them would put a second
|
||||
untracked heap behind three innocuous-looking Flan names, against plan.org's
|
||||
rule that an operation never falls back to a hidden allocator. And 19 `name`
|
||||
lines giving the generated predicates the `?` spelling the hand-written ones
|
||||
already use — `window-ready?` rather than `is-window-ready`, because
|
||||
`key-pressed?` and `is-window-ready` living in one package is precisely the
|
||||
split this change exists to remove. The C symbol is kept verbatim in
|
||||
`Ast.DeclareC` either way, so a rename costs nothing: it is still what is
|
||||
called and still what the check compares against.
|
||||
|
||||
Two bindings the config cannot express stay hand-written, and they are the
|
||||
reason `declare-c` remains the escape hatch: `LoadFontEx` and
|
||||
`LoadImageFromMemory` are bound `-raw` and wrapped by a Flan function of the
|
||||
same name without the suffix, one taking a slice and one answering with an
|
||||
`Option`. The importer refuses them by name collision with those wrappers,
|
||||
which is the correct answer.
|
||||
|
||||
### What a headless FFI test can and cannot pin
|
||||
|
||||
Worth knowing before writing another one, because two plausible tests in a row turned out to check nothing.
|
||||
|
||||
15
DISCUSS.md
15
DISCUSS.md
@ -182,6 +182,10 @@ build, and a permuted `Texture2D` or an `f64` for a `float` is caught by name.
|
||||
|
||||
What is left is two decisions, and both are the author's.
|
||||
|
||||
**6a and 6b are answered. See BUILT.md, "The bindings are committed now".** It became a code generator whose
|
||||
output is committed, and the 172 hand-written lines were *not* migrated — for a reason 6b did not reach. The two
|
||||
sections below are kept as the reasoning that got there.
|
||||
|
||||
### 6a. Does reading the header stay a build-time step, or become a code generator?
|
||||
|
||||
The cost, measured, with the wrappers pruned by `Reach` as they already were:
|
||||
@ -222,6 +226,17 @@ better than the rule's. `IsKeyPressed` is `key-pressed?` by hand and `is-key-pre
|
||||
would become an integer at the call site. A migration is therefore not a deletion; it is a deletion plus a
|
||||
kept list of the lines whose face is deliberately nicer than the header's.
|
||||
|
||||
**Answered: no, and the blocker is not the one above.** Vendoring stopped being the question once the *output* was
|
||||
committed rather than the header — no header is needed at any build. The count was also smaller than feared: 136 of
|
||||
the 172 are exactly what the rule produces, and the other 36 are expressible as `name` overrides in `bindings`.
|
||||
|
||||
What actually decides it is that migration would gut the check. Everything the generator emits agrees with the header
|
||||
by construction, so diffing generated output against its own source proves nothing; the hand-written lines have a
|
||||
different author, so they are the only declarations a header can contradict — and all ten of the 5.1-dev differences
|
||||
came from them. Delete them and the signature half of the check silently becomes a tautology. The enum point above
|
||||
survives intact as a second reason: `(rl/key-down? :space)` keeps its `Key` parameter only because that line is
|
||||
hand-written.
|
||||
|
||||
## 7. Watching variables
|
||||
|
||||
Raised while designing item 1, and deliberately separated from it.
|
||||
|
||||
42
NEXT.md
42
NEXT.md
@ -1,27 +1,9 @@
|
||||
## Queued: commit the generated bindings, with a config beside them
|
||||
## Queued: an idiomatic layer over the generated bindings
|
||||
|
||||
**Decided.** `flan import-c` already exists and prints the lines; kebab-casing is already implemented and reversible
|
||||
(the C symbol is kept verbatim in `Ast.DeclareC`, so the rule never has to be undone).
|
||||
|
||||
**Generate once, commit the result, regenerate when raylib moves.** What changes against today's opt-in header read:
|
||||
no header is needed by anyone, so the `?${FLAN_RAYLIB_H}` split disappears and every build gets all 428 bindings; the
|
||||
bindings become greppable and diffable in the repo; and the hand-written 172 stop being a separate set to maintain.
|
||||
Caching is not the argument — the dump is already cached on disk and in memory.
|
||||
|
||||
**What is given up, and it is the real cost:** the build-time check against the real header stops being automatic and
|
||||
becomes something run at regeneration. That check earned its place — it verified all 172 hand-written declarations and
|
||||
all 16 struct layouts against raylib 5.5 and found them exactly right, and against a 5.1-dev header on the same
|
||||
machine it found ten genuine differences. Keep it as a step, and make regeneration run it.
|
||||
|
||||
**A config file beside `headers`**, because the generated file is committed and therefore any hand-edit is destroyed by
|
||||
the next regeneration — the config is what survives. Wanted: **exclude patterns**, and **name overrides** where
|
||||
kebab-casing gives something ugly. A postprocessing transform pass was considered and rejected: it is a second program
|
||||
to understand and the config covers the real cases.
|
||||
|
||||
**Document it on the web page.** `web/index.html` has no section on the FFI's generated half.
|
||||
|
||||
**Later, not now: an idiomatic layer.** Thin Flan-shaped wrappers *over* the generated bindings, not instead of them —
|
||||
the generated set stays honest to C, and the layer is where a Flan-shaped API lives.
|
||||
Thin Flan-shaped wrappers **over** the generated bindings, not instead of them. The generated set stays honest to C —
|
||||
that is what makes it checkable against the header — and the layer is where a Flan-shaped API lives. Two of these
|
||||
already exist by hand in `vendor/raylib/raylib.flan` and are the shape to copy: `collision-point-poly?` takes a slice
|
||||
and `collision-lines` answers with an `Option`, each wrapping a `-raw` binding of the same name.
|
||||
|
||||
## Queued: a restart is not a transaction, and the docs must say so
|
||||
|
||||
@ -42,13 +24,15 @@ skip a frame and carry on rather than die — exactly the case where a non-idemp
|
||||
|
||||
Three things, in order. The first two are one line each and unblock a real game.
|
||||
|
||||
**1. `DrawTexturePro` is not bound, and it is the one true blocker for `siam-farmer`.** See `PORTING.md`, written
|
||||
against both the Clojure implementation and the WIP Common Lisp port. Every tile in both goes through it;
|
||||
`DrawTextureRec` does not scale and `DrawTextureEx` takes no source rect, so the renderer **cannot be written at all**
|
||||
in a default build. It is reachable only through the opt-in `FLAN_RAYLIB_H` import, which `vendor/raylib/headers`
|
||||
deliberately keeps optional. One `declare-c` line.
|
||||
**1. `DrawTexturePro` — done, and not by a hand-written line.** It was the one true blocker for `siam-farmer`
|
||||
(see `PORTING.md`: every tile in both implementations goes through it, and neither `DrawTextureRec` nor
|
||||
`DrawTextureEx` substitutes). It was reachable only through the opt-in `FLAN_RAYLIB_H` import. The bindings are
|
||||
committed now, so `rl/draw-texture-pro` is in `vendor/raylib/generated.flan` and a default build has it. **Nothing
|
||||
should add it by hand** — a second `declare-c` for the same C symbol is refused for the whole program.
|
||||
|
||||
**2. `Key` has no `left-shift`.** Both implementations use shift+1..5 to pick the tilemap. One enum member.
|
||||
**2. `Key` has no `left-shift`.** Both implementations use shift+1..5 to pick the tilemap. One enum member, and
|
||||
still a hand edit: the importer generates functions and only functions, so no `defenum` comes out of the header.
|
||||
`vendor/raylib/raylib.flan` is where `Key` lives.
|
||||
|
||||
**3. An out-of-bounds index should signal a condition, not `exit(134)`.** `PORTING.md`'s own first recommendation
|
||||
after the binding lines. The game indexes grids straight from mouse coordinates — `game.lisp` had to add an
|
||||
|
||||
@ -1140,15 +1140,77 @@ void flan_shim_get_mouse_position_5ad0e205(flan_ty_Vector2_1bebc5ae *out) {
|
||||
*out = GetMousePosition();
|
||||
}</code></pre>
|
||||
|
||||
<p>No library header is read, deliberately, so a build needs the shared library to be
|
||||
linkable and not the <code>-devel</code> package to be installed.
|
||||
<p>No library header is read during a build, deliberately, so a build needs the shared
|
||||
library to be linkable and not the <code>-devel</code> package to be installed.
|
||||
<strong>Guaranteed:</strong> the C typedef
|
||||
and the Flan struct come from the same <code>defstruct</code>, so they cannot disagree,
|
||||
and clang type-checks the wrapper against the generated prototype.
|
||||
<strong>Trusted:</strong> that the <code>defstruct</code> matches the library's real
|
||||
struct, and that the <code>declare-c</code> signature is the function's real signature.
|
||||
A scalar's width now carries ABI weight — <code>f64</code> where the library says
|
||||
<code>float</code> emits <code>double</code>, and the library reads garbage.</p>
|
||||
<code>float</code> emits <code>double</code>, and the library reads garbage. That
|
||||
trusted half is what the generator below checks.</p>
|
||||
|
||||
<h3>Generated bindings, committed</h3>
|
||||
|
||||
<p>Writing a binding per function by hand does not scale past the ones a program
|
||||
happens to call, so most of raylib's package is not written by hand. A package
|
||||
directory may carry a <code>headers</code> file naming the library's own C header;
|
||||
<code>flan generate-c <package-dir></code> reads it with
|
||||
<code>clang -Xclang -ast-dump=json</code>, turns every function it can represent into
|
||||
the same <code>declare-c</code> line a person would have written, and writes them to
|
||||
<code>generated.flan</code> in the package — which is <em>committed</em>.</p>
|
||||
|
||||
<pre><code class="sh">$ export FLAN_RAYLIB_H=/path/to/raylib-5.5/src/raylib.h
|
||||
$ flan generate-c vendor/raylib
|
||||
wrote vendor/raylib/generated.flan: 253 declarations, 156 refused, of 581 functions.
|
||||
Every defstruct and every hand-written declare-c agrees with it.</code></pre>
|
||||
|
||||
<p>Committing the output rather than generating at build time is what keeps the
|
||||
no-header property honest: the declarations are in the repository, so every build gets
|
||||
all of them, they are greppable, and they show up in a diff when the library moves. The
|
||||
argument is not caching — the clang dump is already cached on disk and in memory.</p>
|
||||
|
||||
<p><strong>Regeneration is the check.</strong> The cost of committing the output is that
|
||||
nothing compares the bindings against reality on every build any more, so the one
|
||||
function that writes the file compares first and <em>refuses to write</em> when the
|
||||
package and the header disagree: every <code>defstruct</code> against the header's
|
||||
record, and every hand-written <code>declare-c</code> against the header's signature.
|
||||
Pointed at a raylib 5.1-dev header while the package is written for 5.5, it reports ten
|
||||
real differences and writes nothing — which is exactly the silent version skew a
|
||||
generated file would otherwise bake in and make look reviewed.</p>
|
||||
|
||||
<p>This is also why the hand-written bindings are kept rather than replaced by generated
|
||||
ones. Everything the generator emits agrees with the header by construction, so diffing
|
||||
generated output against the header it came from proves nothing; the hand-written lines
|
||||
were transcribed by a person, so they are the only declarations a header can actually
|
||||
contradict. All ten of those differences came from them.</p>
|
||||
|
||||
<p>A committed generated file cannot be hand-corrected — the next regeneration destroys
|
||||
the edit without telling anybody — so the corrections live in a <code>bindings</code>
|
||||
file beside <code>headers</code>, which is read <em>while</em> the declarations are made.
|
||||
Two directives:</p>
|
||||
|
||||
<pre><code class="sh"># raylib's own malloc/realloc/free, which would be a second untracked heap
|
||||
# behind three innocuous Flan names.
|
||||
exclude Mem*
|
||||
|
||||
# The kebab rule gives is-window-ready. Lisp spells a predicate with a ?.
|
||||
name IsWindowReady window-ready?</code></pre>
|
||||
|
||||
<p>An excluded function still says it was excluded rather than going quiet, and a name
|
||||
override changes only the Flan face — the C symbol is kept verbatim in the declaration,
|
||||
so it is still what is called and still what the check compares. Renaming is also the
|
||||
way out of a collision: <code>Spin2D</code> and <code>spin2d</code> both kebab to
|
||||
<code>spin-2d</code>, so neither takes the name, because which one won would otherwise
|
||||
depend on the order the header happens to declare them in.</p>
|
||||
|
||||
<p>Anything neither directive can express is a hand-written <code>declare-c</code> in the
|
||||
package's own source, which wins over the generated file and is left alone by the
|
||||
generator. That is the escape hatch for a signature the importer gets wrong and for a
|
||||
Flan face the header cannot describe — raylib keeps two, each a raw binding wrapped by a
|
||||
Flan function of the same name, one taking a slice and one answering with an
|
||||
<code>Option</code>.</p>
|
||||
|
||||
<p>Everything the boundary cannot represent is refused by name with the reason, rather
|
||||
than half-supported: an <code>Option</code>, a union, a fixed array, a map, a returned
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user