Where the generated half is written down, in all four places it belongs
web/index.html had no section on the FFI's generated half at all; it has one now, with the command, the config, and the reason committing the output is what makes the no-header property honest rather than a caveat. BUILT.md gets why the 172 stay, which is the part that is easy to get wrong: 136 of them are exactly what the rule produces and the rest are expressible as overrides, so the superset argument is sound and still leads somewhere bad — deleting them reduces the signature check to a tautology. DISCUSS.md 6a and 6b are answered rather than left open, and 6b's own point about enums turns out to be live in the tree: key-down? keeps its Key parameter because it is hand-written, and the generated key-up? beside it takes an i32.
This commit is contained in:
parent
ecfe180d2e
commit
65fd49f0c6
89
BUILT.md
89
BUILT.md
@ -162,6 +162,8 @@ described is refused with that reason, so `vendor/raylib` describing thirteen
|
|||||||
structs is what makes the import thirteen structs wide, and describing a
|
structs is what makes the import thirteen structs wide, and describing a
|
||||||
fourteenth widens it. Of raylib 5.5's 581 functions, 256 import, 153 are
|
fourteenth widens it. Of raylib 5.5's 581 functions, 256 import, 153 are
|
||||||
refused, and 172 are left alone because the package already binds them by hand.
|
refused, and 172 are left alone because the package already binds them by hand.
|
||||||
|
(253 and 156 once `bindings` excludes raylib's three allocator entry points —
|
||||||
|
see the next section.)
|
||||||
|
|
||||||
Not generating `defstruct`s is what makes the check possible at all. Generate
|
Not generating `defstruct`s is what makes the check possible at all. Generate
|
||||||
them and the header becomes the authority on layout, and comparing the
|
them and the header becomes the authority on layout, and comparing the
|
||||||
@ -355,6 +357,93 @@ header cache and the same build is 0.92s against 0.83s. So the header read is
|
|||||||
compiling a shim with 428 wrappers in it, which is the object cache's business
|
compiling a shim with 428 wrappers in it, which is the object cache's business
|
||||||
and already warm after one build.
|
and already warm after one build.
|
||||||
|
|
||||||
|
### The bindings are committed now — `generated.flan`, `bindings`, `flan generate-c`
|
||||||
|
|
||||||
|
The section above reads the header at build time, behind an opt-in
|
||||||
|
`?${FLAN_RAYLIB_H}` in `headers`, because a build should need libraylib
|
||||||
|
linkable and not raylib-devel installed. That opt-in was doing two jobs and
|
||||||
|
only one of them was defensible: it decided whether a *check* ran, which is
|
||||||
|
fine to make optional, and it decided whether a package had 172 bindings or
|
||||||
|
428, which is not. `DrawTexturePro` being reachable only by exporting an
|
||||||
|
environment variable is the shape of that second job, and it blocked a real
|
||||||
|
game — see PORTING.md.
|
||||||
|
|
||||||
|
**Generate once, commit the result, regenerate when raylib moves.**
|
||||||
|
`flan generate-c vendor/raylib` reads the header named by `headers`, writes
|
||||||
|
`vendor/raylib/generated.flan`, and that file is checked in. No header is
|
||||||
|
needed by anybody: every build gets all 425 declarations, they are greppable,
|
||||||
|
and they show up in a diff when the library moves. **Caching is not the
|
||||||
|
argument** — the dump is already cached on disk and in memory, so a build that
|
||||||
|
reads a header pays for it once either way. The argument is the dependency and
|
||||||
|
the diff.
|
||||||
|
|
||||||
|
**What it costs is the check, so regeneration runs it and the check gates the
|
||||||
|
write.** A file on disk has no second opinion, so nothing compares the bindings
|
||||||
|
against the library on an ordinary build any more. The one function that writes
|
||||||
|
`generated.flan` therefore compares first — every `defstruct` against the
|
||||||
|
header's record, every hand-written `declare-c` against the header's signature
|
||||||
|
— and writes nothing when they disagree. It is not possible to regenerate
|
||||||
|
without comparing, because there is no other way to write the file. Pointed at
|
||||||
|
the 5.1-dev header on this machine while the package is written for 5.5, it
|
||||||
|
reports the same ten real differences the diff above found and writes nothing.
|
||||||
|
|
||||||
|
**The 172 hand-written lines stay, and the reason is not caution.** It was
|
||||||
|
tempting to delete them: 136 of the 172 are exactly what the kebab rule would
|
||||||
|
have produced, and the other 36 could be spelled as name overrides, so the
|
||||||
|
generated set really is a superset. The argument against is the one this
|
||||||
|
section is about. Everything the generator emits agrees with the header *by
|
||||||
|
construction* — the declaration and the prototype come from one dump — so
|
||||||
|
diffing generated output against the header it came from is a tautology, and
|
||||||
|
replacing the hand-written set would quietly reduce the signature half of the
|
||||||
|
check to nothing. The hand-written lines were transcribed from raylib's
|
||||||
|
documentation by a person; they are the only declarations in the package a
|
||||||
|
header can actually contradict. All ten of the 5.1-dev differences came from
|
||||||
|
them. They are not a parallel set to maintain — they are the second opinion,
|
||||||
|
and the check is what maintains them.
|
||||||
|
|
||||||
|
The opt-in did not go away, it stopped deciding anything important. With
|
||||||
|
`FLAN_RAYLIB_H` set, an ordinary build still reads the header, and since every
|
||||||
|
C symbol is now bound — by hand or by generation — the importer generates
|
||||||
|
nothing and the read is purely the check. It is a *better* check than before:
|
||||||
|
425 declarations rather than 172, because `generated.flan` is a package file
|
||||||
|
like any other and is checked like one.
|
||||||
|
|
||||||
|
**`bindings`, beside `headers`, is what survives regeneration.** A committed
|
||||||
|
generated file cannot be hand-corrected — the next run overwrites it and the
|
||||||
|
edit is destroyed without anybody being told, which is the worst shape an edit
|
||||||
|
can have — so the corrections have to live somewhere regeneration *reads*. Two
|
||||||
|
directives, which are the two things the header cannot decide: `exclude <symbol
|
||||||
|
or pattern>` and `name <symbol> <flan-name>`. A postprocessing transform pass
|
||||||
|
was considered and rejected: a second program to understand, run over text the
|
||||||
|
generator had already committed to.
|
||||||
|
|
||||||
|
Both are applied *while* the declarations are made, which is not a detail. The
|
||||||
|
kebab rule is consulted in exactly one place, so collision groups are computed
|
||||||
|
on the name a function will really take — which means renaming one of two
|
||||||
|
colliding symbols dissolves the collision instead of leaving both refused, and
|
||||||
|
`Spin2D`/`spin2d` gains a way out that is not a hand-written line. An excluded
|
||||||
|
symbol still reports that it was excluded rather than going quiet: "there is no
|
||||||
|
such binding" and "the package decided against this binding" are different
|
||||||
|
answers.
|
||||||
|
|
||||||
|
What is actually in raylib's: `exclude Mem*`, because raylib exports
|
||||||
|
malloc/realloc/free under its own names and binding them would put a second
|
||||||
|
untracked heap behind three innocuous-looking Flan names, against plan.org's
|
||||||
|
rule that an operation never falls back to a hidden allocator. And 19 `name`
|
||||||
|
lines giving the generated predicates the `?` spelling the hand-written ones
|
||||||
|
already use — `window-ready?` rather than `is-window-ready`, because
|
||||||
|
`key-pressed?` and `is-window-ready` living in one package is precisely the
|
||||||
|
split this change exists to remove. The C symbol is kept verbatim in
|
||||||
|
`Ast.DeclareC` either way, so a rename costs nothing: it is still what is
|
||||||
|
called and still what the check compares against.
|
||||||
|
|
||||||
|
Two bindings the config cannot express stay hand-written, and they are the
|
||||||
|
reason `declare-c` remains the escape hatch: `LoadFontEx` and
|
||||||
|
`LoadImageFromMemory` are bound `-raw` and wrapped by a Flan function of the
|
||||||
|
same name without the suffix, one taking a slice and one answering with an
|
||||||
|
`Option`. The importer refuses them by name collision with those wrappers,
|
||||||
|
which is the correct answer.
|
||||||
|
|
||||||
### What a headless FFI test can and cannot pin
|
### What a headless FFI test can and cannot pin
|
||||||
|
|
||||||
Worth knowing before writing another one, because two plausible tests in a row turned out to check nothing.
|
Worth knowing before writing another one, because two plausible tests in a row turned out to check nothing.
|
||||||
|
|||||||
15
DISCUSS.md
15
DISCUSS.md
@ -182,6 +182,10 @@ build, and a permuted `Texture2D` or an `f64` for a `float` is caught by name.
|
|||||||
|
|
||||||
What is left is two decisions, and both are the author's.
|
What is left is two decisions, and both are the author's.
|
||||||
|
|
||||||
|
**6a and 6b are answered. See BUILT.md, "The bindings are committed now".** It became a code generator whose
|
||||||
|
output is committed, and the 172 hand-written lines were *not* migrated — for a reason 6b did not reach. The two
|
||||||
|
sections below are kept as the reasoning that got there.
|
||||||
|
|
||||||
### 6a. Does reading the header stay a build-time step, or become a code generator?
|
### 6a. Does reading the header stay a build-time step, or become a code generator?
|
||||||
|
|
||||||
The cost, measured, with the wrappers pruned by `Reach` as they already were:
|
The cost, measured, with the wrappers pruned by `Reach` as they already were:
|
||||||
@ -222,6 +226,17 @@ better than the rule's. `IsKeyPressed` is `key-pressed?` by hand and `is-key-pre
|
|||||||
would become an integer at the call site. A migration is therefore not a deletion; it is a deletion plus a
|
would become an integer at the call site. A migration is therefore not a deletion; it is a deletion plus a
|
||||||
kept list of the lines whose face is deliberately nicer than the header's.
|
kept list of the lines whose face is deliberately nicer than the header's.
|
||||||
|
|
||||||
|
**Answered: no, and the blocker is not the one above.** Vendoring stopped being the question once the *output* was
|
||||||
|
committed rather than the header — no header is needed at any build. The count was also smaller than feared: 136 of
|
||||||
|
the 172 are exactly what the rule produces, and the other 36 are expressible as `name` overrides in `bindings`.
|
||||||
|
|
||||||
|
What actually decides it is that migration would gut the check. Everything the generator emits agrees with the header
|
||||||
|
by construction, so diffing generated output against its own source proves nothing; the hand-written lines have a
|
||||||
|
different author, so they are the only declarations a header can contradict — and all ten of the 5.1-dev differences
|
||||||
|
came from them. Delete them and the signature half of the check silently becomes a tautology. The enum point above
|
||||||
|
survives intact as a second reason: `(rl/key-down? :space)` keeps its `Key` parameter only because that line is
|
||||||
|
hand-written.
|
||||||
|
|
||||||
## 7. Watching variables
|
## 7. Watching variables
|
||||||
|
|
||||||
Raised while designing item 1, and deliberately separated from it.
|
Raised while designing item 1, and deliberately separated from it.
|
||||||
|
|||||||
42
NEXT.md
42
NEXT.md
@ -1,27 +1,9 @@
|
|||||||
## Queued: commit the generated bindings, with a config beside them
|
## Queued: an idiomatic layer over the generated bindings
|
||||||
|
|
||||||
**Decided.** `flan import-c` already exists and prints the lines; kebab-casing is already implemented and reversible
|
Thin Flan-shaped wrappers **over** the generated bindings, not instead of them. The generated set stays honest to C —
|
||||||
(the C symbol is kept verbatim in `Ast.DeclareC`, so the rule never has to be undone).
|
that is what makes it checkable against the header — and the layer is where a Flan-shaped API lives. Two of these
|
||||||
|
already exist by hand in `vendor/raylib/raylib.flan` and are the shape to copy: `collision-point-poly?` takes a slice
|
||||||
**Generate once, commit the result, regenerate when raylib moves.** What changes against today's opt-in header read:
|
and `collision-lines` answers with an `Option`, each wrapping a `-raw` binding of the same name.
|
||||||
no header is needed by anyone, so the `?${FLAN_RAYLIB_H}` split disappears and every build gets all 428 bindings; the
|
|
||||||
bindings become greppable and diffable in the repo; and the hand-written 172 stop being a separate set to maintain.
|
|
||||||
Caching is not the argument — the dump is already cached on disk and in memory.
|
|
||||||
|
|
||||||
**What is given up, and it is the real cost:** the build-time check against the real header stops being automatic and
|
|
||||||
becomes something run at regeneration. That check earned its place — it verified all 172 hand-written declarations and
|
|
||||||
all 16 struct layouts against raylib 5.5 and found them exactly right, and against a 5.1-dev header on the same
|
|
||||||
machine it found ten genuine differences. Keep it as a step, and make regeneration run it.
|
|
||||||
|
|
||||||
**A config file beside `headers`**, because the generated file is committed and therefore any hand-edit is destroyed by
|
|
||||||
the next regeneration — the config is what survives. Wanted: **exclude patterns**, and **name overrides** where
|
|
||||||
kebab-casing gives something ugly. A postprocessing transform pass was considered and rejected: it is a second program
|
|
||||||
to understand and the config covers the real cases.
|
|
||||||
|
|
||||||
**Document it on the web page.** `web/index.html` has no section on the FFI's generated half.
|
|
||||||
|
|
||||||
**Later, not now: an idiomatic layer.** Thin Flan-shaped wrappers *over* the generated bindings, not instead of them —
|
|
||||||
the generated set stays honest to C, and the layer is where a Flan-shaped API lives.
|
|
||||||
|
|
||||||
## Queued: a restart is not a transaction, and the docs must say so
|
## Queued: a restart is not a transaction, and the docs must say so
|
||||||
|
|
||||||
@ -42,13 +24,15 @@ skip a frame and carry on rather than die — exactly the case where a non-idemp
|
|||||||
|
|
||||||
Three things, in order. The first two are one line each and unblock a real game.
|
Three things, in order. The first two are one line each and unblock a real game.
|
||||||
|
|
||||||
**1. `DrawTexturePro` is not bound, and it is the one true blocker for `siam-farmer`.** See `PORTING.md`, written
|
**1. `DrawTexturePro` — done, and not by a hand-written line.** It was the one true blocker for `siam-farmer`
|
||||||
against both the Clojure implementation and the WIP Common Lisp port. Every tile in both goes through it;
|
(see `PORTING.md`: every tile in both implementations goes through it, and neither `DrawTextureRec` nor
|
||||||
`DrawTextureRec` does not scale and `DrawTextureEx` takes no source rect, so the renderer **cannot be written at all**
|
`DrawTextureEx` substitutes). It was reachable only through the opt-in `FLAN_RAYLIB_H` import. The bindings are
|
||||||
in a default build. It is reachable only through the opt-in `FLAN_RAYLIB_H` import, which `vendor/raylib/headers`
|
committed now, so `rl/draw-texture-pro` is in `vendor/raylib/generated.flan` and a default build has it. **Nothing
|
||||||
deliberately keeps optional. One `declare-c` line.
|
should add it by hand** — a second `declare-c` for the same C symbol is refused for the whole program.
|
||||||
|
|
||||||
**2. `Key` has no `left-shift`.** Both implementations use shift+1..5 to pick the tilemap. One enum member.
|
**2. `Key` has no `left-shift`.** Both implementations use shift+1..5 to pick the tilemap. One enum member, and
|
||||||
|
still a hand edit: the importer generates functions and only functions, so no `defenum` comes out of the header.
|
||||||
|
`vendor/raylib/raylib.flan` is where `Key` lives.
|
||||||
|
|
||||||
**3. An out-of-bounds index should signal a condition, not `exit(134)`.** `PORTING.md`'s own first recommendation
|
**3. An out-of-bounds index should signal a condition, not `exit(134)`.** `PORTING.md`'s own first recommendation
|
||||||
after the binding lines. The game indexes grids straight from mouse coordinates — `game.lisp` had to add an
|
after the binding lines. The game indexes grids straight from mouse coordinates — `game.lisp` had to add an
|
||||||
|
|||||||
@ -1140,15 +1140,77 @@ void flan_shim_get_mouse_position_5ad0e205(flan_ty_Vector2_1bebc5ae *out) {
|
|||||||
*out = GetMousePosition();
|
*out = GetMousePosition();
|
||||||
}</code></pre>
|
}</code></pre>
|
||||||
|
|
||||||
<p>No library header is read, deliberately, so a build needs the shared library to be
|
<p>No library header is read during a build, deliberately, so a build needs the shared
|
||||||
linkable and not the <code>-devel</code> package to be installed.
|
library to be linkable and not the <code>-devel</code> package to be installed.
|
||||||
<strong>Guaranteed:</strong> the C typedef
|
<strong>Guaranteed:</strong> the C typedef
|
||||||
and the Flan struct come from the same <code>defstruct</code>, so they cannot disagree,
|
and the Flan struct come from the same <code>defstruct</code>, so they cannot disagree,
|
||||||
and clang type-checks the wrapper against the generated prototype.
|
and clang type-checks the wrapper against the generated prototype.
|
||||||
<strong>Trusted:</strong> that the <code>defstruct</code> matches the library's real
|
<strong>Trusted:</strong> that the <code>defstruct</code> matches the library's real
|
||||||
struct, and that the <code>declare-c</code> signature is the function's real signature.
|
struct, and that the <code>declare-c</code> signature is the function's real signature.
|
||||||
A scalar's width now carries ABI weight — <code>f64</code> where the library says
|
A scalar's width now carries ABI weight — <code>f64</code> where the library says
|
||||||
<code>float</code> emits <code>double</code>, and the library reads garbage.</p>
|
<code>float</code> emits <code>double</code>, and the library reads garbage. That
|
||||||
|
trusted half is what the generator below checks.</p>
|
||||||
|
|
||||||
|
<h3>Generated bindings, committed</h3>
|
||||||
|
|
||||||
|
<p>Writing a binding per function by hand does not scale past the ones a program
|
||||||
|
happens to call, so most of raylib's package is not written by hand. A package
|
||||||
|
directory may carry a <code>headers</code> file naming the library's own C header;
|
||||||
|
<code>flan generate-c <package-dir></code> reads it with
|
||||||
|
<code>clang -Xclang -ast-dump=json</code>, turns every function it can represent into
|
||||||
|
the same <code>declare-c</code> line a person would have written, and writes them to
|
||||||
|
<code>generated.flan</code> in the package — which is <em>committed</em>.</p>
|
||||||
|
|
||||||
|
<pre><code class="sh">$ export FLAN_RAYLIB_H=/path/to/raylib-5.5/src/raylib.h
|
||||||
|
$ flan generate-c vendor/raylib
|
||||||
|
wrote vendor/raylib/generated.flan: 253 declarations, 156 refused, of 581 functions.
|
||||||
|
Every defstruct and every hand-written declare-c agrees with it.</code></pre>
|
||||||
|
|
||||||
|
<p>Committing the output rather than generating at build time is what keeps the
|
||||||
|
no-header property honest: the declarations are in the repository, so every build gets
|
||||||
|
all of them, they are greppable, and they show up in a diff when the library moves. The
|
||||||
|
argument is not caching — the clang dump is already cached on disk and in memory.</p>
|
||||||
|
|
||||||
|
<p><strong>Regeneration is the check.</strong> The cost of committing the output is that
|
||||||
|
nothing compares the bindings against reality on every build any more, so the one
|
||||||
|
function that writes the file compares first and <em>refuses to write</em> when the
|
||||||
|
package and the header disagree: every <code>defstruct</code> against the header's
|
||||||
|
record, and every hand-written <code>declare-c</code> against the header's signature.
|
||||||
|
Pointed at a raylib 5.1-dev header while the package is written for 5.5, it reports ten
|
||||||
|
real differences and writes nothing — which is exactly the silent version skew a
|
||||||
|
generated file would otherwise bake in and make look reviewed.</p>
|
||||||
|
|
||||||
|
<p>This is also why the hand-written bindings are kept rather than replaced by generated
|
||||||
|
ones. Everything the generator emits agrees with the header by construction, so diffing
|
||||||
|
generated output against the header it came from proves nothing; the hand-written lines
|
||||||
|
were transcribed by a person, so they are the only declarations a header can actually
|
||||||
|
contradict. All ten of those differences came from them.</p>
|
||||||
|
|
||||||
|
<p>A committed generated file cannot be hand-corrected — the next regeneration destroys
|
||||||
|
the edit without telling anybody — so the corrections live in a <code>bindings</code>
|
||||||
|
file beside <code>headers</code>, which is read <em>while</em> the declarations are made.
|
||||||
|
Two directives:</p>
|
||||||
|
|
||||||
|
<pre><code class="sh"># raylib's own malloc/realloc/free, which would be a second untracked heap
|
||||||
|
# behind three innocuous Flan names.
|
||||||
|
exclude Mem*
|
||||||
|
|
||||||
|
# The kebab rule gives is-window-ready. Lisp spells a predicate with a ?.
|
||||||
|
name IsWindowReady window-ready?</code></pre>
|
||||||
|
|
||||||
|
<p>An excluded function still says it was excluded rather than going quiet, and a name
|
||||||
|
override changes only the Flan face — the C symbol is kept verbatim in the declaration,
|
||||||
|
so it is still what is called and still what the check compares. Renaming is also the
|
||||||
|
way out of a collision: <code>Spin2D</code> and <code>spin2d</code> both kebab to
|
||||||
|
<code>spin-2d</code>, so neither takes the name, because which one won would otherwise
|
||||||
|
depend on the order the header happens to declare them in.</p>
|
||||||
|
|
||||||
|
<p>Anything neither directive can express is a hand-written <code>declare-c</code> in the
|
||||||
|
package's own source, which wins over the generated file and is left alone by the
|
||||||
|
generator. That is the escape hatch for a signature the importer gets wrong and for a
|
||||||
|
Flan face the header cannot describe — raylib keeps two, each a raw binding wrapped by a
|
||||||
|
Flan function of the same name, one taking a slice and one answering with an
|
||||||
|
<code>Option</code>.</p>
|
||||||
|
|
||||||
<p>Everything the boundary cannot represent is refused by name with the reason, rather
|
<p>Everything the boundary cannot represent is refused by name with the reason, rather
|
||||||
than half-supported: an <code>Option</code>, a union, a fixed array, a map, a returned
|
than half-supported: an <code>Option</code>, a union, a fixed array, a map, a returned
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user