diff --git a/BUILT.md b/BUILT.md index 98d63c6..16bd9a4 100644 --- a/BUILT.md +++ b/BUILT.md @@ -1489,6 +1489,138 @@ And the **accumulation pattern** — `(fn [c] (push errors c) ...)` over an encl capture does not exist at all, and the spec's captured-`Vec`-by-pointer rule has never had to exist because every capturable type today is a value type. It is its own item and should be planned as one. +## Assets are baked in, and the reason it is a compiler feature + +NEXT.md decision 1. `(embed "brush.png")` is a `[u8]`, `(embed "brush.png" string)` is a `string`, and +`(embed-dir "assets")` is a `[n EmbedFile]` sorted by name. Odin's `#load` and `#load_directory` are the model +(`src/parser.cpp`, and `check_load_directive` / `check_load_directory_directive` in `src/check_builtin.cpp`); Odin's +`#` is not imported, because an s-expression language already has a head position for a name and these resolve as +ordinary named calls exactly the way `vec-new` and `heap-allocator` do. + +**The reason for this shape rather than a build flag is the one that decided it.** It is a *compiler* feature, so it +needs no linker arguments and no per-target packaging, and it works identically on desktop and web. That matters more +here than it does for Odin, because `Load` hands out `lflags` only to a directory package and `main` is not exported, +so a program can never be a package: the single file doing `(rl/load-texture "brush.png")` is structurally the one file +with **no link channel at all**. The web lane found that hole and did not invent a flag for it. Embedding has no such +hole, because there is nothing to tell the linker. + +**It costs nothing at run time.** The bytes reach the program as a `Tast.Str` node typed `[u8]`, which emit.ml turns +into the same `private unnamed_addr constant` every string literal already becomes, and its `escape` is byte-exact +across the whole 0–255 range, so a PNG survives the round trip through the `.ll`. Bound with `defconst` at top level an +`embed-dir` is an LLVM constant outright, through emit.ml's `const`. + +**A `Str` node typed `[u8]`, not a `Bytes` prim over a `string`.** This is the one non-obvious choice. `Bytes` is +identity — emit.ml lowers `Types.String` and `Types.Slice _` to the same `%slice` — but wrapping the literal in a prim +makes the node non-constant, and `const` then refuses an `embed-dir` in a `defconst` with *a global's value must be a +compile-time constant*. Both of emit.ml's string emitters take the bytes and ignore the node's type, so it is the same +constant either way and this one is a constant a global can hold. + +**Two spellings, not one form that changes type with its context.** Odin threads a `type_hint` everywhere and can +afford `#load("p")` to mean a `string` here and a `[]u8` there. With structural equality, no implicit widening and no +coercion anywhere, the same text meaning two types would be a wart, so `string` is written down when it is wanted. The +site's expectation is a fallback only and nothing depends on it. + +**The path is a literal and resolves relative to the file the form is written in.** Both are Odin's rules and for +Odin's reasons: the bytes must be in hand before any value exists, which is what makes the result free; and a path +relative to the compiler's working directory would make a package's assets depend on where `flan` was invoked from, +which cannot be right. A missing file is a compile error naming it, never an empty embed — an asset silently absent is +exactly the quiet wrongness this removes. An empty *directory* is not that case and embeds cleanly as `[0 EmbedFile]`. + +**The directory lookup is a linear scan, and that is the chosen answer rather than the fallback one.** `embed-find` is +an ordinary prelude function over a `[EmbedFile]`. A directory embed is tens of entries whose names sit in cache-warm +`.rodata`; a compile-time perfect hash would be a build-time map with its own failure modes that nothing has asked for, +and sort-and-bisect is the next step if a program ever embeds thousands of files — it would not change the type. It +takes a **slice** rather than the array, because an array's length is part of its type and there are no generics, so +the call reads `(embed-find (slice assets 0 (len assets)) "brush.png")`. Entries are sorted by name because `readdir` +order is filesystem-dependent and an unsorted embed would make two builds of identical sources emit different `.ll`. +Non-recursive, files only — Odin again. + +**The sharp edge, inherited and not widened.** The slice points into `.rodata`, so a store through it segfaults at +`-O0` and is deleted as undefined behaviour at `-O2` — the same trap the prelude's ASCII-case note measures for +`(bytes "Hi")`, and the same one NEXT.md tracks as "writing through a string literal". Nothing here makes it worse and +nothing here fixes it; provenance is what would. **To get a mutable copy, clone the bytes into a `Vec`.** It is worth +saying loudly because an embedded asset is precisely the thing someone will try to decode in place. + +**What this does not do.** `sand.flan` still calls `(rl/load-texture "brush.png")`, which hands raylib a path for +raylib to open. Pointing raylib at embedded bytes needs `LoadImageFromMemory` and `LoadTextureFromImage` in place of +`LoadTexture` — a raylib binding question, not an embedding one — so the flagship program is not yet asset-free on the +web. The mechanism it needs is in. + +## `slurp`, `barf`, and the two ways they fail + +NEXT.md decisions 2 and 5. `(slurp path)` and `(slurp path allocator)` read a whole file into a `(Vec u8)`; +`(barf path bytes)` writes one. + +**`slurp` waited for `Vec` because its result has no length until the file is read**, and it obeys spec-memory.md's +rule without an exception: *no allocating operation returns an error*. There is no `Result` here, no out-parameter and +no error code — `slurp`'s type is `(Vec u8)` and `barf`'s is `Unit`. + +**Two failures, two conditions, and the guards nest rather than merge.** Allocation failure is `StorageExhausted` under +`retry`, unchanged and reused. File failure is `FileError {:path :op :reason}` under `retry` and `use-value`. They stay +apart because they ask two different answerable questions: the handler that grows an arena is not the handler that +supplies another path, and collapsing them would make one handler guess which it was looking at. `file_guard` in +check.ml is `alloc_guard`'s shape built from the same nodes — a `while`, a `restart-case` and an `error` — so the +backend learns nothing new. + +**The restarts are Common Lisp's pair for a `file-error`.** `retry` for "the file may be there now"; `use-value [p +string]` for "try this other path". `use-value` is the first restart clause the **compiler itself** emits with a +parameter — typed restarts landed the same session — and its parameter *is* the path slot the attempt reads, so the +clause body is empty. emit.ml's `bind_params` stores the invoker's argument into the slot, the clause falls through, +and the loop re-attempts against the new path. Everything is inside that loop, so a `use-value` naming a different file +re-measures it and re-allocates for *its* size; the `Vec` is freed at the top of each turn, which is why a retry does +not leak, and freeing a `Vec` that never allocated is a no-op. + +**On the web, `barf` signals — every time, with the path in the condition.** This is the decision worth restating, +because two more obvious answers are both wrong here. A **build-time refusal** is unusable: Flan has *no conditional +compilation*, nothing in `parse.ml` or `check.ml` reads the target, so "isolate this to desktop" is not expressible in +source and the refusal would have nowhere to be silenced. A **silent no-op** is worse than either, because that is how +a save file disappears with nothing said. So the program gets a condition and decides — which is the language having +something Odin does not. Odin stubs its whole file API on js/wasm to `.Unsupported` (`core/os/file_js.odin`) so that +importing `core:os` "panics cleanly", and a panic is not a decision. **The restriction was taken; the mechanism was +not.** + +Nothing in the compiler reads the target to do this. The refusal is one `#ifdef __EMSCRIPTEN__` in `flan_rt.c`, which +is where the host ABI is *already* implemented twice. `slurp` keeps working on the web — it compiles, runs, and reports +a missing file honestly — and the bytes a web program actually wants come from an `embed`. + +`test/programs/web-files.flan` is one source built for both targets, and `test/test_web.ml` **runs** it under node +rather than asserting the artifact's shape: an artifact-shape assertion would say nothing about the thing the decision +bought. The test checks that the refusal is printed with its path and reason, and that the desktop's success line is +*absent* — a silent no-op would have taken that branch. + +### What the host ABI grew by, and why that much + +plan.org names the filesystem as the #1 portability risk — "pack assets, one abstraction, never touch paths" — so the +widening is written down rather than assumed. It is **three calls and one reader**: + +| Call | What it does | +|------|--------------| +| `flan_file_size(path, n, &out)` | how many bytes are there | +| `flan_file_read(path, n, buf, cap, &got)` | fill a buffer the caller owns | +| `flan_file_write(path, n, buf, len)` | write a whole file | +| `flan_file_fail_reason()` | which of the four reasons it was | + +They are POSIX-shaped and **Vec-ignorant**: no handle crosses the boundary, nothing is held between calls, and each +takes a path and answers 1/0 the way every allocator entry point already does. `flan_slurp_into` — the part that knows +what a `Vec` is — is runtime *glue* on this side of the ABI, not a fourth host call, so a second target implements +three functions and inherits the rest. The reason is a global rather than an out-parameter for the same reason +`flan_alloc_fail_bytes` is: the condition is a value struct on the failing frame's stack with fixed numeric fields and +no rendered message. + +**These do touch paths, which is the widening plan.org warned about**, and decision 2 took it knowingly. `embed` is the +half that does not: it needs no host ABI at all, so "pack assets" remains the answer for anything known at build time +and `slurp` is for bytes that genuinely are not. + +`FileError` is one type with a `reason` field rather than a family, because conditions have no hierarchy today +(spec-conditions.md §1) and a family would need one handler clause per member to say "any file error". NEXT.md decision +4's parent link is the answer to that and is not built; when it is, these reasons can become types without any call +site changing. + +**One thing `slurp` and `barf` reveal that is not theirs to fix.** A handler that wants "try to save, and carry on if +you cannot" has nowhere to go: `error` is diverging (spec-conditions.md §2), a handler returning normally has not +answered it, and neither `retry` nor `use-value` means *give up*. `web-files.flan` calls `exit` for that reason. A +`continue`-style restart — or `handler-case`, which unwinds — is what the case wants, and neither exists. + ## Where build time goes `flan build calc-me.flan` was ~160ms, and ~95% of it was clang. **The object cache is in**, and it is now ~110ms: diff --git a/NEXT.md b/NEXT.md index d9f4901..deeff41 100644 --- a/NEXT.md +++ b/NEXT.md @@ -229,7 +229,7 @@ $ flan run sand.flan # a window, 120 fps, hold space Five questions were put and answered in one sitting. Each is a decision, not a preference — build against them, and reopen one only with a reason rather than a taste. -**1. Assets are embedded at compile time, one file or one directory.** Odin's answer, and the reason it is the right +~~**1. Assets are embedded at compile time, one file or one directory.**~~ **Built** — `(embed "p")`, `(embed "p" string)`, `(embed-dir "d")`. See BUILT.md, "Assets are baked in". Odin's answer, and the reason it is the right one here: it is a *compiler* feature, so it needs no build flags, no linker arguments and no per-target packaging, and it works identically on desktop and web. That matters more here than it does for Odin, because `Load` gives link flags only to directory packages — the single file doing `(rl/load-texture "brush.png")` is structurally the one file with @@ -238,7 +238,7 @@ no link channel, which is what stopped the web lane from inventing a flag. Embed `--preload-file` stays available later for assets that should load lazily rather than be baked in; the `@web` link line already carries it if wanted. -**2. Reading a file works everywhere; writing is desktop-only and signals on web.** Odin stubs its whole file API on +~~**2. Reading a file works everywhere; writing is desktop-only and signals on web.**~~ **Built** — `barf` on the web signals `FileError` with reason `file-unsupported`, and `test/test_web.ml` runs it under node rather than asserting the artifact's shape. See BUILT.md, "slurp, barf, and the two ways they fail". Odin stubs its whole file API on js/wasm — every operation returns `.Unsupported`, and `core/os/file_js.odin`'s own comment says the stubs exist only so importing `core:os` "panics cleanly". Take the restriction and not the mechanism. **Flan has no conditional compilation** — nothing in `parse.ml` or `check.ml` reads the target — so "isolate this code to desktop" is not @@ -264,7 +264,7 @@ those costs and leaves the frozen model otherwise intact. Real inheritance stays this closes nothing off. That section stays open for the record but is no longer the blocking question for `handler-case`. -**5. File I/O — `slurp` and `barf` — is the next stdlib work**, after `Vec`, because `slurp` returns a string whose +~~**5. File I/O — `slurp` and `barf` — is the next stdlib work**~~ **Built.** It was the next stdlib work, after `Vec`, because `slurp` returns a string whose length is not known until the file is read and therefore cannot exist before an allocator does. ## Blocked and unfinished @@ -375,8 +375,14 @@ equivalent — add the include and the agent compiles into a web build that can `vendor:agent` by name on a web target the way `--dev` is refused. The second is the honest one. Neither was taken here: `vendor/agent/` belonged to another lane this session. -**2. Assets are two questions and only one of them is about emscripten.** `sand.flan` does -`(rl/load-texture "brush.png")` against a bare relative path. +~~**2. Assets are two questions and only one of them is about emscripten.**~~ **Answered by the embed above, and the +answer was the third option neither half here considered: make it a compiler feature and neither question arises.** The +hard half below is exactly right about the problem — the file that needs the asset is structurally the one file that +cannot declare it — and the conclusion drawn from it, that the fix must be a link channel or a new declaration, was +the wrong one. `(embed "brush.png")` needs no channel, because there is nothing to tell the linker. What is *not* done +is `sand.flan` itself: `(rl/load-texture "brush.png")` takes a path and raylib opens it, so pointing raylib at embedded +bytes needs `LoadTextureFromImage` over `LoadImageFromMemory`, which is a raylib binding question and not this one. +The original text follows. `sand.flan` does `(rl/load-texture "brush.png")` against a bare relative path. - The easy half: a bare relative path has no meaning on a target with no filesystem. emscripten's answer is `--embed-file` or `--preload-file` into MEMFS, and both are *linker arguments*, so they are already expressible as an