diff --git a/DISCUSS.org b/DISCUSS.org index 9082ebf..3c4f088 100644 --- a/DISCUSS.org +++ b/DISCUSS.org @@ -194,6 +194,33 @@ as a real user-facing builtin instead of only an internal codegen detail. Not designed or implemented, just an idea. +Built, 2026-09-20, as two builtins rather than one — the author's answer to +the cheap-byte-or-real-pattern question was "why not both? we need some sort +of memset -1 right? and dead-beef can loop, that's fine". They are +(filled BYTE) and (dead-beef), spelled the way [zeroed] is: the value of +whatever type is expected of them, so (set grid (filled 0xFF)) is how a place +is filled and there is no place-taking form to learn beside [set]. + +(dead-beef) writes the default DEADBEEF and (dead-beef 0xBAADF00D) writes the +pattern given, under one byte-order rule: a pattern's ascending bytes are its +big-endian bytes, which is how the hex literal reads left to right. So every +candidate listed above is spellable without the compiler naming any of them, +and the bare form is checked into the spelled-out default rather than being a +case a backend knows about. The pattern may be computed, not only written — +same rule as the byte arm, and both backends byte-reverse at run time when it +is. + +The snag above is why there are two and not one with a wider operand. The +byte fill is one llvm.memset / one rep stosb; the 4-byte pattern is a loop on +both sides, a counted dword loop in emit.ml and rep stosd in x86.ml, because +the intrinsic really does only take a repeated i8. + +What may be filled is numbers, and structs and fixed arrays built out of +numbers — nothing that carries a tag, a length, an owning pointer or a +collector descriptor. That boundary, and why each refusal is the runtime's +rather than a matter of taste, is written up in FIX.org, "The two byte +fills". + ** struct field type change: got a plain type error, not the documented refusal Hit "expected i32, found u8" after changing Cell's row/col from u8 to i32. Turned out to be a leftover (u8 (/ my cell-size)) cast at the construction diff --git a/FIX.org b/FIX.org index d90bb06..9cdda86 100644 --- a/FIX.org +++ b/FIX.org @@ -2236,3 +2236,209 @@ write side. Measured after: 22 runs of ~test_dev.exe~, all exit 0, no fatal exception; 5 runs of ~dune test --root . --force~, all exit 0. ~--force~ because dune caches a test that passed, and a cached pass proves nothing about a race. + +* The two byte fills, 2026-09-20 +DISCUSS.org's "a DEADBEEF-style sentinel-fill builtin", built. The author's +answer to the single-byte-or-four-byte question was "why not both? we need +some sort of memset -1 right? and dead-beef can loop, that's fine", so there +are two builtins and they are siblings of ~zeroed~, not a new shape. + +Revised the same day: the pattern builtin was ~sentinel-filled~ and is now +~dead-beef~, and it gained an optional operand so the pattern is the +program's to choose. What did not change is ~filled~, or the fill boundary, +or the byte-order rule — the revision generalised the pattern, it did not +reopen what may be filled. + +** The spellings +~(filled BYTE)~ and ~(dead-beef)~ / ~(dead-beef PATTERN)~, all value forms +driven by the type expected of them, exactly as ~(zeroed)~ is: + +: (set grid (filled 0xFF)) +: (set frame (dead-beef)) +: (set frame (dead-beef 0xBAADF00D)) + +A place-taking ~(filled place byte)~ was the other candidate and was not +taken. ~zeroed~ already answers "the all-bytes-X value of whatever this is +being stored into", ~set~ already takes the place, and a second spelling for +an operation ~set~ expresses would have been a second thing to learn for +nothing. The cost is real and is paid on purpose: a fill in a position that +expects no type is refused ("filled needs to know the type it is filling"), +which is ~zeroed~'s own refusal worn by both siblings. + +~dead-beef~ takes the pattern or leaves it out, and leaving it out is +*defined as* writing the default: the checker's zero-argument arm builds the +same ~Tast.Int 0xDEADBEEF~ the spelled-out call would have, so ~(dead-beef)~ +and ~(dead-beef 0xDEADBEEF)~ are the same IR node by construction and no +backend has a second path for the bare form. An acceptance row prints both +and pins that they agree. + +The pattern is an ordinary ~u32~ /expression/, not a literal — the byte arm's +rule at four times the width. A literal out of range meets ~in_range~'s +located "does not fit in u32"; anything computed is guaranteed by its type +instead, since a ~u32~ cannot be out of ~u32~ range. Refusing a computed one +would have been a restriction with no mechanism behind it: neither backend +needs the number early. + +/Reading note, for whoever reviews this./ The revision asked for "a u32-range +constant ... decide literal-only vs any constant expression from what the +byte-fill arm already accepts". The byte-fill arm accepts any ~u8~ +expression, runtime ones included, and the same instruction asked that both +backends handle "eax loaded from a value, not an immediate" — which only +exists if a computed pattern is legal. So the operand is any ~u32~ +expression. That is a strict superset of constants-only: every program the +narrower reading allows behaves identically here. Tighten it to literals if +that was the intent; nothing else depends on the breadth. + +** The fill boundary — what may be overwritten with raw bytes +*Numbers, and structs and fixed arrays built out of numbers. Nothing else.* +~Check.unfillable~ is the rule, in one recursive walk, and every refusal +names the type it stopped at and why. + +Zero is a value every type can have; 0xDE is not. That is the whole of why +this rule exists and ~zeroed~ needs none: + +- *dyn* — a struct holding a dyn is rooted on the collector's root stack with + a descriptor naming that word's byte offset. A filled one is a root + pointing at nothing and the next collection follows it. This is the refusal + the feature could not ship without. +- *Vec, Map, Allocator* — an owning header: pointer, length, capacity, + allocator. A filled one frees a wild address the first time it is touched. +- *string, slice* — a pointer and a length that every bounds check believes. +- *Ptr* — not walked by the collector, and a poisoned pointer is arguably the + useful case. Kept out anyway so the rule is one sentence rather than "plain + data, except one kind of address". *This is the arm to relax first if the + question is reopened.* +- *bool* — the one refusal that is about the backends rather than the + runtime. A bool is a byte in memory and an ~i1~ to LLVM, which reads the + low bit, where x86 compares the whole byte against zero: 0xDE is false on + one and true on the other. Byte-identical behaviour across the two backends + is the property this feature is pinned on, so the divergence is refused + rather than documented. +- *a data type* — a tag that names a case, and no byte pattern names a real + one. *An ~(Option T)~* — the same, one bit of it: a filled tag says the + value is there over a payload nobody wrote. *An enum* — its values are the + members it declared, and no byte pattern is one of them. +- *a union* — and this one is not about a tag, because ~env.unions~ is "the + untagged unions". It is that a union's members overlay and ~unfillable~ + walks a struct's fields rather than a union's members, so nothing has + shown every member is plain data; a member that is not would be filled + through the one that is. Relaxable by walking the members, if anyone wants + it. +- *a function value* — a code address, and a call through a filled one jumps + into whatever the pattern happens to address. + +Floats are in: every bit pattern is a float, NaNs included, and both backends +move one as bytes. + +A ~defconst~ of a fill is refused by the existing constant rule and not by +anything of this feature's own — a fill is never a value the linker can write +into the image. A ~defvar~ is fine and goes through the startup function on +both backends, which ~programs/fill.flan~ pins. + +** The byte order, which is the specification +*A pattern's ascending bytes are its big-endian bytes* — exactly how the hex +literal reads left to right. So ~(dead-beef)~ lays down DE AD BE EF and ~xxd~ +reads "deadbeef"; ~(dead-beef 0xBAADF00D)~ lays down BA AD F0 0D. One rule, +both arities. + +On a little-endian machine the word a 4-byte store must therefore leave is +the *byte reversal* of the pattern, which is all ~Emit.word_of_pattern~ is +(~bytes_of_pattern~ beside it is the ascending list). Those two are the one +place the order is written, and ~Tast.dead_beef_default~ is the one place +0xDEADBEEF is written, so the default and the parameterised case cannot +drift. + +x86.ml reads ~word_of_pattern~ out of Emit rather than repeating it. It does +*not* use ~bytes_of_pattern~: its tail walks the bytes out of ~rax~ with +~shr~, which is the same arithmetic the list encodes and is how the computed +path has to do it anyway, so there was no second constant to share. emit.ml +uses both — the list for a folded tail, the word for the loop. + +A literal pattern is reversed at compile time and reaches the loop as an +immediate — the default's generated code is exactly what it was before the +pattern became an operand. A computed one is evaluated and reversed at run +time, by ~llvm.bswap.i32~ on one backend and ~bswap eax~ (0F C8, new) on the +other. + +*Tail behaviour.* A size that is not a multiple of four ends on a prefix of +the ascending bytes: 1 byte over is DE, 2 is DE AD, 3 is DE AD BE. +Equivalently, tail byte k is ~(word >> 8k) & 0xFF~ — which is what the +computed path actually does, by shifting, since there is no constant to fold. +~programs/fill.flan~ has all four lengths (8, 9, 6, 7) and, crucially, runs a +computed pattern over lengths 6 and 7: that is the case a constant-only +implementation would pass by accident. + +** The backends +- *LLVM (emit.ml).* The byte fill is one ~llvm.memset~ with the byte as an + operand instead of a zero — the same call the existing bulk zero makes, and + the reason the single-byte fill is the cheap one. The pattern fill cannot + be a memset at all (the intrinsic takes one repeated i8, which is the snag + DISCUSS.org named), so it is a counted loop over dwords in the + header/body/exit shape ~emit_while~ writes, with the counter as an + entry-block alloca that ~mem2reg~ promotes. Every store is ~align 1~, + because a ~[7 u8]~ array is a legal thing to fill. A computed pattern goes + through ~llvm.bswap.i32~ (newly declared) and the loop stores an SSA value + rather than a constant; the tail then shifts and truncates. +- *x86 (x86.ml).* ~rep stosb~ for the byte fill — ~zero_loc~'s three + registers with the program's byte in ~al~ instead of a zero — and ~rep + stosd~ (new, 0xf3 0xab) for the pattern, with the stored word in ~eax~. A + computed pattern is loaded and run through ~bswap~ (new, 0F C8); the tail + walks the bytes out of ~rax~ with ~shr~ by an immediate (new, C1 /5), which + is used rather than ~shift_cl~ precisely because ~rep stosd~ leaves ~rcx~ + at zero. Either operand is evaluated *before* ~rdi~ is loaded, because + evaluating one may call and a call clobbers ~rdi~; ~rep stosd~ does not + touch ~rax~, which is what lets the tail keep reading the word out of it. +- The one asymmetry: ~emit.ml~ needs a ~Tast.Set~ arm of its own to fill the + place rather than a temporary, because its value path returns an SSA value. + ~x86.ml~ needs none — a ~Set~ there already lowers its value into the + place's location, so filling a place and filling a temporary are the same + line. +- *js.ml* refuses both by name. A struct is an object there, not a run of + bytes, so there is nothing for 0xFF to mean. + +** What was run +~dune test --root .~ green (exit 0, no FAIL lines). Three acceptance rows +over ~test/programs/fill.flan~ — default, ~-O0~ and ~--x86~ — and the three +outputs diffed against each other by hand before the rows were written: +byte-identical, with a fourth build (~--dev~) added at the rename: four-way +identical. The three rows were confirmed to actually run, by breaking one +expectation on purpose and watching all three report. Seventeen checker rows +in ~test_flan.ml~: four accepting (both ~dead-beef~ arities and a computed +pattern among them), and seventeen refusals covering the boundary — one per +reason, since review found the tagged types were sharing a line that was +false for two of them — both arities, both no-expected-type positions, the +byte's range, the pattern's range and the ~defconst~ rule. + +~dune test~ exits 1 on this branch about half the time, with *no FAIL line +anywhere* — the ~Flan.Wire.Closed~ flake an earlier lane wrote up further up +this file. Green runs are real (2 of the last 4 exit 0); the rest are that +race. + +*This lane makes it fire more often, and that is worth saying plainly rather +than filing the whole thing under "known flake".* Measured, because early +runs looked like the lane had broken something: + +| what | full ~dune test~ | +| base commit 1526b6f, none of the lane | 0 failures in 5 | +| this lane | 5 failures in 5 | +| this lane, my 3 acceptance rows off | 1 failure in 3 | + +Isolated, ~test_dev.exe~ alone (15 seconds, not the ten-minute suite) gives 4 +in 6 here against 2 in 6 at the base — much closer, which is the shape you +would expect if the lane is not touching the racy code but *is* changing the +load around it. The three acceptance rows add three compile-and-run jobs to +the pool that ~test_dev~ runs alongside, and a busier machine is slower to +answer the poll that races. + +So: not a new defect, and nothing in ~check.ml~/~emit.ml~/~x86.ml~ here is +implicated — but the next lane to add acceptance rows will push the rate up +again, and the fix the earlier writeup already named (catch ~Closed~ in +~trap_park~'s poll and read it as the program having ended) is now worth +doing rather than noting. + +One detail to add to that earlier writeup, which had only seen the flake on +~dev-trap-null-alloc~: it is not row-specific. Five of my six isolated +failures were that row and the sixth was ~dev-trap-free-all~, so what is racy +is ~trap_park~ itself and every row that calls it — which is exactly what the +mechanism described there predicts. Per the sweep policy the ~@x86~ and +~@sanitize~ sweeps were not run here. diff --git a/lib/check.ml b/lib/check.ml index 59de7e5..79d28f2 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -718,6 +718,64 @@ let rec no_zeroed_fn loc what (t : Types.t) = | Types.Array (_, e) -> no_zeroed_fn loc what e | _ -> () +(* ── What may be overwritten with raw bytes ──────────────────────────── + + [(filled b)] and [(sentinel-filled)] are the only two things in the + language that write a byte pattern over storage the type system has an + opinion about, so the question they raise is which types survive having + arbitrary bytes put in them. The answer here is the narrow one: numbers, + and aggregates built out of numbers. Everything else is refused by name. + + What is being kept out, and why each one is not a matter of taste: + + - [dyn]. A struct that holds a dyn is rooted on the collector's root stack + with a descriptor naming the byte offsets of its dyn words (see the + per-type descriptor note at the bottom of this file). Filling one leaves + a word that is not a dyn at an offset the collector is told to walk, and + the next collection follows it. The refusal is what keeps that from + being reachable at all. + - [Vec], [(Map K V)], [Allocator]. An owning header: a pointer, a length, a + capacity and an allocator the runtime frees through. A filled one is a + free of a wild pointer the first time it is touched. + - [string] and a slice. Two words, the second of which is a length every + bounds check believes. A filled length is a bounds check that passes and + an access that does not. + - [Ptr]. Not walked by the collector, and a poisoned pointer is arguably + the useful case — but it is still a value every [deref] in the language + trusts, and admitting it would make the rule "plain data, except one + kind of address". Kept out so the rule is one sentence. This is the arm + to relax first if the question is reopened. + - [bool]. The one refusal that is about the backends rather than the + runtime: a bool is a byte here and an [i1] to LLVM, which reads the low + bit, where x86 compares the whole byte against zero. 0xDE is false on + one and true on the other, and byte-identical behaviour across the two + backends is the property this feature is pinned on. + - an enum, a data type, a union, an [(Option T)], a function value. Each + carries a tag or a case index that something later reads as a small + number with a meaning, and a filled one names a case that does not + exist. + + Floats are in: every bit pattern is a float, NaNs included, and both + backends move one as bytes. *) +let rec unfillable env seen (t : Types.t) : Types.t option = + match t with + | Types.Int _ | Types.Float _ -> None + | Types.Array (_, e) -> unfillable env seen e + | Types.Named n when not (List.mem n seen) -> + (match Hashtbl.find_opt env.structs n with + | Some s -> + List.fold_left + (fun acc (fl : Tast.field) -> + match acc with + | Some _ -> acc + | None -> unfillable env (n :: seen) fl.Tast.fty) + None s.Tast.fields + (* A data type or a union, which are the two [Named] things that are not + in [structs]. Both overlay their members, so the type itself is what + the refusal names. *) + | None -> Some t) + | _ -> Some t + let rec resolve env ?(seen = []) (t : Ast.texpr) : Types.t = let loc = t.Ast.tloc in match t.Ast.t with @@ -5470,6 +5528,115 @@ and named_call ctx ~want loc name args = "zeroed needs to know the type it is zeroing — use it where one is \ expected, as in (set grid (zeroed))") + (* [zeroed]'s two siblings, and the same shape exactly: a value of whatever + type is expected of it, so [(set grid (filled 0xFF))] is how a place is + filled and there is no second spelling to learn. What they add over + [zeroed] is the bytes — [(filled b)] repeats one the program picks, and + [(dead-beef)] repeats four — and with them the question [zeroed] never + has to ask: zero is a value every type can have, and 0xDE is not. + [unfillable] above is the whole of the answer. + + [dead-beef] takes the pattern or leaves it out, and leaving it out is + defined as writing the default: the [None] arm below builds the same + literal the source would have, so [(dead-beef)] and + [(dead-beef 0xDEADBEEF)] are the same node by construction and no + backend has a second path for the bare form. + + The pattern is an ordinary u32 expression, which is the byte arm's rule + at four times the width — a literal out of range meets [in_range]'s + located "does not fit in u32", and anything computed is guaranteed by + its type instead. Both builtins take a value rather than only a literal + for the same reason: refusing one would be a restriction with no + mechanism behind it, since neither backend needs the number early. *) + | "filled" | "dead-beef" -> + let is_byte = String.equal name "filled" in + if is_byte then arity loc name 1 args + else if List.length args > 1 then + fail loc "%s takes the pattern or nothing at all, given %d arguments" + name (List.length args); + (match want with + | Some ty when ty <> Types.Never -> + (match unfillable ctx.env [] ty with + | Some bad -> + Loc.failk "check/fill-not-plain-data" loc + "%s writes raw bytes over %s, and %s is not plain data — %s. \ + Fill only numbers, and structs and fixed arrays built out of \ + them" + name (Types.to_string ty) + (if Types.equal bad ty then "it" else Types.to_string bad) + (match bad with + | Types.Dyn -> + "a dyn is one word the collector walks by descriptor, and a \ + filled one is a root pointing at nothing" + | Types.Vec _ | Types.Map _ | Types.Alloc -> + "it owns its storage through a pointer and an allocator, and \ + a filled header frees a wild address" + | Types.String | Types.Slice _ -> + "it is a pointer and a length every bounds check believes" + | Types.Ptr _ -> + "it is an address every deref trusts" + | Types.Bool -> + "a bool is an i1 to LLVM and a whole byte to the x86 backend, \ + so a filled one would not even agree with itself across the \ + two" + | Types.Option _ -> + "it carries a tag saying whether the value is there, and a \ + filled one says yes over a payload nobody wrote" + | Types.Fn _ -> + "it is a code address, and a call through a filled one jumps \ + into whatever 0xDE bytes happen to address" + | Types.Named n when Hashtbl.mem ctx.env.datas n -> + "it carries a tag that names a case, and no byte pattern \ + names a real one" + | Types.Named n when Hashtbl.mem ctx.env.unions n -> + (* Untagged, per [env.unions]'s own note — so the reason is + not a tag. It is that a union's members overlay, and this + rule walks a struct's fields rather than a union's members: + nothing here has shown they are all plain data, and a + member that is not would be filled through the one that + is. *) + "a union's members overlay, and this rule does not walk them \ + — so nothing here has shown that every member is plain data" + | Types.Enum _ -> + "an enum's values are the members it declared, and no byte \ + pattern is one of them" + | _ -> + "it is not one of the types this rule admits") + | None -> ()); + if is_byte then + let b = check ctx ~want:(Types.Int Types.U8) (List.hd args) in + mk loc ty (Tast.Fill (ty, b)) + else + let pat = + match args with + | [ a ] -> check ctx ~want:(Types.Int Types.U32) a + (* The bare form, written out. Not a default a backend applies: + the node that leaves here is the one the spelled-out call would + have left, which is what makes the equivalence a fact about the + IR rather than a promise two emitters keep separately. + + Masked to 32 bits, and that is the whole of why this is not + [Int64.of_int32] on its own: [dead_beef_default] is an [int32] + whose top bit is set, so widening it signed would put + -559038737 on a node tagged [u32] — where the same pattern + *written out* arrives as 3735928559, because [in_range] admits + it as the unsigned value it is. Two spellings of one builtin + would then carry two different payloads, and "the same node by + construction" would be false for anything that reads one. *) + | _ -> + mk loc (Types.Int Types.U32) + (Tast.Int + (Int64.logand + (Int64.of_int32 Tast.dead_beef_default) 0xFFFFFFFFL, + Types.U32)) + in + mk loc ty (Tast.DeadBeef (ty, pat)) + | _ -> + fail loc + "%s needs to know the type it is filling — use it where one is \ + expected, as in (set grid (%s))" + name (if is_byte then "filled 0xFF" else name)) + (* The one half of a destructuring [let] that [Parse] cannot do on its own. Everything else about a pattern is bindings and field accesses it already wrote; the arity is a *type* question — how many elements the value has — @@ -7400,6 +7567,15 @@ let builtins : (string * string * string) list = ("zeroed", "zeroed [] T", "The all-bytes-zero value of whatever it is being stored into, so it \ only means anything where a type is expected of it."); + ("filled", "filled [u8] T", + "Every byte of whatever it is being stored into set to one byte, as in \ + (set grid (filled 0xFF)). Numbers only, and structs and fixed arrays \ + built out of them."); + ("dead-beef", "dead-beef [u32?] T", + "A four-byte pattern repeating over whatever it is being stored into, \ + written so a hex dump reads it left to right: 0xDEADBEEF with no \ + argument, the u32 given otherwise. A size that is not a multiple of \ + four ends on a prefix of the pattern. Same types [filled] takes."); ("destructure~nth", "destructure~nth [[n T] i32 i32 i32] T", "Written by the compiler for a destructuring let, and unspellable: the \ reader makes ~ a delimiter, so no source symbol can name this."); diff --git a/lib/emit.ml b/lib/emit.ml index 4cdc1fd..55a0421 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -937,6 +937,47 @@ let emit_bulk_zero f ptr ty = end else false | _ -> false +(* ── The dead-beef pattern ───────────────────────────────────────────── + + [(dead-beef)] writes DE AD BE EF in ascending address order, so [xxd] over + the filled storage reads "deadbeef" and not "efbeadde". That is the whole + reason the builtin exists, so the byte order is the specification and not + an implementation detail. + + [(dead-beef V)] writes V's four bytes under the same rule, and the rule is + what makes the two arities one thing: *a pattern's ascending bytes are its + big-endian bytes*, which is exactly how the hex literal reads left to + right. So [(dead-beef 0xBAADF00D)] lays down BA AD F0 0D. Nothing here + knows about the bare form: the checker writes [Tast.dead_beef_default] in + where the argument would have been, so this file only ever sees a pattern. + + On a little-endian machine the word a 4-byte store must leave is therefore + the byte reversal of the pattern, which is all [word_of_pattern] is. Both + backends go through here — x86 through [rep stosd] with this word in [eax], + this file through a store of it — and the acceptance program reads the + bytes back by index to keep them honest. + + A run whose size is not a multiple of four ends on a prefix of the + ascending bytes: one over is DE, two is DE AD, three is DE AD BE. *) + +(* The pattern's bytes in the order they land in memory, which is + most-significant first — the order the literal is written in. *) +let bytes_of_pattern (v : int32) = + List.map + (fun k -> + Int32.to_int + (Int32.logand (Int32.shift_right_logical v (8 * k)) 0xFFl)) + [ 3; 2; 1; 0 ] + +(* The i32 a little-endian store leaves those bytes as: the same list folded + back the other way, so one definition answers for both and a change to the + byte order cannot reach only half of it. *) +let word_of_pattern (v : int32) = + List.fold_left + (fun acc b -> Int32.logor (Int32.shift_right_logical acc 8) + (Int32.shift_left (Int32.of_int b) 24)) + 0l (bytes_of_pattern v) + let term f fmt = Printf.ksprintf (fun s -> @@ -948,6 +989,7 @@ let label f name = Buffer.add_string f.b (Printf.sprintf "\n%s:\n" name); f.live <- true + (* Every [ret] in a function body goes through here, which is the whole of how the shadow stack's pop is got right. There are five of them — an explicit [return] with a value and without, the [none] arm of [(some x)], the tail of @@ -1156,6 +1198,83 @@ let alloca_raw f lltype = Buffer.add_string f.allocas (Printf.sprintf " %s = alloca %s\n" name lltype); name +(* [(filled b)] over storage this file already has the address of. One + [llvm.memset] with the byte in a register, which is exactly what the + intrinsic is for and the reason the single-byte fill is the cheap one: the + same call the zero fill above makes, with an operand instead of a zero. *) +let emit_byte_fill f ptr ty (byte : string) = + let size, align = lay f.md ty in + if size > 0 then + ins f + "call void @llvm.memset.p0.i64(ptr align %d %s, i8 %s, i64 %d, i1 false)" + align ptr byte size + +(* [(dead-beef PATTERN)] over the same. [llvm.memset] takes one repeated i8 + and nothing else, so a four-byte pattern cannot be a call and has to be a + loop — the snag DISCUSS.org named before this was built, and the reason the + two builtins are two and not one with a wider operand. + + The word is either a folded constant or an SSA value, and [word] below is + just its spelling: a literal pattern is byte-reversed here at compile time + and a computed one by [llvm.bswap.i32] at run time, after which the loop + and the tail are the same instructions either way. That is deliberate — + the case a constant-only implementation would quietly get wrong is a + computed pattern over a length that is not a multiple of four, and there + is only one tail here for it to get wrong. + + The counter is an entry-block alloca, which is how every local in this file + is spelled and what [mem2reg] promotes; the loop is the same + header/body/exit shape [emit_while] writes. The size is a compile-time + constant, so the trip count is one too and LLVM unrolls what it wants to. + + Every store is [align 1]: the pattern is laid down over bytes, and the + storage's own alignment may be 1 — a [7 u8] array is a legal thing to fill. + + The tail is the first up-to-three bytes of the *stored word*, lowest + address first, which is byte k = (word >> 8k) & 0xFF. For a constant that + folds to [bytes_of_pattern]'s list; for a computed word it is a shift and + a truncate, ~tail~ times, unrolled because the count is known here. *) +let emit_dead_beef f ptr ty ~(word : string) ~(folded : int32 option) = + let size, _ = lay f.md ty in + let words = size / 4 and tail = size mod 4 in + if words > 0 then begin + let i = alloca_raw f "i64" in + let lc = fresh_label f "fill" and lb = fresh_label f "fillbody" + and le = fresh_label f "fillend" in + ins f "store i64 0, ptr %s" i; + term f "br label %%%s" lc; + label f lc; + let c = fresh f in + ins f "%s = load i64, ptr %s" c i; + let t = fresh f in + ins f "%s = icmp ult i64 %s, %d" t c words; + term f "br i1 %s, label %%%s, label %%%s" t lb le; + label f lb; + let off = fresh f in + ins f "%s = mul i64 %s, 4" off c; + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %s" p ptr off; + ins f "store i32 %s, ptr %s, align 1" word p; + let n = fresh f in + ins f "%s = add i64 %s, 1" n c; + ins f "store i64 %s, ptr %s" n i; + term f "br label %%%s" lc; + label f le + end; + for k = 0 to tail - 1 do + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %d" p ptr (words * 4 + k); + match folded with + | Some v -> + ins f "store i8 %d, ptr %s, align 1" (List.nth (bytes_of_pattern v) k) p + | None -> + let sh = fresh f in + ins f "%s = lshr i32 %s, %d" sh word (8 * k); + let b = fresh f in + ins f "%s = trunc i32 %s to i8" b sh; + ins f "store i8 %s, ptr %s, align 1" b p + done + (* ── Constants ─────────────────────────────────────────────────────── *) (* LLVM's hex form is exact, which decimal is not: a literal must mean the same @@ -1611,6 +1730,23 @@ and value_at f (e : Tast.expr) : string = | Tast.Str s -> string_const f.md s | Tast.Unit | Tast.Zero _ | Tast.None_ -> "zeroinitializer" | Tast.Uninit _ -> "poison" + (* A fill is a write over storage, so in value position it needs storage to + write over: a temporary, filled and then loaded out of. Every fill the + source actually writes is the value of a [set] or of a [let] binding, and + [Set] below takes the place's own address and skips this; the temporary + is what makes the node mean something everywhere else — a fill passed + straight to a call, say — rather than being a form with a position rule + nobody stated. *) + | Tast.Fill (ty, b) -> + let bv = value f b in + let tmp = alloca f ty in + emit_byte_fill f tmp ty bv; + load f tmp ty + | Tast.DeadBeef (ty, pat) -> + let word, folded = dead_beef_word f pat in + let tmp = alloca f ty in + emit_dead_beef f tmp ty ~word ~folded; + load f tmp ty | Tast.Local _ | Tast.Global _ | Tast.Field _ | Tast.Deref _ -> (* Everything that denotes a location is a load from its address. *) load f (addr f e) e.Tast.ty @@ -1665,6 +1801,14 @@ and value_at f (e : Tast.expr) : string = let ptr, ty = place f p in (match v.Tast.e with | Tast.Zero _ when emit_bulk_zero f ptr ty -> () + (* The shape the source writes: the fill goes straight at the place, + with no temporary and no aggregate load in between. Same short-circuit + the bulk zero above takes, and the reason [(set grid (filled 0xFF))] + is one memset. *) + | Tast.Fill (_, b) -> emit_byte_fill f ptr ty (value f b) + | Tast.DeadBeef (_, pat) -> + let word, folded = dead_beef_word f pat in + emit_dead_beef f ptr ty ~word ~folded | _ -> let v' = value f v in ins f "store %s %s, ptr %s" (ll ty) v' ptr); @@ -1791,6 +1935,24 @@ and load f ptr ty = ins f "%s = load %s, ptr %s" t (ll ty) ptr; t +(* The word a [(dead-beef PATTERN)] store must leave, and whether it is known + here. A literal pattern is reversed at compile time and reaches the loop as + an immediate, which is what keeps the default's code exactly what it was + before the pattern became an operand. Anything else is evaluated and + reversed by [llvm.bswap.i32] — the pattern is a u32 the program computed, + and the byte order it is written in is not a property of how it was + spelled. The [int32] comes back so the tail can fold too. *) +and dead_beef_word f (pat : Tast.expr) : string * int32 option = + match pat.Tast.e with + | Tast.Int (n, _) -> + let v = word_of_pattern (Int64.to_int32 n) in + (Printf.sprintf "%ld" v, Some (Int64.to_int32 n)) + | _ -> + let v = value f pat in + let t = fresh f in + ins f "%s = call i32 @llvm.bswap.i32(i32 %s)" t v; + (t, None) + (* The address of an expression that denotes a location. Anything else is spilled to a temporary first, so [(at (f) 0)] on a returned array works. *) and addr f (e : Tast.expr) : string = @@ -3559,6 +3721,7 @@ let header = {|; Generated by flan. The layout is C's: no object headers anywher @flan_frame_head = external global ptr declare void @llvm.memset.p0.i64(ptr nocapture writeonly, i8, i64, i1 immarg) +declare i32 @llvm.bswap.i32(i32) declare void @flan_rt_init(i32, ptr) declare void @flan_argv(ptr) declare void @flan_write_stdout(ptr, i64) diff --git a/lib/js.ml b/lib/js.ml index 80184cc..62e664f 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -670,6 +670,22 @@ let rec value f (e : Tast.expr) : string = n | Tast.Unit -> "undefined" | Tast.Zero t -> zero f.md e.Tast.loc t + (* A fill is a byte pattern written over storage, and this backend has no + storage to write over: a Flan struct is a JS object here, not a run of + bytes, so there is nothing for 0xFF or DEADBEEF to mean. Refused by name + rather than approximated, which is this file's rule. + + No "js: " on the front — [bin/main.ml] puts it there when it prints an + [Unsupported], and every other refusal in this file leaves it to do + that. *) + | Tast.Fill _ | Tast.DeadBeef _ -> + unsupported + "%s writes a byte pattern over storage, and the JS dialect has no \ + storage to write it over — a struct is an object here, not a run of \ + bytes" + (match e.Tast.e with + | Tast.Fill _ -> "(filled b)" + | _ -> "(dead-beef)") (* [uninit] is the opt-out from zeroing. There is no uninitialised memory here to opt out of, so it is the zero — which is more than the program asked for and never less. *) diff --git a/lib/tast.ml b/lib/tast.ml index 6824a97..6682af8 100644 --- a/lib/tast.ml +++ b/lib/tast.ml @@ -76,6 +76,26 @@ and expr_kind = | Unit | Zero of Types.t (* ZII: all-bytes-zero of this type *) | Uninit of Types.t (* the explicit opt-out *) + (* [(filled b)]: every byte of this type set to [b]. [Zero] with a byte the + program chooses, and a separate node rather than a byte on [Zero] because + that byte is an *expression* — a memset's operand, not a literal — and + every reader of [Zero] treats it as a leaf with nothing under it. + + [(dead-beef PATTERN)]: the pattern's four bytes repeating, ascending + through the storage, so a hex dump reads the pattern left to right — + DEADBEEF by default, and whatever u32 was written otherwise. The operand + is always present by the time it reaches here: [(dead-beef)] is checked + into [(dead-beef 0xDEADBEEF)], so a backend has one shape to lower and + the default cannot acquire a code path of its own. A size that is not a + multiple of four ends on a prefix of the pattern — DE, DE AD, DE AD BE + for the default — which both backends produce identically. + + The operand is an expression and not an [int32] for the same reason + [Fill]'s byte is: it may be computed. A literal one is folded by each + backend into the immediate it always was; anything else is evaluated and + byte-reversed at run time. *) + | Fill of Types.t * expr + | DeadBeef of Types.t * expr | Local of int (* slot index into the frame *) | Global of string | Prim of prim * expr list @@ -361,6 +381,11 @@ let rec walk (f : expr -> unit) (e : expr) = match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | Local _ | Global _ | None_ | FnAddr _ | Break _ | Continue _ -> () + (* Both carry an operand, and it must be walked: a call written inside a + fill's byte or a dead-beef's pattern is a call, and [Reach] roots what it + finds here. A leaf row would drop it silently — the match would still + compile. *) + | Fill (_, b) | DeadBeef (_, b) -> go b | Prim (_, es) | Call (_, es) | Do es | Make (_, es) | MakeCase (_, _, es) | Arr es | InvokeRestart (_, _, es, _, _, _) -> gos es | CallPtr (c, es) -> go c; gos es @@ -398,6 +423,13 @@ and walk_place f (p : place) = rather than an error. A data type case written into the image would need a byte-level encoder that could not encode a string field at all; written as a store at startup it needs nothing. *) +(* The pattern [(dead-beef)] means, and the name the builtin is spelled after. + It lives here rather than in a backend because it is a fact about the + language — what the bare form of a builtin means — and because the checker + is what writes it in: a [DeadBeef] node always carries its pattern, so no + emitter has a default of its own to keep in step with this one. *) +let dead_beef_default = 0xDEADBEEFl + let rec const_init (e : expr) = match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | None_ -> true diff --git a/lib/x86.ml b/lib/x86.ml index 096382f..7da8fd9 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -401,6 +401,26 @@ let push_r b r = if r >= 8 then u8 b 0x41; u8 b (0x50 lor (r land 7)) requires. *) let rep_movsb b = u8 b 0xf3; u8 b 0xa4 let rep_stosb b = u8 b 0xf3; u8 b 0xaa +(* [rep stosd]: [ecx] copies of [eax] at [rdi], four bytes at a time. The + sentinel fill's loop, in one instruction — DISCUSS.org's "a fill loop, not + a memset" is about LLVM's intrinsic taking one repeated byte, and the + string instruction here has no such limit. *) +let rep_stosd b = u8 b 0xf3; u8 b 0xab + +(* [bswap] on a 32-bit register: the byte reversal a computed [(dead-beef V)] + pattern needs, because V's ascending bytes are its big-endian ones and a + store leaves them little-endian. [Emit.word_of_pattern] is the same + operation folded at compile time for a literal. No REX.W — this is the + 32-bit form, and the 64-bit one would reverse eight bytes. *) +let bswap32 b ~dst = + rex b ~w:false ~r:0 ~x:0 ~m:dst; u8 b 0x0f; u8 b (0xc8 lor (dst land 7)) + +(* Logical shift right by an immediate, which the tail of a computed pattern + walks its bytes with. [shift_cl] above is the same opcode group taking the + count in [cl]; this one takes it in the instruction, so it does not need + [rcx] — and [rcx] is exactly what [rep stosd] has just left at zero. *) +let shr_imm b ~dst ~n = + rex b ~w:true ~r:0 ~x:0 ~m:dst; u8 b 0xc1; modrm_r b ~r:5 ~m:dst; u8 b n (* ── SSE ─────────────────────────────────────────────────────────────── *) @@ -1600,6 +1620,11 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = store_int f.b ~src:rax ~mm:(lmem f (shift dst 8) ~scratch:r11) ~size:8 | Tast.Unit -> () | Tast.Zero ty -> zero_value f dst ty + (* No [Set] arm of its own, unlike [emit.ml]: a [Tast.Set] here lowers its + value straight into the place's location, so filling a place and filling + a temporary are already the same line. *) + | Tast.Fill (ty, b) -> fill_value f dst ty b + | Tast.DeadBeef (ty, pat) -> dead_beef_value f dst ty pat | Tast.None_ -> zero_value f dst t (* Reading an uninitialised value gives whatever the slot held: stable garbage rather than LLVM's [poison]. The one construct where the two @@ -2109,6 +2134,74 @@ and emit_invoke_restart f id name (args : Tast.expr list) sg sg_id rloc = xfer_store f ~reg:rax ~scratch:r11; jmp_lbl f.b (current_pad f) +(* [(filled b)] and [(sentinel-filled)], into a location this backend already + has. [zero_loc] above is the same three registers and the same string + instruction with a zero in [al]; these two are what it becomes when the + byte is the program's and when the unit is four bytes rather than one. + + The byte is evaluated *before* [rdi] is loaded, because evaluating it is an + arbitrary expression that may call, and a call clobbers [rdi]. [r11] stays + the scratch every other caller of [lmem] uses, and is never one of the + three this touches. + + The sentinel's word and its tail bytes both come from [Emit] — one list, + read by both backends, so "a hex dump reads DEADBEEF" is stated once. The + tail is up to three straight byte stores, matching [Emit.emit_sentinel]'s + tail exactly: a size that is not a multiple of four ends on DE, DE AD, or + DE AD BE. *) +and fill_value f (dst : loc) (ty : Types.t) (b : Tast.expr) = + let n = sizeof f.md ty in + if n > 0 then begin + let v = eval f b in + load_int f.b ~dst:rax ~mm:(lmem f v ~scratch:r11) ~size:1 ~signed:false; + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rcx (Int64.of_int n); + rep_stosb f.b + end + +and dead_beef_value f (dst : loc) (ty : Types.t) (pat : Tast.expr) = + let n = sizeof f.md ty in + let words = n / 4 and tail = n mod 4 in + (* Whether the word is known here. A literal is reversed at compile time and + loaded as an immediate, which leaves the default's code exactly what it + was before the pattern became an operand; anything else is evaluated and + reversed by [bswap]. The evaluation happens first in both cases, because + it is an arbitrary expression that may call and a call clobbers [rdi]. *) + let folded = + match pat.Tast.e with + | Tast.Int (v, _) -> Some (Int64.to_int32 v) + | _ -> None + in + let load_word () = + match folded with + | Some v -> + movabs f.b ~dst:rax + (Int64.logand (Int64.of_int32 (Emit.word_of_pattern v)) 0xFFFFFFFFL) + | None -> + let l = eval f pat in + load_int f.b ~dst:rax ~mm:(lmem f l ~scratch:r11) ~size:4 ~signed:false; + bswap32 f.b ~dst:rax + in + if n > 0 then begin + load_word (); + if words > 0 then begin + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rcx (Int64.of_int words); + (* [rep stosd] touches rdi and rcx and leaves rax alone, which is what + lets the tail below keep reading the word out of it. *) + rep_stosd f.b + end; + (* The tail is the first up-to-three bytes of the stored word, lowest + address first — byte k is (word >> 8k) & 0xFF either way. A folded word + could store constants instead, but walking [rax] covers both with one + sequence, and the count is at most three. *) + for k = 0 to tail - 1 do + if k > 0 then shr_imm f.b ~dst:rax ~n:8; + store_int f.b ~src:rax + ~mm:(lmem f (shift dst (words * 4 + k)) ~scratch:r11) ~size:1 + done + end + and zero_value f (dst : loc) (ty : Types.t) = if is_agg ty then zero_loc f dst (sizeof f.md ty) else if not (is_void ty) then diff --git a/test/programs/fill.flan b/test/programs/fill.flan new file mode 100644 index 0000000..2597f5b --- /dev/null +++ b/test/programs/fill.flan @@ -0,0 +1,144 @@ +;;;; (filled b) and (dead-beef), the two byte fills, read back as bytes. +;;;; +;;;; The whole point of this program is that every row is a *byte* and not a +;;;; value: the pattern fill's contract is "a hex dump reads the pattern left +;;;; to right", which is a claim about which byte lands at which address, and +;;;; only reading the bytes back in address order can check it. The u32 rows +;;;; are the same claim from the other side — a little-endian load of +;;;; DE AD BE EF is 0xEFBEADDE, which is 4022250974, so a backend that wrote +;;;; the word the other way round would print 3735928559 here and be caught. +;;;; +;;;; The lengths are chosen for the tail. 8 is a whole number of patterns, 9 +;;;; ends on DE, 6 ends on DE AD, and 7 ends on DE AD BE — the three truncated +;;;; endings and the one that is not truncated at all. +;;;; +;;;; The rows that matter most are the last group: a pattern that is *not* a +;;;; literal, over a length that is not a multiple of four. That is the case a +;;;; constant-only implementation would pass by accident — the word reaches +;;;; the loop in a register and the tail bytes have to be shifted out of it +;;;; rather than folded, on both backends. +(defstruct Words [a u32 b u32]) + +;;;; A computed global initialiser: a fill is never a constant the linker can +;;;; write, so this one goes through the startup function on both backends. +(defvar gfill [4 u8] (filled 0x41)) +(defvar gsent [5 u8]) + +(defn bytes4 [b [4 u8]] () + (dotimes [i 4] (print (at b i)) (print " "))) + +;;;; A u32 that no literal rule can see through: (u32 0xBAADF00D) will not do, +;;;; because the cast checks its operand as an i32 first and 0xBAADF00D is not +;;;; one. A parameter's declared type is what makes the literal a u32, and it +;;;; is also what stops the checker folding it at the fill. +(defn u32of [x u32] u32 x) + +;;;; The pattern crosses a call boundary, so nothing can fold it at the fill. +(defn beef7 [pat u32] () + (let [a (array 7 u8)] + (set a (dead-beef pat)) + (dotimes [i 7] (print (at a i)) (print " ")) + (println ""))) + +(defn main [] i32 + ;; One byte, repeated. 0xFF is the -1 fill a debug allocator wants. + (let [a (array 4 u8)] + (set a (filled 0xFF)) + (bytes4 a) (println "")) ; 255 255 255 255 + + ;; The byte is an expression, not a literal: this is a memset with the + ;; operand in a register, and the register path is the one a constant would + ;; otherwise hide. + (let [v (u8 7) + a (array 4 u8)] + (set a (filled v)) + (bytes4 a) (println "")) ; 7 7 7 7 + + ;; (filled 0) is (zeroed), byte for byte, and saying so here is what keeps + ;; the two from drifting. + (let [a (array 4 u8)] + (set a (filled 0)) + (bytes4 a) (println "")) ; 0 0 0 0 + + ;; Four bytes, ascending, with nothing truncated. + (let [a (array 8 u8)] + (set a (dead-beef)) + (dotimes [i 8] (print (at a i)) (print " ")) + (println "")) ; 222 173 190 239 x2 + + ;; The bare form is the spelled-out default, and this is the row that says + ;; so: the same eight bytes from (dead-beef 0xDEADBEEF). + (let [a (array 8 u8)] + (set a (dead-beef 0xDEADBEEF)) + (dotimes [i 8] (print (at a i)) (print " ")) + (println "")) ; identical to the row above + + ;; The three truncated tails. + (let [a (array 9 u8)] + (set a (dead-beef)) + (dotimes [i 9] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 + + (let [a (array 6 u8)] + (set a (dead-beef)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 + + (let [a (array 7 u8)] + (set a (dead-beef)) + (dotimes [i 7] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 190 + + ;; A pattern of the program's own choosing, laid down left to right the same + ;; way: 0xBAADF00D is BA AD F0 0D, which is 186 173 240 13. + (let [a (array 6 u8)] + (set a (dead-beef 0xBAADF00D)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; 186 173 240 13 186 173 + + ;; The same pattern read as words rather than as bytes, which is what says + ;; which end the DE is at. + (let [w (Words {})] + (set w (dead-beef)) + (print (.a w)) (print " ") (print (.b w)) (println "")) + + (let [w (Words {})] + (set w (dead-beef 0xBAADF00D)) + (print (.a w)) (print " ") (print (.b w)) (println "")) + + ;; A nested aggregate: a fixed array of structs is plain data all the way + ;; down, so the fill reaches every byte of it. + (let [g (array 2 Words)] + (set g (filled 0xFF)) + (print (.a (at g 0))) (print " ") (print (.b (at g 1))) (println "")) + + ;; The globals. [gfill] was filled before main ran; [gsent] is filled here. + (dotimes [i 4] (print (at gfill i)) (print " ")) + (println "") ; 65 65 65 65 + (set gsent (dead-beef)) + (dotimes [i 5] (print (at gsent i)) (print " ")) + (println "") ; 222 173 190 239 222 + + ;; A fill in value position rather than as the value of a [set]: the same + ;; bytes, reached through a temporary instead of through the place. + (bytes4 (filled 0xFF)) (println "") ; 255 255 255 255 + + ;; ── The computed pattern ────────────────────────────────────────── + ;; None of these can be folded: the pattern is a parameter, or arithmetic + ;; the checker does not evaluate. Seven bytes is one whole pattern and a + ;; three-byte tail, so the tail bytes have to come out of the register the + ;; word is in rather than out of a constant. + (beef7 0xDEADBEEF) ; 222 173 190 239 222 173 190 + (beef7 0xBAADF00D) ; 186 173 240 13 186 173 240 + (let [p (u32of 0xBAADF00D) + a (array 6 u8)] + (set a (dead-beef p)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; 186 173 240 13 186 173 + ;; A pattern that is genuinely computed, not merely held in a slot, and read + ;; back as a word so the byte order of the runtime path is pinned too. + (let [p (u32of 0xBAAD0000) + w (Words {})] + (set w (dead-beef (bit-or p (u32of 0xF00D)))) + (print (.a w)) (print " ") (print (.b w)) (println "")) + 0) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 8993c41..e982445 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -1256,6 +1256,57 @@ let () = int_float_out; outputs ~x86:true "int and float, --x86" "programs/int-float.flan" int_float_out; + + (* The two byte fills, DISCUSS.org's "a DEADBEEF-style sentinel-fill + builtin". Three rows over one program, and what they are pinning is + byte-identity: a fill is the one operation in the language whose whole + contract is which byte lands at which address, so the LLVM backend's + [llvm.memset] and dword loop and the x86 backend's [rep stosb] and + [rep stosd] have to agree byte for byte, and -O0 has to agree with -O2 + because the loop is a real loop at one and unrolled at the other. + + 222 173 190 239 is DE AD BE EF in decimal, ascending through the + storage, which is the claim "a hex dump reads DEADBEEF". The 4022250974 + rows are the same four bytes read back as a u32 — 0xEFBEADDE — and they + are what a backend that wrote the word the other way round would fail. + The three short rows are the truncated tails a size that is not a + multiple of four ends on. + + Rows four and five are identical on purpose: [(dead-beef)] and + [(dead-beef 0xDEADBEEF)] are the same node by construction, and this is + where that stops being a claim about the checker and becomes one about + the program. + + The last four rows are the ones a constant-only implementation would + pass by accident. Their pattern arrives in a register — a parameter, a + slot, and a [bit-or] the checker does not fold — so the word is + byte-reversed at run time and the tail bytes are shifted out of it + rather than folded. 186 173 240 13 is 0xBAADF00D read left to right, + and the closing 233876922 is that same pattern's stored word, which is + the runtime path's byte order pinned against the literal path's — the + identical number appears above it from a folded [(dead-beef + 0xBAADF00D)]. *) + let fill_out = + "255 255 255 255 \n7 7 7 7 \n0 0 0 0 \n\ + 222 173 190 239 222 173 190 239 \n\ + 222 173 190 239 222 173 190 239 \n\ + 222 173 190 239 222 173 190 239 222 \n\ + 222 173 190 239 222 173 \n\ + 222 173 190 239 222 173 190 \n\ + 186 173 240 13 186 173 \n\ + 4022250974 4022250974\n233876922 233876922\n\ + 4294967295 4294967295\n\ + 65 65 65 65 \n222 173 190 239 222 \n255 255 255 255 \n\ + 222 173 190 239 222 173 190 \n\ + 186 173 240 13 186 173 240 \n\ + 186 173 240 13 186 173 \n\ + 233876922 233876922\n" + in + outputs "byte and dead-beef fills" "programs/fill.flan" fill_out; + outputs ~opt:"-O0" "byte and dead-beef fills, -O0" "programs/fill.flan" + fill_out; + outputs ~x86:true "byte and dead-beef fills, --x86" "programs/fill.flan" + fill_out; (* dyn if: truthiness -- M2 queue item 7. A dyn scrutinee is tested for nil/false vs. everything else, Clojure's rule, on both backends; a typed scrutinee stays strictly bool, which is a checker test diff --git a/test/test_flan.ml b/test/test_flan.ml index c1f08f3..0790e2d 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -2195,6 +2195,114 @@ let () = "(defvar g (Vec u8) uninit) (defn f [] ())" ~needle:"steers every read of it"; + (* ── What may be filled with raw bytes ───────────────────────────── + [(filled b)] and [(dead-beef)] are [zeroed]'s siblings, and the + boundary is the whole of what is new about them: zero is a value every + type can have and 0xDE is not, so the checker says which types survive + arbitrary bytes. The accepting side is programs/fill.flan; what is here + is the catalogue of what it refuses and why each refusal is the runtime's + and not a matter of taste. + + The dyn row is the one that would corrupt the collector: a struct holding + a dyn is rooted with a descriptor naming that word's offset, so a filled + one is a root pointing at nothing. *) + accepts "a fixed array of numbers may be filled" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 0xFF))))"; + accepts "a struct of numbers may be dead-beefed" + "(defstruct S [a i32 b f64]) \ + (defn f [] () (let [s (S {})] (set s (dead-beef))))"; + (* Both arities, and a pattern that is not a literal — the operand is an + ordinary u32 expression, which is the byte arm's rule at four times the + width. *) + accepts "dead-beef takes a pattern" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 0xBAADF00D))))"; + accepts "dead-beef takes a computed pattern" + "(defn f [p u32] () (let [a (array 4 u8)] (set a (dead-beef p))))"; + rejects_check "a struct holding a dyn cannot be filled" + "(defstruct S [a i32 d dyn]) \ + (defn f [] () (let [s (S {})] (set s (dead-beef))))" + ~needle:"a root pointing at nothing"; + rejects_check "a Vec cannot be filled" + "(defn f [] () (let [v (vec-new i32)] (set v (filled 0xFF))))" + ~needle:"frees a wild address"; + rejects_check "a string cannot be filled" + "(defn f [] () (let [s \"hi\"] (set s (filled 0xFF))))" + ~needle:"a length every bounds check believes"; + rejects_check "a pointer field cannot be filled" + "(defstruct S [p (Ptr i32)]) \ + (defn f [] () (let [s (S {})] (set s (filled 0xFF))))" + ~needle:"an address every deref trusts"; + (* The one refusal that is about the two backends rather than the runtime: + LLVM reads a bool's low bit and x86 compares the whole byte, so 0xDE is + false on one and true on the other. Byte-identical behaviour across the + backends is what this feature is pinned on, so the divergence is refused + rather than documented. *) + rejects_check "a bool cannot be filled" + "(defn f [] () (let [b false] (set b (filled 0xFF))))" + ~needle:"would not even agree with itself"; + (* Each of the tagged and address-carrying types names its own reason. They + shared one "it carries a tag that names a case" line until review caught + that it was false for two of them — a union is untagged (env.unions is + "the untagged unions") and a function value is a code pointer, not a + tag. Pinned per type so the reasons cannot quietly re-merge. *) + rejects_check "a union cannot be filled, and not because of a tag" + "(defunion U [a i32 b f64]) \ + (defn f [] () (let [u (U {})] (set u (dead-beef))))" + ~needle:"a union's members overlay"; + rejects_check "a function value cannot be filled" + "(defn g [] ()) (defn f [] () (let [h g] (set h (dead-beef))))" + ~needle:"it is a code address"; + rejects_check "an enum cannot be filled" + "(defenum K [lo 0 hi 1]) \ + (defn f [k K] () (let [e k] (set e (dead-beef))))" + ~needle:"the members it declared"; + rejects_check "an Option cannot be filled" + "(defn f [] () (let [o (Some 1)] (set o (dead-beef))))" + ~needle:"whether the value is there"; + (* A data type's tag is a case index, and no byte pattern names a real + case. The type itself is what the message names, because a data type + overlays its cases. *) + rejects_check "a data type cannot be filled" + "(defdata U [(A [x i32]) (B [y i32])]) \ + (defn f [] () (let [u (U.A {.x 1})] (set u (filled 0xFF))))" + ~needle:"names a case"; + (* [zeroed]'s own refusal, worn by both siblings: a fill is the bytes of + whatever type is expected of it, and in a position that expects nothing + there is no type and nothing to fill. This is the shape's cost and it is + paid on purpose — the alternative was a second, place-taking spelling + for an operation [set] already expresses. *) + rejects_check "a fill in a position with no expected type" + "(defn f [] () (print (filled 0xFF)))" + ~needle:"needs to know the type it is filling"; + rejects_check "a dead-beef in a position with no expected type" + "(defn f [] () (print (dead-beef)))" + ~needle:"needs to know the type it is filling"; + (* The byte is a u8 and the ordinary literal rule applies to it — there is + no range check of this builtin's own, and there does not need to be. *) + rejects_check "a fill byte out of range" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 300))))" + ~needle:"does not fit in u8"; + rejects_check "filled takes exactly one byte" + "(defn f [] () (let [a (array 4 u8)] (set a (filled))))" + ~needle:"takes 1 argument"; + rejects_check "dead-beef takes at most one pattern" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 1 2))))" + ~needle:"takes the pattern or nothing at all, given 2"; + (* The pattern is four bytes, so a wider literal is a typo rather than + something to truncate. The refusal is [in_range]'s, located at the + literal — this builtin has no range check of its own and does not need + one, exactly as the byte arm does not. *) + rejects_check "a dead-beef pattern out of u32 range" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 0x1DEADBEEF))))" + ~needle:"does not fit in u32"; + (* A fill is never a value the linker can write into the image, so a + defconst of one is refused by the constant rule rather than by anything + of this feature's own. A defvar is fine: its initialiser runs at + startup, which programs/fill.flan pins. *) + rejects_check "a defconst cannot be filled" + "(defconst g [4 u8] (filled 0xFF)) (defn f [] ())" + ~needle:"defconst"; + (* ── The third element of a defvar ───────────────────────────────── The rule, 2026-09-20: a type there is the zeroed static global it has always been, and anything else is a dyn global initialised from the