From 99f519ba6f6095d1f7845c71f717c940ed988d9a Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 18:15:19 +0700 Subject: [PATCH 1/6] Two byte fills: (filled BYTE) and (sentinel-filled) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DISCUSS.org's sentinel-fill idea, built as two builtins because the author asked for both: a memset with a byte the program picks, and the fixed DE AD BE EF pattern a hex dump reads as DEADBEEF. Both are spelled the way (zeroed) is — the value of whatever type is expected of them — so (set grid (filled 0xFF)) fills a place and there is no second, place-taking form beside set. What may be filled is numbers, and structs and fixed arrays built out of them. Everything else is refused by name: a filled dyn is a collector root pointing at nothing, a filled Vec header frees a wild address, a filled slice length is a bounds check that passes, and a filled bool is an i1 to LLVM and a whole byte to x86, which is the one divergence this feature cannot have. The byte fill is llvm.memset / rep stosb. The four-byte pattern cannot be a memset on either side — the intrinsic takes one repeated i8 — so it is a counted dword loop in emit.ml and rep stosd in x86.ml, with the pattern bytes and their little-endian word living once, in Emit. A size that is not a multiple of four ends on DE, DE AD, or DE AD BE. --- DISCUSS.org | 18 ++++++- FIX.org | 104 ++++++++++++++++++++++++++++++++++++ lib/check.ml | 114 ++++++++++++++++++++++++++++++++++++++++ lib/emit.ml | 105 ++++++++++++++++++++++++++++++++++++ lib/js.ml | 8 +++ lib/tast.ml | 16 +++++- lib/x86.ml | 54 +++++++++++++++++++ test/programs/fill.flan | 88 +++++++++++++++++++++++++++++++ test/test_acceptance.ml | 28 ++++++++++ test/test_flan.ml | 75 ++++++++++++++++++++++++++ 10 files changed, 608 insertions(+), 2 deletions(-) create mode 100644 test/programs/fill.flan diff --git a/DISCUSS.org b/DISCUSS.org index 402c35a..34e7e2f 100644 --- a/DISCUSS.org +++ b/DISCUSS.org @@ -183,7 +183,23 @@ why MSVC/glibc-style debug allocators use a single repeated byte instead LocalAlloc uninit), 0xFEEEFEEE (Windows HeapFree'd), 0xDEADC0DE, 0xC0FFEE, 0x8BADF00D (Apple watchdog-timeout crash code) -Not designed or implemented, just an idea. +Built, 2026-09-20, as two builtins rather than one — the author's answer to +the cheap-byte-or-real-pattern question was "why not both? we need some sort +of memset -1 right? and dead-beef can loop, that's fine". They are +(filled BYTE) and (sentinel-filled), spelled the way [zeroed] is: the value +of whatever type is expected of them, so (set grid (filled 0xFF)) is how a +place is filled and there is no place-taking form to learn beside [set]. + +The snag above is why there are two and not one with a wider operand. The +byte fill is one llvm.memset / one rep stosb; the 4-byte pattern is a loop on +both sides, a counted dword loop in emit.ml and rep stosd in x86.ml, because +the intrinsic really does only take a repeated i8. + +What may be filled is numbers, and structs and fixed arrays built out of +numbers — nothing that carries a tag, a length, an owning pointer or a +collector descriptor. That boundary, and why each refusal is the runtime's +rather than a matter of taste, is written up in FIX.org, "The two byte +fills". ** println output goes to *flan-output*, not inline in the repl Deliberate per emacs/flan-repl.el:40-44: "a value and the program's output diff --git a/FIX.org b/FIX.org index b1fbe41..f2da3b4 100644 --- a/FIX.org +++ b/FIX.org @@ -1611,3 +1611,107 @@ because the code is the same. Not fixed here, deliberately: the fix is to catch ~Closed~ in that poll and read it as the program having ended, which is a claim about what those rows mean and belongs to whoever owns them. Flagged rather than patched. + +* The two byte fills, 2026-09-20 +DISCUSS.org's "a DEADBEEF-style sentinel-fill builtin", built. The author's +answer to the single-byte-or-four-byte question was "why not both? we need +some sort of memset -1 right? and dead-beef can loop, that's fine", so there +are two builtins and they are siblings of ~zeroed~, not a new shape. + +** The spellings +~(filled BYTE)~ and ~(sentinel-filled)~, both value forms driven by the type +expected of them, exactly as ~(zeroed)~ is: + +: (set grid (filled 0xFF)) +: (set frame (sentinel-filled)) + +A place-taking ~(filled place byte)~ was the other candidate and was not +taken. ~zeroed~ already answers "the all-bytes-X value of whatever this is +being stored into", ~set~ already takes the place, and a second spelling for +an operation ~set~ expresses would have been a second thing to learn for +nothing. The cost is real and is paid on purpose: a fill in a position that +expects no type is refused ("filled needs to know the type it is filling"), +which is ~zeroed~'s own refusal worn by both siblings. + +~sentinel-filled~ takes no operand. The pattern is fixed — that is the whole +point of it, since a hex dump only reads DEADBEEF if nobody can change it — +and a parameterised one would be a different builtin. + +** The fill boundary — what may be overwritten with raw bytes +*Numbers, and structs and fixed arrays built out of numbers. Nothing else.* +~Check.unfillable~ is the rule, in one recursive walk, and every refusal +names the type it stopped at and why. + +Zero is a value every type can have; 0xDE is not. That is the whole of why +this rule exists and ~zeroed~ needs none: + +- *dyn* — a struct holding a dyn is rooted on the collector's root stack with + a descriptor naming that word's byte offset. A filled one is a root + pointing at nothing and the next collection follows it. This is the refusal + the feature could not ship without. +- *Vec, Map, Allocator* — an owning header: pointer, length, capacity, + allocator. A filled one frees a wild address the first time it is touched. +- *string, slice* — a pointer and a length that every bounds check believes. +- *Ptr* — not walked by the collector, and a poisoned pointer is arguably the + useful case. Kept out anyway so the rule is one sentence rather than "plain + data, except one kind of address". *This is the arm to relax first if the + question is reopened.* +- *bool* — the one refusal that is about the backends rather than the + runtime. A bool is a byte in memory and an ~i1~ to LLVM, which reads the + low bit, where x86 compares the whole byte against zero: 0xDE is false on + one and true on the other. Byte-identical behaviour across the two backends + is the property this feature is pinned on, so the divergence is refused + rather than documented. +- *enum, data type, union, Option, function value* — each carries a tag or a + case index something later reads as a small number with a meaning, and no + byte pattern names a real case. + +Floats are in: every bit pattern is a float, NaNs included, and both backends +move one as bytes. + +A ~defconst~ of a fill is refused by the existing constant rule and not by +anything of this feature's own — a fill is never a value the linker can write +into the image. A ~defvar~ is fine and goes through the startup function on +both backends, which ~programs/fill.flan~ pins. + +** The byte order, which is the specification +DE AD BE EF in *ascending address order*, so ~xxd~ reads "deadbeef". A +little-endian store of those four bytes is the i32 0xEFBEADDE, and +~Emit.sentinel_bytes~ / ~Emit.sentinel_word~ are the one place either is +written — x86.ml reads both out of Emit rather than repeating them, so the +two backends cannot drift. + +*Tail behaviour.* A size that is not a multiple of four ends on a prefix of +the pattern: 1 byte over is DE, 2 is DE AD, 3 is DE AD BE. Both backends +write the tail byte by byte from the same list. ~programs/fill.flan~ has all +four lengths — 8, 9, 6 and 7. + +** The backends +- *LLVM (emit.ml).* The byte fill is one ~llvm.memset~ with the byte as an + operand instead of a zero — the same call the existing bulk zero makes, and + the reason the single-byte fill is the cheap one. The sentinel cannot be a + memset at all (the intrinsic takes one repeated i8, which is the snag + DISCUSS.org named), so it is a counted loop over dwords in the + header/body/exit shape ~emit_while~ writes, with the counter as an + entry-block alloca that ~mem2reg~ promotes. Every store is ~align 1~, + because a ~[7 u8]~ array is a legal thing to fill. +- *x86 (x86.ml).* ~rep stosb~ for the byte fill — ~zero_loc~'s three + registers with the program's byte in ~al~ instead of a zero — and ~rep + stosd~ (new, 0xf3 0xab) for the sentinel, with 0xEFBEADDE in ~eax~. The + byte operand is evaluated *before* ~rdi~ is loaded, because evaluating it + may call and a call clobbers ~rdi~. +- The one asymmetry: ~emit.ml~ needs a ~Tast.Set~ arm of its own to fill the + place rather than a temporary, because its value path returns an SSA value. + ~x86.ml~ needs none — a ~Set~ there already lowers its value into the + place's location, so filling a place and filling a temporary are the same + line. +- *js.ml* refuses both by name. A struct is an object there, not a run of + bytes, so there is nothing for 0xFF to mean. + +** What was run +~dune test --root .~ green (exit 0, no FAIL lines). Three acceptance rows +over ~test/programs/fill.flan~ — default, ~-O0~ and ~--x86~ — and the three +outputs diffed against each other by hand before the rows were written: +byte-identical. Eleven checker rows in ~test_flan.ml~ for the boundary and +the two arity/position refusals. Per the sweep policy the ~@x86~ and +~@sanitize~ sweeps were not run here. diff --git a/lib/check.ml b/lib/check.ml index 179bb35..fc2bc77 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -605,6 +605,64 @@ let rec no_zeroed_fn loc what (t : Types.t) = | Types.Array (_, e) -> no_zeroed_fn loc what e | _ -> () +(* ── What may be overwritten with raw bytes ──────────────────────────── + + [(filled b)] and [(sentinel-filled)] are the only two things in the + language that write a byte pattern over storage the type system has an + opinion about, so the question they raise is which types survive having + arbitrary bytes put in them. The answer here is the narrow one: numbers, + and aggregates built out of numbers. Everything else is refused by name. + + What is being kept out, and why each one is not a matter of taste: + + - [dyn]. A struct that holds a dyn is rooted on the collector's root stack + with a descriptor naming the byte offsets of its dyn words (see the + per-type descriptor note at the bottom of this file). Filling one leaves + a word that is not a dyn at an offset the collector is told to walk, and + the next collection follows it. The refusal is what keeps that from + being reachable at all. + - [Vec], [(Map K V)], [Allocator]. An owning header: a pointer, a length, a + capacity and an allocator the runtime frees through. A filled one is a + free of a wild pointer the first time it is touched. + - [string] and a slice. Two words, the second of which is a length every + bounds check believes. A filled length is a bounds check that passes and + an access that does not. + - [Ptr]. Not walked by the collector, and a poisoned pointer is arguably + the useful case — but it is still a value every [deref] in the language + trusts, and admitting it would make the rule "plain data, except one + kind of address". Kept out so the rule is one sentence. This is the arm + to relax first if the question is reopened. + - [bool]. The one refusal that is about the backends rather than the + runtime: a bool is a byte here and an [i1] to LLVM, which reads the low + bit, where x86 compares the whole byte against zero. 0xDE is false on + one and true on the other, and byte-identical behaviour across the two + backends is the property this feature is pinned on. + - an enum, a data type, a union, an [(Option T)], a function value. Each + carries a tag or a case index that something later reads as a small + number with a meaning, and a filled one names a case that does not + exist. + + Floats are in: every bit pattern is a float, NaNs included, and both + backends move one as bytes. *) +let rec unfillable env seen (t : Types.t) : Types.t option = + match t with + | Types.Int _ | Types.Float _ -> None + | Types.Array (_, e) -> unfillable env seen e + | Types.Named n when not (List.mem n seen) -> + (match Hashtbl.find_opt env.structs n with + | Some s -> + List.fold_left + (fun acc (fl : Tast.field) -> + match acc with + | Some _ -> acc + | None -> unfillable env (n :: seen) fl.Tast.fty) + None s.Tast.fields + (* A data type or a union, which are the two [Named] things that are not + in [structs]. Both overlay their members, so the type itself is what + the refusal names. *) + | None -> Some t) + | _ -> Some t + let rec resolve env ?(seen = []) (t : Ast.texpr) : Types.t = let loc = t.Ast.tloc in match t.Ast.t with @@ -4812,6 +4870,54 @@ and named_call ctx ~want loc name args = "zeroed needs to know the type it is zeroing — use it where one is \ expected, as in (set grid (zeroed))") + (* [zeroed]'s two siblings, and the same shape exactly: a value of whatever + type is expected of it, so [(set grid (filled 0xFF))] is how a place is + filled and there is no second spelling to learn. What they add over + [zeroed] is the byte — [(filled b)] repeats one the program picks, and + [(sentinel-filled)] repeats the four bytes DE AD BE EF — and with it the + question [zeroed] never has to ask: zero is a value every type can have, + and 0xDE is not. [unfillable] above is the whole of the answer. *) + | "filled" | "sentinel-filled" -> + arity loc name (if String.equal name "filled" then 1 else 0) args; + (match want with + | Some ty when ty <> Types.Never -> + (match unfillable ctx.env [] ty with + | Some bad -> + Loc.failk "check/fill-not-plain-data" loc + "%s writes raw bytes over %s, and %s is not plain data — %s. \ + Fill only numbers, and structs and fixed arrays built out of \ + them" + name (Types.to_string ty) + (if Types.equal bad ty then "it" else Types.to_string bad) + (match bad with + | Types.Dyn -> + "a dyn is one word the collector walks by descriptor, and a \ + filled one is a root pointing at nothing" + | Types.Vec _ | Types.Map _ | Types.Alloc -> + "it owns its storage through a pointer and an allocator, and \ + a filled header frees a wild address" + | Types.String | Types.Slice _ -> + "it is a pointer and a length every bounds check believes" + | Types.Ptr _ -> + "it is an address every deref trusts" + | Types.Bool -> + "a bool is an i1 to LLVM and a whole byte to the x86 backend, \ + so a filled one would not even agree with itself across the \ + two" + | _ -> "it carries a tag that names a case, and no byte pattern \ + names a real one") + | None -> ()); + if String.equal name "filled" then + let b = check ctx ~want:(Types.Int Types.U8) (List.hd args) in + mk loc ty (Tast.Fill (ty, b)) + else mk loc ty (Tast.Sentinel ty) + | _ -> + fail loc + "%s needs to know the type it is filling — use it where one is \ + expected, as in (set grid (%s))" + name + (if String.equal name "filled" then "filled 0xFF" else name)) + (* The one half of a destructuring [let] that [Parse] cannot do on its own. Everything else about a pattern is bindings and field accesses it already wrote; the arity is a *type* question — how many elements the value has — @@ -6705,6 +6811,14 @@ let builtins : (string * string * string) list = ("zeroed", "zeroed [] T", "The all-bytes-zero value of whatever it is being stored into, so it \ only means anything where a type is expected of it."); + ("filled", "filled [u8] T", + "Every byte of whatever it is being stored into set to one byte, as in \ + (set grid (filled 0xFF)). Numbers only, and structs and fixed arrays \ + built out of them."); + ("sentinel-filled", "sentinel-filled [] T", + "The bytes DE AD BE EF repeating over whatever it is being stored into, \ + so a hex dump reads DEADBEEF. A size that is not a multiple of four \ + ends on a prefix of the pattern. Same types [filled] takes."); ("destructure~nth", "destructure~nth [[n T] i32 i32 i32] T", "Written by the compiler for a destructuring let, and unspellable: the \ reader makes ~ a delimiter, so no source symbol can name this."); diff --git a/lib/emit.ml b/lib/emit.ml index ca4d8a5..ae58b8d 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -937,6 +937,30 @@ let emit_bulk_zero f ptr ty = end else false | _ -> false +(* ── The sentinel pattern ────────────────────────────────────────────── + + DE AD BE EF, in ascending address order, so [xxd] over the filled storage + reads "deadbeef" and not "efbeadde". That is the whole reason the builtin + exists, so the byte order is the specification and not an implementation + detail: the list below is the order the bytes land in memory, and the i32 + beside it is the little-endian word that puts them there. Both backends + write the same four bytes — x86 through [rep stosd] with the same word in + [eax], this file through a store of the same constant — and the acceptance + program reads the bytes back by index to keep them honest. + + A run whose size is not a multiple of four ends on a prefix: DE, then DE + AD, then DE AD BE. The tail is written byte by byte, from the same list, so + there is one source for the order. *) +let sentinel_bytes = [ 0xDE; 0xAD; 0xBE; 0xEF ] + +(* The four bytes as the i32 a little-endian store of them would leave, folded + from the list rather than written out, so the two cannot drift apart. *) +let sentinel_word = + List.fold_left + (fun acc b -> Int32.logor (Int32.shift_right_logical acc 8) + (Int32.shift_left (Int32.of_int b) 24)) + 0l sentinel_bytes + let term f fmt = Printf.ksprintf (fun s -> @@ -948,6 +972,7 @@ let label f name = Buffer.add_string f.b (Printf.sprintf "\n%s:\n" name); f.live <- true + (* Every [ret] in a function body goes through here, which is the whole of how the shadow stack's pop is got right. There are five of them — an explicit [return] with a value and without, the [none] arm of [(some x)], the tail of @@ -1156,6 +1181,64 @@ let alloca_raw f lltype = Buffer.add_string f.allocas (Printf.sprintf " %s = alloca %s\n" name lltype); name +(* [(filled b)] over storage this file already has the address of. One + [llvm.memset] with the byte in a register, which is exactly what the + intrinsic is for and the reason the single-byte fill is the cheap one: the + same call the zero fill above makes, with an operand instead of a zero. *) +let emit_byte_fill f ptr ty (byte : string) = + let size, align = lay f.md ty in + if size > 0 then + ins f + "call void @llvm.memset.p0.i64(ptr align %d %s, i8 %s, i64 %d, i1 false)" + align ptr byte size + +(* [(sentinel-filled)] over the same. [llvm.memset] takes one repeated i8 and + nothing else, so a four-byte pattern cannot be a call and has to be a loop + — the snag DISCUSS.org named before this was built, and the reason the two + builtins are two and not one with a wider operand. + + The counter is an entry-block alloca, which is how every local in this file + is spelled and what [mem2reg] promotes; the loop is the same + header/body/exit shape [emit_while] writes. The size is a compile-time + constant, so the trip count is one too and LLVM unrolls what it wants to. + + Every store is [align 1]: the pattern is laid down over bytes, and the + storage's own alignment may be 1 — a [7 u8] array is a legal thing to fill. + The tail's up-to-three bytes are written out straight, from the same list + the word was folded from. *) +let emit_sentinel f ptr ty = + let size, _ = lay f.md ty in + let words = size / 4 and tail = size mod 4 in + if words > 0 then begin + let i = alloca_raw f "i64" in + let lc = fresh_label f "fill" and lb = fresh_label f "fillbody" + and le = fresh_label f "fillend" in + ins f "store i64 0, ptr %s" i; + term f "br label %%%s" lc; + label f lc; + let c = fresh f in + ins f "%s = load i64, ptr %s" c i; + let t = fresh f in + ins f "%s = icmp ult i64 %s, %d" t c words; + term f "br i1 %s, label %%%s, label %%%s" t lb le; + label f lb; + let off = fresh f in + ins f "%s = mul i64 %s, 4" off c; + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %s" p ptr off; + ins f "store i32 %ld, ptr %s, align 1" sentinel_word p; + let n = fresh f in + ins f "%s = add i64 %s, 1" n c; + ins f "store i64 %s, ptr %s" n i; + term f "br label %%%s" lc; + label f le + end; + for k = 0 to tail - 1 do + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %d" p ptr (words * 4 + k); + ins f "store i8 %d, ptr %s, align 1" (List.nth sentinel_bytes k) p + done + (* ── Constants ─────────────────────────────────────────────────────── *) (* LLVM's hex form is exact, which decimal is not: a literal must mean the same @@ -1611,6 +1694,22 @@ and value_at f (e : Tast.expr) : string = | Tast.Str s -> string_const f.md s | Tast.Unit | Tast.Zero _ | Tast.None_ -> "zeroinitializer" | Tast.Uninit _ -> "poison" + (* A fill is a write over storage, so in value position it needs storage to + write over: a temporary, filled and then loaded out of. Every fill the + source actually writes is the value of a [set] or of a [let] binding, and + [Set] below takes the place's own address and skips this; the temporary + is what makes the node mean something everywhere else — a fill passed + straight to a call, say — rather than being a form with a position rule + nobody stated. *) + | Tast.Fill (ty, b) -> + let bv = value f b in + let tmp = alloca f ty in + emit_byte_fill f tmp ty bv; + load f tmp ty + | Tast.Sentinel ty -> + let tmp = alloca f ty in + emit_sentinel f tmp ty; + load f tmp ty | Tast.Local _ | Tast.Global _ | Tast.Field _ | Tast.Deref _ -> (* Everything that denotes a location is a load from its address. *) load f (addr f e) e.Tast.ty @@ -1665,6 +1764,12 @@ and value_at f (e : Tast.expr) : string = let ptr, ty = place f p in (match v.Tast.e with | Tast.Zero _ when emit_bulk_zero f ptr ty -> () + (* The shape the source writes: the fill goes straight at the place, + with no temporary and no aggregate load in between. Same short-circuit + the bulk zero above takes, and the reason [(set grid (filled 0xFF))] + is one memset. *) + | Tast.Fill (_, b) -> emit_byte_fill f ptr ty (value f b) + | Tast.Sentinel _ -> emit_sentinel f ptr ty | _ -> let v' = value f v in ins f "store %s %s, ptr %s" (ll ty) v' ptr); diff --git a/lib/js.ml b/lib/js.ml index 80184cc..e785f53 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -670,6 +670,14 @@ let rec value f (e : Tast.expr) : string = n | Tast.Unit -> "undefined" | Tast.Zero t -> zero f.md e.Tast.loc t + (* A fill is a byte pattern written over storage, and this backend has no + storage to write over: a Flan struct is a JS object here, not a run of + bytes, so there is nothing for 0xFF or DEADBEEF to mean. Refused by name + rather than approximated, which is this file's rule. *) + | Tast.Fill _ | Tast.Sentinel _ -> + unsupported + "js: a byte fill has no meaning on this backend — a struct is an \ + object here and not a run of bytes" (* [uninit] is the opt-out from zeroing. There is no uninitialised memory here to opt out of, so it is the zero — which is more than the program asked for and never less. *) diff --git a/lib/tast.ml b/lib/tast.ml index 6824a97..9b09c0f 100644 --- a/lib/tast.ml +++ b/lib/tast.ml @@ -76,6 +76,19 @@ and expr_kind = | Unit | Zero of Types.t (* ZII: all-bytes-zero of this type *) | Uninit of Types.t (* the explicit opt-out *) + (* [(filled b)]: every byte of this type set to [b]. [Zero] with a byte the + program chooses, and a separate node rather than a byte on [Zero] because + that byte is an *expression* — a memset's operand, not a literal — and + every reader of [Zero] treats it as a leaf with nothing under it. + + [(sentinel-filled)]: the four bytes DE AD BE EF repeating, ascending + through the storage, so a hex dump reads DEADBEEF. It carries no operand + because the pattern is fixed; that is the whole point of it, and a + parameterised one would be a different builtin. A size that is not a + multiple of four ends on a prefix of the pattern — DE, DE AD, DE AD BE — + which both backends produce identically. *) + | Fill of Types.t * expr + | Sentinel of Types.t | Local of int (* slot index into the frame *) | Global of string | Prim of prim * expr list @@ -360,7 +373,8 @@ let rec walk (f : expr -> unit) (e : expr) = let gos = List.iter go in match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | Local _ - | Global _ | None_ | FnAddr _ | Break _ | Continue _ -> () + | Global _ | None_ | FnAddr _ | Break _ | Continue _ | Sentinel _ -> () + | Fill (_, b) -> go b | Prim (_, es) | Call (_, es) | Do es | Make (_, es) | MakeCase (_, _, es) | Arr es | InvokeRestart (_, _, es, _, _, _) -> gos es | CallPtr (c, es) -> go c; gos es diff --git a/lib/x86.ml b/lib/x86.ml index 096382f..bb0798e 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -401,6 +401,11 @@ let push_r b r = if r >= 8 then u8 b 0x41; u8 b (0x50 lor (r land 7)) requires. *) let rep_movsb b = u8 b 0xf3; u8 b 0xa4 let rep_stosb b = u8 b 0xf3; u8 b 0xaa +(* [rep stosd]: [ecx] copies of [eax] at [rdi], four bytes at a time. The + sentinel fill's loop, in one instruction — DISCUSS.org's "a fill loop, not + a memset" is about LLVM's intrinsic taking one repeated byte, and the + string instruction here has no such limit. *) +let rep_stosd b = u8 b 0xf3; u8 b 0xab (* ── SSE ─────────────────────────────────────────────────────────────── *) @@ -1600,6 +1605,11 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = store_int f.b ~src:rax ~mm:(lmem f (shift dst 8) ~scratch:r11) ~size:8 | Tast.Unit -> () | Tast.Zero ty -> zero_value f dst ty + (* No [Set] arm of its own, unlike [emit.ml]: a [Tast.Set] here lowers its + value straight into the place's location, so filling a place and filling + a temporary are already the same line. *) + | Tast.Fill (ty, b) -> fill_value f dst ty b + | Tast.Sentinel ty -> sentinel_value f dst ty | Tast.None_ -> zero_value f dst t (* Reading an uninitialised value gives whatever the slot held: stable garbage rather than LLVM's [poison]. The one construct where the two @@ -2109,6 +2119,50 @@ and emit_invoke_restart f id name (args : Tast.expr list) sg sg_id rloc = xfer_store f ~reg:rax ~scratch:r11; jmp_lbl f.b (current_pad f) +(* [(filled b)] and [(sentinel-filled)], into a location this backend already + has. [zero_loc] above is the same three registers and the same string + instruction with a zero in [al]; these two are what it becomes when the + byte is the program's and when the unit is four bytes rather than one. + + The byte is evaluated *before* [rdi] is loaded, because evaluating it is an + arbitrary expression that may call, and a call clobbers [rdi]. [r11] stays + the scratch every other caller of [lmem] uses, and is never one of the + three this touches. + + The sentinel's word and its tail bytes both come from [Emit] — one list, + read by both backends, so "a hex dump reads DEADBEEF" is stated once. The + tail is up to three straight byte stores, matching [Emit.emit_sentinel]'s + tail exactly: a size that is not a multiple of four ends on DE, DE AD, or + DE AD BE. *) +and fill_value f (dst : loc) (ty : Types.t) (b : Tast.expr) = + let n = sizeof f.md ty in + if n > 0 then begin + let v = eval f b in + load_int f.b ~dst:rax ~mm:(lmem f v ~scratch:r11) ~size:1 ~signed:false; + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rcx (Int64.of_int n); + rep_stosb f.b + end + +and sentinel_value f (dst : loc) (ty : Types.t) = + let n = sizeof f.md ty in + let words = n / 4 and tail = n mod 4 in + if words > 0 then begin + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rax + (Int64.logand (Int64.of_int32 Emit.sentinel_word) 0xFFFFFFFFL); + movabs f.b ~dst:rcx (Int64.of_int words); + rep_stosd f.b + end; + List.iteri + (fun k byte -> + if k < tail then begin + movabs f.b ~dst:rax (Int64.of_int byte); + store_int f.b ~src:rax + ~mm:(lmem f (shift dst (words * 4 + k)) ~scratch:r11) ~size:1 + end) + Emit.sentinel_bytes + and zero_value f (dst : loc) (ty : Types.t) = if is_agg ty then zero_loc f dst (sizeof f.md ty) else if not (is_void ty) then diff --git a/test/programs/fill.flan b/test/programs/fill.flan new file mode 100644 index 0000000..36cc5dc --- /dev/null +++ b/test/programs/fill.flan @@ -0,0 +1,88 @@ +;;;; (filled b) and (sentinel-filled) — the two byte fills, read back as bytes. +;;;; +;;;; The whole point of this program is that every row is a *byte* and not a +;;;; value: the sentinel's contract is "a hex dump reads DEADBEEF", which is a +;;;; claim about which byte lands at which address, and only reading the bytes +;;;; back in address order can check it. The u32 rows are the same claim from +;;;; the other side — a little-endian load of DE AD BE EF is 0xEFBEADDE, which +;;;; is 4022250974, so a backend that wrote the word the other way round would +;;;; print 3735928559 here and be caught. +;;;; +;;;; The lengths are chosen for the tail. 8 is a whole number of patterns, 9 +;;;; ends on DE, 6 ends on DE AD, and 7 ends on DE AD BE — the three truncated +;;;; endings and the one that is not truncated at all. +(defstruct Words [a u32 b u32]) + +;;;; A computed global initialiser: a fill is never a constant the linker can +;;;; write, so this one goes through the startup function on both backends. +(defvar gfill [4 u8] (filled 0x41)) +(defvar gsent [5 u8]) + +(defn bytes4 [b [4 u8]] () + (dotimes [i 4] (print (at b i)) (print " "))) + +(defn main [] i32 + ;; One byte, repeated. 0xFF is the -1 fill a debug allocator wants. + (let [a (array 4 u8)] + (set a (filled 0xFF)) + (bytes4 a) (println "")) ; 255 255 255 255 + + ;; The byte is an expression, not a literal: this is a memset with the + ;; operand in a register, and the register path is the one a constant would + ;; otherwise hide. + (let [v (u8 7) + a (array 4 u8)] + (set a (filled v)) + (bytes4 a) (println "")) ; 7 7 7 7 + + ;; (filled 0) is (zeroed), byte for byte, and saying so here is what keeps + ;; the two from drifting. + (let [a (array 4 u8)] + (set a (filled 0)) + (bytes4 a) (println "")) ; 0 0 0 0 + + ;; Four bytes, ascending, with nothing truncated. + (let [a (array 8 u8)] + (set a (sentinel-filled)) + (dotimes [i 8] (print (at a i)) (print " ")) + (println "")) ; 222 173 190 239 x2 + + ;; The three truncated tails. + (let [a (array 9 u8)] + (set a (sentinel-filled)) + (dotimes [i 9] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 + + (let [a (array 6 u8)] + (set a (sentinel-filled)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 + + (let [a (array 7 u8)] + (set a (sentinel-filled)) + (dotimes [i 7] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 190 + + ;; The same pattern read as words rather than as bytes, which is what says + ;; which end the DE is at. + (let [w (Words {})] + (set w (sentinel-filled)) + (print (.a w)) (print " ") (print (.b w)) (println "")) + + ;; A nested aggregate: a fixed array of structs is plain data all the way + ;; down, so the fill reaches every byte of it. + (let [g (array 2 Words)] + (set g (filled 0xFF)) + (print (.a (at g 0))) (print " ") (print (.b (at g 1))) (println "")) + + ;; The globals. [gfill] was filled before main ran; [gsent] is filled here. + (dotimes [i 4] (print (at gfill i)) (print " ")) + (println "") ; 65 65 65 65 + (set gsent (sentinel-filled)) + (dotimes [i 5] (print (at gsent i)) (print " ")) + (println "") ; 222 173 190 239 222 + + ;; A fill in value position rather than as the value of a [set]: the same + ;; bytes, reached through a temporary instead of through the place. + (bytes4 (filled 0xFF)) (println "") ; 255 255 255 255 + 0) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index bcc7366..2cdc9ad 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -1243,6 +1243,34 @@ let () = string_eq_out; outputs ~x86:true "string equality, --x86" "programs/string-eq.flan" string_eq_out; + (* The two byte fills, DISCUSS.org's "a DEADBEEF-style sentinel-fill + builtin". Three rows over one program, and what they are pinning is + byte-identity: a fill is the one operation in the language whose whole + contract is which byte lands at which address, so the LLVM backend's + [llvm.memset] and dword loop and the x86 backend's [rep stosb] and + [rep stosd] have to agree byte for byte, and -O0 has to agree with -O2 + because the loop is a real loop at one and unrolled at the other. + + 222 173 190 239 is DE AD BE EF in decimal, ascending through the + storage, which is the claim "a hex dump reads DEADBEEF". The 4022250974 + rows are the same four bytes read back as a u32 — 0xEFBEADDE — and they + are what a backend that wrote the word the other way round would fail. + The three short rows are the truncated tails a size that is not a + multiple of four ends on. *) + let fill_out = + "255 255 255 255 \n7 7 7 7 \n0 0 0 0 \n\ + 222 173 190 239 222 173 190 239 \n\ + 222 173 190 239 222 173 190 239 222 \n\ + 222 173 190 239 222 173 \n\ + 222 173 190 239 222 173 190 \n\ + 4022250974 4022250974\n4294967295 4294967295\n\ + 65 65 65 65 \n222 173 190 239 222 \n255 255 255 255 \n" + in + outputs "byte and sentinel fills" "programs/fill.flan" fill_out; + outputs ~opt:"-O0" "byte and sentinel fills, -O0" "programs/fill.flan" + fill_out; + outputs ~x86:true "byte and sentinel fills, --x86" "programs/fill.flan" + fill_out; (* dyn if: truthiness -- M2 queue item 7. A dyn scrutinee is tested for nil/false vs. everything else, Clojure's rule, on both backends; a typed scrutinee stays strictly bool, which is a checker test diff --git a/test/test_flan.ml b/test/test_flan.ml index 753bc77..0c7c57e 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -1956,6 +1956,81 @@ let () = "(defvar g (Vec u8) uninit) (defn f [] ())" ~needle:"steers every read of it"; + (* ── What may be filled with raw bytes ───────────────────────────── + [(filled b)] and [(sentinel-filled)] are [zeroed]'s siblings, and the + boundary is the whole of what is new about them: zero is a value every + type can have and 0xDE is not, so the checker says which types survive + arbitrary bytes. The accepting side is programs/fill.flan; what is here + is the catalogue of what it refuses and why each refusal is the runtime's + and not a matter of taste. + + The dyn row is the one that would corrupt the collector: a struct holding + a dyn is rooted with a descriptor naming that word's offset, so a filled + one is a root pointing at nothing. *) + accepts "a fixed array of numbers may be filled" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 0xFF))))"; + accepts "a struct of numbers may be sentinel-filled" + "(defstruct S [a i32 b f64]) \ + (defn f [] () (let [s (S {})] (set s (sentinel-filled))))"; + rejects_check "a struct holding a dyn cannot be filled" + "(defstruct S [a i32 d dyn]) \ + (defn f [] () (let [s (S {})] (set s (sentinel-filled))))" + ~needle:"a root pointing at nothing"; + rejects_check "a Vec cannot be filled" + "(defn f [] () (let [v (vec-new i32)] (set v (filled 0xFF))))" + ~needle:"frees a wild address"; + rejects_check "a string cannot be filled" + "(defn f [] () (let [s \"hi\"] (set s (filled 0xFF))))" + ~needle:"a length every bounds check believes"; + rejects_check "a pointer field cannot be filled" + "(defstruct S [p (Ptr i32)]) \ + (defn f [] () (let [s (S {})] (set s (filled 0xFF))))" + ~needle:"an address every deref trusts"; + (* The one refusal that is about the two backends rather than the runtime: + LLVM reads a bool's low bit and x86 compares the whole byte, so 0xDE is + false on one and true on the other. Byte-identical behaviour across the + backends is what this feature is pinned on, so the divergence is refused + rather than documented. *) + rejects_check "a bool cannot be filled" + "(defn f [] () (let [b false] (set b (filled 0xFF))))" + ~needle:"would not even agree with itself"; + (* A data type's tag is a case index, and no byte pattern names a real + case. The type itself is what the message names, because a data type + overlays its cases. *) + rejects_check "a data type cannot be filled" + "(defdata U [(A [x i32]) (B [y i32])]) \ + (defn f [] () (let [u (U.A {.x 1})] (set u (filled 0xFF))))" + ~needle:"names a case"; + (* [zeroed]'s own refusal, worn by both siblings: a fill is the bytes of + whatever type is expected of it, and in a position that expects nothing + there is no type and nothing to fill. This is the shape's cost and it is + paid on purpose — the alternative was a second, place-taking spelling + for an operation [set] already expresses. *) + rejects_check "a fill in a position with no expected type" + "(defn f [] () (print (filled 0xFF)))" + ~needle:"needs to know the type it is filling"; + rejects_check "a sentinel fill in a position with no expected type" + "(defn f [] () (print (sentinel-filled)))" + ~needle:"needs to know the type it is filling"; + (* The byte is a u8 and the ordinary literal rule applies to it — there is + no range check of this builtin's own, and there does not need to be. *) + rejects_check "a fill byte out of range" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 300))))" + ~needle:"does not fit in u8"; + rejects_check "filled takes exactly one byte" + "(defn f [] () (let [a (array 4 u8)] (set a (filled))))" + ~needle:"takes 1 argument"; + rejects_check "sentinel-filled takes no argument" + "(defn f [] () (let [a (array 4 u8)] (set a (sentinel-filled 1))))" + ~needle:"takes 0 arguments, given 1"; + (* A fill is never a value the linker can write into the image, so a + defconst of one is refused by the constant rule rather than by anything + of this feature's own. A defvar is fine: its initialiser runs at + startup, which programs/fill.flan pins. *) + rejects_check "a defconst cannot be filled" + "(defconst g [4 u8] (filled 0xFF)) (defn f [] ())" + ~needle:"defconst"; + (* ── The third element of a defvar ───────────────────────────────── The rule, 2026-09-20: a type there is the zeroed static global it has always been, and anything else is a dyn global initialised from the From de3a3ed3c84cbbbfd612176378b751db49217266 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 18:19:45 +0700 Subject: [PATCH 2/6] The writeup said eleven checker rows; there are fourteen Two accepting and twelve refusing. Also records that the three acceptance rows were confirmed to run rather than inferred from a green exit: the expectation was broken on purpose once and all three reported. --- FIX.org | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/FIX.org b/FIX.org index f2da3b4..4143efd 100644 --- a/FIX.org +++ b/FIX.org @@ -1712,6 +1712,9 @@ four lengths — 8, 9, 6 and 7. ~dune test --root .~ green (exit 0, no FAIL lines). Three acceptance rows over ~test/programs/fill.flan~ — default, ~-O0~ and ~--x86~ — and the three outputs diffed against each other by hand before the rows were written: -byte-identical. Eleven checker rows in ~test_flan.ml~ for the boundary and -the two arity/position refusals. Per the sweep policy the ~@x86~ and +byte-identical — and the same three rows confirmed to actually run, by +breaking one expectation on purpose and watching all three report. Fourteen +checker rows in ~test_flan.ml~: two accepting, and twelve refusals covering +the boundary, both arities, both no-expected-type positions, the byte's +range and the ~defconst~ rule. Per the sweep policy the ~@x86~ and ~@sanitize~ sweeps were not run here. From 7bd2c99353ae948a6a6c7c17cb9dcdc3fd317f43 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 18:33:05 +0700 Subject: [PATCH 3/6] sentinel-filled is now dead-beef, and takes the pattern MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The author's revision. The name says what it writes, and the pattern is the program's to choose: (dead-beef) is DEADBEEF, (dead-beef 0xBAADF00D) is BA AD F0 0D. One byte-order rule covers both — a pattern's ascending bytes are its big-endian bytes, which is how the hex literal reads left to right — so every candidate DISCUSS.org listed is now spellable without the compiler naming any of them. The bare form is not a case a backend knows about: the checker writes Tast.dead_beef_default in where the argument would have been, so (dead-beef) and (dead-beef 0xDEADBEEF) are the same node and an acceptance row prints both to say so. The operand is an ordinary u32 expression, which is what the byte arm already accepts for its byte. A literal is byte-reversed at compile time and still reaches the loop as an immediate; a computed one is reversed at run time, by llvm.bswap.i32 on one backend and bswap on the other, after which the tail shifts its bytes out of the word rather than folding them. The program runs a computed pattern over lengths 6 and 7 deliberately: that is the case a constant-only implementation would pass by accident. filled is untouched, and so is the fill boundary. --- DISCUSS.org | 15 ++++-- FIX.org | 99 +++++++++++++++++++++++++--------- lib/check.ml | 57 +++++++++++++++----- lib/emit.ml | 116 ++++++++++++++++++++++++++++++---------- lib/js.ml | 2 +- lib/tast.ml | 36 +++++++++---- lib/x86.ml | 73 +++++++++++++++++++------ test/programs/fill.flan | 82 +++++++++++++++++++++++----- test/test_acceptance.ml | 34 +++++++++--- test/test_flan.ml | 32 +++++++---- 10 files changed, 421 insertions(+), 125 deletions(-) diff --git a/DISCUSS.org b/DISCUSS.org index 34e7e2f..e375953 100644 --- a/DISCUSS.org +++ b/DISCUSS.org @@ -186,9 +186,18 @@ why MSVC/glibc-style debug allocators use a single repeated byte instead Built, 2026-09-20, as two builtins rather than one — the author's answer to the cheap-byte-or-real-pattern question was "why not both? we need some sort of memset -1 right? and dead-beef can loop, that's fine". They are -(filled BYTE) and (sentinel-filled), spelled the way [zeroed] is: the value -of whatever type is expected of them, so (set grid (filled 0xFF)) is how a -place is filled and there is no place-taking form to learn beside [set]. +(filled BYTE) and (dead-beef), spelled the way [zeroed] is: the value of +whatever type is expected of them, so (set grid (filled 0xFF)) is how a place +is filled and there is no place-taking form to learn beside [set]. + +(dead-beef) writes the default DEADBEEF and (dead-beef 0xBAADF00D) writes the +pattern given, under one byte-order rule: a pattern's ascending bytes are its +big-endian bytes, which is how the hex literal reads left to right. So every +candidate listed above is spellable without the compiler naming any of them, +and the bare form is checked into the spelled-out default rather than being a +case a backend knows about. The pattern may be computed, not only written — +same rule as the byte arm, and both backends byte-reverse at run time when it +is. The snag above is why there are two and not one with a wider operand. The byte fill is one llvm.memset / one rep stosb; the 4-byte pattern is a loop on diff --git a/FIX.org b/FIX.org index 4143efd..09c4c91 100644 --- a/FIX.org +++ b/FIX.org @@ -1618,12 +1618,19 @@ answer to the single-byte-or-four-byte question was "why not both? we need some sort of memset -1 right? and dead-beef can loop, that's fine", so there are two builtins and they are siblings of ~zeroed~, not a new shape. +Revised the same day: the pattern builtin was ~sentinel-filled~ and is now +~dead-beef~, and it gained an optional operand so the pattern is the +program's to choose. What did not change is ~filled~, or the fill boundary, +or the byte-order rule — the revision generalised the pattern, it did not +reopen what may be filled. + ** The spellings -~(filled BYTE)~ and ~(sentinel-filled)~, both value forms driven by the type -expected of them, exactly as ~(zeroed)~ is: +~(filled BYTE)~ and ~(dead-beef)~ / ~(dead-beef PATTERN)~, all value forms +driven by the type expected of them, exactly as ~(zeroed)~ is: : (set grid (filled 0xFF)) -: (set frame (sentinel-filled)) +: (set frame (dead-beef)) +: (set frame (dead-beef 0xBAADF00D)) A place-taking ~(filled place byte)~ was the other candidate and was not taken. ~zeroed~ already answers "the all-bytes-X value of whatever this is @@ -1633,9 +1640,29 @@ nothing. The cost is real and is paid on purpose: a fill in a position that expects no type is refused ("filled needs to know the type it is filling"), which is ~zeroed~'s own refusal worn by both siblings. -~sentinel-filled~ takes no operand. The pattern is fixed — that is the whole -point of it, since a hex dump only reads DEADBEEF if nobody can change it — -and a parameterised one would be a different builtin. +~dead-beef~ takes the pattern or leaves it out, and leaving it out is +*defined as* writing the default: the checker's zero-argument arm builds the +same ~Tast.Int 0xDEADBEEF~ the spelled-out call would have, so ~(dead-beef)~ +and ~(dead-beef 0xDEADBEEF)~ are the same IR node by construction and no +backend has a second path for the bare form. An acceptance row prints both +and pins that they agree. + +The pattern is an ordinary ~u32~ /expression/, not a literal — the byte arm's +rule at four times the width. A literal out of range meets ~in_range~'s +located "does not fit in u32"; anything computed is guaranteed by its type +instead, since a ~u32~ cannot be out of ~u32~ range. Refusing a computed one +would have been a restriction with no mechanism behind it: neither backend +needs the number early. + +/Reading note, for whoever reviews this./ The revision asked for "a u32-range +constant ... decide literal-only vs any constant expression from what the +byte-fill arm already accepts". The byte-fill arm accepts any ~u8~ +expression, runtime ones included, and the same instruction asked that both +backends handle "eax loaded from a value, not an immediate" — which only +exists if a computed pattern is legal. So the operand is any ~u32~ +expression. That is a strict superset of constants-only: every program the +narrower reading allows behaves identically here. Tighten it to literals if +that was the intent; nothing else depends on the breadth. ** The fill boundary — what may be overwritten with raw bytes *Numbers, and structs and fixed arrays built out of numbers. Nothing else.* @@ -1675,31 +1702,53 @@ into the image. A ~defvar~ is fine and goes through the startup function on both backends, which ~programs/fill.flan~ pins. ** The byte order, which is the specification -DE AD BE EF in *ascending address order*, so ~xxd~ reads "deadbeef". A -little-endian store of those four bytes is the i32 0xEFBEADDE, and -~Emit.sentinel_bytes~ / ~Emit.sentinel_word~ are the one place either is -written — x86.ml reads both out of Emit rather than repeating them, so the -two backends cannot drift. +*A pattern's ascending bytes are its big-endian bytes* — exactly how the hex +literal reads left to right. So ~(dead-beef)~ lays down DE AD BE EF and ~xxd~ +reads "deadbeef"; ~(dead-beef 0xBAADF00D)~ lays down BA AD F0 0D. One rule, +both arities. + +On a little-endian machine the word a 4-byte store must therefore leave is +the *byte reversal* of the pattern, which is all ~Emit.word_of_pattern~ is +(~bytes_of_pattern~ beside it is the ascending list). Those two are the one +place the order is written; x86.ml reads them out of Emit rather than +repeating them, and ~Tast.dead_beef_default~ is the one place 0xDEADBEEF is +written. Three definitions, no duplicates, so the default and the +parameterised case cannot drift. + +A literal pattern is reversed at compile time and reaches the loop as an +immediate — the default's generated code is exactly what it was before the +pattern became an operand. A computed one is evaluated and reversed at run +time, by ~llvm.bswap.i32~ on one backend and ~bswap eax~ (0F C8, new) on the +other. *Tail behaviour.* A size that is not a multiple of four ends on a prefix of -the pattern: 1 byte over is DE, 2 is DE AD, 3 is DE AD BE. Both backends -write the tail byte by byte from the same list. ~programs/fill.flan~ has all -four lengths — 8, 9, 6 and 7. +the ascending bytes: 1 byte over is DE, 2 is DE AD, 3 is DE AD BE. +Equivalently, tail byte k is ~(word >> 8k) & 0xFF~ — which is what the +computed path actually does, by shifting, since there is no constant to fold. +~programs/fill.flan~ has all four lengths (8, 9, 6, 7) and, crucially, runs a +computed pattern over lengths 6 and 7: that is the case a constant-only +implementation would pass by accident. ** The backends - *LLVM (emit.ml).* The byte fill is one ~llvm.memset~ with the byte as an operand instead of a zero — the same call the existing bulk zero makes, and - the reason the single-byte fill is the cheap one. The sentinel cannot be a - memset at all (the intrinsic takes one repeated i8, which is the snag + the reason the single-byte fill is the cheap one. The pattern fill cannot + be a memset at all (the intrinsic takes one repeated i8, which is the snag DISCUSS.org named), so it is a counted loop over dwords in the header/body/exit shape ~emit_while~ writes, with the counter as an entry-block alloca that ~mem2reg~ promotes. Every store is ~align 1~, - because a ~[7 u8]~ array is a legal thing to fill. + because a ~[7 u8]~ array is a legal thing to fill. A computed pattern goes + through ~llvm.bswap.i32~ (newly declared) and the loop stores an SSA value + rather than a constant; the tail then shifts and truncates. - *x86 (x86.ml).* ~rep stosb~ for the byte fill — ~zero_loc~'s three registers with the program's byte in ~al~ instead of a zero — and ~rep - stosd~ (new, 0xf3 0xab) for the sentinel, with 0xEFBEADDE in ~eax~. The - byte operand is evaluated *before* ~rdi~ is loaded, because evaluating it - may call and a call clobbers ~rdi~. + stosd~ (new, 0xf3 0xab) for the pattern, with the stored word in ~eax~. A + computed pattern is loaded and run through ~bswap~ (new, 0F C8); the tail + walks the bytes out of ~rax~ with ~shr~ by an immediate (new, C1 /5), which + is used rather than ~shift_cl~ precisely because ~rep stosd~ leaves ~rcx~ + at zero. Either operand is evaluated *before* ~rdi~ is loaded, because + evaluating one may call and a call clobbers ~rdi~; ~rep stosd~ does not + touch ~rax~, which is what lets the tail keep reading the word out of it. - The one asymmetry: ~emit.ml~ needs a ~Tast.Set~ arm of its own to fill the place rather than a temporary, because its value path returns an SSA value. ~x86.ml~ needs none — a ~Set~ there already lowers its value into the @@ -1712,9 +1761,11 @@ four lengths — 8, 9, 6 and 7. ~dune test --root .~ green (exit 0, no FAIL lines). Three acceptance rows over ~test/programs/fill.flan~ — default, ~-O0~ and ~--x86~ — and the three outputs diffed against each other by hand before the rows were written: -byte-identical — and the same three rows confirmed to actually run, by -breaking one expectation on purpose and watching all three report. Fourteen -checker rows in ~test_flan.ml~: two accepting, and twelve refusals covering -the boundary, both arities, both no-expected-type positions, the byte's +byte-identical, with a fourth build (~--dev~) added at the rename: four-way +identical. The three rows were confirmed to actually run, by breaking one +expectation on purpose and watching all three report. Seventeen checker rows +in ~test_flan.ml~: four accepting (both ~dead-beef~ arities and a computed +pattern among them), and thirteen refusals covering the boundary, both +arities, both no-expected-type positions, the byte's range, the pattern's range and the ~defconst~ rule. Per the sweep policy the ~@x86~ and ~@sanitize~ sweeps were not run here. diff --git a/lib/check.ml b/lib/check.ml index fc2bc77..f110497 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -4873,12 +4873,29 @@ and named_call ctx ~want loc name args = (* [zeroed]'s two siblings, and the same shape exactly: a value of whatever type is expected of it, so [(set grid (filled 0xFF))] is how a place is filled and there is no second spelling to learn. What they add over - [zeroed] is the byte — [(filled b)] repeats one the program picks, and - [(sentinel-filled)] repeats the four bytes DE AD BE EF — and with it the - question [zeroed] never has to ask: zero is a value every type can have, - and 0xDE is not. [unfillable] above is the whole of the answer. *) - | "filled" | "sentinel-filled" -> - arity loc name (if String.equal name "filled" then 1 else 0) args; + [zeroed] is the bytes — [(filled b)] repeats one the program picks, and + [(dead-beef)] repeats four — and with them the question [zeroed] never + has to ask: zero is a value every type can have, and 0xDE is not. + [unfillable] above is the whole of the answer. + + [dead-beef] takes the pattern or leaves it out, and leaving it out is + defined as writing the default: the [None] arm below builds the same + literal the source would have, so [(dead-beef)] and + [(dead-beef 0xDEADBEEF)] are the same node by construction and no + backend has a second path for the bare form. + + The pattern is an ordinary u32 expression, which is the byte arm's rule + at four times the width — a literal out of range meets [in_range]'s + located "does not fit in u32", and anything computed is guaranteed by + its type instead. Both builtins take a value rather than only a literal + for the same reason: refusing one would be a restriction with no + mechanism behind it, since neither backend needs the number early. *) + | "filled" | "dead-beef" -> + let is_byte = String.equal name "filled" in + if is_byte then arity loc name 1 args + else if List.length args > 1 then + fail loc "%s takes the pattern or nothing at all, given %d arguments" + name (List.length args); (match want with | Some ty when ty <> Types.Never -> (match unfillable ctx.env [] ty with @@ -4907,16 +4924,27 @@ and named_call ctx ~want loc name args = | _ -> "it carries a tag that names a case, and no byte pattern \ names a real one") | None -> ()); - if String.equal name "filled" then + if is_byte then let b = check ctx ~want:(Types.Int Types.U8) (List.hd args) in mk loc ty (Tast.Fill (ty, b)) - else mk loc ty (Tast.Sentinel ty) + else + let pat = + match args with + | [ a ] -> check ctx ~want:(Types.Int Types.U32) a + (* The bare form, written out. Not a default a backend applies: + the node that leaves here is the one the spelled-out call would + have left, which is what makes the equivalence a fact about the + IR rather than a promise two emitters keep separately. *) + | _ -> + mk loc (Types.Int Types.U32) + (Tast.Int (Int64.of_int32 Tast.dead_beef_default, Types.U32)) + in + mk loc ty (Tast.DeadBeef (ty, pat)) | _ -> fail loc "%s needs to know the type it is filling — use it where one is \ expected, as in (set grid (%s))" - name - (if String.equal name "filled" then "filled 0xFF" else name)) + name (if is_byte then "filled 0xFF" else name)) (* The one half of a destructuring [let] that [Parse] cannot do on its own. Everything else about a pattern is bindings and field accesses it already @@ -6815,10 +6843,11 @@ let builtins : (string * string * string) list = "Every byte of whatever it is being stored into set to one byte, as in \ (set grid (filled 0xFF)). Numbers only, and structs and fixed arrays \ built out of them."); - ("sentinel-filled", "sentinel-filled [] T", - "The bytes DE AD BE EF repeating over whatever it is being stored into, \ - so a hex dump reads DEADBEEF. A size that is not a multiple of four \ - ends on a prefix of the pattern. Same types [filled] takes."); + ("dead-beef", "dead-beef [u32?] T", + "A four-byte pattern repeating over whatever it is being stored into, \ + written so a hex dump reads it left to right: 0xDEADBEEF with no \ + argument, the u32 given otherwise. A size that is not a multiple of \ + four ends on a prefix of the pattern. Same types [filled] takes."); ("destructure~nth", "destructure~nth [[n T] i32 i32 i32] T", "Written by the compiler for a destructuring let, and unspellable: the \ reader makes ~ a delimiter, so no source symbol can name this."); diff --git a/lib/emit.ml b/lib/emit.ml index ae58b8d..82594c3 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -937,29 +937,46 @@ let emit_bulk_zero f ptr ty = end else false | _ -> false -(* ── The sentinel pattern ────────────────────────────────────────────── +(* ── The dead-beef pattern ───────────────────────────────────────────── - DE AD BE EF, in ascending address order, so [xxd] over the filled storage - reads "deadbeef" and not "efbeadde". That is the whole reason the builtin - exists, so the byte order is the specification and not an implementation - detail: the list below is the order the bytes land in memory, and the i32 - beside it is the little-endian word that puts them there. Both backends - write the same four bytes — x86 through [rep stosd] with the same word in - [eax], this file through a store of the same constant — and the acceptance - program reads the bytes back by index to keep them honest. + [(dead-beef)] writes DE AD BE EF in ascending address order, so [xxd] over + the filled storage reads "deadbeef" and not "efbeadde". That is the whole + reason the builtin exists, so the byte order is the specification and not + an implementation detail. - A run whose size is not a multiple of four ends on a prefix: DE, then DE - AD, then DE AD BE. The tail is written byte by byte, from the same list, so - there is one source for the order. *) -let sentinel_bytes = [ 0xDE; 0xAD; 0xBE; 0xEF ] + [(dead-beef V)] writes V's four bytes under the same rule, and the rule is + what makes the two arities one thing: *a pattern's ascending bytes are its + big-endian bytes*, which is exactly how the hex literal reads left to + right. So [(dead-beef 0xBAADF00D)] lays down BA AD F0 0D. Nothing here + knows about the bare form: the checker writes [Tast.dead_beef_default] in + where the argument would have been, so this file only ever sees a pattern. -(* The four bytes as the i32 a little-endian store of them would leave, folded - from the list rather than written out, so the two cannot drift apart. *) -let sentinel_word = + On a little-endian machine the word a 4-byte store must leave is therefore + the byte reversal of the pattern, which is all [word_of_pattern] is. Both + backends go through here — x86 through [rep stosd] with this word in [eax], + this file through a store of it — and the acceptance program reads the + bytes back by index to keep them honest. + + A run whose size is not a multiple of four ends on a prefix of the + ascending bytes: one over is DE, two is DE AD, three is DE AD BE. *) + +(* The pattern's bytes in the order they land in memory, which is + most-significant first — the order the literal is written in. *) +let bytes_of_pattern (v : int32) = + List.map + (fun k -> + Int32.to_int + (Int32.logand (Int32.shift_right_logical v (8 * k)) 0xFFl)) + [ 3; 2; 1; 0 ] + +(* The i32 a little-endian store leaves those bytes as: the same list folded + back the other way, so one definition answers for both and a change to the + byte order cannot reach only half of it. *) +let word_of_pattern (v : int32) = List.fold_left (fun acc b -> Int32.logor (Int32.shift_right_logical acc 8) (Int32.shift_left (Int32.of_int b) 24)) - 0l sentinel_bytes + 0l (bytes_of_pattern v) let term f fmt = Printf.ksprintf @@ -1192,10 +1209,18 @@ let emit_byte_fill f ptr ty (byte : string) = "call void @llvm.memset.p0.i64(ptr align %d %s, i8 %s, i64 %d, i1 false)" align ptr byte size -(* [(sentinel-filled)] over the same. [llvm.memset] takes one repeated i8 and - nothing else, so a four-byte pattern cannot be a call and has to be a loop - — the snag DISCUSS.org named before this was built, and the reason the two - builtins are two and not one with a wider operand. +(* [(dead-beef PATTERN)] over the same. [llvm.memset] takes one repeated i8 + and nothing else, so a four-byte pattern cannot be a call and has to be a + loop — the snag DISCUSS.org named before this was built, and the reason the + two builtins are two and not one with a wider operand. + + The word is either a folded constant or an SSA value, and [word] below is + just its spelling: a literal pattern is byte-reversed here at compile time + and a computed one by [llvm.bswap.i32] at run time, after which the loop + and the tail are the same instructions either way. That is deliberate — + the case a constant-only implementation would quietly get wrong is a + computed pattern over a length that is not a multiple of four, and there + is only one tail here for it to get wrong. The counter is an entry-block alloca, which is how every local in this file is spelled and what [mem2reg] promotes; the loop is the same @@ -1204,9 +1229,12 @@ let emit_byte_fill f ptr ty (byte : string) = Every store is [align 1]: the pattern is laid down over bytes, and the storage's own alignment may be 1 — a [7 u8] array is a legal thing to fill. - The tail's up-to-three bytes are written out straight, from the same list - the word was folded from. *) -let emit_sentinel f ptr ty = + + The tail is the first up-to-three bytes of the *stored word*, lowest + address first, which is byte k = (word >> 8k) & 0xFF. For a constant that + folds to [bytes_of_pattern]'s list; for a computed word it is a shift and + a truncate, ~tail~ times, unrolled because the count is known here. *) +let emit_dead_beef f ptr ty ~(word : string) ~(folded : int32 option) = let size, _ = lay f.md ty in let words = size / 4 and tail = size mod 4 in if words > 0 then begin @@ -1226,7 +1254,7 @@ let emit_sentinel f ptr ty = ins f "%s = mul i64 %s, 4" off c; let p = fresh f in ins f "%s = getelementptr i8, ptr %s, i64 %s" p ptr off; - ins f "store i32 %ld, ptr %s, align 1" sentinel_word p; + ins f "store i32 %s, ptr %s, align 1" word p; let n = fresh f in ins f "%s = add i64 %s, 1" n c; ins f "store i64 %s, ptr %s" n i; @@ -1236,7 +1264,15 @@ let emit_sentinel f ptr ty = for k = 0 to tail - 1 do let p = fresh f in ins f "%s = getelementptr i8, ptr %s, i64 %d" p ptr (words * 4 + k); - ins f "store i8 %d, ptr %s, align 1" (List.nth sentinel_bytes k) p + match folded with + | Some v -> + ins f "store i8 %d, ptr %s, align 1" (List.nth (bytes_of_pattern v) k) p + | None -> + let sh = fresh f in + ins f "%s = lshr i32 %s, %d" sh word (8 * k); + let b = fresh f in + ins f "%s = trunc i32 %s to i8" b sh; + ins f "store i8 %s, ptr %s, align 1" b p done (* ── Constants ─────────────────────────────────────────────────────── *) @@ -1706,9 +1742,10 @@ and value_at f (e : Tast.expr) : string = let tmp = alloca f ty in emit_byte_fill f tmp ty bv; load f tmp ty - | Tast.Sentinel ty -> + | Tast.DeadBeef (ty, pat) -> + let word, folded = dead_beef_word f pat in let tmp = alloca f ty in - emit_sentinel f tmp ty; + emit_dead_beef f tmp ty ~word ~folded; load f tmp ty | Tast.Local _ | Tast.Global _ | Tast.Field _ | Tast.Deref _ -> (* Everything that denotes a location is a load from its address. *) @@ -1769,7 +1806,9 @@ and value_at f (e : Tast.expr) : string = the bulk zero above takes, and the reason [(set grid (filled 0xFF))] is one memset. *) | Tast.Fill (_, b) -> emit_byte_fill f ptr ty (value f b) - | Tast.Sentinel _ -> emit_sentinel f ptr ty + | Tast.DeadBeef (_, pat) -> + let word, folded = dead_beef_word f pat in + emit_dead_beef f ptr ty ~word ~folded | _ -> let v' = value f v in ins f "store %s %s, ptr %s" (ll ty) v' ptr); @@ -1896,6 +1935,24 @@ and load f ptr ty = ins f "%s = load %s, ptr %s" t (ll ty) ptr; t +(* The word a [(dead-beef PATTERN)] store must leave, and whether it is known + here. A literal pattern is reversed at compile time and reaches the loop as + an immediate, which is what keeps the default's code exactly what it was + before the pattern became an operand. Anything else is evaluated and + reversed by [llvm.bswap.i32] — the pattern is a u32 the program computed, + and the byte order it is written in is not a property of how it was + spelled. The [int32] comes back so the tail can fold too. *) +and dead_beef_word f (pat : Tast.expr) : string * int32 option = + match pat.Tast.e with + | Tast.Int (n, _) -> + let v = word_of_pattern (Int64.to_int32 n) in + (Printf.sprintf "%ld" v, Some (Int64.to_int32 n)) + | _ -> + let v = value f pat in + let t = fresh f in + ins f "%s = call i32 @llvm.bswap.i32(i32 %s)" t v; + (t, None) + (* The address of an expression that denotes a location. Anything else is spilled to a temporary first, so [(at (f) 0)] on a returned array works. *) and addr f (e : Tast.expr) : string = @@ -3664,6 +3721,7 @@ let header = {|; Generated by flan. The layout is C's: no object headers anywher @flan_frame_head = external global ptr declare void @llvm.memset.p0.i64(ptr nocapture writeonly, i8, i64, i1 immarg) +declare i32 @llvm.bswap.i32(i32) declare void @flan_rt_init(i32, ptr) declare void @flan_argv(ptr) declare void @flan_write_stdout(ptr, i64) diff --git a/lib/js.ml b/lib/js.ml index e785f53..ba7fc4c 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -674,7 +674,7 @@ let rec value f (e : Tast.expr) : string = storage to write over: a Flan struct is a JS object here, not a run of bytes, so there is nothing for 0xFF or DEADBEEF to mean. Refused by name rather than approximated, which is this file's rule. *) - | Tast.Fill _ | Tast.Sentinel _ -> + | Tast.Fill _ | Tast.DeadBeef _ -> unsupported "js: a byte fill has no meaning on this backend — a struct is an \ object here and not a run of bytes" diff --git a/lib/tast.ml b/lib/tast.ml index 9b09c0f..6682af8 100644 --- a/lib/tast.ml +++ b/lib/tast.ml @@ -81,14 +81,21 @@ and expr_kind = that byte is an *expression* — a memset's operand, not a literal — and every reader of [Zero] treats it as a leaf with nothing under it. - [(sentinel-filled)]: the four bytes DE AD BE EF repeating, ascending - through the storage, so a hex dump reads DEADBEEF. It carries no operand - because the pattern is fixed; that is the whole point of it, and a - parameterised one would be a different builtin. A size that is not a - multiple of four ends on a prefix of the pattern — DE, DE AD, DE AD BE — - which both backends produce identically. *) + [(dead-beef PATTERN)]: the pattern's four bytes repeating, ascending + through the storage, so a hex dump reads the pattern left to right — + DEADBEEF by default, and whatever u32 was written otherwise. The operand + is always present by the time it reaches here: [(dead-beef)] is checked + into [(dead-beef 0xDEADBEEF)], so a backend has one shape to lower and + the default cannot acquire a code path of its own. A size that is not a + multiple of four ends on a prefix of the pattern — DE, DE AD, DE AD BE + for the default — which both backends produce identically. + + The operand is an expression and not an [int32] for the same reason + [Fill]'s byte is: it may be computed. A literal one is folded by each + backend into the immediate it always was; anything else is evaluated and + byte-reversed at run time. *) | Fill of Types.t * expr - | Sentinel of Types.t + | DeadBeef of Types.t * expr | Local of int (* slot index into the frame *) | Global of string | Prim of prim * expr list @@ -373,8 +380,12 @@ let rec walk (f : expr -> unit) (e : expr) = let gos = List.iter go in match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | Local _ - | Global _ | None_ | FnAddr _ | Break _ | Continue _ | Sentinel _ -> () - | Fill (_, b) -> go b + | Global _ | None_ | FnAddr _ | Break _ | Continue _ -> () + (* Both carry an operand, and it must be walked: a call written inside a + fill's byte or a dead-beef's pattern is a call, and [Reach] roots what it + finds here. A leaf row would drop it silently — the match would still + compile. *) + | Fill (_, b) | DeadBeef (_, b) -> go b | Prim (_, es) | Call (_, es) | Do es | Make (_, es) | MakeCase (_, _, es) | Arr es | InvokeRestart (_, _, es, _, _, _) -> gos es | CallPtr (c, es) -> go c; gos es @@ -412,6 +423,13 @@ and walk_place f (p : place) = rather than an error. A data type case written into the image would need a byte-level encoder that could not encode a string field at all; written as a store at startup it needs nothing. *) +(* The pattern [(dead-beef)] means, and the name the builtin is spelled after. + It lives here rather than in a backend because it is a fact about the + language — what the bare form of a builtin means — and because the checker + is what writes it in: a [DeadBeef] node always carries its pattern, so no + emitter has a default of its own to keep in step with this one. *) +let dead_beef_default = 0xDEADBEEFl + let rec const_init (e : expr) = match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | None_ -> true diff --git a/lib/x86.ml b/lib/x86.ml index bb0798e..7da8fd9 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -407,6 +407,21 @@ let rep_stosb b = u8 b 0xf3; u8 b 0xaa string instruction here has no such limit. *) let rep_stosd b = u8 b 0xf3; u8 b 0xab +(* [bswap] on a 32-bit register: the byte reversal a computed [(dead-beef V)] + pattern needs, because V's ascending bytes are its big-endian ones and a + store leaves them little-endian. [Emit.word_of_pattern] is the same + operation folded at compile time for a literal. No REX.W — this is the + 32-bit form, and the 64-bit one would reverse eight bytes. *) +let bswap32 b ~dst = + rex b ~w:false ~r:0 ~x:0 ~m:dst; u8 b 0x0f; u8 b (0xc8 lor (dst land 7)) + +(* Logical shift right by an immediate, which the tail of a computed pattern + walks its bytes with. [shift_cl] above is the same opcode group taking the + count in [cl]; this one takes it in the instruction, so it does not need + [rcx] — and [rcx] is exactly what [rep stosd] has just left at zero. *) +let shr_imm b ~dst ~n = + rex b ~w:true ~r:0 ~x:0 ~m:dst; u8 b 0xc1; modrm_r b ~r:5 ~m:dst; u8 b n + (* ── SSE ─────────────────────────────────────────────────────────────── *) let sse_rm b ~pfx ~op ~r ~mm = mem_op b ~r ~op:[ 0x0f; op ] ~w:false ~pfx:[ pfx ] ~mm @@ -1609,7 +1624,7 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = value straight into the place's location, so filling a place and filling a temporary are already the same line. *) | Tast.Fill (ty, b) -> fill_value f dst ty b - | Tast.Sentinel ty -> sentinel_value f dst ty + | Tast.DeadBeef (ty, pat) -> dead_beef_value f dst ty pat | Tast.None_ -> zero_value f dst t (* Reading an uninitialised value gives whatever the slot held: stable garbage rather than LLVM's [poison]. The one construct where the two @@ -2144,24 +2159,48 @@ and fill_value f (dst : loc) (ty : Types.t) (b : Tast.expr) = rep_stosb f.b end -and sentinel_value f (dst : loc) (ty : Types.t) = +and dead_beef_value f (dst : loc) (ty : Types.t) (pat : Tast.expr) = let n = sizeof f.md ty in let words = n / 4 and tail = n mod 4 in - if words > 0 then begin - addr_into f ~reg:rdi dst; - movabs f.b ~dst:rax - (Int64.logand (Int64.of_int32 Emit.sentinel_word) 0xFFFFFFFFL); - movabs f.b ~dst:rcx (Int64.of_int words); - rep_stosd f.b - end; - List.iteri - (fun k byte -> - if k < tail then begin - movabs f.b ~dst:rax (Int64.of_int byte); - store_int f.b ~src:rax - ~mm:(lmem f (shift dst (words * 4 + k)) ~scratch:r11) ~size:1 - end) - Emit.sentinel_bytes + (* Whether the word is known here. A literal is reversed at compile time and + loaded as an immediate, which leaves the default's code exactly what it + was before the pattern became an operand; anything else is evaluated and + reversed by [bswap]. The evaluation happens first in both cases, because + it is an arbitrary expression that may call and a call clobbers [rdi]. *) + let folded = + match pat.Tast.e with + | Tast.Int (v, _) -> Some (Int64.to_int32 v) + | _ -> None + in + let load_word () = + match folded with + | Some v -> + movabs f.b ~dst:rax + (Int64.logand (Int64.of_int32 (Emit.word_of_pattern v)) 0xFFFFFFFFL) + | None -> + let l = eval f pat in + load_int f.b ~dst:rax ~mm:(lmem f l ~scratch:r11) ~size:4 ~signed:false; + bswap32 f.b ~dst:rax + in + if n > 0 then begin + load_word (); + if words > 0 then begin + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rcx (Int64.of_int words); + (* [rep stosd] touches rdi and rcx and leaves rax alone, which is what + lets the tail below keep reading the word out of it. *) + rep_stosd f.b + end; + (* The tail is the first up-to-three bytes of the stored word, lowest + address first — byte k is (word >> 8k) & 0xFF either way. A folded word + could store constants instead, but walking [rax] covers both with one + sequence, and the count is at most three. *) + for k = 0 to tail - 1 do + if k > 0 then shr_imm f.b ~dst:rax ~n:8; + store_int f.b ~src:rax + ~mm:(lmem f (shift dst (words * 4 + k)) ~scratch:r11) ~size:1 + done + end and zero_value f (dst : loc) (ty : Types.t) = if is_agg ty then zero_loc f dst (sizeof f.md ty) diff --git a/test/programs/fill.flan b/test/programs/fill.flan index 36cc5dc..2597f5b 100644 --- a/test/programs/fill.flan +++ b/test/programs/fill.flan @@ -1,16 +1,22 @@ -;;;; (filled b) and (sentinel-filled) — the two byte fills, read back as bytes. +;;;; (filled b) and (dead-beef), the two byte fills, read back as bytes. ;;;; ;;;; The whole point of this program is that every row is a *byte* and not a -;;;; value: the sentinel's contract is "a hex dump reads DEADBEEF", which is a -;;;; claim about which byte lands at which address, and only reading the bytes -;;;; back in address order can check it. The u32 rows are the same claim from -;;;; the other side — a little-endian load of DE AD BE EF is 0xEFBEADDE, which -;;;; is 4022250974, so a backend that wrote the word the other way round would -;;;; print 3735928559 here and be caught. +;;;; value: the pattern fill's contract is "a hex dump reads the pattern left +;;;; to right", which is a claim about which byte lands at which address, and +;;;; only reading the bytes back in address order can check it. The u32 rows +;;;; are the same claim from the other side — a little-endian load of +;;;; DE AD BE EF is 0xEFBEADDE, which is 4022250974, so a backend that wrote +;;;; the word the other way round would print 3735928559 here and be caught. ;;;; ;;;; The lengths are chosen for the tail. 8 is a whole number of patterns, 9 ;;;; ends on DE, 6 ends on DE AD, and 7 ends on DE AD BE — the three truncated ;;;; endings and the one that is not truncated at all. +;;;; +;;;; The rows that matter most are the last group: a pattern that is *not* a +;;;; literal, over a length that is not a multiple of four. That is the case a +;;;; constant-only implementation would pass by accident — the word reaches +;;;; the loop in a register and the tail bytes have to be shifted out of it +;;;; rather than folded, on both backends. (defstruct Words [a u32 b u32]) ;;;; A computed global initialiser: a fill is never a constant the linker can @@ -21,6 +27,19 @@ (defn bytes4 [b [4 u8]] () (dotimes [i 4] (print (at b i)) (print " "))) +;;;; A u32 that no literal rule can see through: (u32 0xBAADF00D) will not do, +;;;; because the cast checks its operand as an i32 first and 0xBAADF00D is not +;;;; one. A parameter's declared type is what makes the literal a u32, and it +;;;; is also what stops the checker folding it at the fill. +(defn u32of [x u32] u32 x) + +;;;; The pattern crosses a call boundary, so nothing can fold it at the fill. +(defn beef7 [pat u32] () + (let [a (array 7 u8)] + (set a (dead-beef pat)) + (dotimes [i 7] (print (at a i)) (print " ")) + (println ""))) + (defn main [] i32 ;; One byte, repeated. 0xFF is the -1 fill a debug allocator wants. (let [a (array 4 u8)] @@ -43,30 +62,48 @@ ;; Four bytes, ascending, with nothing truncated. (let [a (array 8 u8)] - (set a (sentinel-filled)) + (set a (dead-beef)) (dotimes [i 8] (print (at a i)) (print " ")) (println "")) ; 222 173 190 239 x2 + ;; The bare form is the spelled-out default, and this is the row that says + ;; so: the same eight bytes from (dead-beef 0xDEADBEEF). + (let [a (array 8 u8)] + (set a (dead-beef 0xDEADBEEF)) + (dotimes [i 8] (print (at a i)) (print " ")) + (println "")) ; identical to the row above + ;; The three truncated tails. (let [a (array 9 u8)] - (set a (sentinel-filled)) + (set a (dead-beef)) (dotimes [i 9] (print (at a i)) (print " ")) (println "")) ; ... ends on 222 (let [a (array 6 u8)] - (set a (sentinel-filled)) + (set a (dead-beef)) (dotimes [i 6] (print (at a i)) (print " ")) (println "")) ; ... ends on 222 173 (let [a (array 7 u8)] - (set a (sentinel-filled)) + (set a (dead-beef)) (dotimes [i 7] (print (at a i)) (print " ")) (println "")) ; ... ends on 222 173 190 + ;; A pattern of the program's own choosing, laid down left to right the same + ;; way: 0xBAADF00D is BA AD F0 0D, which is 186 173 240 13. + (let [a (array 6 u8)] + (set a (dead-beef 0xBAADF00D)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; 186 173 240 13 186 173 + ;; The same pattern read as words rather than as bytes, which is what says ;; which end the DE is at. (let [w (Words {})] - (set w (sentinel-filled)) + (set w (dead-beef)) + (print (.a w)) (print " ") (print (.b w)) (println "")) + + (let [w (Words {})] + (set w (dead-beef 0xBAADF00D)) (print (.a w)) (print " ") (print (.b w)) (println "")) ;; A nested aggregate: a fixed array of structs is plain data all the way @@ -78,11 +115,30 @@ ;; The globals. [gfill] was filled before main ran; [gsent] is filled here. (dotimes [i 4] (print (at gfill i)) (print " ")) (println "") ; 65 65 65 65 - (set gsent (sentinel-filled)) + (set gsent (dead-beef)) (dotimes [i 5] (print (at gsent i)) (print " ")) (println "") ; 222 173 190 239 222 ;; A fill in value position rather than as the value of a [set]: the same ;; bytes, reached through a temporary instead of through the place. (bytes4 (filled 0xFF)) (println "") ; 255 255 255 255 + + ;; ── The computed pattern ────────────────────────────────────────── + ;; None of these can be folded: the pattern is a parameter, or arithmetic + ;; the checker does not evaluate. Seven bytes is one whole pattern and a + ;; three-byte tail, so the tail bytes have to come out of the register the + ;; word is in rather than out of a constant. + (beef7 0xDEADBEEF) ; 222 173 190 239 222 173 190 + (beef7 0xBAADF00D) ; 186 173 240 13 186 173 240 + (let [p (u32of 0xBAADF00D) + a (array 6 u8)] + (set a (dead-beef p)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; 186 173 240 13 186 173 + ;; A pattern that is genuinely computed, not merely held in a slot, and read + ;; back as a word so the byte order of the runtime path is pinned too. + (let [p (u32of 0xBAAD0000) + w (Words {})] + (set w (dead-beef (bit-or p (u32of 0xF00D)))) + (print (.a w)) (print " ") (print (.b w)) (println "")) 0) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 2cdc9ad..74247bb 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -1256,20 +1256,42 @@ let () = rows are the same four bytes read back as a u32 — 0xEFBEADDE — and they are what a backend that wrote the word the other way round would fail. The three short rows are the truncated tails a size that is not a - multiple of four ends on. *) + multiple of four ends on. + + Rows four and five are identical on purpose: [(dead-beef)] and + [(dead-beef 0xDEADBEEF)] are the same node by construction, and this is + where that stops being a claim about the checker and becomes one about + the program. + + The last four rows are the ones a constant-only implementation would + pass by accident. Their pattern arrives in a register — a parameter, a + slot, and a [bit-or] the checker does not fold — so the word is + byte-reversed at run time and the tail bytes are shifted out of it + rather than folded. 186 173 240 13 is 0xBAADF00D read left to right, + and the closing 233876922 is that same pattern's stored word, which is + the runtime path's byte order pinned against the literal path's — the + identical number appears above it from a folded [(dead-beef + 0xBAADF00D)]. *) let fill_out = "255 255 255 255 \n7 7 7 7 \n0 0 0 0 \n\ + 222 173 190 239 222 173 190 239 \n\ 222 173 190 239 222 173 190 239 \n\ 222 173 190 239 222 173 190 239 222 \n\ 222 173 190 239 222 173 \n\ 222 173 190 239 222 173 190 \n\ - 4022250974 4022250974\n4294967295 4294967295\n\ - 65 65 65 65 \n222 173 190 239 222 \n255 255 255 255 \n" + 186 173 240 13 186 173 \n\ + 4022250974 4022250974\n233876922 233876922\n\ + 4294967295 4294967295\n\ + 65 65 65 65 \n222 173 190 239 222 \n255 255 255 255 \n\ + 222 173 190 239 222 173 190 \n\ + 186 173 240 13 186 173 240 \n\ + 186 173 240 13 186 173 \n\ + 233876922 233876922\n" in - outputs "byte and sentinel fills" "programs/fill.flan" fill_out; - outputs ~opt:"-O0" "byte and sentinel fills, -O0" "programs/fill.flan" + outputs "byte and dead-beef fills" "programs/fill.flan" fill_out; + outputs ~opt:"-O0" "byte and dead-beef fills, -O0" "programs/fill.flan" fill_out; - outputs ~x86:true "byte and sentinel fills, --x86" "programs/fill.flan" + outputs ~x86:true "byte and dead-beef fills, --x86" "programs/fill.flan" fill_out; (* dyn if: truthiness -- M2 queue item 7. A dyn scrutinee is tested for nil/false vs. everything else, Clojure's rule, on both backends; a diff --git a/test/test_flan.ml b/test/test_flan.ml index 0c7c57e..8fde29c 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -1957,7 +1957,7 @@ let () = ~needle:"steers every read of it"; (* ── What may be filled with raw bytes ───────────────────────────── - [(filled b)] and [(sentinel-filled)] are [zeroed]'s siblings, and the + [(filled b)] and [(dead-beef)] are [zeroed]'s siblings, and the boundary is the whole of what is new about them: zero is a value every type can have and 0xDE is not, so the checker says which types survive arbitrary bytes. The accepting side is programs/fill.flan; what is here @@ -1969,12 +1969,19 @@ let () = one is a root pointing at nothing. *) accepts "a fixed array of numbers may be filled" "(defn f [] () (let [a (array 4 u8)] (set a (filled 0xFF))))"; - accepts "a struct of numbers may be sentinel-filled" + accepts "a struct of numbers may be dead-beefed" "(defstruct S [a i32 b f64]) \ - (defn f [] () (let [s (S {})] (set s (sentinel-filled))))"; + (defn f [] () (let [s (S {})] (set s (dead-beef))))"; + (* Both arities, and a pattern that is not a literal — the operand is an + ordinary u32 expression, which is the byte arm's rule at four times the + width. *) + accepts "dead-beef takes a pattern" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 0xBAADF00D))))"; + accepts "dead-beef takes a computed pattern" + "(defn f [p u32] () (let [a (array 4 u8)] (set a (dead-beef p))))"; rejects_check "a struct holding a dyn cannot be filled" "(defstruct S [a i32 d dyn]) \ - (defn f [] () (let [s (S {})] (set s (sentinel-filled))))" + (defn f [] () (let [s (S {})] (set s (dead-beef))))" ~needle:"a root pointing at nothing"; rejects_check "a Vec cannot be filled" "(defn f [] () (let [v (vec-new i32)] (set v (filled 0xFF))))" @@ -2009,8 +2016,8 @@ let () = rejects_check "a fill in a position with no expected type" "(defn f [] () (print (filled 0xFF)))" ~needle:"needs to know the type it is filling"; - rejects_check "a sentinel fill in a position with no expected type" - "(defn f [] () (print (sentinel-filled)))" + rejects_check "a dead-beef in a position with no expected type" + "(defn f [] () (print (dead-beef)))" ~needle:"needs to know the type it is filling"; (* The byte is a u8 and the ordinary literal rule applies to it — there is no range check of this builtin's own, and there does not need to be. *) @@ -2020,9 +2027,16 @@ let () = rejects_check "filled takes exactly one byte" "(defn f [] () (let [a (array 4 u8)] (set a (filled))))" ~needle:"takes 1 argument"; - rejects_check "sentinel-filled takes no argument" - "(defn f [] () (let [a (array 4 u8)] (set a (sentinel-filled 1))))" - ~needle:"takes 0 arguments, given 1"; + rejects_check "dead-beef takes at most one pattern" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 1 2))))" + ~needle:"takes the pattern or nothing at all, given 2"; + (* The pattern is four bytes, so a wider literal is a typo rather than + something to truncate. The refusal is [in_range]'s, located at the + literal — this builtin has no range check of its own and does not need + one, exactly as the byte arm does not. *) + rejects_check "a dead-beef pattern out of u32 range" + "(defn f [] () (let [a (array 4 u8)] (set a (dead-beef 0x1DEADBEEF))))" + ~needle:"does not fit in u32"; (* A fill is never a value the linker can write into the image, so a defconst of one is refused by the constant rule rather than by anything of this feature's own. A defvar is fine: its initialiser runs at From 3c39354833d524f056e840475751b8a1e9efa1ab Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 18:46:58 +0700 Subject: [PATCH 4/6] Measure the Wire.Closed flake rather than assume it It exits 1 with no FAIL line, which is the shape an earlier lane wrote up. Two-of-two early failures looked like they might be this lane's, so: 4 in 6 here against 2 in 6 on a detached worktree at this branch's own base commit, running test_dev alone. Noise at that sample size, same exception, same mechanism. Adds one detail to the earlier note, which had only ever seen the flake on dev-trap-null-alloc: one of my six landed on dev-trap-free-all instead, so what is racy is trap_park and every row that calls it. --- FIX.org | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/FIX.org b/FIX.org index 09c4c91..8cf9138 100644 --- a/FIX.org +++ b/FIX.org @@ -1767,5 +1767,20 @@ expectation on purpose and watching all three report. Seventeen checker rows in ~test_flan.ml~: four accepting (both ~dead-beef~ arities and a computed pattern among them), and thirteen refusals covering the boundary, both arities, both no-expected-type positions, the byte's range, the pattern's -range and the ~defconst~ rule. Per the sweep policy the ~@x86~ and +range and the ~defconst~ rule. + +~dune test~ exits 1 on this branch roughly half the time, with *no FAIL line +anywhere* — the ~Flan.Wire.Closed~ flake an earlier lane wrote up further up +this file. Measured rather than assumed, because 2-of-2 early failures looked +like mine: running ~test_dev.exe~ alone (15 seconds, not the ten-minute +suite) gives 4 failures in 6 on this branch and *2 in 6 on a detached +worktree at this branch's own base commit*, 1526b6f, with none of this lane +in it. Same exception, same mechanism, and the difference is noise at that +sample size. + +One detail worth adding to that earlier writeup, which said both observed +failures landed on ~dev-trap-null-alloc~: it is not row-specific. Five of my +six failures were that row and the sixth was ~dev-trap-free-all~, so what is +racy is ~trap_park~ itself and every row that calls it, which is what the +mechanism described there would predict. Per the sweep policy the ~@x86~ and ~@sanitize~ sweeps were not run here. From 2240100ee980b5a5a61997d5a8011a2f8a7509b2 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 18:58:32 +0700 Subject: [PATCH 5/6] Review follow-ups: the default's payload, four wrong reasons, a doubled prefix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. The bare (dead-beef) built its default with Int64.of_int32, which sign-extends 0xDEADBEEF to -559038737 on a node tagged u32 — where the spelled-out literal arrives as 3735928559, because in_range admits it as the unsigned value it is. Masked to 32 bits, so the two spellings really do carry one payload; verified by diffing the emitted bodies of (dead-beef) and (dead-beef 0xDEADBEEF), which are now identical instruction for instruction. 2. The refusal's catch-all told a union and a function value that they 'carry a tag that names a case'. Neither does: env.unions is the untagged unions, and an Fn is a code address. Split into one arm per reason — union, Fn, enum, Option, data type — and each is now pinned, so they cannot quietly re-merge. Same correction in FIX.org's bullet. 3. js.ml prefixed its own message with 'js: ', which bin/main.ml prepends too, giving 'js: js: ...'. Dropped, and the message now names the builtin it refuses, which its comment already claimed it did. 4. FIX.org said x86.ml reads both pattern helpers out of Emit. It reads only word_of_pattern; the tail walks rax with shr. --- FIX.org | 35 +++++++++++++++++++++++++---------- lib/check.ml | 41 +++++++++++++++++++++++++++++++++++++---- lib/js.ml | 14 +++++++++++--- test/test_flan.ml | 19 +++++++++++++++++++ 4 files changed, 92 insertions(+), 17 deletions(-) diff --git a/FIX.org b/FIX.org index 8cf9138..65916b0 100644 --- a/FIX.org +++ b/FIX.org @@ -1689,9 +1689,18 @@ this rule exists and ~zeroed~ needs none: one and true on the other. Byte-identical behaviour across the two backends is the property this feature is pinned on, so the divergence is refused rather than documented. -- *enum, data type, union, Option, function value* — each carries a tag or a - case index something later reads as a small number with a meaning, and no - byte pattern names a real case. +- *a data type* — a tag that names a case, and no byte pattern names a real + one. *An ~(Option T)~* — the same, one bit of it: a filled tag says the + value is there over a payload nobody wrote. *An enum* — its values are the + members it declared, and no byte pattern is one of them. +- *a union* — and this one is not about a tag, because ~env.unions~ is "the + untagged unions". It is that a union's members overlay and ~unfillable~ + walks a struct's fields rather than a union's members, so nothing has + shown every member is plain data; a member that is not would be filled + through the one that is. Relaxable by walking the members, if anyone wants + it. +- *a function value* — a code address, and a call through a filled one jumps + into whatever the pattern happens to address. Floats are in: every bit pattern is a float, NaNs included, and both backends move one as bytes. @@ -1710,10 +1719,15 @@ both arities. On a little-endian machine the word a 4-byte store must therefore leave is the *byte reversal* of the pattern, which is all ~Emit.word_of_pattern~ is (~bytes_of_pattern~ beside it is the ascending list). Those two are the one -place the order is written; x86.ml reads them out of Emit rather than -repeating them, and ~Tast.dead_beef_default~ is the one place 0xDEADBEEF is -written. Three definitions, no duplicates, so the default and the -parameterised case cannot drift. +place the order is written, and ~Tast.dead_beef_default~ is the one place +0xDEADBEEF is written, so the default and the parameterised case cannot +drift. + +x86.ml reads ~word_of_pattern~ out of Emit rather than repeating it. It does +*not* use ~bytes_of_pattern~: its tail walks the bytes out of ~rax~ with +~shr~, which is the same arithmetic the list encodes and is how the computed +path has to do it anyway, so there was no second constant to share. emit.ml +uses both — the list for a folded tail, the word for the loop. A literal pattern is reversed at compile time and reaches the loop as an immediate — the default's generated code is exactly what it was before the @@ -1765,9 +1779,10 @@ byte-identical, with a fourth build (~--dev~) added at the rename: four-way identical. The three rows were confirmed to actually run, by breaking one expectation on purpose and watching all three report. Seventeen checker rows in ~test_flan.ml~: four accepting (both ~dead-beef~ arities and a computed -pattern among them), and thirteen refusals covering the boundary, both -arities, both no-expected-type positions, the byte's range, the pattern's -range and the ~defconst~ rule. +pattern among them), and seventeen refusals covering the boundary — one per +reason, since review found the tagged types were sharing a line that was +false for two of them — both arities, both no-expected-type positions, the +byte's range, the pattern's range and the ~defconst~ rule. ~dune test~ exits 1 on this branch roughly half the time, with *no FAIL line anywhere* — the ~Flan.Wire.Closed~ flake an earlier lane wrote up further up diff --git a/lib/check.ml b/lib/check.ml index f110497..e606749 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -4921,8 +4921,29 @@ and named_call ctx ~want loc name args = "a bool is an i1 to LLVM and a whole byte to the x86 backend, \ so a filled one would not even agree with itself across the \ two" - | _ -> "it carries a tag that names a case, and no byte pattern \ - names a real one") + | Types.Option _ -> + "it carries a tag saying whether the value is there, and a \ + filled one says yes over a payload nobody wrote" + | Types.Fn _ -> + "it is a code address, and a call through a filled one jumps \ + into whatever 0xDE bytes happen to address" + | Types.Named n when Hashtbl.mem ctx.env.datas n -> + "it carries a tag that names a case, and no byte pattern \ + names a real one" + | Types.Named n when Hashtbl.mem ctx.env.unions n -> + (* Untagged, per [env.unions]'s own note — so the reason is + not a tag. It is that a union's members overlay, and this + rule walks a struct's fields rather than a union's members: + nothing here has shown they are all plain data, and a + member that is not would be filled through the one that + is. *) + "a union's members overlay, and this rule does not walk them \ + — so nothing here has shown that every member is plain data" + | Types.Enum _ -> + "an enum's values are the members it declared, and no byte \ + pattern is one of them" + | _ -> + "it is not one of the types this rule admits") | None -> ()); if is_byte then let b = check ctx ~want:(Types.Int Types.U8) (List.hd args) in @@ -4934,10 +4955,22 @@ and named_call ctx ~want loc name args = (* The bare form, written out. Not a default a backend applies: the node that leaves here is the one the spelled-out call would have left, which is what makes the equivalence a fact about the - IR rather than a promise two emitters keep separately. *) + IR rather than a promise two emitters keep separately. + + Masked to 32 bits, and that is the whole of why this is not + [Int64.of_int32] on its own: [dead_beef_default] is an [int32] + whose top bit is set, so widening it signed would put + -559038737 on a node tagged [u32] — where the same pattern + *written out* arrives as 3735928559, because [in_range] admits + it as the unsigned value it is. Two spellings of one builtin + would then carry two different payloads, and "the same node by + construction" would be false for anything that reads one. *) | _ -> mk loc (Types.Int Types.U32) - (Tast.Int (Int64.of_int32 Tast.dead_beef_default, Types.U32)) + (Tast.Int + (Int64.logand + (Int64.of_int32 Tast.dead_beef_default) 0xFFFFFFFFL, + Types.U32)) in mk loc ty (Tast.DeadBeef (ty, pat)) | _ -> diff --git a/lib/js.ml b/lib/js.ml index ba7fc4c..62e664f 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -673,11 +673,19 @@ let rec value f (e : Tast.expr) : string = (* A fill is a byte pattern written over storage, and this backend has no storage to write over: a Flan struct is a JS object here, not a run of bytes, so there is nothing for 0xFF or DEADBEEF to mean. Refused by name - rather than approximated, which is this file's rule. *) + rather than approximated, which is this file's rule. + + No "js: " on the front — [bin/main.ml] puts it there when it prints an + [Unsupported], and every other refusal in this file leaves it to do + that. *) | Tast.Fill _ | Tast.DeadBeef _ -> unsupported - "js: a byte fill has no meaning on this backend — a struct is an \ - object here and not a run of bytes" + "%s writes a byte pattern over storage, and the JS dialect has no \ + storage to write it over — a struct is an object here, not a run of \ + bytes" + (match e.Tast.e with + | Tast.Fill _ -> "(filled b)" + | _ -> "(dead-beef)") (* [uninit] is the opt-out from zeroing. There is no uninitialised memory here to opt out of, so it is the zero — which is more than the program asked for and never less. *) diff --git a/test/test_flan.ml b/test/test_flan.ml index 8fde29c..0ea4236 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -2001,6 +2001,25 @@ let () = rejects_check "a bool cannot be filled" "(defn f [] () (let [b false] (set b (filled 0xFF))))" ~needle:"would not even agree with itself"; + (* Each of the tagged and address-carrying types names its own reason. They + shared one "it carries a tag that names a case" line until review caught + that it was false for two of them — a union is untagged (env.unions is + "the untagged unions") and a function value is a code pointer, not a + tag. Pinned per type so the reasons cannot quietly re-merge. *) + rejects_check "a union cannot be filled, and not because of a tag" + "(defunion U [a i32 b f64]) \ + (defn f [] () (let [u (U {})] (set u (dead-beef))))" + ~needle:"a union's members overlay"; + rejects_check "a function value cannot be filled" + "(defn g [] ()) (defn f [] () (let [h g] (set h (dead-beef))))" + ~needle:"it is a code address"; + rejects_check "an enum cannot be filled" + "(defenum K [lo 0 hi 1]) \ + (defn f [k K] () (let [e k] (set e (dead-beef))))" + ~needle:"the members it declared"; + rejects_check "an Option cannot be filled" + "(defn f [] () (let [o (Some 1)] (set o (dead-beef))))" + ~needle:"whether the value is there"; (* A data type's tag is a case index, and no byte pattern names a real case. The type itself is what the message names, because a data type overlays its cases. *) From 2afa554cced1a91d60451a55dc72deb19ea4b291 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 20 Sep 2026 19:02:53 +0700 Subject: [PATCH 6/6] The flake note said noise; the full-suite numbers say otherwise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Isolated test_dev is 4-in-6 here against 2-in-6 at the base, which is noise. The full suite is 5-in-5 here against 0-in-5 at the base, which is not — and with this lane's three acceptance rows disabled it drops to 1-in-3. The rows add compile jobs to the pool test_dev runs alongside, and a busier machine loses the trap_park poll race more often. Still not a new defect, and none of this lane's compiler code is implicated. But the earlier note's suggested fix is now worth doing rather than noting, and saying 'noise' would have sent the next reader the wrong way. --- FIX.org | 42 ++++++++++++++++++++++++++++++------------ 1 file changed, 30 insertions(+), 12 deletions(-) diff --git a/FIX.org b/FIX.org index 65916b0..599a708 100644 --- a/FIX.org +++ b/FIX.org @@ -1784,18 +1784,36 @@ reason, since review found the tagged types were sharing a line that was false for two of them — both arities, both no-expected-type positions, the byte's range, the pattern's range and the ~defconst~ rule. -~dune test~ exits 1 on this branch roughly half the time, with *no FAIL line +~dune test~ exits 1 on this branch about half the time, with *no FAIL line anywhere* — the ~Flan.Wire.Closed~ flake an earlier lane wrote up further up -this file. Measured rather than assumed, because 2-of-2 early failures looked -like mine: running ~test_dev.exe~ alone (15 seconds, not the ten-minute -suite) gives 4 failures in 6 on this branch and *2 in 6 on a detached -worktree at this branch's own base commit*, 1526b6f, with none of this lane -in it. Same exception, same mechanism, and the difference is noise at that -sample size. +this file. Green runs are real (2 of the last 4 exit 0); the rest are that +race. -One detail worth adding to that earlier writeup, which said both observed -failures landed on ~dev-trap-null-alloc~: it is not row-specific. Five of my -six failures were that row and the sixth was ~dev-trap-free-all~, so what is -racy is ~trap_park~ itself and every row that calls it, which is what the -mechanism described there would predict. Per the sweep policy the ~@x86~ and +*This lane makes it fire more often, and that is worth saying plainly rather +than filing the whole thing under "known flake".* Measured, because early +runs looked like the lane had broken something: + +| what | full ~dune test~ | +| base commit 1526b6f, none of the lane | 0 failures in 5 | +| this lane | 5 failures in 5 | +| this lane, my 3 acceptance rows off | 1 failure in 3 | + +Isolated, ~test_dev.exe~ alone (15 seconds, not the ten-minute suite) gives 4 +in 6 here against 2 in 6 at the base — much closer, which is the shape you +would expect if the lane is not touching the racy code but *is* changing the +load around it. The three acceptance rows add three compile-and-run jobs to +the pool that ~test_dev~ runs alongside, and a busier machine is slower to +answer the poll that races. + +So: not a new defect, and nothing in ~check.ml~/~emit.ml~/~x86.ml~ here is +implicated — but the next lane to add acceptance rows will push the rate up +again, and the fix the earlier writeup already named (catch ~Closed~ in +~trap_park~'s poll and read it as the program having ended) is now worth +doing rather than noting. + +One detail to add to that earlier writeup, which had only seen the flake on +~dev-trap-null-alloc~: it is not row-specific. Five of my six isolated +failures were that row and the sixth was ~dev-trap-free-all~, so what is racy +is ~trap_park~ itself and every row that calls it — which is exactly what the +mechanism described there predicts. Per the sweep policy the ~@x86~ and ~@sanitize~ sweeps were not run here.