diff --git a/DISCUSS.org b/DISCUSS.org index 402c35a..34e7e2f 100644 --- a/DISCUSS.org +++ b/DISCUSS.org @@ -183,7 +183,23 @@ why MSVC/glibc-style debug allocators use a single repeated byte instead LocalAlloc uninit), 0xFEEEFEEE (Windows HeapFree'd), 0xDEADC0DE, 0xC0FFEE, 0x8BADF00D (Apple watchdog-timeout crash code) -Not designed or implemented, just an idea. +Built, 2026-09-20, as two builtins rather than one — the author's answer to +the cheap-byte-or-real-pattern question was "why not both? we need some sort +of memset -1 right? and dead-beef can loop, that's fine". They are +(filled BYTE) and (sentinel-filled), spelled the way [zeroed] is: the value +of whatever type is expected of them, so (set grid (filled 0xFF)) is how a +place is filled and there is no place-taking form to learn beside [set]. + +The snag above is why there are two and not one with a wider operand. The +byte fill is one llvm.memset / one rep stosb; the 4-byte pattern is a loop on +both sides, a counted dword loop in emit.ml and rep stosd in x86.ml, because +the intrinsic really does only take a repeated i8. + +What may be filled is numbers, and structs and fixed arrays built out of +numbers — nothing that carries a tag, a length, an owning pointer or a +collector descriptor. That boundary, and why each refusal is the runtime's +rather than a matter of taste, is written up in FIX.org, "The two byte +fills". ** println output goes to *flan-output*, not inline in the repl Deliberate per emacs/flan-repl.el:40-44: "a value and the program's output diff --git a/FIX.org b/FIX.org index b1fbe41..f2da3b4 100644 --- a/FIX.org +++ b/FIX.org @@ -1611,3 +1611,107 @@ because the code is the same. Not fixed here, deliberately: the fix is to catch ~Closed~ in that poll and read it as the program having ended, which is a claim about what those rows mean and belongs to whoever owns them. Flagged rather than patched. + +* The two byte fills, 2026-09-20 +DISCUSS.org's "a DEADBEEF-style sentinel-fill builtin", built. The author's +answer to the single-byte-or-four-byte question was "why not both? we need +some sort of memset -1 right? and dead-beef can loop, that's fine", so there +are two builtins and they are siblings of ~zeroed~, not a new shape. + +** The spellings +~(filled BYTE)~ and ~(sentinel-filled)~, both value forms driven by the type +expected of them, exactly as ~(zeroed)~ is: + +: (set grid (filled 0xFF)) +: (set frame (sentinel-filled)) + +A place-taking ~(filled place byte)~ was the other candidate and was not +taken. ~zeroed~ already answers "the all-bytes-X value of whatever this is +being stored into", ~set~ already takes the place, and a second spelling for +an operation ~set~ expresses would have been a second thing to learn for +nothing. The cost is real and is paid on purpose: a fill in a position that +expects no type is refused ("filled needs to know the type it is filling"), +which is ~zeroed~'s own refusal worn by both siblings. + +~sentinel-filled~ takes no operand. The pattern is fixed — that is the whole +point of it, since a hex dump only reads DEADBEEF if nobody can change it — +and a parameterised one would be a different builtin. + +** The fill boundary — what may be overwritten with raw bytes +*Numbers, and structs and fixed arrays built out of numbers. Nothing else.* +~Check.unfillable~ is the rule, in one recursive walk, and every refusal +names the type it stopped at and why. + +Zero is a value every type can have; 0xDE is not. That is the whole of why +this rule exists and ~zeroed~ needs none: + +- *dyn* — a struct holding a dyn is rooted on the collector's root stack with + a descriptor naming that word's byte offset. A filled one is a root + pointing at nothing and the next collection follows it. This is the refusal + the feature could not ship without. +- *Vec, Map, Allocator* — an owning header: pointer, length, capacity, + allocator. A filled one frees a wild address the first time it is touched. +- *string, slice* — a pointer and a length that every bounds check believes. +- *Ptr* — not walked by the collector, and a poisoned pointer is arguably the + useful case. Kept out anyway so the rule is one sentence rather than "plain + data, except one kind of address". *This is the arm to relax first if the + question is reopened.* +- *bool* — the one refusal that is about the backends rather than the + runtime. A bool is a byte in memory and an ~i1~ to LLVM, which reads the + low bit, where x86 compares the whole byte against zero: 0xDE is false on + one and true on the other. Byte-identical behaviour across the two backends + is the property this feature is pinned on, so the divergence is refused + rather than documented. +- *enum, data type, union, Option, function value* — each carries a tag or a + case index something later reads as a small number with a meaning, and no + byte pattern names a real case. + +Floats are in: every bit pattern is a float, NaNs included, and both backends +move one as bytes. + +A ~defconst~ of a fill is refused by the existing constant rule and not by +anything of this feature's own — a fill is never a value the linker can write +into the image. A ~defvar~ is fine and goes through the startup function on +both backends, which ~programs/fill.flan~ pins. + +** The byte order, which is the specification +DE AD BE EF in *ascending address order*, so ~xxd~ reads "deadbeef". A +little-endian store of those four bytes is the i32 0xEFBEADDE, and +~Emit.sentinel_bytes~ / ~Emit.sentinel_word~ are the one place either is +written — x86.ml reads both out of Emit rather than repeating them, so the +two backends cannot drift. + +*Tail behaviour.* A size that is not a multiple of four ends on a prefix of +the pattern: 1 byte over is DE, 2 is DE AD, 3 is DE AD BE. Both backends +write the tail byte by byte from the same list. ~programs/fill.flan~ has all +four lengths — 8, 9, 6 and 7. + +** The backends +- *LLVM (emit.ml).* The byte fill is one ~llvm.memset~ with the byte as an + operand instead of a zero — the same call the existing bulk zero makes, and + the reason the single-byte fill is the cheap one. The sentinel cannot be a + memset at all (the intrinsic takes one repeated i8, which is the snag + DISCUSS.org named), so it is a counted loop over dwords in the + header/body/exit shape ~emit_while~ writes, with the counter as an + entry-block alloca that ~mem2reg~ promotes. Every store is ~align 1~, + because a ~[7 u8]~ array is a legal thing to fill. +- *x86 (x86.ml).* ~rep stosb~ for the byte fill — ~zero_loc~'s three + registers with the program's byte in ~al~ instead of a zero — and ~rep + stosd~ (new, 0xf3 0xab) for the sentinel, with 0xEFBEADDE in ~eax~. The + byte operand is evaluated *before* ~rdi~ is loaded, because evaluating it + may call and a call clobbers ~rdi~. +- The one asymmetry: ~emit.ml~ needs a ~Tast.Set~ arm of its own to fill the + place rather than a temporary, because its value path returns an SSA value. + ~x86.ml~ needs none — a ~Set~ there already lowers its value into the + place's location, so filling a place and filling a temporary are the same + line. +- *js.ml* refuses both by name. A struct is an object there, not a run of + bytes, so there is nothing for 0xFF to mean. + +** What was run +~dune test --root .~ green (exit 0, no FAIL lines). Three acceptance rows +over ~test/programs/fill.flan~ — default, ~-O0~ and ~--x86~ — and the three +outputs diffed against each other by hand before the rows were written: +byte-identical. Eleven checker rows in ~test_flan.ml~ for the boundary and +the two arity/position refusals. Per the sweep policy the ~@x86~ and +~@sanitize~ sweeps were not run here. diff --git a/lib/check.ml b/lib/check.ml index 179bb35..fc2bc77 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -605,6 +605,64 @@ let rec no_zeroed_fn loc what (t : Types.t) = | Types.Array (_, e) -> no_zeroed_fn loc what e | _ -> () +(* ── What may be overwritten with raw bytes ──────────────────────────── + + [(filled b)] and [(sentinel-filled)] are the only two things in the + language that write a byte pattern over storage the type system has an + opinion about, so the question they raise is which types survive having + arbitrary bytes put in them. The answer here is the narrow one: numbers, + and aggregates built out of numbers. Everything else is refused by name. + + What is being kept out, and why each one is not a matter of taste: + + - [dyn]. A struct that holds a dyn is rooted on the collector's root stack + with a descriptor naming the byte offsets of its dyn words (see the + per-type descriptor note at the bottom of this file). Filling one leaves + a word that is not a dyn at an offset the collector is told to walk, and + the next collection follows it. The refusal is what keeps that from + being reachable at all. + - [Vec], [(Map K V)], [Allocator]. An owning header: a pointer, a length, a + capacity and an allocator the runtime frees through. A filled one is a + free of a wild pointer the first time it is touched. + - [string] and a slice. Two words, the second of which is a length every + bounds check believes. A filled length is a bounds check that passes and + an access that does not. + - [Ptr]. Not walked by the collector, and a poisoned pointer is arguably + the useful case — but it is still a value every [deref] in the language + trusts, and admitting it would make the rule "plain data, except one + kind of address". Kept out so the rule is one sentence. This is the arm + to relax first if the question is reopened. + - [bool]. The one refusal that is about the backends rather than the + runtime: a bool is a byte here and an [i1] to LLVM, which reads the low + bit, where x86 compares the whole byte against zero. 0xDE is false on + one and true on the other, and byte-identical behaviour across the two + backends is the property this feature is pinned on. + - an enum, a data type, a union, an [(Option T)], a function value. Each + carries a tag or a case index that something later reads as a small + number with a meaning, and a filled one names a case that does not + exist. + + Floats are in: every bit pattern is a float, NaNs included, and both + backends move one as bytes. *) +let rec unfillable env seen (t : Types.t) : Types.t option = + match t with + | Types.Int _ | Types.Float _ -> None + | Types.Array (_, e) -> unfillable env seen e + | Types.Named n when not (List.mem n seen) -> + (match Hashtbl.find_opt env.structs n with + | Some s -> + List.fold_left + (fun acc (fl : Tast.field) -> + match acc with + | Some _ -> acc + | None -> unfillable env (n :: seen) fl.Tast.fty) + None s.Tast.fields + (* A data type or a union, which are the two [Named] things that are not + in [structs]. Both overlay their members, so the type itself is what + the refusal names. *) + | None -> Some t) + | _ -> Some t + let rec resolve env ?(seen = []) (t : Ast.texpr) : Types.t = let loc = t.Ast.tloc in match t.Ast.t with @@ -4812,6 +4870,54 @@ and named_call ctx ~want loc name args = "zeroed needs to know the type it is zeroing — use it where one is \ expected, as in (set grid (zeroed))") + (* [zeroed]'s two siblings, and the same shape exactly: a value of whatever + type is expected of it, so [(set grid (filled 0xFF))] is how a place is + filled and there is no second spelling to learn. What they add over + [zeroed] is the byte — [(filled b)] repeats one the program picks, and + [(sentinel-filled)] repeats the four bytes DE AD BE EF — and with it the + question [zeroed] never has to ask: zero is a value every type can have, + and 0xDE is not. [unfillable] above is the whole of the answer. *) + | "filled" | "sentinel-filled" -> + arity loc name (if String.equal name "filled" then 1 else 0) args; + (match want with + | Some ty when ty <> Types.Never -> + (match unfillable ctx.env [] ty with + | Some bad -> + Loc.failk "check/fill-not-plain-data" loc + "%s writes raw bytes over %s, and %s is not plain data — %s. \ + Fill only numbers, and structs and fixed arrays built out of \ + them" + name (Types.to_string ty) + (if Types.equal bad ty then "it" else Types.to_string bad) + (match bad with + | Types.Dyn -> + "a dyn is one word the collector walks by descriptor, and a \ + filled one is a root pointing at nothing" + | Types.Vec _ | Types.Map _ | Types.Alloc -> + "it owns its storage through a pointer and an allocator, and \ + a filled header frees a wild address" + | Types.String | Types.Slice _ -> + "it is a pointer and a length every bounds check believes" + | Types.Ptr _ -> + "it is an address every deref trusts" + | Types.Bool -> + "a bool is an i1 to LLVM and a whole byte to the x86 backend, \ + so a filled one would not even agree with itself across the \ + two" + | _ -> "it carries a tag that names a case, and no byte pattern \ + names a real one") + | None -> ()); + if String.equal name "filled" then + let b = check ctx ~want:(Types.Int Types.U8) (List.hd args) in + mk loc ty (Tast.Fill (ty, b)) + else mk loc ty (Tast.Sentinel ty) + | _ -> + fail loc + "%s needs to know the type it is filling — use it where one is \ + expected, as in (set grid (%s))" + name + (if String.equal name "filled" then "filled 0xFF" else name)) + (* The one half of a destructuring [let] that [Parse] cannot do on its own. Everything else about a pattern is bindings and field accesses it already wrote; the arity is a *type* question — how many elements the value has — @@ -6705,6 +6811,14 @@ let builtins : (string * string * string) list = ("zeroed", "zeroed [] T", "The all-bytes-zero value of whatever it is being stored into, so it \ only means anything where a type is expected of it."); + ("filled", "filled [u8] T", + "Every byte of whatever it is being stored into set to one byte, as in \ + (set grid (filled 0xFF)). Numbers only, and structs and fixed arrays \ + built out of them."); + ("sentinel-filled", "sentinel-filled [] T", + "The bytes DE AD BE EF repeating over whatever it is being stored into, \ + so a hex dump reads DEADBEEF. A size that is not a multiple of four \ + ends on a prefix of the pattern. Same types [filled] takes."); ("destructure~nth", "destructure~nth [[n T] i32 i32 i32] T", "Written by the compiler for a destructuring let, and unspellable: the \ reader makes ~ a delimiter, so no source symbol can name this."); diff --git a/lib/emit.ml b/lib/emit.ml index ca4d8a5..ae58b8d 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -937,6 +937,30 @@ let emit_bulk_zero f ptr ty = end else false | _ -> false +(* ── The sentinel pattern ────────────────────────────────────────────── + + DE AD BE EF, in ascending address order, so [xxd] over the filled storage + reads "deadbeef" and not "efbeadde". That is the whole reason the builtin + exists, so the byte order is the specification and not an implementation + detail: the list below is the order the bytes land in memory, and the i32 + beside it is the little-endian word that puts them there. Both backends + write the same four bytes — x86 through [rep stosd] with the same word in + [eax], this file through a store of the same constant — and the acceptance + program reads the bytes back by index to keep them honest. + + A run whose size is not a multiple of four ends on a prefix: DE, then DE + AD, then DE AD BE. The tail is written byte by byte, from the same list, so + there is one source for the order. *) +let sentinel_bytes = [ 0xDE; 0xAD; 0xBE; 0xEF ] + +(* The four bytes as the i32 a little-endian store of them would leave, folded + from the list rather than written out, so the two cannot drift apart. *) +let sentinel_word = + List.fold_left + (fun acc b -> Int32.logor (Int32.shift_right_logical acc 8) + (Int32.shift_left (Int32.of_int b) 24)) + 0l sentinel_bytes + let term f fmt = Printf.ksprintf (fun s -> @@ -948,6 +972,7 @@ let label f name = Buffer.add_string f.b (Printf.sprintf "\n%s:\n" name); f.live <- true + (* Every [ret] in a function body goes through here, which is the whole of how the shadow stack's pop is got right. There are five of them — an explicit [return] with a value and without, the [none] arm of [(some x)], the tail of @@ -1156,6 +1181,64 @@ let alloca_raw f lltype = Buffer.add_string f.allocas (Printf.sprintf " %s = alloca %s\n" name lltype); name +(* [(filled b)] over storage this file already has the address of. One + [llvm.memset] with the byte in a register, which is exactly what the + intrinsic is for and the reason the single-byte fill is the cheap one: the + same call the zero fill above makes, with an operand instead of a zero. *) +let emit_byte_fill f ptr ty (byte : string) = + let size, align = lay f.md ty in + if size > 0 then + ins f + "call void @llvm.memset.p0.i64(ptr align %d %s, i8 %s, i64 %d, i1 false)" + align ptr byte size + +(* [(sentinel-filled)] over the same. [llvm.memset] takes one repeated i8 and + nothing else, so a four-byte pattern cannot be a call and has to be a loop + — the snag DISCUSS.org named before this was built, and the reason the two + builtins are two and not one with a wider operand. + + The counter is an entry-block alloca, which is how every local in this file + is spelled and what [mem2reg] promotes; the loop is the same + header/body/exit shape [emit_while] writes. The size is a compile-time + constant, so the trip count is one too and LLVM unrolls what it wants to. + + Every store is [align 1]: the pattern is laid down over bytes, and the + storage's own alignment may be 1 — a [7 u8] array is a legal thing to fill. + The tail's up-to-three bytes are written out straight, from the same list + the word was folded from. *) +let emit_sentinel f ptr ty = + let size, _ = lay f.md ty in + let words = size / 4 and tail = size mod 4 in + if words > 0 then begin + let i = alloca_raw f "i64" in + let lc = fresh_label f "fill" and lb = fresh_label f "fillbody" + and le = fresh_label f "fillend" in + ins f "store i64 0, ptr %s" i; + term f "br label %%%s" lc; + label f lc; + let c = fresh f in + ins f "%s = load i64, ptr %s" c i; + let t = fresh f in + ins f "%s = icmp ult i64 %s, %d" t c words; + term f "br i1 %s, label %%%s, label %%%s" t lb le; + label f lb; + let off = fresh f in + ins f "%s = mul i64 %s, 4" off c; + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %s" p ptr off; + ins f "store i32 %ld, ptr %s, align 1" sentinel_word p; + let n = fresh f in + ins f "%s = add i64 %s, 1" n c; + ins f "store i64 %s, ptr %s" n i; + term f "br label %%%s" lc; + label f le + end; + for k = 0 to tail - 1 do + let p = fresh f in + ins f "%s = getelementptr i8, ptr %s, i64 %d" p ptr (words * 4 + k); + ins f "store i8 %d, ptr %s, align 1" (List.nth sentinel_bytes k) p + done + (* ── Constants ─────────────────────────────────────────────────────── *) (* LLVM's hex form is exact, which decimal is not: a literal must mean the same @@ -1611,6 +1694,22 @@ and value_at f (e : Tast.expr) : string = | Tast.Str s -> string_const f.md s | Tast.Unit | Tast.Zero _ | Tast.None_ -> "zeroinitializer" | Tast.Uninit _ -> "poison" + (* A fill is a write over storage, so in value position it needs storage to + write over: a temporary, filled and then loaded out of. Every fill the + source actually writes is the value of a [set] or of a [let] binding, and + [Set] below takes the place's own address and skips this; the temporary + is what makes the node mean something everywhere else — a fill passed + straight to a call, say — rather than being a form with a position rule + nobody stated. *) + | Tast.Fill (ty, b) -> + let bv = value f b in + let tmp = alloca f ty in + emit_byte_fill f tmp ty bv; + load f tmp ty + | Tast.Sentinel ty -> + let tmp = alloca f ty in + emit_sentinel f tmp ty; + load f tmp ty | Tast.Local _ | Tast.Global _ | Tast.Field _ | Tast.Deref _ -> (* Everything that denotes a location is a load from its address. *) load f (addr f e) e.Tast.ty @@ -1665,6 +1764,12 @@ and value_at f (e : Tast.expr) : string = let ptr, ty = place f p in (match v.Tast.e with | Tast.Zero _ when emit_bulk_zero f ptr ty -> () + (* The shape the source writes: the fill goes straight at the place, + with no temporary and no aggregate load in between. Same short-circuit + the bulk zero above takes, and the reason [(set grid (filled 0xFF))] + is one memset. *) + | Tast.Fill (_, b) -> emit_byte_fill f ptr ty (value f b) + | Tast.Sentinel _ -> emit_sentinel f ptr ty | _ -> let v' = value f v in ins f "store %s %s, ptr %s" (ll ty) v' ptr); diff --git a/lib/js.ml b/lib/js.ml index 80184cc..e785f53 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -670,6 +670,14 @@ let rec value f (e : Tast.expr) : string = n | Tast.Unit -> "undefined" | Tast.Zero t -> zero f.md e.Tast.loc t + (* A fill is a byte pattern written over storage, and this backend has no + storage to write over: a Flan struct is a JS object here, not a run of + bytes, so there is nothing for 0xFF or DEADBEEF to mean. Refused by name + rather than approximated, which is this file's rule. *) + | Tast.Fill _ | Tast.Sentinel _ -> + unsupported + "js: a byte fill has no meaning on this backend — a struct is an \ + object here and not a run of bytes" (* [uninit] is the opt-out from zeroing. There is no uninitialised memory here to opt out of, so it is the zero — which is more than the program asked for and never less. *) diff --git a/lib/tast.ml b/lib/tast.ml index 6824a97..9b09c0f 100644 --- a/lib/tast.ml +++ b/lib/tast.ml @@ -76,6 +76,19 @@ and expr_kind = | Unit | Zero of Types.t (* ZII: all-bytes-zero of this type *) | Uninit of Types.t (* the explicit opt-out *) + (* [(filled b)]: every byte of this type set to [b]. [Zero] with a byte the + program chooses, and a separate node rather than a byte on [Zero] because + that byte is an *expression* — a memset's operand, not a literal — and + every reader of [Zero] treats it as a leaf with nothing under it. + + [(sentinel-filled)]: the four bytes DE AD BE EF repeating, ascending + through the storage, so a hex dump reads DEADBEEF. It carries no operand + because the pattern is fixed; that is the whole point of it, and a + parameterised one would be a different builtin. A size that is not a + multiple of four ends on a prefix of the pattern — DE, DE AD, DE AD BE — + which both backends produce identically. *) + | Fill of Types.t * expr + | Sentinel of Types.t | Local of int (* slot index into the frame *) | Global of string | Prim of prim * expr list @@ -360,7 +373,8 @@ let rec walk (f : expr -> unit) (e : expr) = let gos = List.iter go in match e.e with | Int _ | Float _ | Bool _ | Str _ | Unit | Zero _ | Uninit _ | Local _ - | Global _ | None_ | FnAddr _ | Break _ | Continue _ -> () + | Global _ | None_ | FnAddr _ | Break _ | Continue _ | Sentinel _ -> () + | Fill (_, b) -> go b | Prim (_, es) | Call (_, es) | Do es | Make (_, es) | MakeCase (_, _, es) | Arr es | InvokeRestart (_, _, es, _, _, _) -> gos es | CallPtr (c, es) -> go c; gos es diff --git a/lib/x86.ml b/lib/x86.ml index 096382f..bb0798e 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -401,6 +401,11 @@ let push_r b r = if r >= 8 then u8 b 0x41; u8 b (0x50 lor (r land 7)) requires. *) let rep_movsb b = u8 b 0xf3; u8 b 0xa4 let rep_stosb b = u8 b 0xf3; u8 b 0xaa +(* [rep stosd]: [ecx] copies of [eax] at [rdi], four bytes at a time. The + sentinel fill's loop, in one instruction — DISCUSS.org's "a fill loop, not + a memset" is about LLVM's intrinsic taking one repeated byte, and the + string instruction here has no such limit. *) +let rep_stosd b = u8 b 0xf3; u8 b 0xab (* ── SSE ─────────────────────────────────────────────────────────────── *) @@ -1600,6 +1605,11 @@ and lower_at f (e : Tast.expr) (dst : loc) : unit = store_int f.b ~src:rax ~mm:(lmem f (shift dst 8) ~scratch:r11) ~size:8 | Tast.Unit -> () | Tast.Zero ty -> zero_value f dst ty + (* No [Set] arm of its own, unlike [emit.ml]: a [Tast.Set] here lowers its + value straight into the place's location, so filling a place and filling + a temporary are already the same line. *) + | Tast.Fill (ty, b) -> fill_value f dst ty b + | Tast.Sentinel ty -> sentinel_value f dst ty | Tast.None_ -> zero_value f dst t (* Reading an uninitialised value gives whatever the slot held: stable garbage rather than LLVM's [poison]. The one construct where the two @@ -2109,6 +2119,50 @@ and emit_invoke_restart f id name (args : Tast.expr list) sg sg_id rloc = xfer_store f ~reg:rax ~scratch:r11; jmp_lbl f.b (current_pad f) +(* [(filled b)] and [(sentinel-filled)], into a location this backend already + has. [zero_loc] above is the same three registers and the same string + instruction with a zero in [al]; these two are what it becomes when the + byte is the program's and when the unit is four bytes rather than one. + + The byte is evaluated *before* [rdi] is loaded, because evaluating it is an + arbitrary expression that may call, and a call clobbers [rdi]. [r11] stays + the scratch every other caller of [lmem] uses, and is never one of the + three this touches. + + The sentinel's word and its tail bytes both come from [Emit] — one list, + read by both backends, so "a hex dump reads DEADBEEF" is stated once. The + tail is up to three straight byte stores, matching [Emit.emit_sentinel]'s + tail exactly: a size that is not a multiple of four ends on DE, DE AD, or + DE AD BE. *) +and fill_value f (dst : loc) (ty : Types.t) (b : Tast.expr) = + let n = sizeof f.md ty in + if n > 0 then begin + let v = eval f b in + load_int f.b ~dst:rax ~mm:(lmem f v ~scratch:r11) ~size:1 ~signed:false; + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rcx (Int64.of_int n); + rep_stosb f.b + end + +and sentinel_value f (dst : loc) (ty : Types.t) = + let n = sizeof f.md ty in + let words = n / 4 and tail = n mod 4 in + if words > 0 then begin + addr_into f ~reg:rdi dst; + movabs f.b ~dst:rax + (Int64.logand (Int64.of_int32 Emit.sentinel_word) 0xFFFFFFFFL); + movabs f.b ~dst:rcx (Int64.of_int words); + rep_stosd f.b + end; + List.iteri + (fun k byte -> + if k < tail then begin + movabs f.b ~dst:rax (Int64.of_int byte); + store_int f.b ~src:rax + ~mm:(lmem f (shift dst (words * 4 + k)) ~scratch:r11) ~size:1 + end) + Emit.sentinel_bytes + and zero_value f (dst : loc) (ty : Types.t) = if is_agg ty then zero_loc f dst (sizeof f.md ty) else if not (is_void ty) then diff --git a/test/programs/fill.flan b/test/programs/fill.flan new file mode 100644 index 0000000..36cc5dc --- /dev/null +++ b/test/programs/fill.flan @@ -0,0 +1,88 @@ +;;;; (filled b) and (sentinel-filled) — the two byte fills, read back as bytes. +;;;; +;;;; The whole point of this program is that every row is a *byte* and not a +;;;; value: the sentinel's contract is "a hex dump reads DEADBEEF", which is a +;;;; claim about which byte lands at which address, and only reading the bytes +;;;; back in address order can check it. The u32 rows are the same claim from +;;;; the other side — a little-endian load of DE AD BE EF is 0xEFBEADDE, which +;;;; is 4022250974, so a backend that wrote the word the other way round would +;;;; print 3735928559 here and be caught. +;;;; +;;;; The lengths are chosen for the tail. 8 is a whole number of patterns, 9 +;;;; ends on DE, 6 ends on DE AD, and 7 ends on DE AD BE — the three truncated +;;;; endings and the one that is not truncated at all. +(defstruct Words [a u32 b u32]) + +;;;; A computed global initialiser: a fill is never a constant the linker can +;;;; write, so this one goes through the startup function on both backends. +(defvar gfill [4 u8] (filled 0x41)) +(defvar gsent [5 u8]) + +(defn bytes4 [b [4 u8]] () + (dotimes [i 4] (print (at b i)) (print " "))) + +(defn main [] i32 + ;; One byte, repeated. 0xFF is the -1 fill a debug allocator wants. + (let [a (array 4 u8)] + (set a (filled 0xFF)) + (bytes4 a) (println "")) ; 255 255 255 255 + + ;; The byte is an expression, not a literal: this is a memset with the + ;; operand in a register, and the register path is the one a constant would + ;; otherwise hide. + (let [v (u8 7) + a (array 4 u8)] + (set a (filled v)) + (bytes4 a) (println "")) ; 7 7 7 7 + + ;; (filled 0) is (zeroed), byte for byte, and saying so here is what keeps + ;; the two from drifting. + (let [a (array 4 u8)] + (set a (filled 0)) + (bytes4 a) (println "")) ; 0 0 0 0 + + ;; Four bytes, ascending, with nothing truncated. + (let [a (array 8 u8)] + (set a (sentinel-filled)) + (dotimes [i 8] (print (at a i)) (print " ")) + (println "")) ; 222 173 190 239 x2 + + ;; The three truncated tails. + (let [a (array 9 u8)] + (set a (sentinel-filled)) + (dotimes [i 9] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 + + (let [a (array 6 u8)] + (set a (sentinel-filled)) + (dotimes [i 6] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 + + (let [a (array 7 u8)] + (set a (sentinel-filled)) + (dotimes [i 7] (print (at a i)) (print " ")) + (println "")) ; ... ends on 222 173 190 + + ;; The same pattern read as words rather than as bytes, which is what says + ;; which end the DE is at. + (let [w (Words {})] + (set w (sentinel-filled)) + (print (.a w)) (print " ") (print (.b w)) (println "")) + + ;; A nested aggregate: a fixed array of structs is plain data all the way + ;; down, so the fill reaches every byte of it. + (let [g (array 2 Words)] + (set g (filled 0xFF)) + (print (.a (at g 0))) (print " ") (print (.b (at g 1))) (println "")) + + ;; The globals. [gfill] was filled before main ran; [gsent] is filled here. + (dotimes [i 4] (print (at gfill i)) (print " ")) + (println "") ; 65 65 65 65 + (set gsent (sentinel-filled)) + (dotimes [i 5] (print (at gsent i)) (print " ")) + (println "") ; 222 173 190 239 222 + + ;; A fill in value position rather than as the value of a [set]: the same + ;; bytes, reached through a temporary instead of through the place. + (bytes4 (filled 0xFF)) (println "") ; 255 255 255 255 + 0) diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index bcc7366..2cdc9ad 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -1243,6 +1243,34 @@ let () = string_eq_out; outputs ~x86:true "string equality, --x86" "programs/string-eq.flan" string_eq_out; + (* The two byte fills, DISCUSS.org's "a DEADBEEF-style sentinel-fill + builtin". Three rows over one program, and what they are pinning is + byte-identity: a fill is the one operation in the language whose whole + contract is which byte lands at which address, so the LLVM backend's + [llvm.memset] and dword loop and the x86 backend's [rep stosb] and + [rep stosd] have to agree byte for byte, and -O0 has to agree with -O2 + because the loop is a real loop at one and unrolled at the other. + + 222 173 190 239 is DE AD BE EF in decimal, ascending through the + storage, which is the claim "a hex dump reads DEADBEEF". The 4022250974 + rows are the same four bytes read back as a u32 — 0xEFBEADDE — and they + are what a backend that wrote the word the other way round would fail. + The three short rows are the truncated tails a size that is not a + multiple of four ends on. *) + let fill_out = + "255 255 255 255 \n7 7 7 7 \n0 0 0 0 \n\ + 222 173 190 239 222 173 190 239 \n\ + 222 173 190 239 222 173 190 239 222 \n\ + 222 173 190 239 222 173 \n\ + 222 173 190 239 222 173 190 \n\ + 4022250974 4022250974\n4294967295 4294967295\n\ + 65 65 65 65 \n222 173 190 239 222 \n255 255 255 255 \n" + in + outputs "byte and sentinel fills" "programs/fill.flan" fill_out; + outputs ~opt:"-O0" "byte and sentinel fills, -O0" "programs/fill.flan" + fill_out; + outputs ~x86:true "byte and sentinel fills, --x86" "programs/fill.flan" + fill_out; (* dyn if: truthiness -- M2 queue item 7. A dyn scrutinee is tested for nil/false vs. everything else, Clojure's rule, on both backends; a typed scrutinee stays strictly bool, which is a checker test diff --git a/test/test_flan.ml b/test/test_flan.ml index 753bc77..0c7c57e 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -1956,6 +1956,81 @@ let () = "(defvar g (Vec u8) uninit) (defn f [] ())" ~needle:"steers every read of it"; + (* ── What may be filled with raw bytes ───────────────────────────── + [(filled b)] and [(sentinel-filled)] are [zeroed]'s siblings, and the + boundary is the whole of what is new about them: zero is a value every + type can have and 0xDE is not, so the checker says which types survive + arbitrary bytes. The accepting side is programs/fill.flan; what is here + is the catalogue of what it refuses and why each refusal is the runtime's + and not a matter of taste. + + The dyn row is the one that would corrupt the collector: a struct holding + a dyn is rooted with a descriptor naming that word's offset, so a filled + one is a root pointing at nothing. *) + accepts "a fixed array of numbers may be filled" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 0xFF))))"; + accepts "a struct of numbers may be sentinel-filled" + "(defstruct S [a i32 b f64]) \ + (defn f [] () (let [s (S {})] (set s (sentinel-filled))))"; + rejects_check "a struct holding a dyn cannot be filled" + "(defstruct S [a i32 d dyn]) \ + (defn f [] () (let [s (S {})] (set s (sentinel-filled))))" + ~needle:"a root pointing at nothing"; + rejects_check "a Vec cannot be filled" + "(defn f [] () (let [v (vec-new i32)] (set v (filled 0xFF))))" + ~needle:"frees a wild address"; + rejects_check "a string cannot be filled" + "(defn f [] () (let [s \"hi\"] (set s (filled 0xFF))))" + ~needle:"a length every bounds check believes"; + rejects_check "a pointer field cannot be filled" + "(defstruct S [p (Ptr i32)]) \ + (defn f [] () (let [s (S {})] (set s (filled 0xFF))))" + ~needle:"an address every deref trusts"; + (* The one refusal that is about the two backends rather than the runtime: + LLVM reads a bool's low bit and x86 compares the whole byte, so 0xDE is + false on one and true on the other. Byte-identical behaviour across the + backends is what this feature is pinned on, so the divergence is refused + rather than documented. *) + rejects_check "a bool cannot be filled" + "(defn f [] () (let [b false] (set b (filled 0xFF))))" + ~needle:"would not even agree with itself"; + (* A data type's tag is a case index, and no byte pattern names a real + case. The type itself is what the message names, because a data type + overlays its cases. *) + rejects_check "a data type cannot be filled" + "(defdata U [(A [x i32]) (B [y i32])]) \ + (defn f [] () (let [u (U.A {.x 1})] (set u (filled 0xFF))))" + ~needle:"names a case"; + (* [zeroed]'s own refusal, worn by both siblings: a fill is the bytes of + whatever type is expected of it, and in a position that expects nothing + there is no type and nothing to fill. This is the shape's cost and it is + paid on purpose — the alternative was a second, place-taking spelling + for an operation [set] already expresses. *) + rejects_check "a fill in a position with no expected type" + "(defn f [] () (print (filled 0xFF)))" + ~needle:"needs to know the type it is filling"; + rejects_check "a sentinel fill in a position with no expected type" + "(defn f [] () (print (sentinel-filled)))" + ~needle:"needs to know the type it is filling"; + (* The byte is a u8 and the ordinary literal rule applies to it — there is + no range check of this builtin's own, and there does not need to be. *) + rejects_check "a fill byte out of range" + "(defn f [] () (let [a (array 4 u8)] (set a (filled 300))))" + ~needle:"does not fit in u8"; + rejects_check "filled takes exactly one byte" + "(defn f [] () (let [a (array 4 u8)] (set a (filled))))" + ~needle:"takes 1 argument"; + rejects_check "sentinel-filled takes no argument" + "(defn f [] () (let [a (array 4 u8)] (set a (sentinel-filled 1))))" + ~needle:"takes 0 arguments, given 1"; + (* A fill is never a value the linker can write into the image, so a + defconst of one is refused by the constant rule rather than by anything + of this feature's own. A defvar is fine: its initialiser runs at + startup, which programs/fill.flan pins. *) + rejects_check "a defconst cannot be filled" + "(defconst g [4 u8] (filled 0xFF)) (defn f [] ())" + ~needle:"defconst"; + (* ── The third element of a defvar ───────────────────────────────── The rule, 2026-09-20: a type there is the zeroed static global it has always been, and anything else is a dyn global initialised from the