diff --git a/TODO.org b/TODO.org index 0a4bd539..931f09a9 100644 --- a/TODO.org +++ b/TODO.org @@ -10,12 +10,6 @@ pointing at it. A CANCELLED entry carries the one-line reason, because an idea rejected without a record is an idea that gets re-proposed. * Language surface -** NEXT A typed char -Decided 2026-09-26 (127): =char= is a typed code point. A char literal is typed by local -inference like a number literal: u8 or i32 where typed code wants a number (a literal -above 127 is refused as a u8), =char= otherwise; a =char= crossing into dyn stays a char. -Rules out the fork where =f(\a)= printed =\a= and =let c = \a= then =f(c)= printed 97. -Waits on the dyn char lane and the literal inference lane. ** NEXT if let Decided 2026-09-26 (126), Rust's spelling: =if let Some(g) = left= plus a block tests the pattern and binds =g= in that block only; =elif=/=else= follow as for =if=. Any @@ -33,6 +27,15 @@ builders return =String=; =(Vec u8)= stays for raw bytes. Text is UTF-8 everywhe a character is a code point; length and indexing count bytes on str. Waits on the dyn-unless-annotated design. +** DONE A typed char +CLOSED: [2026-09-26] +Decision 127: a char literal is the number typed code wants there, and a =char= otherwise, +an untyped array of char literals included; a =char= crosses into dyn as a dyn char, and +only a dyn char unboxes into one. No arithmetic: =(i32 c)= and =(char n)= convert, the +latter checked. An untyped defconst of one is that literal where a number is wanted. +Printed as its literal, as dyn prints one. =runes-next= and =rune-at= give a char; the +UTF-8 codec (=decode-rune=, =encode-rune=) stays on i32. Rules out the f(\a) fork. + ** DONE Dyn has a char, and dyn text counts characters CLOSED: [2026-09-26] Only a char literal, =at= on a text and =chars= make one, and it prints as its @@ -41,7 +44,8 @@ code point where that fits, into a byte only when ASCII; a dyn int into any widt range-checked, while a cast on either wraps as a typed cast does. length, at and slice on dyn text count code points, a malformed byte counting as one U+FFFD. A non-ASCII literal defaults to i32 and is refused where a byte is wanted. Rules out char -arithmetic, a typed code point turning into a char, and byte offsets on dyn text. +arithmetic, a typed code point turning into a char unless =(char n)= says so, and byte +offsets on dyn text. ** DONE String is a prelude struct over (Vec u8), kept valid by the checker CLOSED: [2026-09-26] diff --git a/emacs/flan-fln-mode.el b/emacs/flan-fln-mode.el index df3ab8f8..7ec7ed3b 100644 --- a/emacs/flan-fln-mode.el +++ b/emacs/flan-fln-mode.el @@ -1888,7 +1888,7 @@ lambda or a `Fn(...)' type, and not after a match arm's." (flan-fln--return-type-matcher 1 font-lock-type-face) ;; The package half of a qualified name, as `flan-mode' draws it. ("\\_<\\([a-zA-Z][a-zA-Z0-9!?*+=<>._-]*/\\)" 1 font-lock-type-face) - ("\\_<\\(?:[iu]\\(?:8\\|16\\|32\\|64\\)\\|f\\(?:32\\|64\\)\\|bool\\|str\\|dyn\\|Never\\|Allocator\\|String\\|Ptr\\|Option\\|Vec\\|Map\\|C?Fn\\)\\_>" + ("\\_<\\(?:[iu]\\(?:8\\|16\\|32\\|64\\)\\|f\\(?:32\\|64\\)\\|bool\\|char\\|str\\|dyn\\|Never\\|Allocator\\|String\\|Ptr\\|Option\\|Vec\\|Map\\|C?Fn\\)\\_>" . font-lock-type-face) ("\\_<\\$[^][ \t\n(){},;\":]*" . font-lock-type-face) ;; A character literal, `\c' or `\space'. diff --git a/emacs/flan-mode.el b/emacs/flan-mode.el index fba945f6..39431c5a 100644 --- a/emacs/flan-mode.el +++ b/emacs/flan-mode.el @@ -179,15 +179,15 @@ face says.") ;; containers and memory "length" "at" "slice" "slice-from" "addr" "deref" ;; options, bytes, the host - "Some" "bytes" "bytes-view" "str" + "Some" "bytes" "bytes-view" "str" "char" "bytes->f64" "bytes->i64" "f64->bytes" "i64->bytes" "write-stdout" "print" "println" "exit" "argv") "The functions the compiler provides, from `lib/check.ml''s `builtins' table. Ordinary calls — nothing here is special to the parser — so they are drawn as -builtins and not as keywords. `str' is in this list and in the type rule -below and means a different thing in each: `(str b)' converts and a bare -`str' names a type, which the rules tell apart by the paren. +builtins and not as keywords. `str' and `char' are in this list and in the +type rule below and mean a different thing in each: `(str b)' converts and a +bare `str' names a type, which the rules tell apart by the paren. `destructure~nth' is in the table and not here: the compiler writes it into a destructuring `let' and nobody types it. @@ -257,7 +257,7 @@ reason and is the odd one — it is legal only as the last item of a `def' or a ;; word outright — unit is spelled `()'. Drawing it as a valid type would ;; advertise a spelling the parser rejects, which is the same reason ;; `find-restart' and `await' are left out of `flan--special'. - ("\\_<\\(?:[iu]\\(?:8\\|16\\|32\\|64\\)\\|f\\(?:32\\|64\\)\\|bool\\|str\\|dyn\\|const\\|int\\|float\\|Never\\|Allocator\\|String\\|Ptr\\|Option\\|Vec\\|Map\\|C?Fn\\)\\_>" + ("\\_<\\(?:[iu]\\(?:8\\|16\\|32\\|64\\)\\|f\\(?:32\\|64\\)\\|bool\\|char\\|str\\|dyn\\|const\\|int\\|float\\|Never\\|Allocator\\|String\\|Ptr\\|Option\\|Vec\\|Map\\|C?Fn\\)\\_>" . font-lock-type-face) ;; A type variable, `$t', which is what a generic `defn' names its ;; parameter types with and what `{:where (ordered? $t)}' constrains. diff --git a/lib/check.ml b/lib/check.ml index c5575e9c..01c8dc8d 100644 --- a/lib/check.ml +++ b/lib/check.ml @@ -571,9 +571,7 @@ let nearest cands n = List.find_opt (fun c -> c <> n && one_edit n c) cands spelling is still a name this language does not have. Short on purpose, and only names with one honest answer. [char] is not - here: C's is a byte, Java's is a UTF-16 unit and Rust's is a scalar value, - and this language has [u8] and rune functions, so there is nothing to - translate it to in three words. Nor [void]: it is a return type and the + here: it is this language's own code point type. Nor [void]: it is a return type and the answer there is the shape [()], which is [parse]'s message to give and not this one's. Nor [usize] and [size_t]: the honest answer is "as wide as a pointer on this target", which is [u64] on x86-64 and [u32] on wasm32, and @@ -2064,6 +2062,7 @@ and resolve_name env ~seen loc n = | None -> match n with | "bool" -> Types.Bool + | "char" -> Types.Char | "str" -> Types.String (* Lowercase and concrete, which the rule three screens down says is a type variable. It is spelled this way because it is a primitive and @@ -2195,7 +2194,7 @@ and array_len env loc = function let is_type_name env n = Types.ikind_of_name n <> None || Types.fkind_of_name n <> None - || List.mem n [ "bool"; "str"; "dyn"; "Unit"; "Never"; "Allocator" ] + || List.mem n [ "bool"; "char"; "str"; "dyn"; "Unit"; "Never"; "Allocator" ] || Hashtbl.mem env.aliases n || Hashtbl.mem env.structs n || Hashtbl.mem env.gstructs n @@ -3107,6 +3106,13 @@ let check_fn_ref : (env -> Ast.fn -> Tast.fn) ref = (* ── Small helpers over the AST ────────────────────────────────────── *) +(* The untyped defconsts whose value is a char literal, by name, with the + code point. A use where typed code wants a number is that literal at the + number's type, as the literal written there would be, so a byte constant + such as (defconst sep \,) still compares with a u8 (decision 127). Filled + by the declaration pass of the program being checked. *) +let char_consts : (string, int) Hashtbl.t = Hashtbl.create 8 + (* Untyped literals: their machine type comes from context, so when one is an operand of a binary operator we look at the *other* operand first. *) let is_literal (e : Ast.expr) = @@ -3728,19 +3734,20 @@ let lit_kind (e : Ast.expr) = reading is the literal's own and not one a use names. *) let lit_default (_ : Ast.expr) = function | `Int -> Types.Int Types.I32 - | `Char -> Types.Int Types.U8 + | `Char -> Types.Char | `Float -> Types.Float (float_default ()) | `Box -> Types.Dyn (* The types a use can give it: any number for an integer or a character, since an untyped integer constant is usable where a float is wanted, and - only a float for a float. A type variable is admitted and left to + only a float for a float. A character may also be a char, its own type. A type variable is admitted and left to [int_literal] to judge against its bound. Anything else — dyn, a struct — says nothing about the literal's type; the local keeps its guess and the use is checked as it always was. *) let lit_admits kind (t : Types.t) = match kind, t with | (`Int | `Char), (Types.Int _ | Types.Float _ | Types.Var _) -> true + | `Char, Types.Char -> true | `Float, (Types.Float _ | Types.Var _) -> true | `Box, t -> not (Types.equal t Types.Dyn) | _ -> false @@ -3818,7 +3825,10 @@ let lit_solve (s : lit_session) = let cons = List.concat_map (fun m -> List.rev (Hashtbl.find_all s.cons m)) ms |> List.filter_map (fun (c, t, l) -> - if kind <> `Box && Types.equal t Types.Dyn then Some (Hint, dyn_width, l) + (* A character meeting dyn stays a char, which crosses as + a dyn char; a number takes the dyn width. *) + if kind = `Char && Types.equal t Types.Dyn then None + else if kind <> `Box && Types.equal t Types.Dyn then Some (Hint, dyn_width, l) else if lit_admits kind t then Some (c, t, l) else None) in @@ -3892,8 +3902,8 @@ let no_dyn_yet loc ~into t extra = (* A typed container crossing into dyn is a view, and the runtime needs to know what one element is: its descriptor, a prefix code runtime/flan_dyn.c documents beside [desc_lay] and reads offsets out of by C's layout rule. - Every number, bool, str, struct of those, and fixed array, slice or Vec of - those can be described. [Error t] names the first type inside that cannot: + Every number, bool, char, str, struct of those, and fixed array, slice or + Vec of those can be described. [Error t] names the first type inside that cannot: a dyn, a pointer, a function, an Option, a map, an enum or a data type. None of those is refused for want of a descriptor letter — each is one a dyn value cannot be read out of or written into without a meaning @@ -3913,6 +3923,7 @@ let rec view_desc ?(into = false) structs (t : Types.t) | Types.Float Types.F32 -> Ok "f" | Types.Float Types.F64 -> Ok "d" | Types.Bool -> Ok "?" + | Types.Char -> Ok "C" | Types.String -> Ok "t" | Types.Array (n, e) -> let* d = view_desc structs e in @@ -4350,6 +4361,8 @@ let box ?ctx loc (e : Tast.expr) : Tast.expr = (* The ABI takes an [int32_t], because a C signature that says [_Bool] is a width argument nobody wants to have. *) | Types.Bool -> dyn "flan_dyn_from_bool" [ widen loc (Types.Int Types.I32) e ] + (* A typed char stays a char on the dyn side (decision 127). *) + | Types.Char -> dyn "flan_dyn_from_char" [ widen loc (Types.Int Types.I32) e ] (* A string is ptr+len and arrives as two arguments, the way every other (ptr, len) entry point in the runtime takes one. The runtime copies: the bytes may be a literal or a slice of a buffer the program goes on to @@ -4483,6 +4496,8 @@ let unbox loc (want : Types.t) (e : Tast.expr) : Tast.expr = language's own cast and cannot fail — the runtime already decided the value was a bool, so what comes back is 0 or 1. *) widen loc Types.Bool (need "flan_dyn_need_bool" (Types.Int Types.I32)) + (* Only a dyn char: an int is a number until (char n) converts it. *) + | Types.Char -> rt loc Types.Char "flan_dyn_need_char" [ e; here loc ] (* Any integer width, checked at run time at this site (TODO.org, "Dyn unless annotated"): an int in the width's range, or a char's code point where it fits — ASCII only into a byte, since a byte past ASCII is not @@ -5018,7 +5033,7 @@ let barrier ctx what f = is the only case that does. *) let rec bytewise_key = function - | Types.Int _ | Types.Enum _ | Types.Bool -> true + | Types.Int _ | Types.Enum _ | Types.Bool | Types.Char -> true | Types.Array (_, t) -> bytewise_key t | _ -> false @@ -6041,8 +6056,7 @@ and check_value ctx ?want (e : Ast.expr) : Tast.expr = n (tyname loc t) gname var (Int64.neg n) n | Ast.Int n -> int_literal loc ~want ~preds:ctx.env.tvpreds n | Ast.UInt (n, s) -> wide_literal loc ~want n s - (* A char literal that ends up dyn is a dyn char, never an int; a typed code - point that crosses later is an int, because only the literal says char. *) + (* A char literal that ends up dyn is a dyn char, never an int. *) | Ast.Byte b when want = Some Types.Dyn -> rt loc Types.Dyn "flan_dyn_from_char" [ mk loc (Types.Int Types.I32) (Tast.Int (Int64.of_int b, Types.I32)) ] @@ -6069,12 +6083,16 @@ and check_value ctx ?want (e : Ast.expr) : Tast.expr = Loc.failk literal_at_want loc "%s is code point %d, which does not fit in a %s. Take its code point \ as an i32" c b t - (* A non-ASCII literal is a code point, which a u8 cannot hold as itself: - its default is the prelude's rune, an i32. *) + (* Where typed code wants a number the literal is that number, as an integer + literal would be; everywhere else it is a char (decision 127). *) + | Ast.Byte b + when (match want with + | Some (Types.Int _ | Types.Float _) -> true + | Some (Types.Var v) -> declares ctx.env.tvpreds v "numeric?" + | _ -> false) -> + int_literal loc ~want ~preds:ctx.env.tvpreds (Int64.of_int b) | Ast.Byte b -> - int_literal loc ~want ~preds:ctx.env.tvpreds - ~default:(if b > 127 then Types.I32 else Types.U8) - (Int64.of_int b) + expect ctx loc ~want (mk loc Types.Char (Tast.Int (Int64.of_int b, Types.U32))) (* The float literal's own dyn case, for the reason the integer's has one: the ABI carries one width and the literal is built at it. f64 is already what an unconstrained float literal defaults to, so this only has to stop @@ -6661,6 +6679,12 @@ and int_literal loc ~want ?(preds = []) ?(default = Types.I32) n = declares $%s numeric. Write {:where (numeric? $%s)} at the head of \ the body" n v v v + | Some Types.Char -> + Loc.failk literal_at_want loc + "the integer literal %Ld is not a char: a char is a character, not a \ + number. Write the character as a char literal, or make one with %s" + n (if Source.indented_at loc then Printf.sprintf "char(%Ld)" n + else Printf.sprintf "(char %Ld)" n) | Some other when other <> Types.Never -> Loc.failk literal_at_want loc "expected %s, found the integer literal %Ld" (tyname loc other) n @@ -6845,6 +6869,12 @@ and var ctx ?(qualified = false) loc ~want name = expect ctx loc ~want (mk loc b.bty (Tast.Local b.slot)) | None -> match Hashtbl.find_opt ctx.env.globals name with + | Some _ + when Hashtbl.mem char_consts name + && (match want with + | Some (Types.Int _ | Types.Float _) -> true + | _ -> false) -> + check ctx ?want { Ast.e = Ast.Byte (Hashtbl.find char_consts name); loc } | Some (ty, _) -> expect ctx loc ~want (mk loc ty (Tast.Global name)) | None -> @@ -9661,7 +9691,7 @@ and check_match ctx ?(tail = false) ?want loc scrutinee arms = two lowerings for the test, so a match over a dyn means what [=] over it means. [is_equatable]'s set minus the enums, which are above, and minus bool, below. *) - | (Types.Int _ | Types.Float _ | Types.String | Types.Dyn) as t -> `Lit t + | (Types.Int _ | Types.Float _ | Types.Char | Types.String | Types.Dyn) as t -> `Lit t (* A bool is a two-member enum spelled true and false: the same chain, and exhaustive without a [_] once both are named. *) | Types.Bool -> `Bool @@ -9741,6 +9771,7 @@ and check_match ctx ?(tail = false) ?want loc scrutinee arms = match e.Ast.e, t with | Ast.Str _, _ -> "is a string" | _, Types.String -> "is a number" + | (Ast.Int _ | Ast.UInt _ | Ast.Float _), Types.Char -> "is a number" | Ast.Float x, Types.Int _ when not (Float.is_integer x) -> "is not a whole number" | Ast.Float _, Types.Int _ -> "is a float" @@ -10865,6 +10896,7 @@ and not_numeric name what (a : Tast.expr) = let where = a.Tast.loc in if a.Tast.ty = Types.Bool && String.equal what "integers" then bool_bits where name + else if a.Tast.ty = Types.Char then char_arith where name else if text then fail where "%s takes %s, and this is %s — there is no %s on text. The prelude \ @@ -10873,6 +10905,16 @@ and not_numeric name what (a : Tast.expr) = else fail where "%s takes %s, found %s" name what (tyname where a.Tast.ty) +(* A char is a character and not a number, so no operator computes with one; + the refusal names the two conversions (decision 127). *) +and char_arith loc name = + let fln = fln_source loc in + Loc.failk "check/char-arithmetic" loc + "%s does no arithmetic on a char: a char is a character, not a number. \ + Take its code point with %s, and make a char of one with %s" + name (if fln then "i32(c)" else "(i32 c)") + (if fln then "char(n)" else "(char n)") + (* ── A conversion whose operand is a type variable ───────────────────── [(i32 x)] where [x] is a [$t]. The concrete question — is this a number — has no answer during the abstract pass, and asking it anyway is what @@ -10946,7 +10988,10 @@ and fold_left_prim ctx ~want loc name p ~needs ok what args = let x, y, rest = match args with x :: y :: rest -> x, y, rest | _ -> assert false in - let a, b = binary ctx ~dyn_ok:true name loc ~want:(numeric_want want) [ x; y ] in + let a, b = + char_operands ctx name [ x; y ] (fun () -> + binary ctx ~dyn_ok:true name loc ~want:(numeric_want want) [ x; y ]) + in (* One dyn operand makes the whole fold dyn, whichever side it is on. The typed side is boxed by [dyn_fold]; a literal was already built at dyn by [binary], so [(+ x 1)] over a dyn x folds an i64 one. *) @@ -10975,6 +11020,25 @@ and fold_left_prim ctx ~want loc name p ~needs ok what args = expect ctx loc ~want acc end +(* A pair an arithmetic operator refused, when one operand is a char: that + is the refusal to give, rather than the mismatch between the two. Asked + only after the refusal, so a pair that checks costs nothing more. *) +and char_operands ctx name (args : Ast.expr list) f = + try f () + with Loc.Error _ as ex -> + (* A char literal beside a number is that number, so it says nothing + unless every operand is a literal. *) + let all_lit = List.for_all is_literal args in + List.iter + (fun (a : Ast.expr) -> + let lit = match a.Ast.e with Ast.Byte _ -> true | _ -> false in + if all_lit || not lit then + match trial ctx (fun () -> check ctx a) with + | Ok e when e.Tast.ty = Types.Char -> char_arith a.Ast.loc name + | _ -> ()) + args; + raise ex + (* The dyn lowering of a fold: one call per operator application, left to right, each taking and answering a dyn word. The typed side of a mixed pair is boxed on the way in — [box] is the identity on something already dyn, so @@ -11036,6 +11100,7 @@ and bits_operand ctx loc name (v : Tast.expr) = | Types.Int _ -> () | t when generic_ty t -> unconstrained ctx.env loc name ~needs:"integer?" t | Types.Bool -> bool_bits v.Tast.loc name + | Types.Char -> char_arith v.Tast.loc name | other -> fail loc "%s takes integers, found %s" name (tyname loc other) (* A bool operand is refused before the operands are joined, and not left to @@ -11796,6 +11861,7 @@ and string_piece ctx what (x : Ast.expr) = false) | t when string_or_ptr t -> `Text (string_bytes ctx loc e, true) | Types.Int Types.I32 -> rune e + | Types.Char -> rune (widen loc (Types.Int Types.I32) e) | Types.Int k when Types.widens_to ~from:(Types.Int k) ~into:(Types.Int Types.I32) -> rune (widen loc (Types.Int Types.I32) e) | Types.Int _ -> @@ -11809,7 +11875,7 @@ and string_piece ctx what (x : Ast.expr) = Write %s, which is checked when it is stored" what (if fln_source loc then "str(b)" else "(str b)") | other -> - fail loc "%s takes a str, a String or a code point, found %s" what + fail loc "%s takes a str, a String, a char or a code point, found %s" what (tyname loc other) (* One allocating store into a String's Vec, under the retry guard, with the @@ -12330,11 +12396,12 @@ and named_call ?(qualified = false) ctx ~want loc name args = (match name with | "=" | "!=" -> fail loc - "%s compares numbers, enums, strings and bools, and %s is none \ - of those" name (tyname loc a.Tast.ty) + "%s compares numbers, chars, enums, strings and bools, and %s \ + is none of those" name (tyname loc a.Tast.ty) | _ -> fail loc - "%s orders machine numbers and enums, and %s is neither" name + "%s orders machine numbers, chars and enums, and %s is none of \ + those" name (tyname loc a.Tast.ty)); match rest with | [] -> prim p Types.Bool [ a; b ] @@ -12467,7 +12534,9 @@ and named_call ?(qualified = false) ctx ~want loc name args = body that declares it — so collapsing them would cost the arity and the evaluation rule and buy nothing. *) unconstrained ctx.env loc name ~needs:"ordered?" a.Tast.ty; - if not (Types.is_numeric a.Tast.ty || generic_ty a.Tast.ty) then + (* A char orders, so it has a least and a greatest too. *) + if not (Types.is_numeric a.Tast.ty || a.Tast.ty = Types.Char + || generic_ty a.Tast.ty) then not_numeric name "numbers" a; let ty = a.Tast.ty in let cmp = if String.equal name "min" then Tast.Lt else Tast.Gt in @@ -14650,6 +14719,44 @@ and named_call ?(qualified = false) ctx ~want loc name args = ~what:"a number or an enum" ~is:"a number" v | t -> fail loc "%s converts a number, found %s" name (tyname loc t)); prim (Tast.Cast target) target [ a ] + (* (char n): a code point made a char. Only a Unicode scalar value is one, + so a literal is checked here and anything else at run time. *) + | "char" when List.length args = 1 -> + let x = List.hd args in + let char_lit n = mk loc Types.Char (Tast.Int (n, Types.U32)) in + let fln = fln_source loc in + let not_scalar n = + Loc.failk literal_at_want loc + "%Ld is not a Unicode scalar value, so it is not a char. A char is a \ + code point from 0 to 0x10FFFF, outside 0xD800 to 0xDFFF" n + in + (match x.Ast.e, literal_arith x with + | Ast.Byte b, _ -> expect ctx loc ~want (char_lit (Int64.of_int b)) + | _, Some n -> + if Int64.compare n 0L >= 0 && Int64.compare n 0x10ffffL <= 0 + && not (Int64.compare n 0xd800L >= 0 && Int64.compare n 0xdfffL <= 0) + then expect ctx loc ~want (char_lit n) + else not_scalar n + | _ -> + let a = check ctx x in + let checked i64 = + rt loc Types.Char "flan_char_of" [ i64; here loc ] + in + (match a.Tast.ty with + | Types.Char -> expect ctx loc ~want a + | Types.Int _ -> expect ctx loc ~want (checked (widen loc dyn_i64 a)) + (* Explicit, so a dyn int converts as a typed one does. *) + | Types.Dyn -> + expect ctx loc ~want + (checked (rt loc dyn_i64 "flan_dyn_int_of" [ a ])) + | t -> + fail loc "char makes a char from an integer code point, found %s%s" + (tyname loc t) + (match t with + | Types.Float _ -> + if fln then " — convert it with i32(x) first" + else " — convert it with (i32 x) first" + | _ -> ""))) | _ when is_cast name && List.length args = 1 -> let target = resolve_name ctx.env ~seen:[] loc name in (* An integer literal too wide for the i32 it would default to is checked @@ -14662,6 +14769,9 @@ and named_call ?(qualified = false) ctx ~want loc name args = when Int64.compare n (-2147483648L) < 0 || Int64.compare n 2147483647L > 0 -> Some target | Ast.UInt _, (Types.Int _ | Types.Float _) -> Some target + (* A char literal is the number at the target, so (u8 \é) is refused + as a u8 literal is. *) + | Ast.Byte _, (Types.Int _ | Types.Float _) -> Some target (* A float literal likewise: (f64 0.1) is the f64 nearest 0.1 and not the f32 one widened, and (u64 1.8e19) converts the f64 it says. *) | _, Types.Float _ when lit_kind (List.hd args) = Some `Float -> Some target @@ -14679,6 +14789,12 @@ and named_call ?(qualified = false) ctx ~want loc name args = admits. *) | Types.Dyn -> () | t when Types.is_numeric t -> () + (* A char's code point, into any integer width. *) + | Types.Char when Types.is_integer target -> () + | Types.Char -> + fail loc "%s converts a number, and a char converts only to an integer, \ + as %s" name + (if fln_source loc then "i32(c)" else "(i32 c)") (* The operand of a conversion inside a generic body. The target is a machine type, so what is in question is only the operand, and the [where] clause is what answers it. *) @@ -15801,6 +15917,15 @@ and lit_operands ctx (x : Ast.expr) (y : Ast.expr) f = | _ -> f () and binary_pair ctx ~dyn_ok ~join loc ~want (x : Ast.expr) (y : Ast.expr) = + (* A char defconst no local shadows reads as the literal it names. *) + let is_literal (e : Ast.expr) = + is_literal e + || (match e.Ast.e with + | Ast.Var n -> + Hashtbl.mem char_consts n && lookup ctx n = None + && peek_outer ctx n = None + | _ -> false) + in let y_decides = (is_literal x && not (is_literal y)) || (match x.Ast.e, y.Ast.e with @@ -16146,7 +16271,7 @@ let builtins : (string * string * string) list = "Takes out the character at position i, counting characters, and \ answers its code point. A position past the end signals BoundsError."); ("runes", "runes [str|String|[const u8]] Runes", - "A cursor over the text's code points: (runes-next (addr it)) answers \ + "A cursor over the text's chars: (runes-next (addr it)) answers \ the next one, or None at the end. A malformed byte in a str comes back \ as U+FFFD."); ("rune-count", "rune-count [str|String|[const u8]] i32", @@ -16283,6 +16408,10 @@ let builtins : (string * string * string) list = A String's str lasts until the String next changes. It does not check \ UTF-8, because `str` does not claim UTF-8 — valid-utf8? is an \ ordinary function you call when you care."); + ("char", "char [int|char|dyn] char", + "A code point as a char. Only a Unicode scalar value is one — 0 to \ + 0x10FFFF, outside 0xD800 to 0xDFFF: a literal is checked when it \ + compiles and any other value when it runs. (i32 c) is the way back."); ("bytes->f64", "bytes->f64 [[const u8]] f64", "Parses a float out of the bytes."); ("bytes->i64", "bytes->i64 [[const u8]] i64", "Parses an integer out of the bytes."); @@ -16703,6 +16832,7 @@ let collect env (decls : Ast.decl list) = every other signature in hand — so they are deferred to a pass of their own below. *) let untyped = ref [] in + Hashtbl.reset char_consts; (* Enums come first, in a pass of their own: a signature below may name one, and [resolve] has to find it before it resolves that signature. *) List.iter @@ -17042,6 +17172,9 @@ let collect env (decls : Ast.decl list) = Hashtbl.replace env.globals n (resolve env t, true); Hashtbl.replace env.global_locs n loc | Ast.Defconst (n, None, v) -> + (match v.Ast.e with + | Ast.Byte b -> Hashtbl.replace char_consts n b + | _ -> ()); defconst_type_shaped env n v; Hashtbl.replace env.global_locs n loc; untyped := (n, v) :: !untyped diff --git a/lib/emit.ml b/lib/emit.ml index 5b8950fc..475b7e92 100644 --- a/lib/emit.ml +++ b/lib/emit.ml @@ -338,6 +338,8 @@ let rec ll (t : Types.t) = | Types.Named n -> sname n (* A C enum is an i32 — its own type in the checker, nothing at all here. *) | Types.Enum _ -> "i32" + (* A char is a u32 code point; only the checker tells it from one. *) + | Types.Char -> "i32" | Types.Array (n, e) -> Printf.sprintf "[%Ld x %s]" n (ll e) | Types.Ptr _ -> "ptr" (* An [Allocator] is the runtime's [flan_allocator] record and the @@ -611,7 +613,7 @@ let rec lay m (t : Types.t) : int * int = | Types.Bool -> 1, 1 | Types.String | Types.Slice _ -> 16, 8 | Types.Unit | Types.Never -> 0, 1 - | Types.Enum _ -> 4, 4 + | Types.Enum _ | Types.Char -> 4, 4 | Types.Ptr _ -> 8, 8 | Types.Alloc -> 16, 8 | Types.Fn _ -> 16, 8 @@ -1067,6 +1069,7 @@ let rec dty m d (t : Types.t) : int = | Types.Float k -> basic (Types.to_string t) (Types.bits_f k) "DW_ATE_float" | Types.Bool -> basic "bool" 8 "DW_ATE_boolean" | Types.Enum e -> basic e 32 "DW_ATE_signed" + | Types.Char -> basic "char" 32 "DW_ATE_UTF" | Types.Unit | Types.Never -> composite (Types.to_string t) [] | Types.Ptr (_, e) -> let id = dalloc d in @@ -1497,7 +1500,10 @@ let settled_prim (p : Tast.prim) = ordinary aggregate literal there is pay for a copy. *) let cast_checks (src : Types.t) (target : Types.t) = let concrete (t : Types.t) = - match t with Types.Enum _ -> Types.Int Types.I32 | t -> t + match t with + | Types.Enum _ -> Types.Int Types.I32 + | Types.Char -> Types.Int Types.U32 + | t -> t in match concrete src, concrete target with | Types.Float _, Types.Int _ -> true @@ -3813,6 +3819,9 @@ and prim f (e : Tast.expr) (p : Tast.prim) (args : Tast.expr list) = location. Signed, because a member may be declared negative. *) | Types.Enum _ -> ins f "%s = icmp %s %s %s, %s" t (icmp_op true p) (ll x.Tast.ty) a b + (* A code point orders as the unsigned number it is. *) + | Types.Char -> + ins f "%s = icmp %s i32 %s, %s" t (icmp_op false p) a b (* A bool is an i1 here, and only [=]/[!=] reach it: [<] on a bool is refused in check.ml. *) | Types.Bool -> @@ -4161,6 +4170,7 @@ and cast f ~guard (x : Tast.expr) target = let concrete (t : Types.t) = match t with | Types.Enum _ -> Types.Int Types.I32 + | Types.Char -> Types.Int Types.U32 | t -> t in let src = concrete x.Tast.ty and target = concrete target in @@ -5133,6 +5143,8 @@ declare i32 @flan_dyn_need_bool(i64) declare i32 @flan_dyn_need_i32(i64, ptr, i64) declare i64 @flan_dyn_need_int(i64, i32, ptr, i64) declare i64 @flan_dyn_int_of(i64) +declare i32 @flan_dyn_need_char(i64, ptr, i64) +declare i32 @flan_char_of(i64, ptr, i64) ; A numeric cast written on a dyn answers which numeric tag the box holds; ; check.ml's [cast_dyn] branches on it and each arm is an ordinary need plus ; the ordinary cast. The two slices are the site's location and the target's diff --git a/lib/inspect.ml b/lib/inspect.ml index c6114151..878e7b77 100644 --- a/lib/inspect.ml +++ b/lib/inspect.ml @@ -221,6 +221,8 @@ let rec walk c b depth addr (ty : Types.t) = (* An [i1] in memory is a byte, and a load keeps its low bit. *) | Types.Bool -> put b (if u8 c addr land 1 <> 0 then "true" else "false") | Types.Unit -> put b "()" + (* As the runtime spells a dyn char, which [Form.byte_repr] mirrors. *) + | Types.Char -> put b (Form.byte_repr (Int32.to_int (i32 c addr) land 0x1fffff)) (* The prelude's String is a (Vec u8), whose header starts with the same pointer and length a str is. *) | Types.String | Types.Slice (_, Types.Int Types.U8) | Types.Named "String" -> diff --git a/lib/js.ml b/lib/js.ml index 7ab9f151..ebf92617 100644 --- a/lib/js.ml +++ b/lib/js.ml @@ -247,6 +247,8 @@ let rec refuse_ty loc (t : Types.t) = "dyn is not in the JS dialect yet — every JavaScript value is already \ dynamic, so this is a matter of lowering the dyn operations onto the \ host's own, and that work has not been done" + | Types.Char -> + at loc "char is not in the JS dialect yet — take its code point as an i32" | Types.Var n -> at loc "a type variable (%s) reached the backend, which cannot happen" n | Types.Len _ | Types.LArray _ -> diff --git a/lib/prelude.ml b/lib/prelude.ml index 3997843c..19eced32 100644 --- a/lib/prelude.ml +++ b/lib/prelude.ml @@ -1422,10 +1422,10 @@ let source = {flan| ;; 0xf0, 0xf4 second byte 0x90..0xbf and 0x80..0x8f: overlong below, ;; and past U+10FFFF above. 0xf5..0xff lead nothing at all. ;; -;; A rune is an i32 and not a type of its own. That is Odin's answer too — -;; its `rune` is a four-byte integer distinguished only by a flag on the -;; basic-type row (src/types.cpp, the Basic_rune entry) — so nothing in the -;; checker has to learn a new type for any of this. +;; The codec works on a code point as an i32, the number it is built from +;; with shifts. What hands a character to a caller — rune-at and runes-next — +;; hands back a char, converted once decoding has made it a scalar value, +;; so nothing that walks text has to treat a number as a character. ;; One deliberate divergence from Odin, and it is the parse-i64 argument over ;; again. Odin's decode_rune answers RUNE_ERROR — U+FFFD — for malformed @@ -1499,11 +1499,11 @@ let source = {flan| ;; Decode at a byte offset. None when the offset is not on a rune boundary or ;; the bytes there are malformed, which is stricter than Odin's rune_at — that ;; one hands back RUNE_ERROR and the caller carries on with a wrong character. -(defn rune-at [s [const u8] i i32] (Option i32) +(defn rune-at [s [const u8] i i32] (Option char) (if (or (< i 0) (>= i (length s))) None (let [r (decode-rune (slice s i (length s)))] - (if (.ok r) (Some (.code r)) None)))) + (if (.ok r) (Some (char (.code r))) None)))) ;; Counted through decode-rune rather than through a second walk of its own. ;; Odin keeps a separate rune_count_in_bytes that re-implements the size @@ -1744,17 +1744,17 @@ let source = {flan| ;; A cursor over the code points of some UTF-8 bytes, which owns nothing: the ;; shape split-on-byte has. (runes s) makes one over a str, a String or a -;; [const u8], and runes-next hands back one code point at a time. A +;; [const u8], and runes-next hands back one char at a time. A ;; malformed byte in a str or a [const u8] comes back as U+FFFD and counts ;; as one, as rune-count counts it; a String has none. (defstruct Runes [rest [const u8]]) -(defn runes-next [it (Ptr Runes)] (Option i32) +(defn runes-next [it (Ptr Runes)] (Option char) (if (= (length (.rest it)) 0) None (let [r (decode-rune (.rest it))] (set (.rest it) (slice (.rest it) (.width r) (length (.rest it)))) - (Some (if (.ok r) (.code r) 0xfffd))))) + (Some (char (if (.ok r) (.code r) 0xfffd)))))) ;; append — onto a String, or a run of bytes onto a (Vec u8) — is the ;; checker's (check.ml, "append"), because what it takes decides what it diff --git a/lib/render.ml b/lib/render.ml index d86a0c6b..e40d65bc 100644 --- a/lib/render.ml +++ b/lib/render.ml @@ -155,6 +155,12 @@ let rec render ?(refuse = print_refusal) c depth (e : Tast.expr) : Tast.expr lis | Types.Float _ -> [ c.emit.ef64 (cast (Types.Float Types.F64) e) ] | Types.Bool -> [ unit_ (Tast.If (e, lit "true", lit "false")) ] + (* A char prints as a dyn char does, as its literal: the runtime's one + spelling, so the typed and dyn sides cannot drift apart. *) + | Types.Char -> + [ c.emit.edyn + { Tast.e = Tast.Prim (Tast.Rt "flan_dyn_from_char", [ cast (Types.Int Types.I32) e ]); + ty = Types.Dyn; loc } ] (* Evaluated *and then* reported. A Unit expression is almost always a call made for its effect — (println "x") is the REPL's most ordinary input — so emitting the literal without running it would make the prompt diff --git a/lib/types.ml b/lib/types.ml index d7942343..644625a4 100644 --- a/lib/types.ml +++ b/lib/types.ml @@ -33,6 +33,10 @@ type t = | Int of ikind | Float of fkind | Bool + (* [char]: a Unicode scalar value, a u32 at run time and its own type here, + so a code point is never mistaken for a number. It compares, orders and + hashes; it does no arithmetic, and [i32 c] / [char n] convert. *) + | Char | String | Unit (* the zero-sized type, not C's void *) | Never (* return, exit, error: no value at all *) @@ -175,7 +179,7 @@ let fkind_of_name = function and the identity is a half one. *) let primitive_names = [ "i8"; "i16"; "i32"; "i64"; "u8"; "u16"; "u32"; "u64"; - "f32"; "f64"; "bool"; "str"; "dyn"; "Unit"; "Never"; "Allocator"; + "f32"; "f64"; "bool"; "char"; "str"; "dyn"; "Unit"; "Never"; "Allocator"; "int"; "float" ] let ikind_name k = @@ -198,7 +202,7 @@ let rec equal a b = different things at run time, which is the point of the type and is not this function's question: this is identity of *static* types, and there is one dyn type the way there is one string type. *) - | Bool, Bool | String, String | Unit, Unit | Never, Never | Dyn, Dyn -> true + | Bool, Bool | Char, Char | String, String | Unit, Unit | Never, Never | Dyn, Dyn -> true | Named x, Named y | Enum x, Enum y -> String.equal x y | Slice (a, x), Slice (b, y) -> a = b && equal x y | Array (n, x), Array (m, y) -> Int64.equal n m && equal x y @@ -272,6 +276,7 @@ and to_string = function | Int k -> ikind_name k | Float k -> fkind_name k | Bool -> "bool" + | Char -> "char" | String -> "str" | Unit -> "()" | Never -> "Never" @@ -320,7 +325,7 @@ let is_integer = function Int _ -> true | _ -> false key would hash an address, and hashing an address is a different operation from hashing what it points at. *) let rec keyable = function - | Int _ | Enum _ | Bool | String -> true + | Int _ | Enum _ | Bool | Char | String -> true | Float _ -> false (* NaN /= NaN, and 0.0 and -0.0 differ bytewise *) | Array (_, t) -> keyable t | Named _ -> true (* [Check] decides, by walking the fields *) @@ -331,7 +336,7 @@ let rec keyable = function what every type supports (plan.org, Types). A string has no ordering either: there is no true answer to whether one string is less than another until the language picks a collation, and byte order is not it. *) -let is_comparable = function Enum _ -> true | t -> is_numeric t +let is_comparable = function Enum _ | Char -> true | t -> is_numeric t (* Equality admits one type ordering does not: a string, grown in by the M2 queue's item 5 — bytewise, by content and not by address, so two diff --git a/lib/x86.ml b/lib/x86.ml index c8d053e0..f6dbf3f3 100644 --- a/lib/x86.ml +++ b/lib/x86.ml @@ -537,8 +537,8 @@ let is_agg (t : Types.t) = match t with (* A [(CFn ...)] is one word and crosses exactly as a pointer does, which is the whole of its reason for existing. *) - | Types.Int _ | Types.Float _ | Types.Bool | Types.Ptr _ | Types.Enum _ - | Types.CFn _ -> false + | Types.Int _ | Types.Float _ | Types.Bool | Types.Char | Types.Ptr _ + | Types.Enum _ | Types.CFn _ -> false (* The record and its incarnation — see [Emit]'s %alloc. *) | Types.Alloc -> true | Types.Unit | Types.Never -> false @@ -3673,7 +3673,10 @@ and slice_in_out f sym (src : loc) dst = width, so one pair covers all sixty-four pairings. *) and cast f (a : Tast.expr) (target : Types.t) dst = let concrete (t : Types.t) = - match t with Types.Enum _ -> Types.Int Types.I32 | t -> t + match t with + | Types.Enum _ -> Types.Int Types.I32 + | Types.Char -> Types.Int Types.U32 + | t -> t in let src_t = concrete a.Tast.ty and dst_t = concrete target in let l = eval f a in diff --git a/runtime/flan_dyn.c b/runtime/flan_dyn.c index 32d05937..981cdd08 100644 --- a/runtime/flan_dyn.c +++ b/runtime/flan_dyn.c @@ -2875,6 +2875,22 @@ int64_t flan_dyn_int_of(flan_dyn v) { return flan_dyn_need_i64(v); } +/* A typed char wanted: only a char. An int is refused rather than taken as a + * code point, because a number is not a character until (char n) says so. */ +uint32_t flan_dyn_need_char(flan_dyn v, const uint8_t *loc, int64_t loclen) { + int32_t t = flan_dyn_tag(v); + char sv[SAY_MAX]; + if (t == FLAN_DYN_TAG_CHAR) return (uint32_t)dyn_payload(v); + if (t == FLAN_DYN_TAG_NIL) sv[0] = '\0'; + else say(sv, SAY_MAX, v); + flan_say(loc, loclen, "dyn: a char is wanted here, and this is %s%s%s%s%s%s", + t == FLAN_DYN_TAG_NIL ? "" : an(tag_of(v)), + t == FLAN_DYN_TAG_NIL ? "" : " ", tag_of(v), + t == FLAN_DYN_TAG_NIL ? "" : ", ", sv, + t == FLAN_DYN_TAG_INT ? ". Convert a code point with (char n)" : ""); + flan_trap((const uint8_t *)"DynType", 7); +} + uint8_t flan_dyn_need_bool(flan_dyn v) { if (flan_dyn_tag(v) != FLAN_DYN_TAG_BOOL) trap1(NULL, 0, TYPE_TRAP, "bool", "a bool was wanted", v); @@ -3560,6 +3576,7 @@ static inline int is_map(flan_dyn v) { * * b B h H i I l L i8 u8 i16 u16 i32 u32 i64 u64 * f d ? f32 f64 bool + * C char, a u32 code point * t str (read as a copy; never written from here) * a;T a fixed [n T] * sT a slice [T] @@ -3611,7 +3628,7 @@ static void desc_lay(const uint8_t *d, int64_t *size, int64_t *align) { switch (*d) { case 'b': case 'B': case '?': *size = 1; *align = 1; return; case 'h': case 'H': *size = 2; *align = 2; return; - case 'i': case 'I': case 'f': *size = 4; *align = 4; return; + case 'i': case 'I': case 'f': case 'C': *size = 4; *align = 4; return; case 't': case 's': case 'c': *size = 16; *align = 8; return; case 'v': *size = 40; *align = 8; return; case 'a': { @@ -3647,7 +3664,7 @@ static inline int64_t desc_size(const uint8_t *d) { switch (*d) { /* the scalars, without the walk */ case 'b': case 'B': case '?': return 1; case 'h': case 'H': return 2; - case 'i': case 'I': case 'f': return 4; + case 'i': case 'I': case 'f': case 'C': return 4; case 'l': case 'L': case 'd': return 8; default: break; } @@ -3706,10 +3723,10 @@ static int64_t desc_nfields(const uint8_t *d) { /* The Flan spelling of a descriptor's type, for a sentence. */ static void desc_spell(const uint8_t *d, char *buf, size_t cap) { - static const char scalars[] = "bBhHiIlLfd?t"; + static const char scalars[] = "bBhHiIlLfd?tC"; static const char *const words[] = { "i8", "u8", "i16", "u16", "i32", "u32", "i64", "u64", "f32", "f64", "bool", - "str" }; + "str", "char" }; const char *w; char inner[96]; if (cap == 0) return; @@ -3971,6 +3988,7 @@ static flan_dyn view_read(const uint8_t *loc, int64_t loclen, const char *op, case 'f': { float x; memcpy(&x, p, 4); return flan_dyn_from_f64((double)x); } case 'd': { double x; memcpy(&x, p, 8); return flan_dyn_from_f64(x); } case '?': return flan_dyn_from_bool(*p ? 1 : 0); + case 'C': { uint32_t x; memcpy(&x, p, 4); return flan_dyn_from_char((int32_t)x); } case 't': { const uint8_t *s; int64_t n; @@ -4177,6 +4195,17 @@ static void view_write(const uint8_t *loc, int64_t loclen, const char *op, } *p = dyn_payload(x) ? 1 : 0; return; + /* Only a char: an int is a number until (char n) converts it. */ + case 'C': + if (flan_dyn_tag(x) != FLAN_DYN_TAG_CHAR) { + if (field) { + value_is(why, sizeof why, x); + field_refuse(loc, loclen, op, v, key, d, x, "DynType", why); + } + trap2(loc, loclen, TYPE_TRAP, op, "this view's elements are char", v, x); + } + { uint32_t w = (uint32_t)dyn_payload(x); memcpy(p, &w, 4); } + return; case 't': if (field) field_refuse(loc, loclen, op, v, key, d, x, "DynType", @@ -4644,6 +4673,13 @@ static void into_put(into_site *s, const uint8_t *d, flan_dyn x, uint8_t *p) { } *p = dyn_payload(x) ? 1 : 0; return; + case 'C': + if (flan_dyn_tag(x) != FLAN_DYN_TAG_CHAR) { + into_wanted(why, sizeof why, d); + into_wrong(s, x, why); + } + { uint32_t w = (uint32_t)dyn_payload(x); memcpy(p, &w, 4); } + return; case 't': if (!is_text(x)) into_wrong(s, x, "a str is wanted there, which only a text becomes"); @@ -4767,10 +4803,10 @@ static void into_put(into_site *s, const uint8_t *d, flan_dyn x, uint8_t *p) { /* The copy a [const T] reads, of a plain vec or a view of other elements. */ static void into_copy(into_site *s, const uint8_t *e, flan_obj *o, uint8_t *out) { - static const char scalars[] = "bBhHiIlLfd?t"; + static const char scalars[] = "bBhHiIlLfd?tC"; static const char *const words[] = { "i8", "u8", "i16", "u16", "i32", "u32", "i64", "u64", "f32", "f64", "bool", - "str" }; + "str", "char" }; const char *w = *e != '\0' ? strchr(scalars, *e) : NULL; /* The registry keeps the name by pointer, so it is static text. */ const char *type = w != NULL ? words[w - scalars] : "element"; diff --git a/runtime/flan_dyn.h b/runtime/flan_dyn.h index 5579dd26..2b73f3ec 100644 --- a/runtime/flan_dyn.h +++ b/runtime/flan_dyn.h @@ -302,6 +302,8 @@ uint8_t flan_dyn_need_bool(flan_dyn v); int64_t flan_dyn_need_int(flan_dyn v, int32_t kind, const uint8_t *loc, int64_t loclen); int32_t flan_dyn_need_i32(flan_dyn v, const uint8_t *loc, int64_t loclen); +/* For a typed char: a char's code point. An int traps; (char n) converts. */ +uint32_t flan_dyn_need_char(flan_dyn v, const uint8_t *loc, int64_t loclen); /* A numeric cast's int arm: an int's value or a char's code point. */ int64_t flan_dyn_int_of(flan_dyn v); diff --git a/runtime/flan_rt.c b/runtime/flan_rt.c index 475cfc9e..f4b91a4f 100644 --- a/runtime/flan_rt.c +++ b/runtime/flan_rt.c @@ -2971,6 +2971,16 @@ void flan_rune_check(int32_t c, const uint8_t *loc, int64_t loclen) { rt_trap((const uint8_t *)"InvalidRune", 11); } +/* (char n) on a value only known at run time: n itself when it is a Unicode + * scalar value. Taken as an i64 so every integer width arrives unchanged. */ +uint32_t flan_char_of(int64_t n, const uint8_t *loc, int64_t loclen) { + if (n >= 0 && n <= 0x10ffff && !(n >= 0xd800 && n <= 0xdfff)) + return (uint32_t)n; + flan_say(loc, loclen, "%lld is not a Unicode scalar value, so it is not a char", + (long long)n); + rt_trap((const uint8_t *)"InvalidChar", 11); +} + /* [n] elements from [src] onto the end of a Vec, growing it once. [src] may * point into the Vec's own block — (append s (str s)) — so where it lies is * found before the grow and read again after it: the grow frees the old diff --git a/test/programs/char.flan b/test/programs/char.flan new file mode 100644 index 00000000..5d58768e --- /dev/null +++ b/test/programs/char.flan @@ -0,0 +1,53 @@ +;;;; A typed char (decision 127): a char literal is a char unless typed code +;;;; wants a number, and a char crossing into dyn stays a char. It compares, +;;;; orders and hashes; (i32 c) and (char n) convert. With "dyn-int" a dyn +;;;; int at a char parameter traps, and with "surrogate" (char n) does. + +(defn show [x] () (println x)) +(defn take-char [c char] char c) +(defn next-char [c char] char (char (+ (i32 c) 1))) +(defn a-dyn [x] dyn x) + +(defn main [args [str]] i32 + ;; A direct literal and a let-bound one print alike through dyn. + (show \a) + (let [c \a] (show c)) + ;; Printed as its literal, as a dyn char is, alone and inside a value. + (let [e \é] + (println e [e \z \space]) + (println (i32 e) (u32 e))) + ;; Equality and ordering by code point. + (println (< \a \b) (= \a \a) (!= \a \b) (>= \é \z) (min \q \c) (max \q \c)) + ;; Conversions both ways. + (println (char 65) (char 0x1F600) (i32 \A) (u8 \A) (next-char \y)) + (let [n 66] (println (char n))) + ;; A map keyed by char. + (let [m (map-new char i32)] + (put m \x 1) + (put m \é 2) + (println (get m \x) (get m \é) (has-key? m \y))) + ;; A match over a char. + (let [k \b] + (println (match k \a 1 \b 2 _ 0))) + ;; Byte code is unchanged: a literal beside a byte is the byte. + (let [b (u8 97) + v (vec-new u8) + s (string-new)] + (println (= b \a) (= \a b)) + (push v \z) + (println (at v 0)) + ;; And a char appends to a String as its UTF-8. + (append s \é) + (append s (char 0x65E5)) + (println s)) + ;; A dyn char into a typed char, and typed chars viewed from dyn. + (println (take-char (a-dyn \q))) + (let [cs [\x \y]] + (show cs) + (show (at (a-dyn cs) 1))) + (when (> (length args) 1) + (if (= (at args 1) "dyn-int") + (println (take-char (a-dyn 97))) + (let [n (+ 0xD800 (length args))] + (println (char n))))) + 0) diff --git a/test/programs/slices.flan b/test/programs/slices.flan index 0c027bce..c5428b4e 100644 --- a/test/programs/slices.flan +++ b/test/programs/slices.flan @@ -102,7 +102,7 @@ (let [a [6 2 4 9 1 9 4 5]] (sort (slice a)) (show (slice a))) ; 1 2 4 4 5 6 9 9 - (let [cs [\I \N \S \E \R \T \I \O \N \S \O \R \T]] + (let [cs [(u8 \I) \N \S \E \R \T \I \O \N \S \O \R \T]] (sort (slice cs)) (println (str (slice cs)))) ; EIINNOORRSSTT diff --git a/test/programs/string-owned.flan b/test/programs/string-owned.flan index 7b1bc099..8a23f3e8 100644 --- a/test/programs/string-owned.flan +++ b/test/programs/string-owned.flan @@ -33,7 +33,7 @@ n 0] (while going (match (runes-next (addr it)) - (Some c) (do (when (> c 127) (print c "")) (set n (+ n 1))) + (Some c) (do (when (> (i32 c) 127) (print (i32 c) "")) (set n (+ n 1))) None (set going false))) (println n)) ;; A copy is independent of the original. diff --git a/test/programs/utf8.flan b/test/programs/utf8.flan index 799d12e3..0d083908 100644 --- a/test/programs/utf8.flan +++ b/test/programs/utf8.flan @@ -64,6 +64,10 @@ (print (match o (Some v) v None -1)) (print " ")) +(defn show-char-opt [o (Option char)] () + (print (match o (Some c) (i32 c) None -1)) + (print " ")) + ;; Encode into the scratch buffer and decode straight back out of it. A round ;; trip is the only check that catches an encoder and a decoder that are ;; wrong in the same direction — printing the bytes would not. @@ -145,11 +149,11 @@ ;; rune-at: on a boundary, off a boundary, and out of range. Off a boundary ;; is None rather than a replacement character, which is where this is ;; stricter than Odin's rune_at. - (show-opt (rune-at (bytes-view "日本") 0)) ; 26085 - (show-opt (rune-at (bytes-view "日本") 3)) ; 26412 - (show-opt (rune-at (bytes-view "日本") 1)) ; -1, mid-character - (show-opt (rune-at (bytes-view "日本") 6)) ; -1, past the end - (show-opt (rune-at (bytes-view "") 0)) ; -1 + (show-char-opt (rune-at (bytes-view "日本") 0)) ; 26085 + (show-char-opt (rune-at (bytes-view "日本") 3)) ; 26412 + (show-char-opt (rune-at (bytes-view "日本") 1)) ; -1, mid-character + (show-char-opt (rune-at (bytes-view "日本") 6)) ; -1, past the end + (show-char-opt (rune-at (bytes-view "") 0)) ; -1 (println "") ;; rune-size, at every boundary and on both sides of it. diff --git a/test/test_acceptance.ml b/test/test_acceptance.ml index 6b13cab9..9f073d98 100644 --- a/test/test_acceptance.ml +++ b/test/test_acceptance.ml @@ -5526,6 +5526,37 @@ level "1" outputs "dyn: chars" "programs/dyn-char.flan" dyn_char_out; outputs ~opt:"-O0" "dyn: chars, -O0" "programs/dyn-char.flan" dyn_char_out; outputs ~x86:true "dyn: chars, --x86" "programs/dyn-char.flan" dyn_char_out; + (* A typed char: the fork case, printing, ordering, conversions, a map + key, a match, byte code beside it, a String append, and the crossings + both ways. Then a dyn int at a char parameter and (char n) on a + surrogate, each trapping at its own form. *) + let char_out = + "\\a\n\\a\n\\é [\\é \\z \\space]\n233 233\n\ + true true true true \\c \\q\n\\A \\😀 65 65 \\z\n\\B\n\ + (some 1) (some 2) false\n2\ntrue true\n122\né日\n\\q\n[\\x \\y]\n\\y\n" + in + outputs "char: a typed char" "programs/char.flan" char_out; + outputs ~opt:"-O0" "char: a typed char, -O0" "programs/char.flan" char_out; + outputs ~x86:true "char: a typed char, --x86" "programs/char.flan" char_out; + List.iter + (fun x86 -> + let exe = compile ~x86 "programs/char.flan" in + List.iter + (fun (arg, want) -> + let code, text = run exe (Some arg) in + if code <> 134 || not (contains text want) then begin + incr failures; + Printf.printf + "FAIL char: %s traps%s\n got: %S (exit %d)\n \ + wanted: %S (exit 134)\n" + arg (if x86 then ", --x86" else "") text code want + end) + [ ("dyn-int", "programs/char.flan:50:27: dyn: a char is wanted \ + here, and this is an int, 97. Convert a code point \ + with (char n)"); + ("surrogate", "programs/char.flan:52:18: 55298 is not a Unicode \ + scalar value, so it is not a char") ]) + [ false; true ]; (* A String, and a str made from one, cross into dyn as text measured like any other: characters counted, ASCII or not. *) let string_char_out = diff --git a/test/test_flan.ml b/test/test_flan.ml index 48c10224..525eb056 100644 --- a/test/test_flan.ml +++ b/test/test_flan.ml @@ -1076,7 +1076,7 @@ let () = (* ── Literal defaulting and inference ──────────────────────────── *) infers "int defaults to i32" "42" "i32"; infers "float defaults to f64" "0.5" "f64"; - infers "byte is u8" "\\space" "u8"; + infers "a char literal is a char" "\\space" "char"; infers "string" "\"hi\"" "str"; infers "bool" "true" "bool"; infers "arithmetic keeps kind" "(+ 1 2)" "i32"; @@ -3198,6 +3198,58 @@ let () = ~needle:"\\😀 is code point 128512, which does not fit in a u16"; accepts "an ASCII char is a u8" "(defn main [] i32 (let [b (the u8 97)] (if (= b \\a) 0 1)))"; + (* Decision 127: a char literal is a char unless typed code wants a + number, and a char is a character: it compares, orders and hashes, and + only a conversion computes with it. *) + accepts "a let-bound char literal beside a u8 is the u8" + "(defn main [] i32 (let [b (the u8 97) c \\a] (if (= b c) 0 1)))"; + accepts "a char literal pushed into bytes is a byte" + "(defn main [] i32 (let [v (vec-new u8)] (push v \\a) 0))"; + accepts "a char parameter, compared and ordered" + "(defn f [c char] bool (and (= c \\a) (< c \\z) (!= c (char 98))))"; + accepts "a char defconst beside a byte is the byte, either side of =" + "(defconst sep \\,)\n\ + (defn main [] i32 (let [b (the u8 44)] (if (and (= b sep) (= sep b)) 0 1)))"; + accepts "a char as a map key" + "(defn main [] i32 (let [m (map-new char i32)] (put m \\a 1) 0))"; + accepts "a char converts to an integer and back" + "(defn f [c char] char (char (+ (i32 c) 1)))"; + rejects_check "a char does no arithmetic" + "(defn f [c char] char (+ c c))" + ~needle:"+ does no arithmetic on a char: a char is a character, not a \ + number. Take its code point with (i32 c)"; + rejects_check "nor beside a number" + "(defn f [c char] i32 (let [n 3] (* n c)))" + ~needle:"* does no arithmetic on a char"; + rejects_check "nor with an integer literal" + "(defn f [c char] char (- c 1))" + ~needle:"- does no arithmetic on a char"; + rejects_check "an integer literal is not a char" + "(defn f [c char] bool (= c 97))" + ~needle:"the integer literal 97 is not a char"; + rejects_check "nor negated" + "(defn f [c char] char (- c))" ~needle:"- does no arithmetic on a char"; + rejects_check "nor bitwise" + "(defn f [c char] char (bit-and c c))" + ~needle:"bit-and does no arithmetic on a char"; + rejects_check "a surrogate is not a char" + "(defn main [] i32 (let [c (char 0xD800)] 0))" + ~needle:"55296 is not a Unicode scalar value, so it is not a char"; + rejects_check "nor is a negative number" + "(defn main [] i32 (let [c (char -1)] 0))" + ~needle:"-1 is not a Unicode scalar value"; + rejects_check "nor anything past U+10FFFF" + "(defn main [] i32 (let [c (char 0x110000)] 0))" + ~needle:"1114112 is not a Unicode scalar value"; + rejects_check "a char converts to an integer, not a float" + "(defn f [c char] f64 (f64 c))" + ~needle:"a char converts only to an integer"; + rejects_check "char takes an integer" + "(defn f [x f64] char (char x))" + ~needle:"char makes a char from an integer code point, found f64"; + rejects_check "a non-ASCII char literal cast to a byte" + "(defn main [] i32 (let [b (u8 \\é)] 0))" + ~needle:"\\é is 2 bytes in UTF-8, not one, so it is not a u8"; rejects_check "type-of takes one argument" "(defn main [] i32 (let [k (type-of 1 2)] 0))" ~needle:"type-of"; (* The constructor is an ordinary function, so its arity is the ordinary @@ -3358,13 +3410,13 @@ let () = accepts "typed = on strings" "(defn f [] bool (= \"a\" \"b\"))"; accepts "typed != on strings" "(defn f [] bool (!= \"a\" \"b\"))"; rejects_check "no built-in < on strings" - "(defn f [] bool (< \"a\" \"b\"))" ~needle:"orders machine numbers and enums"; + "(defn f [] bool (< \"a\" \"b\"))" ~needle:"orders machine numbers, chars and enums"; rejects_check "no built-in <= on strings" - "(defn f [] bool (<= \"a\" \"b\"))" ~needle:"orders machine numbers and enums"; + "(defn f [] bool (<= \"a\" \"b\"))" ~needle:"orders machine numbers, chars and enums"; rejects_check "no built-in > on strings" - "(defn f [] bool (> \"a\" \"b\"))" ~needle:"orders machine numbers and enums"; + "(defn f [] bool (> \"a\" \"b\"))" ~needle:"orders machine numbers, chars and enums"; rejects_check "no built-in >= on strings" - "(defn f [] bool (>= \"a\" \"b\"))" ~needle:"orders machine numbers and enums"; + "(defn f [] bool (>= \"a\" \"b\"))" ~needle:"orders machine numbers, chars and enums"; (* (Vec T) is built. What is still refused is the arity: one element type, and a near-miss there would otherwise resolve to a type variable and come back as generics. *) @@ -5050,11 +5102,11 @@ let () = accepts "!= on bools, chained" "(defn f [a bool b bool] bool (!= a b true))"; rejects_check "< on bools" "(defn f [a bool b bool] bool (< a b))" - ~needle:"< orders machine numbers and enums, and bool is neither"; + ~needle:"< orders machine numbers, chars and enums, and bool is none of those"; rejects_check "= on a struct names what it compares" "(defstruct P [x i32])\n(defn f [a P b P] bool (= a b))" - ~needle:"= compares numbers, enums, strings and bools, and P is none of \ - those"; + ~needle:"= compares numbers, chars, enums, strings and bools, and P is \ + none of those"; (* A destructuring pattern in an arm's binds is a name position like any other. *) rejects_check "a pattern inside a match arm's binds"