JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

A number with a precision, which %g cannot be asked for

Browse Source 
f64->bytes is snprintf "%g": six significant digits, exponent notation of its
own accord, and no precision to pass it. A frame time of 1/60 comes back as
0.0166667 and a score past a million as 1.23457e+06. format-f64 returns a Vec
instead, so it inherits neither that nor the shared static scratch buffer --
and it is the reason append-i64! exists, because it renders the integer part
and the fraction through that one buffer in strict sequence.

Half away from zero at the last digit kept, which is round-f32's rule and not
printf's. 0.125 at two places is 0.13 here and 0.12 there; matching printf
would mean pinning a particular libc's nearest-even on the binary value, and
that answer is not the same on every target anyway.

The three cases that ship broken are each one line and each tested: the
carry, where the rounded fraction equals the scale and is the next integer
(0.999995 at five places prints "0.100000" without it); the zero padding,
without which 1.005 at three places prints "1.5"; and the sign, which belongs
to the number rather than to its integer part, since -0.5 has an integer part
of 0 and 0 carries no sign.

The clamp on the precision is spelled (min 9 (max 0 prec)) and not with the
clamp macro, and the reason is a finding: the prelude is never
macro-expanded. macro.ml's pass runs over the file being compiled, and the
prelude arrives at the checker through Check.program's own prepend, so a
prelude function calling a prelude macro resolves the macro's underlying
defn -- the one that takes a [Form] -- and reports an arity error.
This commit is contained in:
Joseph Ferano 2026-09-12 21:55:17 +07:00
parent 02850d7282
commit b5e7351c7e
3 changed files with 214 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

101
lib/prelude.ml
View File

@ -227,9 +227,8 @@ let source = {flan|
;;
;; Only the ones that encode a decision. abs is (max x (- 0 x)); a wrapper over
;; that is a function emitted into every program to save a caller nothing. The
;; one honest caveat
;; on that abs: at the least representable integer it answers itself, because
;; the negation wraps. That is what every two's-complement abs does, a
;; one honest caveat on that abs: at the least representable integer it
;; answers itself, because the negation wraps. That is what every two's-complement abs does, a
;; function here would do it too, and the only fix is not to hand it that
;; value — so it is written down rather than wrapped.
;;
@ -951,6 +950,102 @@ let source = {flan|
None (set going false)))
v))
;; ── A number with a precision ─────────────────────────────────────────
;;
;; The one formatting job the runtime cannot do. f64->bytes is snprintf "%g",
;; which is six significant digits and switches to exponent notation on its
;; own: a frame time of 0.0166667 is what a caller wanted two decimals of, and
;; 1.23457e+06 is what a score looks like once it passes a million. There is no
;; precision to pass it, and there cannot be — it renders into one shared
;; static buffer in the runtime, which is the same reason two of its results
;; cannot be held at once.
;;
;; This returns a Vec, so neither problem is inherited. It uses i64->bytes
;; twice and the two calls are strictly sequential — the integer part is copied
;; into the Vec before the fraction is rendered — which is the discipline the
;; shared buffer requires and the one append-i64! exists to make automatic.
;;
;; Half away from zero, the same rule round-f32 follows, applied at the last
;; digit kept. That is not bit-for-bit printf: printf rounds the *binary* value
;; to nearest-even at the decimal digit, and this rounds the decimal expansion
;; half-up, so a value sitting exactly on a half — 0.999995 at five places —
;; comes out 1.00000 here and may come out 0.99999 there. Choosing the rule the
;; rest of this file already uses beats matching a libc whose answer is not the
;; same on every target anyway.
;;
;; Precision is clamped to 0..9 rather than refused. 10^9 is the largest power
;; of ten that leaves room in the f64 product below, and a precision argument
;; is almost always a literal, so a refusal would be a run-time condition for a
;; mistake visible in the source.
;;
;; The clamp is written out as (min 9 (max 0 prec)) and not as the `clamp`
;; macro two hundred lines up, and that is a limit rather than a preference:
;; **the prelude is not macro-expanded**. macro.ml's pass runs over the file
;; being compiled, and the prelude reaches the checker through Check.program's
;; own prepend, having never been through the expander — so a prelude function
;; calling a prelude macro resolves the macro's underlying defn, which takes
;; one [Form] argument, and the report is an arity error at the call. It is
;; written down in NEXT.md beside the other macro gaps.
;;
;; Three inputs do not have decimal expansions and are named before the cast
;; that would be undefined on them: NaN, which fails every comparison and is
;; therefore tested with (not (= x x)) and nothing else, and the two
;; infinities, which are the values satisfying (= x (* x 2.0)) away from zero.
;; A magnitude past 9e18 has no fractional bits left at all and would not fit
;; in the i64 the integer part is carried in, so it falls back to f64->bytes —
;; which is the honest answer there rather than an approximation of one.
;;
;; -0.0 prints as "0.00": the sign test is (< x 0.0), which -0.0 fails. A
;; caller that needs the sign of a zero should not be reading it out of text.
(defn format-f64 [x f64 prec i32] (Vec u8)
(let [b (vec-new u8)
p (min 9 (max 0 prec))]
(cond
(not (= x x))
(append! (addr b) (bytes "nan"))
(and (= x (* x 2.0)) (!= x 0.0))
(append! (addr b) (bytes (if (< x 0.0) "-inf" "inf")))
:else
(let [neg (< x 0.0)
m (if neg (- 0.0 x) x)]
(if (>= m 9.0e18)
(append! (addr b) (f64->bytes x))
(let [scale (i64 1)]
(dotimes [i p]
(set scale (* scale 10)))
;; The split is exact: (i64 m) truncates toward zero and m is
;; non-negative here, and the subtraction of an integer from the
;; float it came from is exact at every magnitude an f64 can hold.
;; Only the scaling below rounds, and it rounds a value already
;; under 1.
(let [ip (i64 m)
fr (i64 (+ (* (- m (f64 ip)) (f64 scale)) 0.5))]
;; The carry, which is the bug this shape is otherwise written
;; with: 0.999995 at five places scales to exactly 100000, which
;; is not a fraction at all — it is the next integer, and without
;; this line it prints as "0.100000".
(when (>= fr scale)
(set fr 0)
(set ip (+ ip 1)))
;; The sign goes on separately, because the integer part is a
;; magnitude: -0.5 at one place has an integer part of 0, and
;; i64->bytes of 0 has no sign to carry.
(when neg
(push b \-))
(append-i64! (addr b) ip)
(when (> p 0)
(push b \.)
;; Left-padded with zeros to exactly p digits. fr is under
;; scale by the carry above, so it never needs more, and
;; without the padding 1.005 at three places prints "1.5".
(let [d (i64->bytes fr)]
(dotimes [i (- p (len d))]
(push b \0))
(append! (addr b) d))))))))
b))
;; ── Refused, by name ──────────────────────────────────────────────────
;;
;; Every one of these needs to produce bytes that did not exist in its input,

86
test/programs/format.flan Normal file
View File

@ -0,0 +1,86 @@
;;;; format-f64: a number rendered to a fixed number of decimal places.
;;;;
;;;; The runtime's f64->bytes is snprintf "%g" and there is no precision to
;;;; pass it, so this is the first number formatter in the language that a
;;;; caller can steer. Every case below is one a plausible wrong version gets
;;;; wrong, and three of them are the ones that actually ship broken: the
;;;; carry, where the rounded fraction equals the scale and is not a fraction
;;;; at all; the zero padding, without which 1.005 prints as "1.5"; and the
;;;; sign, which belongs to the number and not to its integer part, because
;;;; -0.5 has an integer part of 0 and 0 carries no sign.
(defn show [x f64 p i32]
(let [v (format-f64 x p)]
(println (string (as-slice v)))
(free v)))
(defn main [] i32
;; The ordinary cases, and the one %g cannot do at all: 1/60 wanted to two
;; places is a frame time, and "%g" answers 0.0166667.
(show 3.14159 2) ; 3.14
(show 0.0166667 2) ; 0.02
(show 1234.5 1) ; 1234.5
(show 2.0 0) ; 2
(show 2.0 3) ; 2.000
;; Rounding is half away from zero at the last digit kept, on both signs.
(show 0.125 2) ; 0.13
(show -0.125 2) ; -0.13
(show 2.5 0) ; 3
(show -2.5 0) ; -3
;; The carry. 0.999995 scaled by 10^5 rounds to exactly 100000, which is the
;; next integer; without the carry this prints "0.100000".
(show 0.999995 5) ; 1.00000
(show 9.99 1) ; 10.0
(show -9.99 1) ; -10.0
(show 0.99 0) ; 1
;; Zero padding. The fraction of 1.005 at three places is 5, and five digits
;; is not the same number as 005.
(show 1.005 3) ; 1.005
(show 1.0001 4) ; 1.0001
(show 7.0 6) ; 7.000000
;; The sign lives on the number, not on the integer part: both of these have
;; an integer part of 0, which i64->bytes renders without a sign.
(show -0.5 2) ; -0.50
(show -0.004 2) ; -0.00
;; -0.0 prints as a plain zero. The sign test is (< x 0.0), which -0.0 fails,
;; and text is not where the sign of a zero should be read from.
(show 0.0 2) ; 0.00
(show -0.0 2) ; 0.00
;; Precision is clamped rather than refused, at both ends.
(show 1.5 -3) ; 2
(show 1.5 40) ; 1.500000000
;; The three inputs with no decimal expansion.
(show (/ 0.0 0.0) 2) ; nan
(show (/ 1.0 0.0) 2) ; inf
(show (/ -1.0 0.0) 2) ; -inf
;; Past 9e18 an f64 has no fractional bits and the integer part does not fit
;; in an i64, so this falls back to %g rather than approximating.
(show 1e20 2) ; 1e+20
;; A large magnitude that does fit, where the fraction is genuinely gone: an
;; f64 has no bits below 1 up there, so the padding produces the zeros.
(show 1234567890123.0 2) ; 1234567890123.00
;; And the thing it is for: a formatted number inside a built string, which
;; needs the integer part copied out before the fraction is rendered, because
;; both come through the runtime's one shared scratch buffer.
(let [b (vec-new u8)]
(append! (addr b) (bytes "fps "))
(let [f (format-f64 59.94 1)]
(append! (addr b) (as-slice f))
(free f))
(append! (addr b) (bytes " / frame "))
(let [f (format-f64 0.0166667 4)]
(append! (addr b) (as-slice f))
(free f))
(println (string (as-slice b))) ; fps 59.9 / frame 0.0167
(free b))
0)

30
test/test_acceptance.ml
View File

@ -478,6 +478,36 @@ let () =
strings_out;
outputs ~dev:true "string building, dev" "programs/strings.flan"
strings_out;
(* format-f64, the first number formatter a caller can steer. The three
lines that would ship wrong are pinned deliberately: 0.999995 at five
places, where the rounded fraction equals the scale and is the next
integer rather than a fraction; 1.005 at three, where dropping the zero
padding prints "1.5"; and -0.5, where the sign belongs to the number and
the integer part it would otherwise ride on is 0, which i64->bytes
renders unsigned.
0.125 at two places answers 0.13 and printf's "%.2f" answers 0.12. That
is not a defect: this rounds the decimal expansion half away from zero,
which is round-f32's rule and the rest of this file's, where printf
rounds the binary value to nearest-even. Pinning 0.12 here would be
pinning a libc.
The last line is the one the whole shape is for — two numbers in one
built string, which the runtime's single shared scratch buffer makes
impossible for a formatter that answers a slice. *)
let format_out =
"3.14\n0.02\n1234.5\n2\n2.000\n\
0.13\n-0.13\n3\n-3\n\
1.00000\n10.0\n-10.0\n1\n\
1.005\n1.0001\n7.000000\n\
-0.50\n-0.00\n0.00\n0.00\n\
2\n1.500000000\n\
nan\ninf\n-inf\n1e+20\n1234567890123.00\n\
fps 59.9 / frame 0.0167\n"
in
outputs "a number with a precision" "programs/format.flan" format_out;
outputs ~opt:"-O0" "a number with a precision, -O0" "programs/format.flan"
format_out;
(* A debug build, because [dty] is a separate path from everything above:
[outputs ~dev:true] goes through the cells, not through DWARF, and a
type with no arm there dies at emit rather than being merely undebugged.