JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

The corpus numbers, and the rows they came from

Browse Source
This commit is contained in:
Joseph Ferano 2026-09-13 23:26:48 +07:00
parent 11474ecbda
commit 65eeb09378
3 changed files with 150 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
spike/x86/COST.md
View File

@ -12,11 +12,14 @@ for speed). Both are documented in their own headers. Machine: 16-core x86-64, F
linker on both sides, and other work running on it throughout — which is why every time below is the *minimum* of
five or seven runs and why nothing here rests on a difference of a few percent.
The rows behind the tables are committed beside this file as `cost-corpus.tsv` and `cost-bench.tsv`, so a later
lane can recompute a ratio rather than believe one.
## What is being compared, and against what
The interesting column is not the size of the executable. A Flan binary is mostly `flan_rt.o` and libc glue, the
same object on both sides, and it drowns the signal: over the corpus the whole file is only **RATIO_FILE×** bigger
through this backend and `.text` only **RATIO_TEXT×**, which would be a reassuring number and a meaningless one.
same object on both sides, and it drowns the signal: over the corpus the whole file is only **1.09×** bigger
through this backend and `.text` only **1.18×**, which would be a reassuring number and a meaningless one.
So the measurement is the sum of the sizes of the defined symbols the compiler *named itself* — everything called
`flan.<something>`. The runtime's C is `flan_<something>` with an underscore, so the two never collide, and a
@ -32,7 +35,48 @@ and not in `.text`, checked, so it does not contaminate the symbol sums.)
## The corpus: size
CORPUS_TABLE
Ninety-seven programs — the same set `survey.sh` matches on, minus the two that run forever. Summed over all of
them:
| | LLVM -O2 | LLVM -O0 | x86 |
|---|---|---|---|
| own code, all 97 programs | 221,608 | 444,504 | 851,638 |
| against LLVM -O2 | 1.00× | 2.01× | **3.84×** |
| against LLVM -O0 | | 1.00× | **1.92×** |
So the headline is two numbers, not one. **This backend emits 3.8× the code LLVM does at `-O2`, and half of that
factor is the optimiser Flan ships with rather than anything about the backend; against LLVM with the optimiser
off it is 1.9×.** Per program the second ratio is tight — median 2.21, quartiles 1.94 and 2.98, the whole range
1.32 to 5.44 — which is itself a finding: the cost is not a few bad nodes, it is a constant tax on everything.
The ten largest programs, which are where the bytes actually are:
| program | LLVM -O2 | LLVM -O0 | x86 | x86 / -O2 | x86 / -O0 |
|---|---|---|---|---|---|
| `strings` | 13,395 | 22,681 | 33,265 | 2.48× | 1.47× |
| `maps` | 12,833 | 19,850 | 40,445 | 3.15× | 2.04× |
| `edn` | 12,057 | 34,556 | 48,532 | 4.03× | 1.40× |
| `generics` | 10,137 | 20,427 | 39,241 | 3.87× | 1.92× |
| `slurp` | 9,744 | 14,813 | 24,656 | 2.53× | 1.66× |
| `into` | 8,277 | 12,345 | 21,320 | 2.58× | 1.73× |
| `vec` | 8,039 | 12,089 | 22,770 | 2.83× | 1.88× |
| `map-iter` | 7,211 | 10,374 | 21,715 | 3.01× | 2.09× |
| `algorithms` | 6,776 | 16,616 | 30,875 | 4.56× | 1.86× |
| `slices` | 2,393 | 9,057 | 17,753 | 7.42× | 1.96× |
And the two ends of the distribution, both of which are more interesting than the middle:
| program | LLVM -O2 | LLVM -O0 | x86 | x86 / -O2 | x86 / -O0 | why |
|---|---|---|---|---|---|---|
| `bounds` | 363 | 1,951 | 4,039 | **11.13×** | 2.07× | LLVM at `-O2` proves the indices and deletes the checks |
| `array-ctor` | 357 | 1,840 | 3,880 | 10.87× | 2.11× | the same, over a constructor's worth of stores |
| `p2-loop-print` | 283 | 106 | 577 | 2.04× | **5.44×** | a `-O0` build *smaller* than `-O2`: no inlining of the printer |
| `pkg-return` | 7,024 | 23,635 | 31,716 | 4.52× | 1.34× | mostly prelude, where `-O0` is already fat |
`bounds` is the clearest case in the table of why the `-O0` column had to exist. Eleven times is a shocking
number and it is not about this backend at all: the program's whole point is indexing, LLVM at `-O2` can see the
indices are in range and removes the check, and neither LLVM at `-O0` nor this backend can. Against the compiler
that also keeps every check, `bounds` is 2.07× — a completely ordinary row.
## Where the size goes

5
spike/x86/cost-bench.tsv Normal file
View File

@ -0,0 +1,5 @@
name llvm_us o0_us x86_us llvm_nbc_us x86_nbc_us llvm_dev_us x86_dev_us llvm_own o0_own x86_own x86_dev_own
b1-calls 2108 36764 104266 1959 103336 26827 100874 82 200 807 83018
b2-bounds 4711 27047 112567 4548 104951 4663 111295 355 407 1390 83596
b3-spill 22184 28982 137347 22772 132053 22659 132913 168 248 1117 83323
b4-copy 2043 7170 20792 1980 19559 2058 18497 140 348 877 83083
1 name llvm_us o0_us x86_us llvm_nbc_us x86_nbc_us llvm_dev_us x86_dev_us llvm_own o0_own x86_own x86_dev_own
2 b1-calls 2108 36764 104266 1959 103336 26827 100874 82 200 807 83018
3 b2-bounds 4711 27047 112567 4548 104951 4663 111295 355 407 1390 83596
4 b3-spill 22184 28982 137347 22772 132053 22659 132913 168 248 1117 83323
5 b4-copy 2043 7170 20792 1980 19559 2058 18497 140 348 877 83083

98
spike/x86/cost-corpus.tsv Normal file
View File

@ -0,0 +1,98 @@
name llvm_file llvm_text llvm_own o0_own x86_file x86_text x86_own llvm_us x86_us
agent-longname 82224 42002 79 134 82264 42450 519 -1 -1
agent-queue 82352 42306 340 567 86488 43698 1747 2491 2546
agent 82336 42434 454 798 86472 44162 2218 2609 2496
algorithms 76680 39282 6776 16616 101296 63234 30875 2499 2605
allocators 67760 33650 1297 1719 71896 37490 5130 3170 3034
array-ctor 67784 32722 357 1840 71920 36242 3880 2442 2664
bounds-condition 76416 37074 4641 6963 84648 45858 13493 2673 2469
bounds 67720 32738 363 1951 71856 36418 4039 3194 2654
break 82296 42754 819 1182 86432 44482 2549 -1 -1
bytes2 72120 37010 4576 10893 88544 52018 19654 2573 2510
cleanup 68176 33986 1583 2077 72312 36946 4583 2351 2402
conditions 68064 33586 1212 1209 68104 35346 2984 2436 2513
debug-permuted 67720 32610 261 410 67760 33794 1433 2479 2442
debug 67720 32610 261 429 67760 33794 1433 2494 2494
defer-let 67928 34018 1625 1784 72064 36754 4396 2548 2543
destructure 67792 33650 1290 4577 80120 40946 8583 2534 2523
dev-break-bounds 82440 42546 588 648 86576 43762 1830 -1 -1
dev-break 82368 42418 459 715 86504 43682 1755 -1 -1
dev-globals 82456 42162 206 534 86584 43538 1614 -1 -1
dev-inspect 82408 42610 652 704 86544 43698 1765 -1 -1
dev-locals 82336 42418 465 623 86472 43650 1721 -1 -1
dev-noagent 67720 32434 83 109 67760 32866 507 3116 2939
dev-pause 82336 42066 112 244 82376 42802 877 -1 -1
dev-ptr 86632 43954 1970 2859 90768 47490 5543 -1 -1
dev-repl 82336 42066 112 244 82376 42802 877 -1 -1
dev-robust 82336 42066 112 244 82376 42802 877 -1 -1
edn 91208 44706 12057 34556 124016 80898 48532 3218 3165
embed 67800 33330 977 2461 71936 37906 5550 2440 2650
enum-compare 67752 32562 201 581 67792 33970 1611 2607 2805
enum-convert 67752 33202 830 1522 71888 37042 4675 2737 2512
error 67744 32498 131 163 67784 32946 591 2614 2587
exhausted-unhandled 67688 33090 749 1156 67728 34850 2491 2442 2591
exhausted 76192 35778 3415 5139 80336 42674 10321 2409 2469
fn-values 68296 34226 1745 5124 80632 43026 10672 2405 2406
format 76032 36802 4413 7076 84264 45394 13038 2449 2388
frame-rollback 68440 34466 2020 3424 80768 39762 7402 2383 2394
free-all-refused 67688 32370 28 28 67728 32658 299 2418 2497
generics 81368 42786 10137 20427 114184 71602 39241 2474 2516
handles 75880 35826 3482 6289 84112 46338 13981 2520 2470
higher-order 76648 37154 4652 10329 88976 51314 18948 2413 2439
into 80256 40658 8277 12345 92584 53682 21320 2428 2412
loops 67736 33378 1016 1720 71872 37634 5269 2370 2315
machine 67960 33186 821 1925 72096 37394 5042 2515 2456
macro-unless 67720 32642 288 615 67760 34610 2243 2466 2457
macros 67688 32802 461 793 67728 35250 2896 2532 2409
map-exhausted 76152 36162 3805 5994 84392 44530 12171 2393 2506
map-iter 76096 39586 7211 10374 92520 54082 21715 2541 2531
map-stale-region 67688 33650 1307 2106 71824 36898 4537 2354 2407
maps 88504 45266 12833 19850 117216 72802 40445 3162 3082
math 67832 33986 1611 3288 72024 39362 7000 2384 2405
math2 67784 33506 1148 2862 72024 39010 6644 2300 2482
pkg-diamond 67880 32626 252 616 67928 34162 1801 2353 2227
pkg-macro-idle 67688 32338 3 3 67728 32562 210 2440 2410
pkg-macro 67800 32722 354 450 67840 33986 1626 2455 2429
pkg-return 78416 39570 7024 23635 103032 64082 31716 2432 2375
pkg-shadow 67848 32642 277 371 67888 33618 1258 2408 2236
pkg-shared 74560 34066 1676 5593 86888 45650 13283 2422 2648
pkg-unused 69704 32386 35 35 69744 34658 2298 2351 2271
pool-stale-region 67688 33282 932 1675 71824 35746 3379 2389 2466
printers 82464 42098 168 452 82504 43282 1348 -1 -1
println 76256 36098 3737 7587 88584 50114 17753 2312 2460
raylib-audio 79536 35522 2478 5610 87768 44178 11225 2574 2769
raylib-ffi 85848 39538 6245 11506 102272 55650 22537 2607 2852
raylib-font 79088 35538 2622 6260 83232 43186 10350 2661 2658
raylib-image 80152 37698 4504 8285 88384 47058 13966 2684 2946
raylib-imported 70344 33202 612 1003 74480 36658 4066 2688 2602
reach-walk 67992 33170 774 1082 68032 34802 2443 2552 2308
recur 67792 34594 2233 4070 76024 40674 8314 2468 62674
registry 72008 35330 2940 4604 80240 41010 8608 2515 2329
reload-generic 67944 32930 520 1390 67992 35442 3076 2341 2241
restarts 77200 39010 6430 9268 89528 49442 17065 2356 2304
rl-with 69704 32386 35 35 69744 34658 2298 2550 2461
sand-headless 74728 34546 2139 6326 87056 47634 15269 2783 9172
signedness 67688 32594 246 386 67728 33938 1576 2418 2411
slice-from-ptr 67752 33106 755 2549 75984 37458 5101 2421 2367
slices 68128 34802 2393 9057 88648 50114 17753 2504 2271
slurp-unhandled 67688 33522 1173 1601 67728 35298 2945 2340 2354
slurp 80288 42114 9744 14813 100808 57010 24656 2358 2408
stale-region 67688 33490 1148 1681 71824 35842 3483 2332 2286
string-of-bytes 67856 33490 981 1925 71992 36338 3817 2319 2353
strings 88912 45890 13395 22681 109432 65618 33265 2455 2391
text 72296 37122 4666 8007 84624 49378 17018 2310 2397
unions 76016 36498 4131 17714 92440 55810 23456 2323 2413
unit-main 67688 32386 33 33 67728 32674 307 2459 2377
utf8 81216 39714 7170 21515 114024 73106 40745 2563 2380
values 67720 32530 180 667 67760 34082 1719 2531 2378
vec 80080 40402 8039 12089 92408 55122 22770 2370 2389
virtual-controls-headless 70896 33666 1271 1295 75032 38962 6601 2319 3201
web-files 67736 33058 706 1000 67776 34610 2246 2321 2382
p1-exit 67688 32338 3 3 67728 32562 210 2436 2377
p2-loop-print 67688 32626 283 106 67728 32930 577 2269 2364
p3-fizz 67720 32530 169 225 67760 33282 928 2512 2359
p4-convention 67824 32786 405 1005 67864 35138 2778 2436 2332
p5-core 67792 33346 993 1507 71928 37218 4865 2346 2444
p6-transfer 68096 34354 1944 2801 72232 37618 5252 2386 2310
p7-slice-from-ptr 67840 33698 1336 1498 71984 35682 3318 2493 2392
p8-cell 67752 32514 146 270 67800 33202 847 2402 2381
1 name llvm_file llvm_text llvm_own o0_own x86_file x86_text x86_own llvm_us x86_us
2 agent-longname 82224 42002 79 134 82264 42450 519 -1 -1
3 agent-queue 82352 42306 340 567 86488 43698 1747 2491 2546
4 agent 82336 42434 454 798 86472 44162 2218 2609 2496
5 algorithms 76680 39282 6776 16616 101296 63234 30875 2499 2605
6 allocators 67760 33650 1297 1719 71896 37490 5130 3170 3034
7 array-ctor 67784 32722 357 1840 71920 36242 3880 2442 2664
8 bounds-condition 76416 37074 4641 6963 84648 45858 13493 2673 2469
9 bounds 67720 32738 363 1951 71856 36418 4039 3194 2654
10 break 82296 42754 819 1182 86432 44482 2549 -1 -1
11 bytes2 72120 37010 4576 10893 88544 52018 19654 2573 2510
12 cleanup 68176 33986 1583 2077 72312 36946 4583 2351 2402
13 conditions 68064 33586 1212 1209 68104 35346 2984 2436 2513
14 debug-permuted 67720 32610 261 410 67760 33794 1433 2479 2442
15 debug 67720 32610 261 429 67760 33794 1433 2494 2494
16 defer-let 67928 34018 1625 1784 72064 36754 4396 2548 2543
17 destructure 67792 33650 1290 4577 80120 40946 8583 2534 2523
18 dev-break-bounds 82440 42546 588 648 86576 43762 1830 -1 -1
19 dev-break 82368 42418 459 715 86504 43682 1755 -1 -1
20 dev-globals 82456 42162 206 534 86584 43538 1614 -1 -1
21 dev-inspect 82408 42610 652 704 86544 43698 1765 -1 -1
22 dev-locals 82336 42418 465 623 86472 43650 1721 -1 -1
23 dev-noagent 67720 32434 83 109 67760 32866 507 3116 2939
24 dev-pause 82336 42066 112 244 82376 42802 877 -1 -1
25 dev-ptr 86632 43954 1970 2859 90768 47490 5543 -1 -1
26 dev-repl 82336 42066 112 244 82376 42802 877 -1 -1
27 dev-robust 82336 42066 112 244 82376 42802 877 -1 -1
28 edn 91208 44706 12057 34556 124016 80898 48532 3218 3165
29 embed 67800 33330 977 2461 71936 37906 5550 2440 2650
30 enum-compare 67752 32562 201 581 67792 33970 1611 2607 2805
31 enum-convert 67752 33202 830 1522 71888 37042 4675 2737 2512
32 error 67744 32498 131 163 67784 32946 591 2614 2587
33 exhausted-unhandled 67688 33090 749 1156 67728 34850 2491 2442 2591
34 exhausted 76192 35778 3415 5139 80336 42674 10321 2409 2469
35 fn-values 68296 34226 1745 5124 80632 43026 10672 2405 2406
36 format 76032 36802 4413 7076 84264 45394 13038 2449 2388
37 frame-rollback 68440 34466 2020 3424 80768 39762 7402 2383 2394
38 free-all-refused 67688 32370 28 28 67728 32658 299 2418 2497
39 generics 81368 42786 10137 20427 114184 71602 39241 2474 2516
40 handles 75880 35826 3482 6289 84112 46338 13981 2520 2470
41 higher-order 76648 37154 4652 10329 88976 51314 18948 2413 2439
42 into 80256 40658 8277 12345 92584 53682 21320 2428 2412
43 loops 67736 33378 1016 1720 71872 37634 5269 2370 2315
44 machine 67960 33186 821 1925 72096 37394 5042 2515 2456
45 macro-unless 67720 32642 288 615 67760 34610 2243 2466 2457
46 macros 67688 32802 461 793 67728 35250 2896 2532 2409
47 map-exhausted 76152 36162 3805 5994 84392 44530 12171 2393 2506
48 map-iter 76096 39586 7211 10374 92520 54082 21715 2541 2531
49 map-stale-region 67688 33650 1307 2106 71824 36898 4537 2354 2407
50 maps 88504 45266 12833 19850 117216 72802 40445 3162 3082
51 math 67832 33986 1611 3288 72024 39362 7000 2384 2405
52 math2 67784 33506 1148 2862 72024 39010 6644 2300 2482
53 pkg-diamond 67880 32626 252 616 67928 34162 1801 2353 2227
54 pkg-macro-idle 67688 32338 3 3 67728 32562 210 2440 2410
55 pkg-macro 67800 32722 354 450 67840 33986 1626 2455 2429
56 pkg-return 78416 39570 7024 23635 103032 64082 31716 2432 2375
57 pkg-shadow 67848 32642 277 371 67888 33618 1258 2408 2236
58 pkg-shared 74560 34066 1676 5593 86888 45650 13283 2422 2648
59 pkg-unused 69704 32386 35 35 69744 34658 2298 2351 2271
60 pool-stale-region 67688 33282 932 1675 71824 35746 3379 2389 2466
61 printers 82464 42098 168 452 82504 43282 1348 -1 -1
62 println 76256 36098 3737 7587 88584 50114 17753 2312 2460
63 raylib-audio 79536 35522 2478 5610 87768 44178 11225 2574 2769
64 raylib-ffi 85848 39538 6245 11506 102272 55650 22537 2607 2852
65 raylib-font 79088 35538 2622 6260 83232 43186 10350 2661 2658
66 raylib-image 80152 37698 4504 8285 88384 47058 13966 2684 2946
67 raylib-imported 70344 33202 612 1003 74480 36658 4066 2688 2602
68 reach-walk 67992 33170 774 1082 68032 34802 2443 2552 2308
69 recur 67792 34594 2233 4070 76024 40674 8314 2468 62674
70 registry 72008 35330 2940 4604 80240 41010 8608 2515 2329
71 reload-generic 67944 32930 520 1390 67992 35442 3076 2341 2241
72 restarts 77200 39010 6430 9268 89528 49442 17065 2356 2304
73 rl-with 69704 32386 35 35 69744 34658 2298 2550 2461
74 sand-headless 74728 34546 2139 6326 87056 47634 15269 2783 9172
75 signedness 67688 32594 246 386 67728 33938 1576 2418 2411
76 slice-from-ptr 67752 33106 755 2549 75984 37458 5101 2421 2367
77 slices 68128 34802 2393 9057 88648 50114 17753 2504 2271
78 slurp-unhandled 67688 33522 1173 1601 67728 35298 2945 2340 2354
79 slurp 80288 42114 9744 14813 100808 57010 24656 2358 2408
80 stale-region 67688 33490 1148 1681 71824 35842 3483 2332 2286
81 string-of-bytes 67856 33490 981 1925 71992 36338 3817 2319 2353
82 strings 88912 45890 13395 22681 109432 65618 33265 2455 2391
83 text 72296 37122 4666 8007 84624 49378 17018 2310 2397
84 unions 76016 36498 4131 17714 92440 55810 23456 2323 2413
85 unit-main 67688 32386 33 33 67728 32674 307 2459 2377
86 utf8 81216 39714 7170 21515 114024 73106 40745 2563 2380
87 values 67720 32530 180 667 67760 34082 1719 2531 2378
88 vec 80080 40402 8039 12089 92408 55122 22770 2370 2389
89 virtual-controls-headless 70896 33666 1271 1295 75032 38962 6601 2319 3201
90 web-files 67736 33058 706 1000 67776 34610 2246 2321 2382
91 p1-exit 67688 32338 3 3 67728 32562 210 2436 2377
92 p2-loop-print 67688 32626 283 106 67728 32930 577 2269 2364
93 p3-fizz 67720 32530 169 225 67760 33282 928 2512 2359
94 p4-convention 67824 32786 405 1005 67864 35138 2778 2436 2332
95 p5-core 67792 33346 993 1507 71928 37218 4865 2346 2444
96 p6-transfer 68096 34354 1944 2801 72232 37618 5252 2386 2310
97 p7-slice-from-ptr 67840 33698 1336 1498 71984 35682 3318 2493 2392
98 p8-cell 67752 32514 146 270 67800 33202 847 2402 2381