The eight attempts at reading the table now wait between themselves
Retrying immediately looked like eight chances and was one: a walk that bails at the epoch check costs almost nothing, so all eight fit inside the single compaction they were all losing to, and the listing refused having waited for nothing. A quarter of a millisecond between attempts -- the agent's break-loop idiom, legal here because the waiter is the listener thread and never the game loop -- bounds the whole refusal at two milliseconds. Measured with a writer noting and freeing on top of three thousand live blocks: 8 right answers in 200 without the pause, 200 in 200 with it. It is not magic, and the comment says so: a writer that spends most of its time rearranging the table still gets refused, which is the honest answer and used to be a zero-row lie. The two cases the last commit left unwired are wired now, and a third joins them: a listing taken while the table really is being compacted, which nothing covered -- the full-of-live case never compacts and the churn case is single-threaded, so the retry itself was exercised by nothing. It asserts only what a slower machine cannot change: never zero rows, never a count that is neither right nor a refusal. How the rest divides is printed, not pinned.
This commit is contained in:
parent
5caf455e66
commit
739b871ff1
@ -5119,6 +5119,14 @@ asking is asking because they suspect the opposite. A slot the writer kept winni
|
||||
would be a leak report quietly missing the leak — and the agent turns a refusal into an `err` line naming how many
|
||||
slots would not hold still, which the daemon already renders as the editor's message.
|
||||
|
||||
**And the eight attempts wait between themselves**, which is most of what makes them eight attempts rather than one.
|
||||
A walk that bails at the epoch check costs almost nothing, so eight of them back to back fit inside the single
|
||||
compaction they are all losing to. A quarter of a millisecond between them -- `flan_agent.c`'s break-loop idiom, and
|
||||
legal here because the waiter is the listener thread and never the game loop -- bounds the whole refusal at two
|
||||
milliseconds, and under a writer churning on top of 3000 live blocks it was the difference between 8 right answers in
|
||||
200 and 200 in 200. It is not magic: a writer that spends most of its time rearranging the table still gets refused,
|
||||
which is the honest answer to a question asked of a table that is never still.
|
||||
|
||||
**"At exit" is not a hook, and the honest reason is that a game is killed.** A program stopped by a signal runs no
|
||||
`atexit` handler, no destructor, nothing — so no code written inside the program could report anything about the run
|
||||
that matters most. The authoritative reader is therefore `(:op "leaks")`, which reads the same table over the agent
|
||||
|
||||
@ -33,6 +33,10 @@
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
/* For the one wait in this file: the pause between a listing's attempts at
|
||||
* reading the table. See flan_reg_scan_wait. Nothing on the writer's side
|
||||
* waits for anything. */
|
||||
#include <time.h>
|
||||
|
||||
#define FLAN_DEV_MAX 4096
|
||||
|
||||
@ -1152,6 +1156,33 @@ static int flan_reg_scan_ok(uint64_t at) {
|
||||
return __atomic_load_n(&flan_reg_epoch, __ATOMIC_ACQUIRE) == at;
|
||||
}
|
||||
|
||||
/* And a pause between a reader's attempts, which is most of what makes eight
|
||||
* attempts worth more than one.
|
||||
*
|
||||
* Retrying immediately looks like eight chances and is not. A walk that bails
|
||||
* at the epoch check costs almost nothing, so eight of them back to back fit
|
||||
* inside the single compaction they are all losing to, and the reader refuses
|
||||
* having waited for nothing. Measured, with a writer allocating and freeing on
|
||||
* top of three thousand live blocks: without this pause 8 answers in 200 were
|
||||
* right and 192 were refusals; with it, 200 of 200. The rate matters and the
|
||||
* pause is not magic — a writer rearranging the table more than half the time
|
||||
* still gets refused, which is the honest answer to a question asked of a
|
||||
* table that is never still, and is what the refusal sentence is for.
|
||||
*
|
||||
* Waiting is legal here and only here. The reader is the agent's listener
|
||||
* thread, or the exiting program's own; the writer is a game loop and never
|
||||
* reaches this. A quarter of a millisecond is well over one compaction and far
|
||||
* under what a person waiting for a keypress to answer would notice, and eight
|
||||
* of them bound the whole refusal at two milliseconds. It is [flan_agent.c]'s
|
||||
* break-loop idiom — nanosleep a step, look again — for its reason too: a spin
|
||||
* would take a core from the thread being waited on. */
|
||||
static void flan_reg_scan_wait(void) {
|
||||
struct timespec step;
|
||||
step.tv_sec = 0;
|
||||
step.tv_nsec = 250000; /* 250us */
|
||||
nanosleep(&step, NULL);
|
||||
}
|
||||
|
||||
/* Allocated by flan_dev_reg_enable and null until then, which is the whole of
|
||||
* what a release build carries: a null pointer, a zero flag, and the load and
|
||||
* not-taken branch each of the hooks below begins with. A fixed array here
|
||||
@ -1612,7 +1643,7 @@ int64_t flan_dev_reg_by_type(int32_t live_only, int64_t *counts,
|
||||
uint64_t at;
|
||||
n = 0;
|
||||
missed = 0;
|
||||
if (!flan_reg_scan_open(&at)) continue;
|
||||
if (!flan_reg_scan_open(&at)) { flan_reg_scan_wait(); continue; }
|
||||
for (i = 0; i < FLAN_REG_CAP; i++) {
|
||||
flan_reg_entry e;
|
||||
int64_t j;
|
||||
@ -1640,13 +1671,14 @@ int64_t flan_dev_reg_by_type(int32_t live_only, int64_t *counts,
|
||||
}
|
||||
n++;
|
||||
}
|
||||
if (!flan_reg_scan_ok(at)) continue;
|
||||
if (!flan_reg_scan_ok(at)) { flan_reg_scan_wait(); continue; }
|
||||
/* A stable epoch and every slot copied: this is a table that existed. */
|
||||
if (missed == 0) return n;
|
||||
/* A stable epoch but slots that would not hold still. Worth another walk —
|
||||
the writer moves on — and worth remembering the closest one, because it
|
||||
is the number the refusal quotes. */
|
||||
if (fewest < 0 || missed < fewest) fewest = missed;
|
||||
flan_reg_scan_wait();
|
||||
}
|
||||
/* Eight walks, and not one of them saw the whole table. Answering with the
|
||||
last walk's rows would be answering with a table that never existed, and
|
||||
|
||||
@ -286,10 +286,88 @@ static int regoverflow(void) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
/* ── A listing taken while the table is actually being rearranged ────
|
||||
*
|
||||
* [regfull] is a table of live blocks, where the fix is that no compaction
|
||||
* runs at all. This is the other race: a writer churning hard enough that
|
||||
* compactions do run, under a reader asking the leak question through them.
|
||||
*
|
||||
* What it pins is not a rate. Rates here are the machine's, and the counts
|
||||
* printed are deliberately the two that cannot come out any other way on any
|
||||
* machine: [zero], an answer of no rows, which is the original lie and the one
|
||||
* shape a program holding three thousand blocks must never produce; and
|
||||
* [low], a count that is neither right nor a refusal, which is what a torn
|
||||
* read or a walk that straddled a compaction would produce. Right-versus-
|
||||
* refused is the interesting number and it is printed rather than asserted,
|
||||
* because a refusal is an honest answer and how often it comes up is the
|
||||
* writer's business. It moves a lot: on the machine this was written on, the
|
||||
* paced retry turned 8 right in 200 into 200 in 200 at one churn rate and
|
||||
* changed nothing at all at another.
|
||||
*
|
||||
* The throttle is a spin and not a sleep because the whole interesting range
|
||||
* is under nanosleep's floor of tens of microseconds. */
|
||||
enum { RACE_LIVE = 3000, RACE_ASKS = 200, RACE_SPIN = 1000 };
|
||||
|
||||
static volatile int race_stop;
|
||||
|
||||
static void *race_writer(void *arg) {
|
||||
long long i = 0;
|
||||
(void)arg;
|
||||
while (!race_stop) {
|
||||
/* Noted and freed, which is what makes the dead count climb and the
|
||||
* compaction actually run — the difference from [regfull]'s writer, which
|
||||
* re-notes live blocks and therefore never earns a sweep. */
|
||||
void *p = (void *)(uintptr_t)(0x40000000 + (uintptr_t)(i % 100000) * 64);
|
||||
flan_dev_reg_note(p, 32, 4, "Bullet", 6);
|
||||
flan_dev_reg_dead(p);
|
||||
for (volatile long k = 0; k < RACE_SPIN; k++) { }
|
||||
i++;
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
|
||||
static int regrace(void) {
|
||||
enum { ROWS = 16 };
|
||||
int64_t counts[ROWS], bytes[ROWS], typelens[ROWS], unread;
|
||||
const char *types[ROWS];
|
||||
pthread_t w;
|
||||
int right = 0, refused = 0, zero = 0, low = 0;
|
||||
|
||||
flan_dev_reg_enable();
|
||||
for (int i = 0; i < RACE_LIVE; i++)
|
||||
flan_dev_reg_note((void *)(uintptr_t)(0x100000 + (uintptr_t)i * 64), 64, 8,
|
||||
"Enemy", 5);
|
||||
if (pthread_create(&w, NULL, race_writer, NULL) != 0) {
|
||||
printf("no writer thread\n");
|
||||
return 2;
|
||||
}
|
||||
for (int a = 0; a < RACE_ASKS; a++) {
|
||||
int64_t n =
|
||||
flan_dev_reg_by_type(1, counts, bytes, types, typelens, ROWS, &unread);
|
||||
if (n < 0) refused++;
|
||||
else if (n == 0) zero++;
|
||||
else {
|
||||
int64_t enemies = 0;
|
||||
for (int64_t j = 0; j < n && j < ROWS; j++)
|
||||
if (typelens[j] == 5) enemies = counts[j];
|
||||
if (enemies == RACE_LIVE) right++; else low++;
|
||||
}
|
||||
}
|
||||
race_stop = 1;
|
||||
pthread_join(w, NULL);
|
||||
|
||||
printf("zero %d\n", zero);
|
||||
printf("low %d\n", low);
|
||||
printf("answered %d refused %d\n", right, refused);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
flan_rt_init(argc, argv);
|
||||
if (argc < 2) {
|
||||
fprintf(stderr, "usage: %s cap|names|regfull|regchurn|regoverflow\n",
|
||||
fprintf(stderr,
|
||||
"usage: %s cap|names|regfull|regchurn|regrace|regoverflow\n",
|
||||
argv[0]);
|
||||
return 2;
|
||||
}
|
||||
@ -297,6 +375,7 @@ int main(int argc, char **argv) {
|
||||
if (strcmp(argv[1], "names") == 0) return names();
|
||||
if (strcmp(argv[1], "regfull") == 0) return regfull();
|
||||
if (strcmp(argv[1], "regchurn") == 0) return regchurn();
|
||||
if (strcmp(argv[1], "regrace") == 0) return regrace();
|
||||
if (strcmp(argv[1], "regoverflow") == 0) return regoverflow();
|
||||
fprintf(stderr, "unknown mode %s\n", argv[1]);
|
||||
return 2;
|
||||
|
||||
@ -560,6 +560,70 @@ let () =
|
||||
"the registry read under a writer\n got: %S (exit %d, err %S)\n wanted: %S"
|
||||
out code err want_reg;
|
||||
|
||||
(* The other half of the same trigger: it still compacts when compacting
|
||||
would reclaim something. This one would fail silently if the dead count
|
||||
the trigger reads ever drifted low — nothing crashes, the table simply
|
||||
stops reclaiming, pins at its cap, and every listing after that is a
|
||||
floor being read as a count. [overflowed 0] is the line that catches it:
|
||||
3000 blocks held live with 600 more noted and freed on top of them, over
|
||||
and over, fills a 4096-slot table several times over unless the sweep is
|
||||
running. *)
|
||||
let code, out, err = mode "regchurn" in
|
||||
let want_churn = "live 3000\nrows 1\nblocks 3000\noverflowed 0\n" in
|
||||
if code <> 0 || out <> want_churn then
|
||||
fail
|
||||
"the registry under churn\n got: %S (exit %d, err %S)\n wanted: %S"
|
||||
out code err want_churn;
|
||||
|
||||
(* A listing taken while the table really is being rearranged, which
|
||||
neither case above covers: [regfull]'s table never compacts, and the
|
||||
churn above is single-threaded, so the epoch retry — the thing that
|
||||
decides between an answer and a refusal — was exercised by nothing.
|
||||
|
||||
What is asserted is only what cannot come out differently on a slower
|
||||
machine: never an answer of no rows, which is the original lie, and
|
||||
never a count that is neither right nor refused, which is what a walk
|
||||
that straddled a compaction would produce. How the rest divides between
|
||||
answered and refused is the writer's business and is printed rather
|
||||
than pinned — a refusal is an honest answer, and a case that went red
|
||||
because a shared runner was busy would teach whoever runs this to skim
|
||||
past red. *)
|
||||
let code, out, err = mode "regrace" in
|
||||
let first_two =
|
||||
match String.split_on_char '\n' out with
|
||||
| a :: b :: _ -> a ^ "\n" ^ b ^ "\n"
|
||||
| _ -> out
|
||||
in
|
||||
if code <> 0 || first_two <> "zero 0\nlow 0\n" then
|
||||
fail
|
||||
"a listing taken during a compaction\n got: %S (exit %d, err %S)\n wanted no zero-row and no wrong-count answers"
|
||||
out code err;
|
||||
|
||||
(* And a table that is genuinely full, which is the state the trigger above
|
||||
must not paper over. The note is dropped — a diagnostic that killed the
|
||||
program because it ran out of room would be the diagnostic shooting the
|
||||
patient — and the decision this pins is that the drop is *said*, once.
|
||||
Once matters: this is the game thread inside the allocation hook, and a
|
||||
line per dropped note would be sixty a second down a pipe nobody drains
|
||||
while a request is being served. *)
|
||||
let code, out, err = mode "regoverflow" in
|
||||
let want_over = "live 4096\noverflowed 0\noverflowed 1\nlive 4096\n" in
|
||||
if code <> 0 || out <> want_over then
|
||||
fail "a full registry\n got: %S (exit %d)\n wanted: %S" out
|
||||
code want_over;
|
||||
let said_full =
|
||||
let needle = "the allocation registry is full" in
|
||||
let rec go i n =
|
||||
if i + String.length needle > String.length err then n
|
||||
else if String.sub err i (String.length needle) = needle then
|
||||
go (i + 1) (n + 1)
|
||||
else go (i + 1) n
|
||||
in
|
||||
go 0 0
|
||||
in
|
||||
if said_full <> 1 then
|
||||
fail "a full registry said so %d times, not once: %S" said_full err;
|
||||
|
||||
Printf.printf
|
||||
"reload: emit %.1fms llc %.1fms ld %.1fms (v2: emit %.1fms llc %.1fms ld %.1fms) host run %.1fms\n"
|
||||
emit_ms t1.Build.llc_ms t1.Build.link_ms emit2_ms t2.Build.llc_ms
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user