JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
flan/vendor/agent/flan_agent.c

2207 lines
110 KiB
C
Raw Blame History

/* flan_agent — the half of the dev loop that lives in the running program.
*
* A redefinition arrives as a path to a .so (runtime/flan_dev.c and
* Emit.redefinition are what put it there). Two things have to happen to it,
* and they must happen on different threads:
*
* dlopen relocates the module and runs the loader. It is milliseconds,
* unbounded, and takes the loader lock. Doing it on the game thread
* is a dropped frame.
* install is one store per redefined function. It is sub-microsecond, and
* it must happen at a point where no redefined function is on the
* stack — a frame boundary — or a frame runs half in the old code
* and half in the new.
*
* So the listener thread does the loading and hands over a function pointer;
* the game thread calls flan-poll or flan-wait when it is between frames and
* that is when the swap becomes visible. Nothing else in the program needs to
* know the agent exists.
*
* Nothing that published anything is ever dlclosed: a cell holds an address
* inside a module's text, and unloading it would leave every call site
* pointing at unmapped memory. The two modules that are closed are the ones
* nothing can point into — a transient thunk, which installs no bodies, and a
* module refused before it was queued.
*
* This is not a protocol. One line per request, the path to load, and a one
* line answer. The daemon and its nREPL are a separate program that will speak
* to a socket, not something this file grows into.
*
* The compiler is no longer a separate *process*, though. [flan dev] builds one
* binary that is the program and holds the whole compiler, so a request from it
* is a call to [flan_agent_request] below and not a connect on a socket that
* loops back into this address space. It is the same verb table either way —
* [handle_line] writes into a [sink] that is an fd or a buffer — and the same
* hand-off: a module is published to the ring, and the game thread installs it
* at a frame boundary. The socket stays for [--two-process] and for a person
* with nc.
*/
#include <dlfcn.h>
#include <errno.h>
#include <stdlib.h>
#include <pthread.h>
#include <signal.h>
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <time.h>
#include <unistd.h>
#if defined(__linux__)
#include <sys/prctl.h>
#endif
typedef void (*install_fn)(void);
/* A module may also carry a thunk to run once — that is C-x C-e, an expression
* compiled into a function with nowhere to be called from. It runs where the
* install happens, on the game thread between frames, because an expression
* that reads the program's state has to see it at a point the program agrees
* is consistent. */
typedef void (*call_fn)(void);
/* The value of the last evaluated expression, copied out under the seqlock in
* runtime/flan_dev.c rather than borrowed: a reader gets bytes of its own, and
* a read that loses the race reports the last complete generation and no
* bytes, which leaves the compiler polling rather than showing it half a
* value.
*
* How big that copy has to be is the runtime's number, and it is asked for
* rather than written down here as well. This file used to carry its own copy
* of the bound and a check that the two had not drifted — which is what you do
* when one of them is sizing a buffer to send through. Nothing here sends
* anything any more, so the bound belongs to whoever owns the storage, and
* changing it is one file's decision. */
int flan_dev_result_read(char *dst, uint64_t cap, uint64_t *gen,
uint64_t *len);
uint64_t flan_dev_result_cap(void);
/* The watch table, same arrangement and for the same reason: the game thread
* writes it mid-frame into fixed storage that belongs to flan_dev.c, and this
* file only ever copies out of it, on the listener thread. The one addition is
* [enable] — the table is written only while somebody is reading it, so
* opening and closing a watch buffer is a message that arrives here. */
void flan_dev_watch_enable(int on);
/* And [reset], which opens a new accumulation window for the numeric slots.
* It moves one counter and touches no slot, so the game thread stays the only
* writer of the table. */
void flan_dev_watch_reset(void);
uint32_t flan_dev_watch_count(void);
int flan_dev_watch_overflowed(void);
int flan_dev_watch_read(uint32_t i, char *nd, uint64_t ncap,
char *vd, uint64_t vcap, uint64_t *vlen);
uint64_t flan_dev_watch_name_cap(void);
uint64_t flan_dev_watch_val_cap(void);
/* The allocation registry, read the same way and on the same thread. The
* renderer's two questions — is this address live, and what died here — are
* asked from inside a compiled thunk and never come through this file; these
* three are the *reader's* side, which has no (Ptr T) to read a type off and
* so has to ask the table what it recorded.
*
* Formatting lives here rather than in flan_dev.c for the reason [watch] and
* [result] already have: this runs on the listener thread, where allocating
* and snprintf are legal, and what the game thread writes stays a fixed
* table nobody has to format to fill. */
int32_t flan_dev_reg_at(const void *p, const char **type, int64_t *typelen,
int64_t *off, int64_t *bytes, int64_t *elem,
int64_t *seq, int64_t *died);
int64_t flan_dev_reg_by_type(int32_t live_only, int64_t *counts,
int64_t *bytes, const char **types,
int64_t *typelens, int64_t cap, int64_t *unread);
int flan_dev_reg_enabled(void);
int flan_dev_reg_overflowed(void);
/* A ring the listener writes and the game thread reads. One producer, one
* consumer, so two atomics and no lock — the game thread must never block on
* the loader.
*
* A full ring is *refused*, at the sender, with an error. The previous comment
* here claimed it dropped the oldest request, and nothing did that: [publish]
* never read [tail], so the 65th module overwrote the slot the game thread was
* reading — twenty-four bytes of function pointers, copied field by field with
* no atomic anywhere near them, so the consumer could take half of one job and
* half of another and call it. Silent corruption of the one thing in this file
* that gets *called*.
*
* Of the three honest answers, refusing is the only one that reaches the
* person who asked. Dropping loses a reload the sender was told was ok — the
* same lie in quieter clothes. Blocking stalls the accept loop, which serves
* connections inline, so a program that has stopped polling would also stop
* answering [status] and [abort]: the dev loop would have no way to say
* anything to a program that had stopped listening to it. A refusal is a line
* the daemon can show, and the fix is to call agent/poll.
*
* The check is safe to make separately from the store because there is exactly
* one producer at a time — [request_lock] is what keeps that true now that the
* compiler can call in as well as connect — so room, once seen, cannot be
* taken away by anyone: the consumer only ever makes more of it. */
#define QUEUE 64
/* [handle] is set only for a module that declared itself transient — one that
* ran a thunk and left nothing behind. Everything else is kept mapped forever:
* a cell holds an address inside a module's text, and unloading it would leave
* every call site pointing at unmapped memory. */
/* [stopped_only] is the second half of a promise the *daemon* made and cannot
* keep on its own. A module built to render what is at a raw address has that
* address baked into it as an integer literal: the registry blessed the
* address as live at one moment, and the thunk dereferences it at another,
* three round trips and a ~300ms build later. Nothing in between held the
* break. A [restart] arriving mid-build resumes the game thread, and this ring
* would then install and run the thunk at the very next frame boundary —
* mid-frame, against an address the program may have freed since.
*
* So the job carries the condition it was built under, and [flan_agent_poll]
* re-asks it at the moment of truth instead of trusting the answer the sender
* got. The gate lives *here* and nowhere else on purpose: a second check on
* the listener thread, at delivery, would only shrink the window — [depth] can
* flip between that check and this one — and two places emitting the same
* refusal is how two places stop agreeing, which is the note the struct
* comment further down already makes about layouts.
*
* What makes the check in [flan_agent_poll] sound rather than narrower: only
* the game thread polls (agent.flan's [poll-raw] at a frame boundary, and the
* break loop below, which is the same thread parked), and only the game thread
* raises [depth]. So [depth > 0] read from inside a poll means *this* thread
* is inside the break loop right now — it cannot be running a frame at the
* same time — and the stop the daemon asked under is still the stop in force.
* A false negative is possible in one direction only and is the harmless one:
* a break entered after the read refuses a job that would have been legal, and
* refusing is right anyway, because that is a different stop from the one the
* registry answered under.
*
* [handle] is set only for a module that declared itself transient — one that
* ran a thunk and left nothing behind. Everything else is kept mapped forever:
* a cell holds an address inside a module's text, and unloading it would leave
* every call site pointing at unmapped memory.
*
* ── [at_stop], and why [stopped_only] is not enough for a *write* ──────
*
* [stopped_only] asks "is the program stopped", and for a read that is the
* whole question: the worst a render can do against the wrong stop is print
* something that was true of a different frame, and it is printed into a
* buffer nobody stores anywhere.
*
* A write is not that. A module that *stores* into frame N slot I reaches its
* target through [flan_agent_frame_slot], which reads whatever snapshot is on
* top at the moment the store runs. Resume and stop again inside the build
* window — ~300ms of llc, and a game loop that breaks every frame closes that
* window without trying — and [depth] is above zero again, the job is
* accepted, and the store lands in frame N of a *different* stack. Not a
* fault: a plausible shape, in the wrong place, silently.
*
* So a write names the stop it was addressed against. [snap_push] mints a
* generation that is monotone and never reused, precisely so that "resumed
* and stopped again" is distinguishable from "still the same stop" — the
* restart machinery already leans on it for the same reason. [at_stop] is
* that number, asked for by the daemon through [stop] and handed back on the
* request, and checked here, on the game thread, at the moment the job is
* claimed. Zero means the job does not care, which is every read.
*
* It sits beside [stopped_only] rather than subsuming it because they are two
* different questions and one of them has no answer to give: a render rooted
* at a raw address wants "stopped at all" and has no stop to name, since the
* registry that blessed the address is not a stack. */
typedef struct {
install_fn install;
call_fn call;
void *handle;
int stopped_only;
int32_t at_stop;
} job;
/* Said once, in one place, and shipped to the daemon over [refusals] rather
* than written down again at the other end. A refusal is a sentence naming
* what actually happened, and the thing that actually happened is not "the
* gate failed" — it is that the program the person was inspecting started
* running again while the inspector was being compiled. */
static const char *RESUMED =
"the program resumed while this inspection was being built — stop it again "
"and re-ask";
/* The other way a job's stop can stop being the job's stop, and it needs its
* own sentence because the fix is a different one. Above, the program is
* running and the reader has to stop it. Here it *is* stopped — at a stop
* that came after the one the request named — so stopping it again would do
* nothing, and what is wanted is to look at what is there now. A write built
* against a render of the old stop would otherwise land in storage the reader
* never saw. */
static const char *RESTOPPED =
"the program was resumed and stopped again while this was being built, so it "
"is no longer at the stop this was addressed to — look again and re-ask";
/* Which of the two the last drop was. One counter and two sentences rather
* than two counters, because the daemon's question is "did a drop happen
* between these two reads" and that is a count; the text is only what it says
* afterwards. Last writer wins, which is the residual [refused_while_running]
* already documents below for two inspections in flight at once. */
static const char *_Atomic refused_why = NULL;
/* How many stopped-only jobs have been dropped, ever. A count and not a flag:
* the daemon reads it before it delivers and again while it waits, and what it
* wants to know is whether one happened *in between*, which a flag somebody
* else could have cleared cannot say.
*
* It is the whole of the channel because the sentence is fixed. One residual,
* left alone deliberately: two inspections in flight at once would let either
* one's refusal surface to the other's waiter. The result buffer they both
* write into has exactly that property already — flan_dev.c says so where the
* seqlock is — and fixing it here without fixing it there would be half a
* repair wearing the whole one's clothes. */
static atomic_ullong refused_while_running;
static job queue[QUEUE];
static atomic_uint head; /* written by the listener */
static atomic_uint tail; /* written by the game thread */
static int listen_fd = -1;
static pthread_t listener;
static atomic_int started;
/* Room for one more. Unsigned subtraction, so the answer survives [head] and
* [tail] wrapping; only their difference means anything. */
static int queue_room(void) {
unsigned h = atomic_load_explicit(&head, memory_order_relaxed);
unsigned t = atomic_load_explicit(&tail, memory_order_acquire);
return (h - t) < QUEUE;
}
/* 0 if the ring is full, having published nothing. Checked here as well as at
* the caller, because a producer that forgot would otherwise reintroduce
* exactly the overwrite this replaced. */
static int publish(job j) {
unsigned h = atomic_load_explicit(&head, memory_order_relaxed);
unsigned t = atomic_load_explicit(&tail, memory_order_acquire);
if (h - t >= QUEUE) return 0;
queue[h % QUEUE] = j;
/* Release: the store to the slot must be visible before the index that
* advertises it. */
atomic_store_explicit(&head, h + 1, memory_order_release);
return 1;
}
/* Returns how many modules were installed. Call it between frames. */
/* ── The break loop, spec-conditions.md §2 ──────────────────────────── */
/* Where "a crash kills the program" stops being true. An unhandled error runs
* this instead of rt_die(), on the frame that erred, with nothing unwound — so
* the condition and every restart between here and the top are still live.
*
* It is the *poll* loop, run from here instead of from the frame boundary, and
* that is the whole design. An expression evaluated while stopped is a module
* the loader thread queues and the game thread runs; if this loop did not
* drain that queue, C-x C-e would hang exactly when it is most wanted.
*
* Installing while stopped is deliberately allowed. The rule that a redefined
* function must not be swapped while it is on the stack is about *mid-frame
* consistency* — half a frame of old code and half of new — and there is no
* frame in progress here. The old body on the stack keeps running; a retry
* restart calls through the cell and reaches the new one. That is the fix-it-
* and-retry loop, and refusing the install would remove the point. */
/* The runtime's end of it. flan_rt.c holds the hook and the lookup; the loop
* itself is here, because it needs the socket and the install queue and the
* runtime must not depend on an optional package for either. */
extern void (*flan_break_hook)(const uint8_t *name, int64_t namelen,
void *condition, void *xfer);
/* The other hook, for the traps that call with no channel — see rt_trap in
* flan_rt.c, which carries the argument for why they are two hooks and not
* one. This end of it is [trap_stop] below. */
extern void (*flan_trap_hook)(const uint8_t *name, int64_t namelen);
extern int32_t flan_restart_count(void);
/* The shadow stack (runtime/flan_dev.c). The compiler pushes a frame per Flan
* call in a dev build; these read one, and only ever on the thread that owns
* it. The frame is opaque here on purpose: its shape belongs to flan_dev.c,
* and two files each declaring the struct is how the two stop agreeing. */
extern int32_t flan_dev_frame_count(void);
extern void *flan_dev_frame_at(int32_t i);
extern const char *flan_dev_frame_name(const void *frame, int64_t *len);
extern const char *flan_dev_frame_loc(const void *frame, int64_t *len);
extern int32_t flan_dev_frame_nslots(const void *frame);
extern int32_t flan_dev_frame_slotsig(const void *frame);
extern int32_t flan_dev_frame_refsig(const void *frame);
extern void *flan_dev_frame_slot(const void *frame, int32_t i);
extern const uint8_t *flan_restart_name(int32_t i, int64_t *len);
extern void *flan_restart_frame(int32_t i);
extern void flan_restart_take(void *frame, void *xfer);
/* Where the expression that trapped is written (runtime/flan_rt.c). Set by
* the trap sites around their call into the break hook and NULL otherwise,
* so it is read here exactly once, while the snapshot is being taken on the
* thread the trap stopped. */
extern const uint8_t *flan_break_site;
extern int64_t flan_break_site_len;
/* A restart frame with no Flan function under it, which is what the boundary
* below is made of. The storage belongs to flan_rt.c for the reason the
* shadow-stack frame's shape does: the struct is declared in one file. */
extern void *flan_restart_push_c(const uint8_t *name, int64_t namelen);
extern void flan_restart_pop_c(void *frame);
/* -- How far down a transfer can actually land ----------------------- */
/* A thunk run from this loop is called through [flan_reload_call], which
* allocates its *own* transfer channel and drops it on return. So a restart
* chosen at a break inside a thunk unwinds as far as the thunk and no
* further: every frame below that C boundary is on the list, is accepted, and
* is silently not taken. Three statements that the program will resume, none
* of them true - which is worse than either resuming or refusing.
*
* The boundary is recorded where it is *made*, at the call, not at the break:
* the frames below it are exactly the ones already on the stack when the
* thunk started, and a restart-case the thunk enters itself is above it and
* reachable. Recording the depth on entering the break loop instead would
* count those too and refuse restarts that work.
*
* Game thread only, saved and restored around the call, so a thunk that
* breaks and whose break runs another thunk nests correctly. */
static int32_t restart_floor;
/* The same boundary, counted in shadow-stack frames. A break inside a C-x C-e
* thunk has that thunk's frames on top of the program's, and "where is my
* program" answered with [eval/7] is not the question anyone asked. Recorded
* at the call for the same reason [restart_floor] is: the frames below it are
* exactly the ones that were already on the stack when the thunk started. */
/* -1, not 0, and the distinction is the whole of it: [restart_floor] can be
* zero meaning "no restarts were on the stack when the thunk started", and
* outside a thunk zero is also the right answer. Frames are the other way
* round — outside a thunk *every* frame is the program's, and zero would say
* none of them are. So "not inside a thunk" gets a value of its own. */
static int32_t frame_floor = -1;
/* -- The way out that the floors left missing ------------------------- */
/* Everything above says which restarts a break inside a thunk cannot take. It
* took a report from someone whose game died to notice what that leaves when
* the answer is *all of them*: an expression evaluated with C-x C-e signals,
* nothing above the boundary established a restart — a bad index establishes
* none — and the whole list is either empty or below the floor. Then the only
* live choice at the break is [abort], and abort ends the process. A mistyped
* index cost a session that had been running for an hour.
*
* So the boundary offers a restart of its own. It is a real restart frame, on
* the real restart list, pushed by the agent immediately *after* the floor is
* read — which is what puts it above the floor and makes it reachable, where
* pushing it first would have marked it as the program's and refused it.
*
* Taking it aims the transfer at this frame. Nothing compares against it, so
* the unwind runs to the top of the thunk, [flan_reload_call] drops the
* channel it holds, and [flan_agent_poll] returns to whatever called it — the
* game loop, or an outer break. That is the same path a below-the-floor
* restart used to take by accident; the difference is that this one is what
* was asked for, and is reported as what happened.
*
* What it does not do, and nothing at this boundary could: undo. The thunk
* ran until it signalled, and every global it set and every byte it allocated
* on the way is still set and still allocated. Abandoning is "stop running
* this expression", not "unmake what it did".
*
* NULL when no thunk is in progress, which is also the answer to "is there an
* evaluation to abandon": a program that broke on its own is not inside one,
* and a break there must not offer this. Game thread only, saved and restored
* around the call like the floors, so nesting names the innermost. */
static void *eval_boundary;
static const uint8_t abandon_name[] = "abandon-evaluation";
/* The three of them, dropped between two runs of [main]. The counterpart of
* flan_rt.c's [flan_condition_stacks_reset] and flan_dev.c's
* [flan_dev_frames_reset], called from the same one place and for the same
* reason: a merged dev build re-enters [main] by longjmp, which pops no frame,
* so a thunk that was in progress when the run ended leaves a floor counting
* frames that are gone and a boundary naming a [c_restarts] slot the runtime
* has just released.
*
* Harmless today — a program's restart frames are allocas and can never
* compare equal to that address, so a stale boundary marks nothing — and left
* in that state it is one changed representation away from marking the wrong
* entry. The floors go with it because they are the same kind of state and it
* would be strange to empty two thirds of it. */
void flan_agent_run_reset(void) {
eval_boundary = NULL;
restart_floor = 0;
frame_floor = -1;
}
/* Nothing counts how many evaluations have been abandoned, and that is a
* decision rather than an omission. The editor is told twice already: the
* reply to the evaluation says the expression stopped before it produced a
* value, and the reply to the restart says taking that one abandoned it. A
* counter here would be a third telling, read by nobody, which is how a wire
* grows a verb whose answer drifts from what happened. */
/* What the listener thread hands the stopped game thread. One slot, because
* only one thread is ever stopped. */
/* A *depth*, not a flag. A thunk this loop runs may itself error, and the
* break loop that catches that one is nested inside this one — so a flag is
* wrong twice over: the inner loop clearing it on resume tells the world the
* program is running while the outer loop is still stopped, and every verb
* then answers "not stopped" while the outer loop spins forever with no
* protocol path out. Only kill recovered it. Counting fixes both. */
static _Atomic int depth;
#define BREAK_MAX 8 /* deep enough to nest, shallow
* enough that a loop of breaks
* stops rather than grinds */
static _Atomic int chosen_index;
/* Which snapshot that index is an index *into*. The listener validates a
* choice against the snapshot on top when the request arrives, and the game
* thread resolves it against the snapshot on top when it next looks - and
* those are two reads of a stack that can move between them. An evaluation
* this loop runs may error, push a break of its own, and reach [chosen_ready]
* first, which would take that break's index 2 for the one someone chose from
* the outer list. Depth is not enough to tell them apart: an outer break
* resuming and a new one starting reuses the number. A generation does not.
*
* A mismatch is *left alone* rather than discarded. The listener already
* answered ok for it, so the break it was meant for must still be able to
* take it; the inner loop simply does not claim what is not addressed to it. */
static _Atomic int chosen_gen;
static _Atomic int chosen_ready;
static _Atomic int aborting;
/* -- The snapshot ---------------------------------------------------- */
/* The stopped thread is not holding still. This loop runs [flan_agent_poll],
* which runs arbitrary Flan, and every restart-case that runs pushes and pops
* the one global restart list. Serving names straight off that list hands the
* listener a pointer into a frame that may already have been popped; serving
* *indices* off it is worse still, because an index carries no evidence of
* what it meant - the list and the choice can disagree and nothing can tell.
*
* So the list is read once, on entry, while the thread that owns it is in
* this function and not in Flan, and copied: names into a buffer of our own,
* frames as the addresses a transfer carries. Everything the listener answers
* comes from here and nothing re-reads the live stack. */
#define SNAP_MAX 64 /* restarts offered at one break */
#define SNAP_NAMES 4096 /* bytes of names behind them */
#define FRAME_MAX 64 /* frames listed in a backtrace */
#define FRAME_TEXT 8192 /* bytes of names and locations */
typedef struct {
int32_t gen; /* never reused, never 0 */
/* Whether *any* restart on this list can be taken, which is a property of
* the break and not of the restarts. [reachable] answers a different
* question — that one is per restart, and it is about the thunk boundary.
* A break taken by a trap with no transfer channel has a perfectly good
* list of live restarts below it and no way to aim at one, so the whole
* snapshot is marked instead of each entry: the refusal sentence differs,
* and a reader of this struct should not have to infer which case a run of
* zeroes in [reachable] meant. */
int32_t resumable;
int32_t n;
int32_t total; /* before SNAP_MAX truncated it */
void *frame[SNAP_MAX];
int32_t off[SNAP_MAX], len[SNAP_MAX];
int32_t reachable[SNAP_MAX];
/* Which entry is [eval_boundary]'s, or -1 when this break is not inside an
* evaluation. An index rather than a per-entry flag, because there is at
* most one on any list: the outer thunk's boundary is below this one's
* floor and is already marked unreachable. Recorded here rather than
* recomputed at reply time for the reason everything else is — the listener
* answers from the snapshot and never reads the live stack.
*
* A reader must not look for the *name* instead. Nothing stops a program
* establishing a restart called [abandon-evaluation] of its own, and a
* client that matched on the name would offer the program's restart as the
* way out of an evaluation. The address is what makes it this one. */
int32_t boundary;
int32_t used;
char names[SNAP_NAMES];
/* Where the stopped thread is, taken at the same moment and for the same
* reason: the chain is the game thread's, and it is holding still only
* because it is parked in this loop. [fframe] is kept as well as the text,
* because reading a frame's locals means going back to that frame — and to
* that frame rather than to whatever is at index 2 by then. */
int32_t fn; /* frames listed */
int32_t ftotal; /* before FRAME_MAX truncated it */
int32_t fused;
void *fframe[FRAME_MAX];
int32_t fnoff[FRAME_MAX], fnlen[FRAME_MAX];
int32_t floff[FRAME_MAX], fllen[FRAME_MAX];
int32_t fslots[FRAME_MAX];
int32_t fsig[FRAME_MAX]; /* the slot fingerprint of the body
* this frame was compiled from */
int32_t frsig[FRAME_MAX]; /* and the fingerprint of the
* globals that body names */
int32_t fmine[FRAME_MAX]; /* 0 = the evaluation's, not the
* program's */
char ftext[FRAME_TEXT];
/* The condition this break was entered with, or NULL for a trap that
* carries none. An address into the signalling frame, which is live for
* exactly as long as this snapshot is on top — nothing unwound — so a
* render thunk aimed at it through [flan_agent_condition] reads storage
* that is still there. Never dereferenced here: its type is the daemon's
* to know, and rendering it is the daemon-built thunk's job. */
void *cond;
/* Where the expression that trapped is written, copied from the runtime's
* [flan_break_site] at the same held-still moment as everything else.
* Empty for a stop with no site — a user (error ...), a (pause). */
int32_t sitelen;
char site[512];
} snapshot;
/* One per nested break loop, because an inner break must not answer with the
* outer one's restarts and must not destroy them either - the outer loop is
* still going to need them when the inner one resumes. Same reason [depth] is
* a count and not a flag. */
static snapshot snaps[BREAK_MAX];
static _Atomic int snap_depth; /* published last; 0 = none */
static snapshot *snap_top(void);
/* Where slot [slot] of frame [frame] lives, resolved against the snapshot this
* break took and not against the live chain.
*
* This is what a locals thunk calls. The thunk runs on the stopped game
* thread, from inside this break loop's own poll, and it pushes frames of its
* own while it runs — so "frame 2" means the third frame of the backtrace the
* daemon was shown, not the third frame of whatever the stack looks like by
* the time the thunk is executing. Resolving against the snapshot is the whole
* of the difference, and it is the same reason the restarts are answered from
* there.
*
* NULL for anything it cannot place, and the thunk is built to ask only about
* slots the same snapshot already reported as bound. A NULL would be
* dereferenced, so this is the one place that must not answer optimistically:
* an index that is out of range, a frame that is not in this snapshot, or a
* slot the binding for which had not run, are each a null here and a refusal
* before the thunk is ever built. */
void *flan_agent_frame_slot(int64_t frame, int64_t slot) {
snapshot *s = snap_top();
if (s == NULL) return NULL;
if (frame < 0 || frame >= s->fn) return NULL;
if (slot < 0 || slot > 0x7fffffff) return NULL;
return flan_dev_frame_slot(s->fframe[frame], (int32_t)slot);
}
/* The condition this break holds, for the render thunk the daemon builds to
* show its fields. Same contract as [flan_agent_frame_slot]: called on the
* stopped game thread, resolved against the snapshot on top *when the thunk
* runs*, NULL for a break that carries none — and the daemon delivers the
* thunk at-stop, so a resume between the asking and the running drops it
* rather than rendering one break's type over another break's pointer. */
void *flan_agent_condition(void) {
snapshot *s = snap_top();
return s == NULL ? NULL : s->cond;
}
static snapshot *snap_top(void) {
int d = atomic_load(&snap_depth);
return d <= 0 ? NULL : &snaps[d - 1];
}
/* Called on the game thread with the stack held still. 0 if there is no room
* to nest, which the caller reports rather than serving a stale one. */
static int32_t snap_gen; /* monotone; 0 is "no snapshot" */
static int snap_push(int resumable, void *cond) {
int d = atomic_load(&snap_depth);
if (d >= BREAK_MAX) return 0;
snapshot *s = &snaps[d];
int32_t n = flan_restart_count();
s->gen = ++snap_gen;
s->resumable = resumable;
s->cond = cond;
s->sitelen = 0;
if (flan_break_site != NULL && flan_break_site_len > 0) {
int64_t k = flan_break_site_len;
/* Truncation would be silent and would produce a *plausible* site — a
* path cut short still parses as one — so a site too long to hold is
* dropped instead. No site is a state the reader already handles; a
* wrong one is not. 512 is far past any real file:line:col. */
if (k > (int64_t)sizeof s->site) k = 0;
memcpy(s->site, flan_break_site, (size_t)k);
s->sitelen = (int32_t)k;
/* Consumed, not just read. The trap site clears the global when its hook
* returns — but this loop *is* the hook, so a break nested inside it (a
* fix candidate evaluated at a bounds stop raises its own error) would
* otherwise copy the outer trap's site under the inner condition's name,
* which is a caret pointing at an unrelated line. Each snapshot owns its
* copy; a nested entry that set no fresh site gets none. */
flan_break_site = NULL;
flan_break_site_len = 0;
}
s->total = n;
s->used = 0;
s->n = 0;
s->boundary = -1;
/* One slot and one name's worth of bytes kept back for the boundary, and the
* arithmetic below is the whole of why.
*
* The walk runs innermost first and stops at the first limit it meets, so the
* *outermost* entries are the ones truncation drops — and the boundary is the
* outermost entry of the evaluation, which makes it the first casualty. A
* thunk that established 64 restarts of its own would therefore reproduce the
* bug this change exists to fix, exactly: a full list, nothing on it that
* leaves the evaluation, and abort as the only live choice.
*
* So it is placed rather than found when the walk does not reach it. The cost
* is one listed restart out of sixty-four whenever an evaluation is in
* progress, which is a straight trade against losing the way out. */
const int32_t held = (eval_boundary != NULL) ? 1 : 0;
const int32_t held_bytes = held ? (int32_t)sizeof abandon_name : 0;
for (int32_t i = 0; i < n && s->n < SNAP_MAX - held; i++) {
int64_t len = 0;
const uint8_t *nm = flan_restart_name(i, &len);
void *fr = flan_restart_frame(i);
if (nm == NULL || fr == NULL) continue;
if (len < 0) len = 0;
if ((int64_t)s->used + len + 1 > SNAP_NAMES - held_bytes) break;
s->frame[s->n] = fr;
s->off[s->n] = s->used;
s->len[s->n] = (int32_t)len;
/* The outermost [restart_floor] frames are below the thunk boundary. */
s->reachable[s->n] = (i < n - restart_floor);
if (fr == eval_boundary && eval_boundary != NULL) s->boundary = s->n;
memcpy(s->names + s->used, nm, (size_t)len);
s->used += (int32_t)len;
s->names[s->used++] = 0;
s->n++;
}
/* The slot kept back, used only when the walk ran out before reaching it.
* Appended rather than inserted at its live position: every index that
* crosses the wire is a position in *this* array — [restart-at] resolves
* through [s->frame] — so the snapshot's order is the only order there is,
* and the entry is the same frame wherever it sits. The name is the one this
* file pushed; the runtime is not asked for it back. */
if (held && s->boundary < 0 && s->n < SNAP_MAX
&& (int32_t)s->used + held_bytes <= SNAP_NAMES) {
int32_t len = (int32_t)sizeof abandon_name - 1;
s->frame[s->n] = eval_boundary;
s->off[s->n] = s->used;
s->len[s->n] = len;
/* Reachable by construction: it is pushed above the floor, and the floor
* is what [reachable] is measured against. */
s->reachable[s->n] = 1;
s->boundary = s->n;
memcpy(s->names + s->used, abandon_name, (size_t)len);
s->used += len;
s->names[s->used++] = 0;
s->n++;
}
/* And the frames, from the same held-still stack. A deep recursion is
* truncated rather than followed: the innermost frames are the ones the
* question is about, and the count says how many were left out. */
{
int32_t fn = flan_dev_frame_count();
s->ftotal = fn;
s->fused = 0;
s->fn = 0;
for (int32_t i = 0; i < fn && s->fn < FRAME_MAX; i++) {
void *fr = flan_dev_frame_at(i);
int64_t nl = 0, ll = 0;
const char *nm, *lc;
if (fr == NULL) break;
nm = flan_dev_frame_name(fr, &nl);
lc = flan_dev_frame_loc(fr, &ll);
if (nl < 0) nl = 0;
if (ll < 0) ll = 0;
if ((int64_t)s->fused + nl + ll + 2 > FRAME_TEXT) break;
s->fframe[s->fn] = fr;
s->fnoff[s->fn] = s->fused;
s->fnlen[s->fn] = (int32_t)nl;
if (nm != NULL && nl > 0) memcpy(s->ftext + s->fused, nm, (size_t)nl);
s->fused += (int32_t)nl;
s->ftext[s->fused++] = 0;
s->floff[s->fn] = s->fused;
s->fllen[s->fn] = (int32_t)ll;
if (lc != NULL && ll > 0) memcpy(s->ftext + s->fused, lc, (size_t)ll);
s->fused += (int32_t)ll;
s->ftext[s->fused++] = 0;
s->fslots[s->fn] = flan_dev_frame_nslots(fr);
/* Snapshotted with the rest of the frame rather than read later: the
* module this description lives in can be unloaded once the daemon
* installs a replacement, and the comparison happens after that. */
s->fsig[s->fn] = flan_dev_frame_slotsig(fr);
s->frsig[s->fn] = flan_dev_frame_refsig(fr);
/* The outermost [frame_floor] frames are the program's; anything above
* them belongs to the evaluation this break is inside. */
s->fmine[s->fn] = (frame_floor < 0) || (i >= fn - frame_floor);
s->fn++;
}
}
atomic_store(&snap_depth, d + 1);
return 1;
}
static void snap_pop(void) {
int d = atomic_load(&snap_depth);
if (d > 0) atomic_store(&snap_depth, d - 1);
}
/* The condition's class name, so an editor can say what stopped rather than
* only that something did. The pointer beside it goes into the snapshot: this
* side still cannot render a value whose type it does not know, but the
* daemon knows the type — it compiled it — and builds a thunk that reads the
* fields through [flan_agent_condition]. Written before [broken] is set and
* read only while [broken] is 1, so the listener never sees half of it. */
static char condition_name[128];
int32_t flan_agent_poll(void);
/* The path this process actually bound, NUL-terminated, or "" if it never
* bound one. Written once in [start_on] between the successful bind and
* anything that could read it — the listener thread, the atexit handler and
* the orphan signal handler are all arranged after it is set.
*
* It is a copy rather than a [getenv] at use time because one of its readers
* is a signal handler: getenv is not async-signal-safe, and [orphan_die] can
* run on any thread at any instruction. It is also the only record of the path
* when the zero-argument form chose it, where there is no environment variable
* to read back. sun_path's size is the bound because that is where the string
* came from. */
static char bound_sock[sizeof(((struct sockaddr_un *)0)->sun_path)];
/* A socket file outlives the process that bound it, and a stale one answers
* the next client with ECONNREFUSED — which reads like a program that is there
* and refusing rather than one that has gone. So the bind registers its own
* removal, and the three ways out of a program each unlink it:
*
* ordinary exit this handler, via atexit
* break-loop abort die_now, by hand, because it takes _exit
* orphaned child orphan_die, by hand, for the same reason
*
* Outside a daemon that is the whole story, and the path is under /tmp where
* nothing else would ever reclaim it.
*
* Under [flan dev] none of the three is how a session usually ends, and this
* is worth being exact about rather than claiming cover it does not give: the
* two-process daemon kills its child with SIGTERM, whose default disposition
* runs no atexit, and the merged session leaves by [Unix._exit 0]. So the
* socket there is left in the daemon's temp directory — which is itself never
* removed today. FIX.org, "The daemon leaves its temp directory behind": the
* session-end cleanup that item asks for takes the socket with it, and until
* it lands the socket outlives the session. Nothing below can fix that from
* here; a program that is killed does not get to tidy up. */
static void unlink_bound_sock(void) {
if (bound_sock[0] != '\0') unlink(bound_sock);
}
/* Every way out of the break loop that is not a resume. [_exit] and not
* [exit], because this runs on the game thread while the listener thread may
* be inside [dlopen] holding the loader lock — and [exit] runs the atexit
* chain and the ELF destructors, which want that same lock. A program asked to
* abort would hang instead of dying, which is the failure mode the break loop
* exists to replace. Nothing here needs an orderly teardown: the streams are
* flushed by hand above every call.
*
* 134 is kept because that is what a trap exits with; see rt_die in
* flan_rt.c. */
static _Noreturn void die_now(void) {
fflush(stdout);
fflush(stderr);
/* The editor's socket, by hand. In a merged `flan dev' this process is the
* listener as well as the program, and _exit skips the atexit handler that
* would otherwise remove it -- deliberately, for the loader-lock reason
* above. A socket file left behind with nothing accepting on it answers the
* next client with ECONNREFUSED, which reads like a daemon that is there and
* refusing rather than one that died. FLAN_DEV_SOCK is unset in an ordinary
* run, and then this does nothing. */
{
const char *sock = getenv("FLAN_DEV_SOCK");
if (sock != NULL && *sock != '\0') unlink(sock);
}
/* And this program's own agent socket, for the same reason and skipped by
* the same _exit. Empty unless [start_on] bound one, and then this does
* nothing. */
unlink_bound_sock();
_exit(134);
}
/* The loop itself, with one bit of what the caller knows: whether a restart
* taken here has anywhere to land. [break_loop] passes 1 — it holds the
* signaller's channel, and writing a frame into it is how §2 resumes.
* [trap_stop] passes 0, and then this is a place to *stand and read* and
* nothing more: the stack, the locals, the globals and the restart list are
* all still there to be looked at, and only the resume is refused. That is
* strictly more than the alternative, which was the whole process exiting
* before anyone could ask a question. */
static void break_loop_at(const uint8_t *name, int64_t namelen, void *condition,
void *xfer, int resumable) {
struct timespec step = { 0, 2000000 }; /* 2ms */
fflush(stdout);
fprintf(stderr, "\nflan: unhandled %.*s — stopped, not dead.\n",
(int)namelen, (const char *)name);
/* Taken before anything is printed, and before [depth] says there is a
* break to ask about: the list on the terminal and the list on the socket
* are then the same list, numbered the same way, and the numbers are what a
* choice is made of. */
int32_t my_gen;
if (!snap_push(resumable, condition)) {
fflush(stdout);
fprintf(stderr, "flan: %d nested break loops - giving up rather than "
"spinning\n", BREAK_MAX);
fflush(stderr);
die_now();
}
{
snapshot *s = snap_top();
my_gen = s->gen;
/* Said once, above the list, rather than appended to every line: it is
* one fact about the break, and repeating it per restart would read as
* each restart having its own separate problem. The list is still
* printed, because what is on offer is part of where the program is even
* when none of it can be taken — and because the same names come back
* from a `restarts' query, and the terminal and the socket must not be
* describing two different programs. */
if (!s->resumable)
fprintf(stderr,
" nothing here can be resumed into; read the frame, then fix "
"and reload, or abort\n");
else if (s->n == 0)
fprintf(stderr, " no restarts are active; abort, or fix and reload\n");
for (int32_t i = 0; i < s->n; i++)
/* Numbered, because that is how one is taken now, and marked when it is
* not takeable - a restart below the thunk boundary is shown rather
* than hidden, since "why can I not have that one" is a fair question
* and silence is how this went wrong the first time. */
fprintf(stderr, " %2d. restart: %s%s\n", i, s->names + s->off[i],
!s->resumable ? " (cannot be taken from this trap)"
: i == s->boundary
? " (stop running the expression; the program carries on)"
: s->reachable[i] ? ""
: " (below this break; cannot be taken)");
if (s->total > s->n)
fprintf(stderr, " ... and %d more, not listed\n", s->total - s->n);
}
fflush(stderr);
/* What the *outer* loop was reporting, restored on the way out: resuming an
* inner break must not leave the outer one describing a condition that has
* already been answered. */
char outer_name[sizeof condition_name];
memcpy(outer_name, condition_name, sizeof outer_name);
{
size_t k = namelen < 0 ? 0 : (size_t)namelen;
if (k >= sizeof condition_name) k = sizeof condition_name - 1;
memcpy(condition_name, name, k);
condition_name[k] = '\0';
}
/* Published last: the name has to be whole before anything advertises that
* there is one to read. */
if (atomic_fetch_add(&depth, 1) + 1 > BREAK_MAX) {
/* Printed without going near the hook: whatever is erroring is erroring
* inside the machinery that reports errors. */
fflush(stdout);
fprintf(stderr,
"flan: %d nested break loops — giving up rather than spinning\n",
BREAK_MAX);
fflush(stderr);
die_now();
}
for (;;) {
flan_agent_poll();
if (atomic_load(&aborting)) {
fflush(stdout);
fprintf(stderr, "flan: aborted at the break loop\n");
die_now();
}
if (atomic_load(&chosen_ready)) {
/* Claimed into a local and the flag cleared *before* the attempt. The
* other order loses a request that was already answered ok: a [restart]
* arriving during the attempt passes its own check, writes a new name
* and sets the flag, and the store below then erases it. Copying also
* keeps strlen off a buffer the listener may be writing. */
/* Read before the claim: a choice addressed to some other break is
* not this one's to consume. */
if (atomic_load(&chosen_gen) != my_gen) { nanosleep(&step, NULL); continue; }
int take = atomic_load(&chosen_index);
atomic_store(&chosen_ready, 0);
snapshot *s = snap_top();
int ok = s != NULL && s->gen == my_gen && take >= 0 && take < s->n
&& s->resumable && s->reachable[take];
if (ok) {
/* Which of the two things a take is, decided here because this is the
* only place that holds both the choice and the boundary. The transfer
* itself is identical — a frame address into the channel — and only
* the sentence differs. */
flan_restart_take(s->frame[take], xfer);
if (take == s->boundary)
fprintf(stderr,
"flan: the evaluation is abandoned; the program carries on "
"from where it was called. Anything it changed before it "
"stopped stays changed.\n");
else
fprintf(stderr, "flan: resuming at restart %d. %s\n", take,
s->names + s->off[take]);
fflush(stderr);
memcpy(condition_name, outer_name, sizeof condition_name);
/* Cleared with the resume: an abort that passed its check just as the
* game thread resumed would otherwise stay armed and kill the program
* at the *next* unhandled error, minutes later, in unrelated code,
* giving nobody the chance to choose. */
atomic_store(&aborting, 0);
/* Popped *before* the depth comes down. The other order leaves a
* window where [depth] says the outer break is the current one and
* [snap_top] still answers with the inner one's list, so a request
* arriving in it is validated against a list nobody is looking at. */
snap_pop();
atomic_fetch_sub(&depth, 1);
return;
}
/* The listener checks all of this before answering ok, so reaching here
* means the two disagreed - worth saying loudly rather than looping on
* in silence, which is the failure this whole change is about. */
fprintf(stderr, "flan: restart %d is not one this break can take\n", take);
fflush(stderr);
}
nanosleep(&step, NULL);
}
}
/* §2's break, which can resume, and the trap's, which cannot. Two names over
* one loop because the two hooks have different signatures and different
* promises, and a caller of either should not have to pass a flag it does not
* understand.
*
* [trap_stop] is not marked _Noreturn even though it never returns: the loop
* above leaves only by resuming, which needs [resumable], or by [die_now].
* Saying so in the type would oblige this function to prove it, and the honest
* proof is the [die_now] below — reached only if the loop is ever given a way
* out that this call did not ask for, and dying there is what the program did
* before any of this existed. */
static void break_loop(const uint8_t *name, int64_t namelen, void *condition,
void *xfer) {
break_loop_at(name, namelen, condition, xfer, 1);
}
static void trap_stop(const uint8_t *name, int64_t namelen) {
break_loop_at(name, namelen, NULL, NULL, 0);
die_now();
}
/* Re-entrant, and it has to be: a thunk this runs may itself error, and the
* break loop that catches it polls again from inside that very call. So a job
* is *claimed* — tail advanced past it — before it is run, and both indices
* are re-read each time round rather than cached across the work. Caching them
* and storing tail at the end would rewind it over everything the nested poll
* consumed, and running a C-x C-e thunk a second time is the one thing the
* whole dev loop is careful never to do. Still single-consumer: only the game
* thread writes tail, nesting included. */
int32_t flan_agent_poll(void) {
int32_t n = 0;
for (;;) {
unsigned t = atomic_load_explicit(&tail, memory_order_relaxed);
unsigned h = atomic_load_explicit(&head, memory_order_acquire);
if (t == h) return n;
job j = queue[t % QUEUE];
atomic_store_explicit(&tail, t + 1, memory_order_relaxed);
/* The gate the [job] comment argues for, asked at the only moment whose
* answer is worth anything. Claimed first and then dropped, rather than
* left in the ring for a later stop: the address it was built around was
* blessed by a stop that has already ended, and a job that waits for the
* *next* break would run against a blessing older still.
*
* The handle is closed for the reason the listener closes one it refuses
* for having no installer: a transient module nothing ever points into is
* unreachable the moment this returns, and dlopen refcounts by path, so
* dropping the handle on the floor here would leave a count that re-asking
* the same inspection bumps and nothing brings down. [n] is not bumped —
* nothing was installed, and [n] is what a caller polls to find out that
* something was. */
if (j.stopped_only && atomic_load(&depth) <= 0) {
atomic_store(&refused_why, RESUMED);
atomic_fetch_add(&refused_while_running, 1);
if (j.handle != NULL) { dlclose(j.handle); }
continue;
}
/* And the same gate for a job that named a stop. Read from [snap_top] and
* not from [snap_gen], which is the counter and not the stop: after a
* resume [snap_gen] still holds the generation of the break that ended,
* so comparing against it would accept a job whose stop is over. The
* snapshot on top is the stop that is in force. */
if (j.at_stop != 0) {
snapshot *s = snap_top();
if (s == NULL || s->gen != j.at_stop) {
atomic_store(&refused_why, s == NULL ? RESUMED : RESTOPPED);
atomic_fetch_add(&refused_while_running, 1);
if (j.handle != NULL) { dlclose(j.handle); }
continue;
}
}
if (j.install != NULL) { j.install(); n++; }
/* After the install, so a thunk sees the bodies its own module published.
*
* The floor moves for the duration. [flan_reload_call] holds its own
* transfer channel and drops it on return, so every restart frame that
* was on the stack before this call is unreachable from a break inside
* it. Saved and restored rather than set and cleared: this runs from
* inside break loops, which run from inside thunks. */
if (j.call != NULL) {
int32_t outer = restart_floor;
int32_t oframe = frame_floor;
void *obound = eval_boundary;
restart_floor = flan_restart_count();
frame_floor = flan_dev_frame_count();
/* After the floor is read and not before: the floor counts the frames
* that were there when the thunk started, and this one is the thunk's.
* Pushed first it would be below its own boundary and refused. */
eval_boundary = flan_restart_push_c(abandon_name, sizeof abandon_name - 1);
j.call();
/* Popped whichever way the thunk left — returning with a value, or
* unwinding past this frame because someone abandoned it. */
flan_restart_pop_c(eval_boundary);
eval_boundary = obound;
restart_floor = outer;
frame_floor = oframe;
}
if (j.handle != NULL) { dlclose(j.handle); }
}
}
/* The same, but waits up to [ms] for something to arrive first. A game loop
* does not want this; a headless test does, because it makes the reload
* deterministic instead of a race against the frame rate. */
int32_t flan_agent_wait(int32_t ms) {
struct timespec step = { 0, 1000000 }; /* 1ms */
for (int32_t i = 0; i < ms; i++) {
int32_t n = flan_agent_poll();
if (n > 0) return n;
nanosleep(&step, NULL);
}
return flan_agent_poll();
}
/* Where an answer goes. Two kinds, and the verb table below cannot tell them
* apart: a socket, which is how the two-process daemon asks, and a buffer,
* which is how the compiler asks when it is a thread in this same process.
*
* One sink rather than two copies of the verb table. Every answer here is
* assembled from several pieces — a header, a name, a newline — so the seam
* had to be the writing and not the handler, or the handlers would have had
* to be duplicated and would drift. */
typedef struct {
int fd; /* -1 when this is a buffer */
char *buf;
size_t len;
size_t cap;
} sink;
/* MSG_NOSIGNAL rather than write(2). A reply goes out in more than one piece,
* and a sender that has read enough and closed leaves the rest of it writing
* into a closed socket — which is SIGPIPE, whose default action would kill the
* program the agent is embedded in. Suppressing it per call rather than
* installing a handler, because the disposition belongs to the program and not
* to us. */
static void emit(sink *o, const void *p, size_t n) {
if (o->fd >= 0) {
const char *s = (const char *)p;
while (n > 0) {
ssize_t k = send(o->fd, s, n, MSG_NOSIGNAL);
if (k <= 0) return;
s += k;
n -= (size_t)k;
}
return;
}
if (o->len + n + 1 > o->cap) {
size_t want = o->cap ? o->cap : 1024;
while (want < o->len + n + 1) want *= 2;
char *g = realloc(o->buf, want);
/* Nothing useful to do with a failed realloc here: the caller reads
* [len], so a short answer is what it sees, and that is the same shape as
* a socket the peer closed early. */
if (g == NULL) return;
o->buf = g;
o->cap = want;
}
memcpy(o->buf + o->len, p, n);
o->len += n;
o->buf[o->len] = '\0';
}
static void reply(sink *o, const char *s) { emit(o, s, strlen(s)); }
/* A dlopen failure the compiler's two backends are responsible for, turned
* into a sentence that says so.
*
* Flan has two native backends. They agree about every scalar and disagree
* about every aggregate — the x86 dev backend passes a struct by pointer with
* a hidden sret, LLVM classifies it per the SysV psABI — so a redefinition
* module built by one and loaded into a host built by the other links, loads,
* and then dies with SIGSEGV at the first call into a redefined function that
* takes or returns a struct. A dev build therefore defines a marker naming its
* backend and a module holds a pointer to the marker it was built for, which
* is a relocation the loader must resolve while it maps the object. A crossed
* pair has no such symbol and is refused here, before any of the new code
* runs.
*
* What the loader says at that point is "undefined symbol: flan.abi.x86",
* which is true and tells nobody anything. So the marker's name is matched —
* the name, not the loader's phrasing, which is libc's to change — and the
* reason is stated instead. Which marker is missing says which backend built
* the module, and the host is necessarily the other one.
*
* Returns NULL for a failure that is about something else, which is then
* passed through as the loader wrote it. */
static const char *abi_mismatch(const char *err) {
if (err == NULL) return NULL;
if (strstr(err, "flan.abi.x86") != NULL)
return "the module and the running program were built by different "
"backends: the module came from the x86 dev backend and needs "
"flan.abi.x86, which this program does not define. The two "
"backends pass every struct differently, so the pair would die at "
"the first call into a redefined function that takes or returns "
"one. Rebuild the program with --x86 so that both halves agree.";
if (strstr(err, "flan.abi.llvm") != NULL)
return "the module and the running program were built by different "
"backends: the module came from LLVM and needs flan.abi.llvm, "
"which an --x86 program does not define. The two backends pass "
"every struct differently, so the pair would die at the first call "
"into a redefined function that takes or returns one. Rebuild the "
"program without --x86: there is no --x86 spelling for building a "
"redefinition module yet, so the program is the half that has to "
"move.";
return NULL;
}
/* One line, one answer, one module. This is the whole of what the agent is
* asked, and it is reached two ways: from the socket below, and — in a build
* where the compiler is a thread in this same process — by being called. The
* two share this function rather than a protocol.
*
* What does not change with the caller is where the work lands. [dlopen] runs
* here, on whichever thread asked; the install is only ever *published* to the
* ring, and the game thread picks it up at a frame boundary. A direct call
* that installed on the spot would be a frame running half in the old code and
* half in the new. */
/* One request at a time, whoever asks. Until the compiler could call in, the
* accept loop was the only caller and was single-threaded, and the ring's
* "exactly one producer" was true by construction. It is not any more: a
* direct call runs on the compiler thread while the accept loop can be serving
* the same verbs. This lock puts that back rather than weakening the ring —
* [publish]'s room check is separate from its store, and two producers
* interleaving there is the silent overwrite the ring comment above describes.
*
* It is held across the [dlopen], which is milliseconds. That is what the
* accept loop already did to itself by serving connections inline, so no
* caller waits longer than it did before. The game thread never takes it. */
static pthread_mutex_t request_lock = PTHREAD_MUTEX_INITIALIZER;
static void handle_line(char *line, sink *o) {
/* The one verb that is not a module: read back the value of the last
* expression evaluated, with the counter that says whether it is a new
* one. The daemon polls this rather than the agent holding a connection
* open across a frame boundary it does not control. */
/* Only while stopped: what is on offer, and which one to take. Both are
* refused when the program is running, by name, rather than silently
* doing nothing — there is no restart stack to walk from here. */
/* Whether the program is stopped, and on what. Answered while it is
* running too — "running" is an answer, not a refusal — because this is
* the one question an editor asks without knowing the state already, and
* refusing it would leave nothing to poll. */
if (strcmp(line, "status") == 0) {
if ((atomic_load(&depth) > 0)) {
reply(o, "stopped ");
reply(o, condition_name);
reply(o, "\n");
} else
reply(o, "running\n");
return;
}
/* Whether the break on top holds a condition value a render thunk could be
* aimed at. [+] or [-] and nothing else: the pointer itself never crosses
* the wire — an address in another process's frame is not something the
* daemon can read — and the *type* is already answered by [status]. The
* daemon asks this before spending a build on a thunk that would render
* nothing. */
if (strcmp(line, "condition") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no snapshot\n"); return; }
reply(o, s->cond != NULL ? "+\n" : "-\n");
return;
}
/* Where the expression that trapped is written — file:line:col, or [-] for
* a stop that has no site (a user (error ...), a (pause)). The frame lines
* say where each call was; this is the only record of the indexing or the
* division itself. */
if (strcmp(line, "site") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no snapshot\n"); return; }
if (s->sitelen > 0) emit(o, s->site, (size_t)s->sitelen);
else reply(o, "-");
reply(o, "\n");
return;
}
/* One line per restart, innermost first: the index it is taken by, a flag
* for whether it can be taken at all, and the name. The index leads
* because it is the identity - two frames can offer [retry] and only one
* of them is the one meant, which is the whole reason this is not a list
* of names any more. Read from the snapshot, never from the live stack. */
if (strcmp(line, "restarts") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no restart snapshot\n"); return; }
/* Ahead of the entries, and only at a trap: [!] on a line of its own says
* that this break has no transfer channel, so nothing below can be taken
* whatever its flag says.
*
* A line rather than a fourth flag value, because it is a fact about the
* *break* and not about any entry — and because a trap with an empty
* restart list has no entry to carry it, which is exactly the shape
* [dev-trap-null-alloc] has. The terminal listing says the same thing in
* words above its own list; an editor that had to infer the reason from
* the absence of a boundary would caption a segfault with a sentence about
* an evaluation that is not there.
*
* Safely ignored by a client that does not know it: the line does not
* start with an index, so a parser looking for [I F NAME] drops it. */
if (!s->resumable) reply(o, "!\n");
for (int32_t i = 0; i < s->n; i++) {
char hdr[32];
/* Both facts fold into the one flag, because the flag answers one
* question — can this be taken — and a break taken by a trap can take
* none of them. The terminal listing above says the same thing in
* words; the two must not describe different programs.
*
* [*] is a third value of that same flag and not a fourth column: the
* boundary restart is takeable, so it answers the flag's question with
* yes, and the extra thing it says is what taking it means. A client
* that only knows [+] and [-] gets it wrong in the safe direction — it
* reads a takeable restart as takeable, and only misses that this one
* is the way out. */
int k = snprintf(hdr, sizeof hdr, "%d %c ", i,
!(s->resumable && s->reachable[i]) ? '-'
: i == s->boundary ? '*'
: '+');
if (k > 0) emit(o, hdr, (size_t)k);
emit(o, s->names + s->off[i], (size_t)s->len[i]);
reply(o, "\n");
}
reply(o, ".\n");
return;
}
/* Where the stopped thread is. One line per frame, innermost first:
* the index, whether the frame is the program's or the evaluation's, how
* many slots it has, where it is written, and its name. Served from the
* snapshot, never from the live chain — the game thread is parked in the
* break loop, but the loop polls, and a poll runs Flan.
*
* Refused while running, like every other break verb and for the same
* reason: a chain read by one thread while another pushes and pops it is
* not a backtrace, it is a race with a plausible shape. */
if (strcmp(line, "backtrace") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no frame snapshot\n"); return; }
if (s->fn == 0 && s->ftotal == 0) {
/* Not "no frames": a release build has no shadow stack at all, and
* answering with an empty backtrace would read as a program with an
* empty stack, which is not a thing that can be stopped. */
reply(o, "err this program was not built with --dev, so it has no "
"shadow stack to walk\n");
return;
}
for (int32_t i = 0; i < s->fn; i++) {
char hdr[64];
/* Both fingerprints go before the location and the location before
* the name, because the name is the one field that can contain a
* space and so has to be last. */
int k = snprintf(hdr, sizeof hdr, "%d %c %d %d %d ", i,
s->fmine[i] ? '+' : '-', s->fslots[i], s->fsig[i],
s->frsig[i]);
if (k > 0) emit(o, hdr, (size_t)k);
if (s->fllen[i] > 0)
emit(o, s->ftext + s->floff[i], (size_t)s->fllen[i]);
else
reply(o, "?");
reply(o, " ");
emit(o, s->ftext + s->fnoff[i], (size_t)s->fnlen[i]);
reply(o, "\n");
}
if (s->ftotal > s->fn) {
char more[64];
int k = snprintf(more, sizeof more, "... %d\n", s->ftotal - s->fn);
if (k > 0) emit(o, more, (size_t)k);
}
reply(o, ".\n");
return;
}
/* Which of a frame's slots have been bound at the point it stopped. One
* line per slot: the index and [+] or [-].
*
* The daemon asks this before it builds a thunk, and that order is the
* safety: an unbound slot is a null address, a thunk that rendered one
* would dereference it, and a program stopped in a break loop is the last
* place to take a fault. It is answered from the snapshot, so the set the
* daemon is told about is the set the thunk will resolve against.
*
* It says nothing about *what* a slot holds, or what it is called. Those
* are facts about the build, and the daemon owns the build — [Tast.fn]
* carries [slots] and [snames] beside each other. Sending them from here
* would be a second copy of them that could drift. */
if (strncmp(line, "locals ", 7) == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no frame snapshot\n"); return; }
char *end = NULL;
long at = strtol(line + 7, &end, 10);
if (end == line + 7) { reply(o, "err locals wants a frame index\n"); return; }
if (at < 0 || at >= s->fn) { reply(o, "err no frame at that index\n"); return; }
if (s->fslots[at] == 0) {
reply(o, "err that frame records no slots; it has no named local, or "
"this build does not record them\n");
return;
}
for (int32_t i = 0; i < s->fslots[at]; i++) {
char l[32];
int k = snprintf(l, sizeof l, "%d %c\n", i,
flan_dev_frame_slot(s->fframe[at], i) ? '+' : '-');
if (k > 0) emit(o, l, (size_t)k);
}
reply(o, ".\n");
return;
}
/* Take the i'th, optionally checking that the caller and this snapshot
* still agree on what the i'th is called. The name is not the lookup -
* that is the bug - it is a receipt: a client that listed, prompted, and
* chose has the name in hand for nothing, and sending it turns a bare
* integer into something that can be wrong out loud. */
if (strncmp(line, "restart-at ", 11) == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no restart snapshot\n"); return; }
char *end = NULL;
long idx = strtol(line + 11, &end, 10);
if (end == line + 11) { reply(o, "err restart-at wants an index\n"); return; }
if (idx < 0 || idx >= s->n) {
reply(o, "err no restart at that index\n");
return;
}
while (*end == ' ') end++;
if (*end != '\0') {
size_t k = strlen(end);
if (k != (size_t)s->len[idx] ||
memcmp(end, s->names + s->off[idx], k) != 0) {
reply(o, "err that index is now ");
reply(o, s->names + s->off[idx]);
reply(o, ", not what you named; list the restarts again\n");
return;
}
}
/* Checked before [reachable], because it is the stronger fact and the one
* with the better sentence: at a trap every restart is unreachable, and
* answering with the thunk-boundary reason would send the reader looking
* for an evaluation that is not there. */
if (!s->resumable) {
reply(o, "err this break was taken by a trap with no transfer channel, "
"so no restart can be taken from it; read the frame, then fix "
"and reload, or abort\n");
return;
}
if (!s->reachable[idx]) {
/* Refused, with the reason, rather than accepted and dropped. The
* transfer would unwind to the thunk this break is inside and stop
* there, and the program would carry on as if nothing had been
* chosen. */
reply(o, "err restart ");
reply(o, s->names + s->off[idx]);
reply(o, " is below the evaluation this break is inside, so a "
"transfer to it has nowhere to land; choose one offered "
"above it, or abort\n");
return;
}
atomic_store(&chosen_index, (int)idx);
atomic_store(&chosen_gen, s->gen);
/* Published last, so the game thread never reads an index that is about
* to change, or one whose generation has not arrived yet. */
atomic_store(&chosen_ready, 1);
/* "ok", and for the boundary "ok abandon". Said on the acceptance rather
* than looked up afterwards, because afterwards there is nothing to look
* it up in: the take resumes the stopped thread and the snapshot it was
* resolved against is popped. The daemon used to re-ask [restarts] before
* sending, purely to word its note — a second round trip on the verb a
* person is waiting on, for a fact this end already holds. */
reply(o, idx == s->boundary ? "ok abandon\n" : "ok\n");
return;
}
/* By name, still, for a person at a raw socket - and now defined as
* exactly [restart-at] on the first index offering the name, which is
* what §4's walk already meant. So the two verbs cannot disagree, and a
* shadowed name is reachable through the other one rather than nowhere. */
if (strncmp(line, "restart ", 8) == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(o, "err no restart snapshot\n"); return; }
size_t k = strlen(line + 8);
if (k == 0) { reply(o, "err bad restart name\n"); return; }
int32_t at = -1;
for (int32_t i = 0; i < s->n && at < 0; i++)
if ((size_t)s->len[i] == k && memcmp(s->names + s->off[i], line + 8, k) == 0)
at = i;
if (at < 0) {
reply(o, "err no restart named ");
reply(o, line + 8);
reply(o, " is active\n");
return;
}
if (!s->resumable) {
reply(o, "err this break was taken by a trap with no transfer channel, "
"so no restart can be taken from it; read the frame, then fix "
"and reload, or abort\n");
return;
}
if (!s->reachable[at]) {
reply(o, "err restart ");
reply(o, line + 8);
reply(o, " is below the evaluation this break is inside, so a "
"transfer to it has nowhere to land; choose one offered "
"above it, or abort\n");
return;
}
atomic_store(&chosen_index, at);
atomic_store(&chosen_gen, s->gen);
atomic_store(&chosen_ready, 1);
/* The same two answers as [restart-at], because this verb is defined as
* that one on the first index offering the name. Two verbs that resolve to
* the same frame must not report it differently. */
reply(o, at == s->boundary ? "ok abandon\n" : "ok\n");
return;
}
if (strcmp(line, "abort") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(o, "err not stopped\n"); return; }
reply(o, "ok\n");
atomic_store(&aborting, 1);
return;
}
/* "watch on" / "watch off" arm and disarm the table; bare "watch" reads it.
*
* Arming is a message rather than something the daemon infers, because the
* program is the writer and it has to be told. Nothing writes the table
* while it is off, which is the whole of "watching costs nothing when nobody
* is watching" — see flan_dev.c. */
if (strcmp(line, "watch on") == 0 || strcmp(line, "watch off") == 0) {
flan_dev_watch_enable(line[6] == 'o' && line[7] == 'n');
reply(o, "ok\n");
return;
}
/* "watch reset" opens a new window for the numeric accumulators: their
* count, range and mean start again from the next sample, while a text slot
* is untouched.
*
* A separate command rather than a side effect of the read, which was the
* tempting shape and is the wrong one. A destructive read makes *looking*
* change what is there, so anything that polls — a test's [await], a second
* editor, a person pressing the read twice — silently shortens the window
* and gets a count that is noise. Reading is free and resetting is a
* decision; the editor makes it once per tick, after the read. */
if (strcmp(line, "watch reset") == 0) {
flan_dev_watch_reset();
reply(o, "ok\n");
return;
}
if (strcmp(line, "watch") == 0) {
/* One line per slot: NAME<tab>VALUE. The name cannot contain a tab — it is
* a C identifier-ish string the program passed — and the value cannot
* contain a raw tab or newline, because everything that reaches it goes
* through an emitter that escapes both. So no framing is needed beyond
* this, which is the same bet render_locals makes on the same grounds.
*
* A header first: the count actually written, and how many names were
* whether any name ever found no slot, so an overflow is reported rather
* than showing up as a value that never appears. A flag rather than a
* count, because the count would be of *writes* — see flan_dev.c. */
uint64_t ncap = flan_dev_watch_name_cap();
uint64_t vcap = flan_dev_watch_val_cap();
char *nb = malloc((size_t)ncap + 1);
char *vb = malloc((size_t)vcap + 1);
if (nb == NULL || vb == NULL) {
free(nb); free(vb);
reply(o, "err out of memory reading the watch table\n");
return;
}
/* Allocating here is fine, for [result]'s reason: this is the listener
* thread. The table the game thread writes is a fixed static. */
uint32_t n = flan_dev_watch_count();
char hdr[64];
int k = snprintf(hdr, sizeof hdr, "%lu %d\n", (unsigned long)n,
flan_dev_watch_overflowed());
if (k > 0) emit(o, hdr, (size_t)k);
for (uint32_t i = 0; i < n; i++) {
uint64_t vlen = 0;
/* A torn slot is *still listed*, with an empty value. Dropping the row
* would make the buffer's rows move under the reader every time the
* game happened to be mid-write, which is far worse to look at than one
* value that is blank for a tick. */
flan_dev_watch_read(i, nb, ncap, vb, vcap, &vlen);
emit(o, nb, strlen(nb));
emit(o, "\t", 1);
if (vlen > 0) emit(o, vb, (size_t)vlen);
emit(o, "\n", 1);
}
free(nb); free(vb);
return;
}
/* How many stopped-only jobs the game thread has dropped for arriving at a
* frame boundary instead of at a break, and the sentence to say about it.
* Answered while running and while stopped alike — it is a count of things
* that have already happened, not a claim about the program's state now.
*
* The count leads and the text follows, because the daemon needs the count
* to decide and the text only to speak. It reads this before it delivers a
* stopped-only module and again while it waits for the value, and a number
* that moved in between is the answer. */
if (strcmp(line, "refusals") == 0) {
char hdr[32];
int k = snprintf(hdr, sizeof hdr, "%llu\n",
(unsigned long long)atomic_load(&refused_while_running));
if (k > 0) emit(o, hdr, (size_t)k);
const char *why = atomic_load(&refused_why);
reply(o, why != NULL ? why : RESUMED);
reply(o, "\n");
return;
}
/* Which stop the program is at, as a number that is never reused and never
* zero — zero being "it is not stopped". [status] cannot answer this: two
* stops at the same [(error (Boom …))] are both "stopped Boom", and telling
* them apart is the whole question a write has to ask before it stores into
* a frame somebody rendered a moment ago.
*
* Its own verb rather than a field on [status] or [backtrace], because both
* of those have readers in flight and a reply format is a thing two ends
* agree on. Answered while running as well, for [status]'s reason: an
* editor polls this without knowing the state already. */
if (strcmp(line, "stop") == 0) {
snapshot *s = (atomic_load(&depth) > 0) ? snap_top() : NULL;
char hdr[32];
int k = snprintf(hdr, sizeof hdr, "%d\n", s == NULL ? 0 : s->gen);
if (k > 0) emit(o, hdr, (size_t)k);
return;
}
if (strcmp(line, "result") == 0) {
uint64_t gen = 0, len = 0;
uint64_t cap = flan_dev_result_cap();
char *v = malloc(cap ? (size_t)cap : 1);
if (v == NULL) { reply(o, "err out of memory reading the result\n"); return; }
/* Allocating here is fine and is the distinction that matters: this runs
* on the listener thread, or on the compiler thread in a merged build.
* What the game thread writes into is a fixed static in flan_dev.c
* precisely so that *it* allocates nothing. */
flan_dev_result_read(v, cap, &gen, &len);
char hdr[64];
int k = snprintf(hdr, sizeof hdr, "%llu %llu\n", (unsigned long long)gen,
(unsigned long long)len);
if (k > 0) {
emit(o, hdr, (size_t)k);
if (len > 0) emit(o, v, (size_t)len);
}
free(v);
return;
}
/* "reg at ADDR" — what the allocation registry records for one address.
*
* The reader's side of the table, and the one question the renderer's own
* two cannot answer. Inside a thunk the type at the far end of a pointer is
* already static — (Ptr Enemy) says Enemy — so [reg-live] and [reg-emit]
* only ever needed permission. Somebody pointing at a bare address has no
* (Ptr T) to read a type off, and the recorded name is the whole of what
* there is to go on.
*
* It used to say here that this is answered while the program runs as well
* as while it is stopped, on the grounds that a table is not a stack. The
* table is not a stack and it is still written by the other thread, and the
* answer to one address is a claim that stops being true as it is made — so
* the gate below now says what the daemon already said.
*
* ADDR is read with base 0, so both 0x-hex and decimal arrive; an editor
* that has an address as text has it in one of those two spellings. */
if (strncmp(line, "reg at ", 7) == 0) {
const char *type = NULL;
int64_t typelen = 0, off = 0, bytes = 0, elem = 0, seq = 0, died = 0;
char *end = NULL;
unsigned long long a;
/* Stopped only, like every break verb. Whether an address is still live is
* exactly what a running program is changing, so the answer would describe
* a table the game thread has already moved on from — the daemon refuses
* the question for that reason before it ever reaches here
* (lib/dev.ml, [inspect_addr]), and this says the same thing to anything
* else that asks. The two listing verbs below are the opposite case and
* stay answerable while running: a breakdown is a description of the
* program and not a claim about one address, and the table carries its own
* seqlock so that reading it while it is written is safe. */
if (!(atomic_load(&depth) > 0)) {
reply(o, "err not stopped: whether one address is still live is what a "
"running program is changing, so this is read from a stopped "
"one\n");
return;
}
if (!flan_dev_reg_enabled()) {
reply(o, "err the allocation registry is off; this is not a dev build\n");
return;
}
a = strtoull(line + 7, &end, 0);
if (end == line + 7 || a == 0) {
reply(o, "err reg at wants an address\n");
return;
}
if (!flan_dev_reg_at((const void *)(uintptr_t)a, &type, &typelen, &off,
&bytes, &elem, &seq, &died)) {
/* Never heard of it, which is a fact and not a failure: a stack local,
* a global, or a pointer from C. The daemon says which of those it
* might be; this says only that the table has nothing. */
reply(o, "none\n");
return;
}
{
char hdr[128];
int k = snprintf(hdr, sizeof hdr, "ok %d %lld %lld %lld %lld %lld\t",
died == 0 ? 1 : 0, (long long)off, (long long)bytes,
(long long)elem, (long long)seq, (long long)died);
if (k > 0) emit(o, hdr, (size_t)k);
/* Last, and after a tab, because a type spelling holds spaces —
* "(Vec i32)" — and nothing else on the line does. It cannot hold a tab
* or a newline: it is Types.to_string of a type the programmer wrote. */
if (typelen > 0) emit(o, type, (size_t)typelen);
reply(o, "\n");
}
return;
}
/* "reg types" — the whole table grouped by type spelling; "reg leaks" — the
* same walk with the dead left out.
*
* One verb would have done with a flag, and two exist because the two
* questions are asked at different moments and read differently: a
* breakdown is "what is this program made of", a leak report is "what is
* still held". The *walk* is one function in flan_dev.c for exactly that
* reason — two of them would drift.
*
* A header first, like [watch]: how many rows follow, and whether the table
* ever overflowed. The second is not decoration — an overflowed table has
* blocks in the program that are in nobody's row, so every number below is
* a floor, and a reader that could not tell would quote them as counts. */
if (strcmp(line, "reg types") == 0 || strcmp(line, "reg leaks") == 0) {
enum { REG_ROWS = 256 };
/* Allocated here and freed before the reply is finished, not declared as
* static arrays. 256 rows of four words is 8KB, and static would put that
* 8KB in the BSS of every build this package is linked into — including a
* release build of a game that imports the agent, which never writes a
* row. That is flan_dev.c's own argument against a fixed table, at a
* thirty-second of the size, and it is the same rule: a release build
* carries a null pointer and the declarations.
*
* Allocating is legal here for [watch]'s reason and no other: this runs
* on the listener thread, or on the compiler thread in a merged build.
* The table the game thread writes is a fixed static in flan_dev.c
* precisely so that *it* allocates nothing. */
int64_t *counts = malloc(REG_ROWS * sizeof *counts);
int64_t *bytes = malloc(REG_ROWS * sizeof *bytes);
int64_t *typelens = malloc(REG_ROWS * sizeof *typelens);
const char **types = malloc(REG_ROWS * sizeof *types);
int64_t n, i, unread = 0;
int live_only = line[4] == 'l';
if (counts == NULL || bytes == NULL || typelens == NULL || types == NULL) {
free(counts); free(bytes); free(typelens); free(types);
reply(o, "err out of memory reading the allocation registry\n");
return;
}
if (!flan_dev_reg_enabled()) {
free(counts); free(bytes); free(typelens); free(types);
reply(o, "err the allocation registry is off; this is not a dev build\n");
return;
}
n = flan_dev_reg_by_type(live_only ? 1 : 0, counts, bytes, types, typelens,
REG_ROWS, &unread);
/* A read that did not settle is refused in a sentence rather than sent as
* a header saying zero rows. The header cannot carry the difference: a row
* count is a number, and the number that means "I could not look" is the
* same number that means "nothing is held" — which is the answer this verb
* is most often asked to disprove. An [err] line is already a shape the
* daemon turns into the editor's message, so the distinction survives all
* the way to the person who asked.
*
* Both sentences name the *program's* behaviour and not the mechanism,
* because the mechanism is not actionable and the behaviour is: a table
* being rewritten this hard is a game thread allocating flat out, and the
* useful next move is to ask again, or to ask it while stopped. */
if (n < 0) {
free(counts); free(bytes); free(typelens); free(types);
if (unread > 0) {
char msg[192];
int k = snprintf(msg, sizeof msg,
"err the allocation registry could not be read whole: "
"%lld slot%s were being written through every one of "
"eight attempts, so any rows here would be missing "
"blocks; ask again\n",
(long long)unread, unread == 1 ? "" : "s");
/* Clamped, because [snprintf] returns the length it wanted rather
* than the length it wrote — [flan_dev_reg_emit]'s rule. */
if (k > (int)sizeof msg - 1) k = (int)sizeof msg - 1;
if (k > 0) emit(o, msg, (size_t)k);
} else {
reply(o, "err the allocation registry was being rearranged through "
"every one of eight attempts; no listing taken, rather than "
"one of a table that never existed — ask again\n");
}
return;
}
{
char hdr[64];
int k = snprintf(hdr, sizeof hdr, "%lld %d\n",
(long long)(n < REG_ROWS ? n : REG_ROWS),
flan_dev_reg_overflowed());
if (k > 0) emit(o, hdr, (size_t)k);
}
for (i = 0; i < n && i < REG_ROWS; i++) {
char row[64];
int k = snprintf(row, sizeof row, "%lld %lld\t", (long long)counts[i],
(long long)bytes[i]);
if (k > 0) emit(o, row, (size_t)k);
if (typelens[i] > 0) emit(o, types[i], (size_t)typelens[i]);
reply(o, "\n");
}
free(counts); free(bytes); free(typelens); free(types);
return;
}
/* A module that may only run from a break, and the word for it goes in front
* of the path rather than inside the module. The alternative was a fourth
* exported symbol beside [flan_reload_transient], which would have put the
* decision in both emitters and in the .so's ABI — for a fact that is not
* about the module's *contents* at all. What is stopped-only is the
* *question*, not the code: the same rendering machinery, rooted at a frame
* slot instead of a raw address, is perfectly safe to run while the program
* runs. So it travels with the request. See [job].
*
* Everything after the prefix is the ordinary path, gate included, so a
* stopped-only module with no installer is still refused for having no
* installer. */
int stopped_only = 0;
if (strncmp(line, "stopped-only ", 13) == 0) {
stopped_only = 1;
line += 13;
}
/* [at-stop N ] travels the same way and for the same reason, and it goes
* after [stopped-only ] so that a module which is both spells it
* "stopped-only at-stop 7 /path". Nothing sends both today — naming a stop
* already implies one — but the two prefixes answer different questions and
* a parser that made them exclusive would have to be revisited the first
* time something wants the pair. The number is parsed here rather than
* trusted: a path beginning "at-stop " with no number after it is a path,
* and treating it as a malformed prefix would lose the module. */
int32_t at_stop = 0;
if (strncmp(line, "at-stop ", 8) == 0) {
char *end = NULL;
long n = strtol(line + 8, &end, 10);
if (end != line + 8 && *end == ' ' && n > 0 && n <= 0x7fffffff) {
at_stop = (int32_t)n;
line = end + 1;
}
}
/* Before the dlopen, not after it: a module there is no room to queue is
* one there is no point relocating, and refusing here means no handle is
* taken for it at all. Only one producer runs at a time, so room seen now is
* room still there at [publish] below. */
if (!queue_room()) {
reply(o, "err reload queue full; the program is not calling "
"agent/poll\n");
return;
}
void *h = dlopen(line, RTLD_NOW | RTLD_LOCAL);
if (h == NULL) {
/* [dlerror] is one-shot and the next dl call may clobber what it returned,
* so the pointer is taken once and used for both the test and the reply. */
const char *err = dlerror();
const char *why = abi_mismatch(err);
reply(o, "err ");
reply(o, why != NULL ? why : err);
reply(o, "\n");
return;
}
install_fn f = (install_fn)(uintptr_t)dlsym(h, "flan_reload_install");
if (f == NULL) {
/* Closed, and this is not an exception to "nothing is ever dlclosed".
* That rule is about a module something *points into* — a cell holding
* an address in its text. This one published nothing: no installer ran,
* so no cell names it, and it is unreachable the moment this function
* returns. What leaked before was the handle value rather than the
* mapping — dlopen refcounts by path, so re-sending the same bad file
* bumped a count nothing could ever bring down, and the one reference
* that could was dropped on the floor here. */
dlclose(h);
reply(o, "err no flan_reload_install\n");
return;
}
/* Optional: only an expression evaluation has one. */
call_fn c = (call_fn)(uintptr_t)dlsym(h, "flan_reload_call");
/* And only one that leaves nothing behind may be unloaded. */
void *transient =
(c == NULL) ? NULL : dlsym(h, "flan_reload_transient");
/* Answer before queueing, not after. The game thread can install and run
* to completion between the two, and a program that exits there would tear
* down this connection with the reply still unwritten — which reaches the
* sender as a reset, not as an answer. */
reply(o, "ok\n");
/* "queued", not "installed": the store happens on the game thread, at a
* time this thread does not get to choose. The room was checked before the
* dlopen and only this thread consumes it, so this cannot fail; it is
* asserted rather than assumed because a silent [publish] that did nothing
* is the failure being fixed. */
if (!publish((job){ .install = f, .call = c,
.handle = transient == NULL ? NULL : h,
.stopped_only = stopped_only, .at_stop = at_stop }))
fprintf(stderr, "flan: reload queue full after it was checked\n");
return;
}
/* One connection, one line, one module. Loading here rather than in the game
* thread is the whole reason this thread exists. */
static void serve(int fd) {
/* A deadline on the read. The accept loop is single-threaded and serves each
* connection inline, so a client that connects and then sends nothing - an
* editor killed mid-request, a daemon that crashed between connect and send -
* blocks every later request, including the [abort] that would end a stopped
* program. Worse, the requests the client had already given up on are served
* when its socket finally closes, so an abandoned abort can kill the program
* minutes later against a state that has moved on. Two seconds is generous
* for one line. */
{
struct timeval tv = { 2, 0 };
setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof tv);
}
sink o = { fd, NULL, 0, 0 };
char line[4096];
size_t n = 0;
for (;;) {
ssize_t k = read(fd, line + n, sizeof line - n - 1);
if (k <= 0) return;
n += (size_t)k;
line[n] = '\0';
char *nl = strchr(line, '\n');
if (nl == NULL) {
if (n == sizeof line - 1) { reply(&o, "err path too long\n"); return; }
continue;
}
*nl = '\0';
pthread_mutex_lock(&request_lock);
handle_line(line, &o);
pthread_mutex_unlock(&request_lock);
return;
}
}
/* The direct hand-off. In a [flan dev] build the compiler is a thread in this
* process, so a request that used to be a connect, a write and a read on a
* unix socket looping back into this address space is now this call — the same
* verb table, answered into a buffer instead of onto an fd.
*
* The buffer is malloc'd here and freed by the caller. It is not a static,
* because the socket's accept loop can be answering [status] on another thread
* while this runs, and two answers into one buffer is a race with a plausible
* shape.
*
* The rule this must not break is the one that keeps the collector off the
* frame thread: nothing here calls into OCaml and nothing here runs on the
* game thread. A delivery is left in the ring exactly as the listener leaves
* it, and the game thread picks it up when it reaches a frame boundary. */
char *flan_agent_request(const char *line, uint64_t *len) {
char stack[4096];
sink o = { -1, NULL, 0, 0 };
size_t n = strlen(line);
if (n >= sizeof stack) {
/* The same answer the socket gives for the same input. The point of one
* verb table is that the two callers cannot be told apart, and a silent
* empty reply here would be the first place they could. */
reply(&o, "err path too long\n");
*len = (uint64_t)o.len;
return o.buf;
}
memcpy(stack, line, n + 1);
pthread_mutex_lock(&request_lock);
handle_line(stack, &o);
pthread_mutex_unlock(&request_lock);
*len = (uint64_t)o.len;
return o.buf;
}
void flan_agent_request_free(char *p) { free(p); }
static void *accept_loop(void *arg) {
(void)arg;
for (;;) {
int fd = accept(listen_fd, NULL, NULL);
if (fd < 0) { if (errno == EINTR) continue; return NULL; }
serve(fd);
close(fd);
}
}
/* ── Outliving the daemon, which this program must not do ──────────── */
/* Under [--two-process] the compiled program is a child of the daemon, and
* for the whole of its life the daemon is the only thing that will ever end
* it: the program is a game loop or a listener, it has no reason of its own to
* stop, and lib/dev.ml's [Fun.protect ~finally] is what kills it when the
* session closes. That finally block runs when the daemon exits. It does not
* run when the daemon is SIGKILLed — by a test harness tearing down after a
* failure, by a watchdog that fired, by anybody at all — and what is left
* behind is a program with ppid 1, sleeping, holding a window and a socket,
* waiting for requests from a process that no longer exists. Eight of those
* were found on one machine, the oldest six days old. Nothing would ever have
* woken them: there is no timeout anywhere in this file, and none should be
* added — a program between frames is supposed to wait indefinitely.
*
* So the child is told to die with its parent instead, which on Linux is one
* call. PR_SET_PDEATHSIG asks the kernel to deliver a signal to *this* process
* when the process that is currently its parent dies. It is exactly the right
* primitive and it has exactly one sharp edge, which is that it is armed from
* the child and so is armed a moment *after* the fork: if the parent died in
* that moment, the signal it would have sent has already not been sent, and
* the child waits forever having done everything right. The standard answer is
* the one below — arm, then ask who the parent is now, and if it is not the
* one that was expected, deliver the signal by hand. [getppid() == 1] is the
* form this idiom is usually written in and it is wrong here: under a systemd
* user session, or in a container with a subreaper, an orphan is reparented to
* the subreaper and not to init, so the test would pass and the child would
* still be stranded. The daemon therefore puts its own pid in the environment
* and this compares against that, which cannot be fooled by who does the
* adopting.
*
* FLAN_DEV_PARENT is a positive gate and is set by [two_process] and by
* nothing else. A merged build runs this same file and must never arm any of
* this: there the daemon *is* this process, its parent is whoever typed [flan
* dev] — a shell, an emacs, a terminal that is about to be closed — and none
* of those is the session's owner. A person who starts a daemon in one
* terminal and attaches an editor to it from another is doing a normal thing,
* and killing their live session because the shell exited would be a worse bug
* than the leak. Gating on the *absence* of a merged-build variable would have
* covered the same cases today and quietly stopped covering them the first
* time somebody added a third shape; this way the only process that arms is
* the one the daemon explicitly told to.
*
* The one gap left is the window before [agent/start]: a child SIGKILLed out
* from under during the first few milliseconds of its own startup is still
* stranded, because nothing has armed yet. It is not closed here because
* closing it means arming from flan_rt.c, which every program links and which
* would put a Linux-only prctl in the one file that also has to compile for
* wasm. The daemon already refuses to run a program that never reaches
* [agent/start], so every child that lives long enough to matter passes
* through here. */
#if defined(__linux__) && defined(SIGPWR)
#define FLAN_ORPHAN_SIG SIGPWR
#endif
#if defined(FLAN_ORPHAN_SIG)
/* Three calls, and the restraint is the point. This runs on whichever thread
* the kernel picked, which may be the game thread halfway through a printf,
* holding stdio's lock — so fprintf here would deadlock against itself and
* fflush(NULL) would deadlock against the other thread. write, unlink and
* _exit are on the async-signal-safe list and nothing else here is.
*
* Not flushing is a real cost and a small one: flan_rt_init sets stdout line
* buffered precisely so that a program's output is observable as it runs, so
* what is lost is at most a partial line. Losing it beats hanging.
*
* The socket is unlinked because this process bound it and no other process
* knows it is stale — the daemon that would have cleaned up is the thing that
* just died. Its directory is left alone: that belongs to the daemon, and
* tidying up someone else's directory from a signal handler is how a signal
* handler becomes a bug.
*
* Status 0, because nothing went wrong. The program did what it was told for
* as long as there was anybody to tell it. */
static void orphan_die(int sig) {
static const char said[] =
"flan dev: the daemon that owns this program is gone, so the program is "
"stopping too\n";
ssize_t ignored;
(void)sig;
ignored = write(2, said, sizeof said - 1);
(void)ignored;
if (bound_sock[0] != '\0') unlink(bound_sock);
_exit(0);
}
#endif
/* Called after the bind: the handler it arms unlinks [bound_sock], so it must
* not be able to fire before that is set. */
static void watch_the_daemon(void) {
#if defined(FLAN_ORPHAN_SIG)
const char *want_s = getenv("FLAN_DEV_PARENT");
char *end;
long want;
struct sigaction sa;
if (want_s == NULL || want_s[0] == '\0') return;
want = strtol(want_s, &end, 10);
if (end == want_s || *end != '\0' || want <= 0) return;
/* SIGPWR and not SIGTERM, and the collision it avoids is not hypothetical:
* lib/dev.ml's ordinary teardown kills this child with SIGTERM, and in
* --two-process the child's stderr *is* the daemon's stderr. A handler on
* SIGTERM would print "the daemon is gone" into the daemon's own output on
* every clean session close, which is both noise and a sentence that is only
* half true. A signal the program will never otherwise receive keeps the two
* deaths distinguishable, and leaves SIGTERM on its default disposition
* where the existing teardown already relies on it. */
memset(&sa, 0, sizeof sa);
sa.sa_handler = orphan_die;
sigfillset(&sa.sa_mask);
if (sigaction(FLAN_ORPHAN_SIG, &sa, NULL) != 0) return;
if (prctl(PR_SET_PDEATHSIG, FLAN_ORPHAN_SIG) != 0) return;
/* The race, closed. Armed above, so from here on the kernel will tell us;
* this asks whether it already should have. */
if ((long)getppid() != want) raise(FLAN_ORPHAN_SIG);
#endif
}
/* Bind [path], start the listener, and arrange the two things that depend on
* having bound: the socket's removal and the orphan watch.
*
* Three answers, not two, because the callers below need to tell "bound it
* just now" from "somebody already had": 1 is already started, 0 is bound
* here, -1 could not listen. Only the caller that chose the path itself cares,
* and it cares because it is the one that prints it.
*
* Starting twice is a no-op answering the first socket. That is the whole of
* the idempotence the zero-argument form needs: a program that gets the
* listener for free under [flan dev] and *also* writes [(agent/start ...)] of
* its own must not end up with two listeners, and the second call is the one
* that has to give way — the first is the one whose socket the daemon is
* already talking to.
*
* [started] is what makes that true, and it is claimed at the top because two
* threads may arrive at once. A FAILURE MUST THEREFORE GIVE IT BACK. This is
* not tidiness: the constructor runs before main and does not report anything,
* so a latched flag would mean an unbindable path silently disarms every later
* start — the program's own [(agent/start ...)] would answer 0 with no socket,
* no listener and no hooks, which is worse than the error it replaced. So
* every way out below that is not a listening socket puts it back and answers
* -1, and a second attempt is a real attempt. */
static int32_t start_on(const char *path) {
struct sockaddr_un addr;
size_t len;
int fd = -1;
/* Whether the bind got as far as making the file, which is what has to be
* taken away again on a later failure. Nothing else knows it is there: it is
* not in [bound_sock] yet, so no handler would remove it. */
int made = 0;
if (atomic_exchange(&started, 1)) return 1;
len = strlen(path);
if (len == 0 || len >= sizeof addr.sun_path) goto failed;
memset(&addr, 0, sizeof addr);
addr.sun_family = AF_UNIX;
memcpy(addr.sun_path, path, len);
unlink(addr.sun_path);
fd = socket(AF_UNIX, SOCK_STREAM, 0);
if (fd < 0) goto failed;
if (bind(fd, (struct sockaddr *)&addr, sizeof addr) < 0) goto failed;
made = 1;
if (listen(fd, 4) < 0) goto failed;
/* Published together: the fd the accept loop reads and the path the three
* exits unlink. Both before the listener thread and before either handler,
* so that nothing which uses them can run while they are still empty. */
listen_fd = fd;
memcpy(bound_sock, addr.sun_path, len + 1);
atexit(unlink_bound_sock);
if (pthread_create(&listener, NULL, accept_loop, NULL) != 0) {
/* Published above and now taken back, in the reverse order. The atexit
stays registered — there is no way to withdraw one — and an empty
[bound_sock] is what makes it a no-op. */
bound_sock[0] = '\0';
listen_fd = -1;
goto failed;
}
/* After the bind, because a program that is going to fail to listen should
fail on its own terms rather than arrange its death first. */
watch_the_daemon();
/* From here an unhandled error stops rather than dying. Installed with the
* socket and not before it: without a listener there is nobody to ask what
* to do, and stopping forever is worse than the abort it replaces. */
flan_break_hook = break_loop;
flan_trap_hook = trap_stop;
return 0;
failed:
if (made) unlink(addr.sun_path);
if (fd >= 0) close(fd);
atomic_store(&started, 0);
return -1;
}
/* [path] is a Flan string: ptr and len, not NUL-terminated.
*
* FLAN_AGENT_SOCKET overrides it. A program's source has to name some path,
* and the daemon that launches the program is the one that knows where it
* wants to talk to it — without the override the daemon would have to guess,
* and guessing wrong fails silently: everything compiles, the module is built,
* and nothing ever receives it. */
int32_t flan_agent_start(const uint8_t *path, int64_t len) {
char buf[sizeof(((struct sockaddr_un *)0)->sun_path)];
const char *env = getenv("FLAN_AGENT_SOCKET");
if (env != NULL && env[0] != '\0') return start_on(env) < 0 ? -1 : 0;
if (len <= 0 || (size_t)len >= sizeof buf) return -1;
memcpy(buf, path, (size_t)len);
buf[len] = '\0';
return start_on(buf) < 0 ? -1 : 0;
}
/* The zero-argument form: (agent/start), with nowhere named.
*
* Under [flan dev] there is a right answer and the daemon has already written
* it down — the same FLAN_AGENT_SOCKET the explicit form honours — so this
* binds there and says nothing. The daemon knows where it put the program's
* socket; a line about it would only go down the pipe the editor reads.
*
* Outside the daemon there is no right answer, so it invents one that no other
* program will collide with and prints it, because a socket nobody can name is
* a socket nobody can connect to. The pid makes it this process's; the clock
* makes a second run of the same program a different path rather than one that
* silently reuses a file it may not own. stderr rather than stdout, so a
* program whose output is data stays data. */
int32_t flan_agent_start_auto(void) {
char path[sizeof(((struct sockaddr_un *)0)->sun_path)];
struct timespec ts;
int32_t r;
const char *env = getenv("FLAN_AGENT_SOCKET");
if (env != NULL && env[0] != '\0') return start_on(env) < 0 ? -1 : 0;
if (clock_gettime(CLOCK_REALTIME, &ts) != 0) ts.tv_nsec = 0;
snprintf(path, sizeof path, "/tmp/flan-agent-%ld-%08lx.sock",
(long)getpid(), (unsigned long)(ts.tv_nsec & 0xffffffffL));
r = start_on(path);
/* Only when this call is the one that bound: a second [(agent/start)] would
* otherwise print a path it did not bind and nothing is listening on. */
if (r == 0) {
fprintf(stderr, "flan agent: listening on %s\n", path);
fflush(stderr);
}
return r < 0 ? -1 : 0;
}
/* And the call itself, gone. A program under [flan dev] that imports this
* package gets the listener before main, without asking.
*
* FLAN_AGENT_SOCKET is the whole condition, and it is the right one: the
* daemon sets it in both shapes — before the fork in --two-process, before the
* exec in the merged build — and nothing else on a machine sets it. So an
* ordinary run of an ordinary program falls straight through here and this
* costs it one getenv. (Not FLAN_DEV_PARENT, which is deliberately unset in
* the merged build; gating on it would quietly skip half the daemon.)
*
* WHAT THIS DOES NOT REACH, because it is a fact about linking rather than a
* choice made here: [Reach] prunes a package nothing calls into, and an
* executable that never mentions the agent does not link this file at all — so
* there is no constructor in it to run. Auto-start covers a program that calls
* [(agent/poll)] and has dropped its [(agent/start)], which is the ceremony
* this was asked to remove. A program that wants a dev loop while calling
* nothing at all would need the package force-linked into every --dev build,
* which is a decision for Load and Build and not for this file.
*
* The window it closes is real and was costing seconds: a program that opened
* a window before its [(agent/start)] left the daemon waiting on a socket that
* did not exist yet (DISCUSS.org). Bound here, it exists before main.
*
* A pthread from a constructor is fine — it is this executable's own init, not
* a dlopen, so nothing holds the loader lock against it — and the thread it
* starts does nothing until something connects. The narrow hazard is a request
* arriving before [flan_rt_init]: the verbs read runtime state, and a person
* with nc and very quick fingers could ask before there is any. The daemon
* cannot, in either shape — it has an editor to hear from first, and a module
* to compile after that. */
__attribute__((constructor)) static void auto_start(void) {
const char *env = getenv("FLAN_AGENT_SOCKET");
if (env == NULL || env[0] == '\0') return;
/* The answer is dropped because there is nobody to give it to: this is ELF
* init, before main, before the program has decided anything. What matters
* is that a failure here is not final — [start_on] gives [started] back, so
* a program's own [(agent/start ...)] still tries for itself and still says
* -1 if it cannot listen either. A daemon that named a path nothing can bind
* finds out the way it always did, from the program. */
(void)start_on(env);
}
Reference in New Issue View Git Blame Copy Permalink