JosephFerano/flan
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

A frame per call in a dev build, and a stopped program can say where it is

Browse Source 
plan.org has specified a shadow stack in the dev column since the beginning
and nothing had ever built it. A frame is four words on the calling
function's own stack: the one it displaced, a pointer to a static
description of the function, and two words reserved for its locals. The
name and the location travel on the frame, so a backtrace needs no debug
information, no symbol table, and nothing from the platform unwinder that
plan.org deliberately does not use.

The pop is at every ret, the landing block a transfer leaves through
included. That is the half that is easy to get wrong: a pop written only on
the normal path leaves a dead frame behind every handled error, and the
test takes five breaks and resumes all of them by transfer before asking
for two frames.

(:op "backtrace") answers from a snapshot the stopped thread takes, beside
the restarts and for the same reason, and marks which frames belong to the
program and which to the evaluation the break is inside. It is refused
while the program runs.

Measured, interleaved, three pairs of binaries: 29% on 600 frames of sand,
7.6% on a benchmark that is nothing but calls -- 32us per frame of sand, a
fifth of a percent of a frame at 60fps. An array with a stack pointer was
built and timed as the alternative and is worse on both.
This commit is contained in:
Joseph Ferano 2026-09-12 11:49:25 +07:00
parent ce59f90707
commit 6ce4282337
7 changed files with 581 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
BUILT.md
View File

@ -1094,6 +1094,80 @@ a socket. A refusal is nil, not an error: the buffer already draws a section exp
turning `C-c C-b` into an error would take away the restarts — the decision the buffer exists for — over a missing
annotation.
### The shadow stack, and `backtrace`
plan.org's "Dev vs release builds" table has had **Frames: shadow stack** in the dev column since the project began and
nothing had ever built it. This is it. It is what `(:op "backtrace")` is made of, and it is dev-only, so a shipped game
pays nothing — the same bargain the indirection cells already make.
Chosen over the DWARF route deliberately, and the author's reason is the one that decides it: **the more a break loop
can show, the less often a real debugger is needed.** DWARF buys frames in lldb; this buys them in the break loop,
where someone already is when they want them.
A frame is four words on the calling function's own stack — the frame it displaced, a pointer to a static description
of the function, and two words reserved for locals. The description is per function and not per call, because nothing
about a function changes between two calls to it: the qualified name, `file:line:col` as `Loc` spells it, and how many
slots the frame has. So the name in a backtrace comes off the frame itself and needs no debug information, no symbol
table and no agreement with the optimiser about what a stack frame looks like. It is the same mechanism on wasm32,
which is the other reason it is not `.eh_frame`: plan.org lowers every non-local exit explicitly rather than through
platform unwinding, so there is no unwinder here to borrow.
The compiler emits the push and the pop **inline** rather than calling into the runtime. This is on every call in a dev
build, and a call made to record a call would be most of what it costs.
**The pop is at every `ret`, and the transfer path is the one that matters.** There are five: an explicit `return` with
a value and without, the `none` arm of `(some x)`, the tail of the body, and the landing block a transfer leaves
through. `emit.ml` funnels all five through one `ret`, because the way to get this wrong is to write the pop on the
normal path and not on the other one — and then every *handled* error leaves a dead frame behind, and the backtrace
after the fifth one is five frames of fiction. `test_dev.ml` takes five breaks and resumes all of them by condition
transfer before asking for a backtrace, and the answer has to be two frames. Same lesson `with-allocator` learned about
restoring at the pad.
**A plain global, not a thread-local.** It matches what the handler stack and the restart stack in `flan_rt.c` already
assume: one thread runs Flan, and the listener thread runs C and the loader and never enters a Flan body. If the
language grows threads this becomes thread-local and the compiler's two stores become TLS-relative, which is the whole
of the change.
**The chain is snapshotted on the stopped thread**, into the same `snapshot` the restarts are copied into, at the same
moment and for exactly the same reason: the break loop *polls*, a poll runs Flan, and a chain read by the listener
thread is a chain that can be popped underneath the reader. Names are copied as bytes rather than kept as pointers,
because a transient `C-x C-e` module does get `dlclose`d and its rodata with it. Frame *addresses* are kept beside the
text, because reading a frame's locals means going back to that frame and not to whatever is at index 2 by then.
```
(:op "backtrace") → (:status "ok" :frames (("fetch" "/game.flan:15:7" "program" 0)
("main" "/game.flan:27:7" "program" 0))
:more 0 :stopped t :condition "Missing")
```
Innermost first. `:more` is how many frames deep recursion left off the end — the innermost ones are what the question
is about. **`origin` is `"program"` or `"eval"`**: a break inside a `C-x C-e` thunk has that thunk's frames on top of
the program's, and "where is my program" answered with `eval/7` is true and not the question. The boundary is recorded
at the call, in `flan_agent_poll`, exactly where `restart_floor` is and for the same reason — except that its
out-of-a-thunk value is `-1` rather than `0`, because zero restarts on the stack is a real answer and zero frames
belonging to the program is not.
**Refused while the program is running**, like every other break verb. The chain is the game thread's and it is pushed
and popped on every call; a walk of it from the daemon would have the shape of a backtrace and the contents of a race.
**What it costs, measured rather than assumed.** Three compilers built, three pairs of binaries, timed interleaved so
that machine load falls on all of them:
| Dev build, -O2 | without frames | with |
|---|---|---|
| 600 frames of sand's simulation | 64.9 ms | 84.0 ms (+29%) |
| fib(30) plus 20M calls in a loop | 90.6 ms | 97.5 ms (+7.6%) |
That is **32 µs per frame of sand**, or 0.19% of a 16.6 ms frame at 60fps, and about 0.3 ns per call. The dev loop's
premise is that redefinition does not stutter a running game, and a fifth of a percent of a frame does not.
The shape was chosen by that measurement and not before it. An array with a stack pointer — no alloca, no address
escaping — was built and timed as the obvious alternative, and it is *worse* on both benchmarks (sand +34%, fib +27%):
the frame record on the stack is already hot, and the array's indexed store into a megabyte of BSS is not. It also has
a fixed depth, which the chain does not. One measurement in between said the opposite, loudly, and was an artefact of
comparing a 40-frame binary with a 600-frame one — which is why all six numbers above come from binaries built in one
sitting and run alternately.
### Conditions — step 2: `restart-case` and `invoke-restart`
`spec-conditions.md` §3 to §6: the transfer. A handler runs where the signal was, decides, and control resumes at a

10
NEXT.md
View File

@ -689,10 +689,12 @@ sanitized sweep (`@sanitize`) is under the same watchdog but has never been obse
it for free: the string the break loop already reports *is* that name, because `Emit.struct_name_of` writes
`Types.Named` into `flan_error`. It is still open for **locals**, where DWARF gives a name and the name a debugger
reads is not qualified by anything.
- **`(:op "backtrace")` is blocked** on frame metadata — unlocked by the DWARF work, then a new agent verb. Locals are
blocked twice: DWARF for the frame layout, *and* the pointer-rooted render thunk. Restart source locations and
arity are blocked too — `flan_restart` carries `prev`, `name_id`, `name` and `namelen`, so both need a new field in
the frame, which means the compiler emitting it.
- ~~**`(:op "backtrace")` is blocked** on frame metadata.~~ **Built**, and not out of DWARF: decision 3's shadow
stack carries the name and the location on the frame itself, so a backtrace needs no debug information at all. See
BUILT.md, "The shadow stack, and `backtrace`", for what it costs — 0.19% of a 60fps frame. Locals are no longer
blocked on a frame layout either; what is left of them is the pointer-rooted render thunk. Restart source locations
and arity are still blocked — `flan_restart` carries `prev`, `name_id`, `name` and `namelen`, so both need a new
field in the frame, which means the compiler emitting it.
### One line away

92
lib/dev.ml
View File

@ -231,6 +231,55 @@ let restarts t =
end
| exception Unix.Unix_error (e, _, _) -> Error (Unix.error_message e)
(* Where a stopped program is, one frame per line, innermost first — the same
framing [restarts] uses, terminated by a lone dot, because it comes back
over the same one-line-out socket.
Each line is [I ± NSLOTS LOC NAME]. The flag says whether the frame belongs
to the program or to the C-x C-e thunk the break happens to be inside: a
break inside an evaluation has that evaluation's frames on top, and
answering "where is my program" with [eval/7] would be true and useless.
[LOC] is the frame's own — it travels in the module that defined the body,
so a redefined function reports where the *installed* body is written and
not where the one this daemon first built was. A frame with none says [?].
A truncated backtrace ends [... N] before the dot; deep recursion is the
case, and the innermost frames are the ones the question is about. *)
let backtrace t =
match ask t "backtrace" with
| text ->
let lines =
List.map String.trim (String.split_on_char '\n' text)
in
if List.exists (fun l -> String.length l >= 3 && String.sub l 0 3 = "err") lines
then Error (String.trim text)
else begin
let more = ref 0 in
let parse line =
if String.length line > 4 && String.sub line 0 4 = "... " then begin
(match int_of_string_opt (String.sub line 4 (String.length line - 4)) with
| Some n -> more := n
| None -> ());
None
end
else
match String.split_on_char ' ' line with
| idx :: flag :: nslots :: loc :: rest when rest <> [] ->
(match int_of_string_opt idx, int_of_string_opt nslots with
| Some _, Some k ->
Some (String.concat " " rest, (if loc = "?" then "" else loc),
flag = "+", k)
| _ -> None)
| _ -> None
in
let frames =
List.filter_map parse
(List.filter (fun l -> l <> "" && l <> ".") lines)
in
Ok (frames, !more)
end
| exception Unix.Unix_error (e, _, _) -> Error (Unix.error_message e)
let alive t =
match Unix.waitpid [ Unix.WNOHANG ] t.child with
| 0, _ -> true
@ -594,6 +643,48 @@ let break t =
rs) ]
| Error m -> error ("the program refused to list its restarts: " ^ m))
(* [(:op "backtrace")] — the frames of a stopped program, innermost first.
NEXT.md's "Asked for by the editor lanes" had this blocked on exactly the
frame metadata the shadow stack now carries.
Refused while the program is running, and that is not a gap in the feature:
the chain is the game thread's, it is pushed and popped on every call, and
a walk of it from this end while that thread runs would produce a plausibly
shaped answer that was never true. Stopped, the thread is parked in the
break loop and the program itself takes the snapshot.
Each frame is [(name loc origin nslots)] — four fields in the shape [defs]
already uses, so an editor reads it with [read] and nothing else. [origin]
is "program" or "eval": a break inside a C-x C-e thunk has the thunk's
frames above the program's, and they are shown and labelled rather than
hidden, the same decision [:unreachable] makes for the restarts under one.
[nslots] is how many slots the frame has, which is what a client asks about
before asking for any of them. *)
let backtrace_op t =
if not (alive t) then error "the program exited; restart flan dev"
else
match state t with
| Running ->
error
"the program is running; a backtrace is only taken while it is stopped, \
because the frame chain is the game thread's and it is changing"
| Unreachable m -> error ("cannot ask the program where it is: " ^ m)
| Stopped _ ->
(match backtrace t with
| Ok (frames, more) ->
ok
[ ":frames "
^ Wire.list
(List.map
(fun (name, loc, mine, nslots) ->
Wire.list
[ Wire.quote name; Wire.quote loc;
Wire.quote (if mine then "program" else "eval");
string_of_int nslots ])
frames);
Printf.sprintf ":more %d" more ]
| Error m -> error ("the program refused to say where it is: " ^ m))
(* A choice is validated by the *program*, on its listener thread, against a
stack the stopped game thread is holding still — not here. The daemon has no
copy of that stack and anything it checked would be a guess that was true a
@ -982,6 +1073,7 @@ let handle t req =
| Some "describe" -> describe t
| Some "defs" -> defs t
| Some "break" -> break t
| Some "backtrace" -> backtrace_op t
| Some "layout" ->
(match Wire.string_field req "type" with
| Some ty -> layout t ~ty

113
lib/emit.ml
View File

@ -216,6 +216,15 @@ type m = {
and nothing else. *)
sanitize : bool;
mutable nstr : int;
(* The frame descriptors a dev build's shadow stack points at, counted apart
from [nstr] deliberately. [nstr] is the test [redefinition] uses to decide
whether an expression thunk's module may be unloaded — a string literal in
the module image is something the program may still be pointing at after
the thunk returns. A frame descriptor is not: the frames that named it
were popped on the way out, and the break loop copies the bytes it shows
rather than keeping the pointer. Counting these in [nstr] would silently
stop every C-x C-e module from ever being unloaded. *)
mutable nfi : int;
}
(* The attribute group every emitted function names, empty unless sanitizing.
@ -400,6 +409,11 @@ type f = {
declared on -- the fallback for a node the checker made up. *)
dsub : int option;
dline : int;
(* The value the shadow-stack head held when this function was entered, in a
dev build: [Some %fprev]. Every [ret] restores it — see [ret] — which is
what makes the pop happen on the transfer path as well as the normal one.
[None] in a release build, where there is no frame at all. *)
mutable frame : string option;
(* The [, !dbg !N] suffix every instruction in this function carries, or "".
Uniform rather than only on the instructions that want a line: LLVM's
verifier rejects a call without a location inside a function that has
@ -429,6 +443,20 @@ let label f name =
Buffer.add_string f.b (Printf.sprintf "\n%s:\n" name);
f.live <- true
(* Every [ret] in a function body goes through here, which is the whole of how
the shadow stack's pop is got right. There are five of them — an explicit
[return] with a value and without, the [none] arm of [(some x)], the tail of
the body, and the landing block a transfer leaves through — and the last of
those is the one that matters: a condition handled further out unwinds past
this frame, so a pop written only on the normal path leaves a dead frame on
the stack after every handled error, and the next backtrace is a lie. Same
lesson [emit_with_alloc] learned about the context allocator. *)
let ret f v =
(match f.frame with
| Some prev -> ins f "store ptr %s, ptr @flan_frame_head" prev
| None -> ());
term f "ret %s %s" (ll f.ret) v
let alloca f ty =
let name = fresh f in
Buffer.add_string f.allocas (Printf.sprintf " %s = alloca %s\n" name (ll ty));
@ -485,6 +513,34 @@ let cstring m s =
id (String.length s + 1) (escape s));
id
(* ── The shadow stack's descriptors ──────────────────────────────────── *)
(* One [flan_fninfo] per function in a dev build: the name and the location,
as bytes and lengths, plus how many slots the frame has. It is static data —
nothing about a function changes between two calls to it — so a frame stores
a pointer to this and not six fields of its own.
The strings go through [m.nfi] rather than [string_bytes], which is the
whole of why that counter exists; see [m.nfi]. *)
let fi_bytes m s =
let id = Printf.sprintf "@\".fi.%d\"" m.nfi in
m.nfi <- m.nfi + 1;
Buffer.add_string m.strs
(Printf.sprintf "%s = private unnamed_addr constant [%d x i8] c\"%s\"\n"
id (String.length s) (escape s));
id, String.length s
let fninfo m (fn : Tast.fn) ~nslots =
let nid, nlen = fi_bytes m fn.Tast.name in
let lid, llen = fi_bytes m (Loc.to_string fn.Tast.floc) in
let id = Printf.sprintf "@\".fi.%d\"" m.nfi in
m.nfi <- m.nfi + 1;
Buffer.add_string m.strs
(Printf.sprintf
"%s = private unnamed_addr constant %%fninfo { ptr %s, i64 %d, ptr %s, i64 %d, i32 %d, i32 0 }\n"
id nid nlen lid llen nslots);
id
(* ── Bounds checks ───────────────────────────────────────────────────── *)
(* A failure is a branch to a [noreturn] call and then [unreachable] — the same
@ -616,10 +672,10 @@ and value_at f (e : Tast.expr) : string =
| Tast.While (c, body) -> emit_while f c body; "zeroinitializer"
| Tast.Return v ->
(match v with
| None -> term f "ret %s zeroinitializer" (ll f.ret)
| None -> ret f "zeroinitializer"
| Some v ->
let v' = value f v in
term f "ret %s %s" (ll f.ret) v');
ret f v');
"zeroinitializer"
| Tast.Set (p, v) ->
let ptr, ty = place f p in
@ -1264,7 +1320,7 @@ and emit_unwrap f ty v =
let ln = fresh_label f "none" and lc = fresh_label f "some" in
term f "br i1 %s, label %%%s, label %%%s" isnone ln lc;
label f ln;
term f "ret %s zeroinitializer" (ll f.ret);
ret f "zeroinitializer";
label f lc;
let out = fresh f in
ins f "%s = extractvalue %s %s, 1" out oty ov;
@ -1590,6 +1646,7 @@ let emit_fn m ?(hidden = false) ?(pnames = []) (fn : Tast.fn) =
slots = Array.init n (fun i -> Printf.sprintf "%%s%d" i);
slot_tys = fn.Tast.slots;
pads = []; unwind = "unwind"; unwound = false; defers = fn.Tast.fdefers;
frame = None;
dsub;
dline = (if fn.Tast.floc.Loc.line = 0 then 1 else fn.Tast.floc.Loc.line);
dloc = "";
@ -1608,6 +1665,43 @@ let emit_fn m ?(hidden = false) ?(pnames = []) (fn : Tast.fn) =
Buffer.add_string f.allocas
(Printf.sprintf " store %s %%p%d, ptr %s\n" (ll ty) i f.slots.(i)))
fn.Tast.params;
(* The shadow stack's push, in the entry block, and the pop is at every
[ret] (see [ret]). plan.org has had *Frames: shadow stack* in the dev
column since the beginning; this is it, and it is dev-only, so a shipped
game pays nothing for it.
Inline rather than a call in either direction: this is on every call in a
dev build, and a call to record a call would be most of what it costs. The
record is four words on this frame's own stack, of which two are stored
from static data and two are reserved for the slots [locals] needs -- they
are written, not left as whatever the stack held, because a frame with a
garbage [slots] pointer is one the break loop could follow.
A lifted handler-bind clause pushes one like any other function, which is
right: it really is on the stack, and a backtrace that skipped it would
show a gap exactly where the handler ran. *)
if m.dev then begin
let info = fninfo m fn ~nslots:0 in
let prev = fresh f in
Buffer.add_string f.allocas " %frame = alloca %flanframe
";
Buffer.add_string f.allocas
(Printf.sprintf " %s = load ptr, ptr @flan_frame_head
" prev);
Buffer.add_string f.allocas
(Printf.sprintf " store ptr %s, ptr %%frame
" prev);
List.iter
(fun line -> Buffer.add_string f.allocas (" " ^ line ^ "\n"))
[ "%frame.i = getelementptr inbounds %flanframe, ptr %frame, i32 0, i32 1";
Printf.sprintf "store ptr %s, ptr %%frame.i" info;
"%frame.s = getelementptr inbounds %flanframe, ptr %frame, i32 0, i32 2";
"store ptr null, ptr %frame.s";
"%frame.k = getelementptr inbounds %flanframe, ptr %frame, i32 0, i32 3";
"store i64 0, ptr %frame.k";
"store ptr %frame, ptr @flan_frame_head" ];
f.frame <- Some prev
end;
(* One [llvm.dbg.declare] per slot, in the entry block beside the alloca it
describes. This is the whole of what lldb needs to print a local: the slot
is ordinary stack storage of an ordinary machine type, so there is no
@ -1668,7 +1762,7 @@ let emit_fn m ?(hidden = false) ?(pnames = []) (fn : Tast.fn) =
(* A Unit function's body may end on a form of any type — the value is
discarded, so the return is the Unit constant rather than that value. *)
if Types.equal fn.Tast.ret Types.Unit then last := "zeroinitializer";
term f "ret %s %s" (ll fn.Tast.ret) !last;
ret f !last;
(* The transfer exit, spec-conditions.md §5 and §6. A transfer that reached
the top of this function without a restart-case to catch it leaves the
same way a [return] does — which is what reuses the existing return path,
@ -1689,7 +1783,7 @@ let emit_fn m ?(hidden = false) ?(pnames = []) (fn : Tast.fn) =
f.pads <- [];
ins f "store ptr %s, ptr %s" tgt xfer_param
end;
term f "ret %s zeroinitializer" (ll fn.Tast.ret);
ret f "zeroinitializer";
(* A defer that starts a *second* transfer while the first is unwinding.
§6's per-frame slot nests, but nothing here does: the first transfer's
target is in hand and the defers are half run. Refused loudly rather
@ -1769,6 +1863,13 @@ let header = {|; Generated by flan. The layout is C's: no object headers anywher
; ends disagree. The first four fields are what the runtime's own
; [flan_restart] declares and their offsets do not move.
%restart = type { ptr, i32, ptr, i64, ptr, i32, i32, i32, ptr, i64 }
; A shadow-stack frame and the static description of the function that pushed
; it (runtime/flan_dev.c). Dev builds only: [emit_fn] pushes one on entry and
; every [ret] restores the head, the transfer path included. A release build
; emits neither, and the head below is then a symbol nothing in the .ll names.
%fninfo = type { ptr, i64, ptr, i64, i32, i32 }
%flanframe = type { ptr, ptr, ptr, i64 }
@flan_frame_head = external global ptr
declare void @flan_rt_init(i32, ptr)
declare void @flan_argv(ptr)
@ -1892,7 +1993,7 @@ let new_module ~checks ~dev ~known ?(debug = false) ?(sanitize = false)
out = Buffer.create 8192; strs = Buffer.create 512;
structs = Hashtbl.create 16; globals = Hashtbl.create 16;
externs = Hashtbl.create 32;
checks; dev; known; nstr = 0; sanitize;
checks; dev; known; nstr = 0; nfi = 0; sanitize;
dbg = (if debug then Some (new_dbg p) else None);
} in
List.iter (fun (s : Tast.structure) -> Hashtbl.replace m.structs s.Tast.sname s)

92
runtime/flan_dev.c
View File

@ -297,3 +297,95 @@ int flan_dev_result_read(char *dst, uint64_t cap, uint64_t *gen,
*len = 0;
return 0;
}
/* ── The shadow stack ───────────────────────────────────────────────── */
/* plan.org's "Dev vs release builds" has had *Frames: shadow stack* in the dev
* column since the beginning. This is it, and it exists for one reason: the
* more a break loop can show, the less often a real debugger is needed. A
* stopped program that cannot say where it is sends someone to lldb.
*
* Native unwinding would be the other route and is deliberately not taken:
* plan.org lowers every non-local exit explicitly rather than through platform
* unwinding, so there is no .eh_frame walk to borrow, and a frame pointer walk
* gives addresses that only DWARF can turn back into names. A pushed record
* carries the name itself, is the same on wasm32 as it is here, and needs no
* agreement with the optimiser about what a frame looks like.
*
* The compiler emits the push and the pop inline (emit.ml, [emit_fn]) rather
* than calling in here: this is on every call in a dev build, and a call to
* record a call would double the thing being measured.
*
* A plain global, not a _Thread_local. It matches what the handler stack and
* the restart stack in flan_rt.c already assume — one thread runs Flan; the
* listener thread runs C and the loader and never enters a Flan body. If the
* language ever grows threads this becomes thread-local and the compiler's
* two stores become TLS-relative, which is the only change.
*
* Nothing in here walks the chain while the game thread is running. The break
* loop snapshots it on the stopped thread, exactly as it snapshots the restart
* list and for exactly the same reason: a chain read by another thread is a
* chain that can be popped underneath the reader. The accessors below are the
* snapshot's, and take the frame they were given rather than re-reading the
* head. */
typedef struct {
const char *name; /* the Flan name, qualified; not NUL-terminated */
int64_t namelen;
const char *loc; /* file:line:col, as Loc spells it */
int64_t loclen;
int32_t nslots;
int32_t spare;
} flan_fninfo;
typedef struct flan_frame {
struct flan_frame *prev;
const flan_fninfo *info;
/* Where each slot lives, and which of them have been bound at the point the
* frame was interrupted. Both are null/zero unless the build records them;
* see emit.ml. Read through [flan_dev_frame_slot], never directly. */
void **slots;
uint64_t init;
} flan_frame;
/* The compiler names this symbol directly. A redefinition module reaches it
* the same way it reaches any other host global — through the dynamic symbol
* table, which [--dev] links with -rdynamic. */
flan_frame *flan_frame_head;
/* [i] counts from the innermost. NULL past the end, which is how a caller
* learns the depth without a second walk. */
void *flan_dev_frame_at(int32_t i) {
flan_frame *f = flan_frame_head;
while (f != NULL && i > 0) { f = f->prev; i--; }
return f;
}
int32_t flan_dev_frame_count(void) {
int32_t n = 0;
for (flan_frame *f = flan_frame_head; f != NULL; f = f->prev) {
n++;
if (n > 100000) break; /* a corrupt chain says so rather than hanging */
}
return n;
}
const char *flan_dev_frame_name(const void *frame, int64_t *len) {
const flan_frame *f = frame;
if (f == NULL || f->info == NULL) { *len = 0; return NULL; }
*len = f->info->namelen;
return f->info->name;
}
const char *flan_dev_frame_loc(const void *frame, int64_t *len) {
const flan_frame *f = frame;
if (f == NULL || f->info == NULL) { *len = 0; return NULL; }
*len = f->info->loclen;
return f->info->loc;
}
int32_t flan_dev_frame_nslots(const void *frame) {
const flan_frame *f = frame;
return (f == NULL || f->info == NULL) ? 0 : f->info->nslots;
}

91
test/test_dev.ml
View File

@ -416,6 +416,49 @@ let () =
fail "restarts on offer: %s" (String.concat ", " names)
| _ -> fail "break did not list the restarts");
(* Where it is, which is the other half of what a stopped program can
be asked. The shadow stack is dev-only and the daemon owns the
build, so a frame per Flan call is there to be walked; the names
come off the frames themselves rather than out of any DWARF, which
is what makes this work in the break loop rather than in lldb.
Innermost first, [main] last, and both marked as the program's:
nothing is being evaluated here, so nothing is the evaluation's. *)
let frames r =
match Wire.field r "frames" with
| Some { Form.v = Form.List l; _ } ->
List.filter_map
(fun (e : Form.t) ->
match e.Form.v with
| Form.List
({ Form.v = Form.Str n; _ }
:: { Form.v = Form.Str loc; _ }
:: { Form.v = Form.Str origin; _ } :: _) ->
Some (n, loc, origin)
| _ -> None)
l
| _ -> []
in
let r = ask "(:op \"backtrace\")" in
if status r <> "ok" then
fail "backtrace: %s"
(Option.value ~default:(status r) (Wire.string_field r "message"))
else begin
match frames r with
| [ ("fetch", floc, "program"); ("main", _, "program") ] ->
(* Absolute and pointing into the program's own source, for the
same reason [defs] is: an editor is not in this process's
working directory. It comes off the frame, not off this end's
session, so a redefined body reports where the *installed* one
is written. *)
if String.length floc = 0 || floc.[0] <> '/' then
fail "a frame's location is not absolute: %s" floc
| fs ->
fail "backtrace of a stopped program: %s"
(String.concat ", "
(List.map (fun (n, _, o) -> n ^ "/" ^ o) fs))
end;
(* The payoff. The break loop is the poll loop, so an expression
evaluated here is a module the listener queues and the *stopped*
thread runs — which is the only reason C-x C-e works at the one
@ -483,6 +526,19 @@ let () =
if unreachable <> [ 2; 3 ] then
fail "positions below the thunk: %s"
(String.concat ", " (List.map string_of_int unreachable));
(* And the backtrace says the same thing the restart list does, in its
own words: the two frames on top belong to the evaluation, the two
below them to the program. A backtrace that did not draw that line
would answer "where is my program" with [eval/1], which is true and
not the question. *)
(match frames (ask "(:op \"backtrace\")") with
| [ ("fetch", _, "eval"); (thunk, _, "eval"); ("fetch", _, "program");
("main", _, "program") ]
when String.length thunk > 5 && String.sub thunk 0 5 = "eval/" -> ()
| fs ->
fail "backtrace at a break inside a thunk: %s"
(String.concat ", "
(List.map (fun (n, _, o) -> n ^ "/" ^ o) fs)));
(* Refused, and refused *here* — not accepted and dropped. *)
let r = ask "(:op \"restart-at\" :index 2 :name \"retry\")" in
if status r <> "error" then
@ -537,6 +593,13 @@ let () =
let r = ask "(:op \"abort\")" in
if status r <> "error" then
fail "an abort was accepted by a running program";
(* Refused for the same reason, and it is not a missing feature: the
frame chain is the game thread's and it is pushed and popped on
every call, so a walk from this end would have the shape of a
backtrace and the contents of a race. *)
let r = ask "(:op \"backtrace\")" in
if status r <> "error" then
fail "a running program answered with a backtrace";
(* ...and an ordinary evaluation works again on the far side of it. *)
let r =
ask "(:op \"eval-expr\" :code \"(+ 1 1)\" :file \"/tmp/buf.flan\")"
@ -596,6 +659,34 @@ let () =
if not (await (fun () -> stopped (ask "(:op \"describe\")"))) then
fail "the program never stopped on the body that errors"
else begin
(* The claim this test exists for, and the one thing about a shadow
stack that is easy to get wrong: **the pop happens on the
transfer path too**. Five breaks have been taken and resumed by
now, every one of them by a condition transfer that unwound past
the frame that erred. A pop written only on the normal return
path would have left one dead frame behind each time, and this
backtrace would be [step, main] with a pile of stale [fetch]es
under it. It is two frames or the feature is a liar. *)
(match
List.map (fun (n, _, o) -> (n, o))
(match Wire.field (ask "(:op \"backtrace\")") "frames" with
| Some { Form.v = Form.List l; _ } ->
List.filter_map
(fun (e : Form.t) ->
match e.Form.v with
| Form.List
({ Form.v = Form.Str n; _ }
:: { Form.v = Form.Str loc; _ }
:: { Form.v = Form.Str o; _ } :: _) ->
Some (n, loc, o)
| _ -> None)
l
| _ -> [])
with
| [ ("step", "program"); ("main", "program") ] -> ()
| fs ->
fail "frames left on the shadow stack by five handled errors: %s"
(String.concat ", " (List.map (fun (n, o) -> n ^ "/" ^ o) fs)));
let r = ask "(:op \"abort\")" in
if status r <> "ok" then
fail "abort was refused by a stopped program: %s"

119
vendor/agent/flan_agent.c vendored
View File

@ -146,6 +146,15 @@ extern void (*flan_break_hook)(const uint8_t *name, int64_t namelen,
extern int32_t flan_break_resume(const uint8_t *name, int64_t namelen,
void *xfer);
extern int32_t flan_restart_count(void);
/* The shadow stack (runtime/flan_dev.c). The compiler pushes a frame per Flan
* call in a dev build; these read one, and only ever on the thread that owns
* it. The frame is opaque here on purpose: its shape belongs to flan_dev.c,
* and two files each declaring the struct is how the two stop agreeing. */
extern int32_t flan_dev_frame_count(void);
extern void *flan_dev_frame_at(int32_t i);
extern const char *flan_dev_frame_name(const void *frame, int64_t *len);
extern const char *flan_dev_frame_loc(const void *frame, int64_t *len);
extern int32_t flan_dev_frame_nslots(const void *frame);
extern const uint8_t *flan_restart_name(int32_t i, int64_t *len);
extern void *flan_restart_frame(int32_t i);
extern void flan_restart_take(void *frame, void *xfer);
@ -169,6 +178,18 @@ extern void flan_restart_take(void *frame, void *xfer);
* breaks and whose break runs another thunk nests correctly. */
static int32_t restart_floor;
/* The same boundary, counted in shadow-stack frames. A break inside a C-x C-e
* thunk has that thunk's frames on top of the program's, and "where is my
* program" answered with [eval/7] is not the question anyone asked. Recorded
* at the call for the same reason [restart_floor] is: the frames below it are
* exactly the ones that were already on the stack when the thunk started. */
/* -1, not 0, and the distinction is the whole of it: [restart_floor] can be
* zero meaning "no restarts were on the stack when the thunk started", and
* outside a thunk zero is also the right answer. Frames are the other way
* round — outside a thunk *every* frame is the program's, and zero would say
* none of them are. So "not inside a thunk" gets a value of its own. */
static int32_t frame_floor = -1;
/* What the listener thread hands the stopped game thread. One slot, because
* only one thread is ever stopped. */
/* A *depth*, not a flag. A thunk this loop runs may itself error, and the
@ -213,6 +234,8 @@ static _Atomic int aborting;
* comes from here and nothing re-reads the live stack. */
#define SNAP_MAX 64 /* restarts offered at one break */
#define SNAP_NAMES 4096 /* bytes of names behind them */
#define FRAME_MAX 64 /* frames listed in a backtrace */
#define FRAME_TEXT 8192 /* bytes of names and locations */
typedef struct {
int32_t gen; /* never reused, never 0 */
@ -223,6 +246,21 @@ typedef struct {
int32_t reachable[SNAP_MAX];
int32_t used;
char names[SNAP_NAMES];
/* Where the stopped thread is, taken at the same moment and for the same
* reason: the chain is the game thread's, and it is holding still only
* because it is parked in this loop. [fframe] is kept as well as the text,
* because reading a frame's locals means going back to that frame — and to
* that frame rather than to whatever is at index 2 by then. */
int32_t fn; /* frames listed */
int32_t ftotal; /* before FRAME_MAX truncated it */
int32_t fused;
void *fframe[FRAME_MAX];
int32_t fnoff[FRAME_MAX], fnlen[FRAME_MAX];
int32_t floff[FRAME_MAX], fllen[FRAME_MAX];
int32_t fslots[FRAME_MAX];
int32_t fmine[FRAME_MAX]; /* 0 = the evaluation's, not the
* program's */
char ftext[FRAME_TEXT];
} snapshot;
/* One per nested break loop, because an inner break must not answer with the
@ -267,6 +305,42 @@ static int snap_push(void) {
s->names[s->used++] = 0;
s->n++;
}
/* And the frames, from the same held-still stack. A deep recursion is
* truncated rather than followed: the innermost frames are the ones the
* question is about, and the count says how many were left out. */
{
int32_t fn = flan_dev_frame_count();
s->ftotal = fn;
s->fused = 0;
s->fn = 0;
for (int32_t i = 0; i < fn && s->fn < FRAME_MAX; i++) {
void *fr = flan_dev_frame_at(i);
int64_t nl = 0, ll = 0;
const char *nm, *lc;
if (fr == NULL) break;
nm = flan_dev_frame_name(fr, &nl);
lc = flan_dev_frame_loc(fr, &ll);
if (nl < 0) nl = 0;
if (ll < 0) ll = 0;
if ((int64_t)s->fused + nl + ll + 2 > FRAME_TEXT) break;
s->fframe[s->fn] = fr;
s->fnoff[s->fn] = s->fused;
s->fnlen[s->fn] = (int32_t)nl;
if (nm != NULL && nl > 0) memcpy(s->ftext + s->fused, nm, (size_t)nl);
s->fused += (int32_t)nl;
s->ftext[s->fused++] = 0;
s->floff[s->fn] = s->fused;
s->fllen[s->fn] = (int32_t)ll;
if (lc != NULL && ll > 0) memcpy(s->ftext + s->fused, lc, (size_t)ll);
s->fused += (int32_t)ll;
s->ftext[s->fused++] = 0;
s->fslots[s->fn] = flan_dev_frame_nslots(fr);
/* The outermost [frame_floor] frames are the program's; anything above
* them belongs to the evaluation this break is inside. */
s->fmine[s->fn] = (frame_floor < 0) || (i >= fn - frame_floor);
s->fn++;
}
}
atomic_store(&snap_depth, d + 1);
return 1;
}
@ -434,9 +508,12 @@ int32_t flan_agent_poll(void) {
* inside break loops, which run from inside thunks. */
if (j.call != NULL) {
int32_t outer = restart_floor;
int32_t oframe = frame_floor;
restart_floor = flan_restart_count();
frame_floor = flan_dev_frame_count();
j.call();
restart_floor = outer;
frame_floor = oframe;
}
if (j.handle != NULL) { dlclose(j.handle); }
}
@ -539,6 +616,48 @@ static void serve(int fd) {
reply(fd, ".\n");
return;
}
/* Where the stopped thread is. One line per frame, innermost first:
* the index, whether the frame is the program's or the evaluation's, how
* many slots it has, where it is written, and its name. Served from the
* snapshot, never from the live chain — the game thread is parked in the
* break loop, but the loop polls, and a poll runs Flan.
*
* Refused while running, like every other break verb and for the same
* reason: a chain read by one thread while another pushes and pops it is
* not a backtrace, it is a race with a plausible shape. */
if (strcmp(line, "backtrace") == 0) {
if (!(atomic_load(&depth) > 0)) { reply(fd, "err not stopped\n"); return; }
snapshot *s = snap_top();
if (s == NULL) { reply(fd, "err no frame snapshot\n"); return; }
if (s->fn == 0 && s->ftotal == 0) {
/* Not "no frames": a release build has no shadow stack at all, and
* answering with an empty backtrace would read as a program with an
* empty stack, which is not a thing that can be stopped. */
reply(fd, "err this program was not built with --dev, so it has no "
"shadow stack to walk\n");
return;
}
for (int32_t i = 0; i < s->fn; i++) {
char hdr[64];
int k = snprintf(hdr, sizeof hdr, "%d %c %d ", i,
s->fmine[i] ? '+' : '-', s->fslots[i]);
if (k > 0) send(fd, hdr, (size_t)k, MSG_NOSIGNAL);
if (s->fllen[i] > 0)
send(fd, s->ftext + s->floff[i], (size_t)s->fllen[i], MSG_NOSIGNAL);
else
reply(fd, "?");
reply(fd, " ");
send(fd, s->ftext + s->fnoff[i], (size_t)s->fnlen[i], MSG_NOSIGNAL);
reply(fd, "\n");
}
if (s->ftotal > s->fn) {
char more[64];
int k = snprintf(more, sizeof more, "... %d\n", s->ftotal - s->fn);
if (k > 0) send(fd, more, (size_t)k, MSG_NOSIGNAL);
}
reply(fd, ".\n");
return;
}
/* Take the i'th, optionally checking that the caller and this snapshot
* still agree on what the i'th is called. The name is not the lookup -
* that is the bug - it is a receipt: a client that listed, prompted, and