Three follow-ups to the registry reader.
The agent's "reg types" handler held its four row buffers as statics. The lock
argument for that was right — request_lock is held across the whole of
handle_line from both entry points — but the BSS was not: 8KB in every build
this package is linked into, including a release build of a game that imports
the agent and never writes a row. That is flan_dev.c's own argument against a
fixed table at a thirty-second of the size. Malloc'd and freed per request,
like the watch handler beside it.
A followed pointer parses as an opaque node, so the inspector refused it with
"no structure for this type" — false and unhelpful at once. It has structure,
it is drawn, and the reason you cannot step in is that the step would start
from the pointee. Said that way instead.
And the three new commands have elisp tests: the address root's wire, that a
path is refused rather than dropped, the listing's rows and totals, the
overflow warning, and a build with no registry refusing rather than showing an
empty table that reads like a program holding nothing.
The allocation registry had a recording side and half a reader. This is the
rest of the reader: point at any heap address, a breakdown by type, what is
still held, and the test that stops dev-ptr.flan's header from being read by
hand.
The recorded name, back to a type. The table records a string and has to —
the note is built where the concrete type exists and what crosses into the
runtime is bytes. What closes it is that the string is Types.to_string, which
is the source spelling, so the round trip is the language's own reader,
Parse.texpr and Check.resolve. No table of spellings is written down, so
nothing can fall behind Types.to_string, and a name that is not a type —
"pool slots" — is refused with the name quoted rather than defaulted.
The address root renders a (Ptr T) and not the pointee, which puts it through
render.ml's pointer arm: permission is asked in one place in the compiler, and
an address root and a slot root reach the same two answers by the same code.
Flan has no integer-to-pointer cast, so flan_dev_reg_addr is an extern beside
flan_agent_frame_slot, for the same reason.
One walk and two questions: a leak report is a breakdown with the dead left
out, so flan_dev_reg_by_type is one function and the agent formats it.
"At exit" is not a hook. A program killed by a signal runs no handler, which
is how a game under the editor ends, so (:op "leaks") is the authoritative
reader and can be asked at any moment including the one before the kill. The
atexit hook is for the program that returns from main, is registered from
inside flan_dev_reg_enable rather than by a file-scope destructor so that a
release build does not grow a third not-free place, and is off unless
FLAN_DEV_LEAKS is set because the acceptance table reads stderr.
The memcheck half of item 6 is deliberately not here.
The watch accumulator clears a slot lazily, on its next sample, and the reader
never compares epochs. A previous lane read that as a defect and left
watch_render_num's n=0 arm behind as dead code against the day the reader
became epoch-aware.
It should not. A stopped program takes no samples, so an epoch-aware reader
would report every slot as empty for as long as the program sat in a break
loop — and reading the numbers from the moment you stopped is the whole point
of stopping. The lazy clear is the right answer there.
What was wrong was narrower and lives in the editor: flan-watch--tick sent
`:reset t` five times a second at a program that could not answer it. The read
still goes out every tick; only the reset field drops, guarded on
flan-dev--stopped, which flan-dev.el's background poll already keeps current.
The n=0 arm is deleted rather than commented, since the only way to reach it is
the epoch check just rejected and dead code is an invitation to add one. n is
at least 1 whenever watch_render_num runs, so sum/n cannot divide by zero.
test-flan-watch.el asserts both halves with no daemon: a running tick carries
:reset, a stopped one does not, both still read the table and both leave a
reply in flight. Verified by mutation. What it cannot reach is the daemon
honouring the absent field; test_dev.ml drives a real program for that.
`flan-watch-ghost-call-regexp' allowed one hyphenated segment until this
lane's sibling widened it, so `watch-num-i64' matched as far as
`watch-num', required whitespace, found `-i64' and backtracked to
failure: the row appeared in the watch buffer as usual and the call site
got no inline value at all. Nothing automated covered the widening. It
does now — a block pinning both accumulator heads beside the four
one-segment ones, verified by putting the `?' back and watching exactly
those two checks fail.
The other half of that handoff asked for a test of `watch_render_num''s
`n=0' branch. It cannot be written: `s->num' is set in one place,
`watch_record', which always falls through to `s->n += 1', and the clear
and the increment sit inside one odd-generation window. The reader
never compares the slot's epoch to the global one, so a reset does not
make a slot read as empty either. The branch is dead, and the live
consequence is that a read taken after `:reset t' and before the
program's next sample reports the previous window — which is why the
existing test waits for the count to drop rather than reading once.
Recorded in HANDOFF-f3.md with the two ways out, both of which belong to
whoever owns runtime/flan_dev.c.
Finishes DISCUSS.md §9's `pause` marking: the daemon half was already
built, this is the editor half plus the one daemon path it was missing.
`C-u C-c C-c` marks the form point is inside, `C-u C-u C-c C-c` the
top-level form (stop on entry), `C-u C-x C-e` the expression before
point. The buffer is never edited — the position rides beside the code
and the `(pause)` call goes into the tree after parsing, so no source
location moves.
`C-x C-e`'s path needed the daemon: its 5s `wait` answered "the program
did not reach a frame boundary", which is exactly what a thunk parked at
a breakpoint looks like from out here. `wait` is now three-way and asks
`state t = Stopped` only when a pause was requested, so the no-pause
shape `test_dev.ml` pins is unchanged.
The overlay is an annotation and not feedback, so unlike an error marker
it survives `pre-command-hook`; what takes it down is an accepted
evaluation with no `:pause` on it, which is the same thing that takes
the mark itself down.
Tests: a `test_dev.ml` block over the new `dev-pause.flan` that marks,
stops, re-evaluates plainly and then polls half a second confirming it
does not stop again — one sample after `continue` proves nothing, the
resumed frame is still in the old body — and an `emacs/test-flan-dev.el`
block for which form a prefix picks, the byte column, the overlay's
lifetime, and one live round trip.
PORTING.md Tier 1 item 5. The spy half of the watch was already built — the
pushed table, the buffer, the inline ghost text. What was missing is spy-num,
which is the part that item calls least obvious and most valuable, and it is
what this is.
A slot keeps count, min, max, last and mean. Each answers a question you can
ask without building a query: n is the first thing wrong when a loop is wrong,
the range is what one sample can never show you, last is what the scalar watch
would have given you, and the mean is a running sum divided at read time
because a mean accumulated as a mean drifts. A small ring of the last N
samples was the other candidate and loses — N out of 91,200 is a sample of the
tail of the loop rather than of the loop, and past five numbers every richer
answer is a UI for building a query.
The write path does no formatting, which is the feature rather than an
optimisation: a snprintf per sample at thousands a frame is a HUD that costs
more than the game. A sample is a load, five compares and the slot's seqlock;
the listener thread renders once per editor tick.
The window is since the editor's last tick, and that is a deliberate
divergence from watch.clj, where the stats are cumulative until reset-spies!.
Cumulative min and max reach the session's extremes within seconds of play and
then never move again, so the two most useful of the five go dead exactly when
you start interacting with the thing you are debugging — and this tool exists
to show you a number while you drag the mouse. Reset is its own message and
never a side effect of reading, because a destructive read makes looking
change what is there and anything that polls would shorten the window under
the editor that owns it. It bumps one epoch counter and clears no slot; a slot
clears itself on its next sample, so the reader never writes the table.
Ghost text needed one character. The call regexp allowed one hyphenated
segment, so watch-num-i64 backtracked to failure and a numeric watch got no
inline value while appearing normally in the buffer.
dune test is green, run twice. HANDOFF-f3.md carries the reasoning, the two
small gaps left behind it, and what did not work on the way.
flan-watch-ghost-mode paints each watched value inline, after the line holding
the call that wrote it. An addition to the watch buffer and not a replacement:
both can be on at once, and turning either off leaves the other running.
The earlier note said ghost text was gated on a (watch ...) form in check.ml,
because nothing in the table carries a source location. That is true of the
table and the conclusion did not follow. The call site is in the buffer, and
the name in the table is the string literal in it, so the anchor is searched
for rather than reported. Nothing new is asked of the daemon. The head of the
call is a defcustom regexp, because watch-i64 is a name the program's author
chose in their own declare-c and only the C symbol behind it is fixed.
Both pictures are painted from one reply in flan-watch--absorb, so they cannot
disagree and there is no second watch request in flight. That meant the watch
buffer could no longer be the subscription: arming and the timer now hang off
flan-watch--consumers, and only the last consumer out disarms the table.
Overlays are replaced wholesale on every repaint rather than followed through
edits, which is the whole answer to invalidating one whose line moved. Only
buffers shown in a window are scanned.
Settled and written down: two sites of one name both show it and say so,
because the table has one slot and the last writer wins; a watch in a loop
shows the last value written, as the buffer does, because every better answer
is the query UI this design exists to avoid; a stopped program's values say
"last frame" and change face, since inline they sit in code that looks live;
a site with no row is annotated only when the table reports overflow.
syntax-ppss moves point and clobbers the match data, so calling it inside a
re-search-forward loop and then reading match-string restarts the scan and the
loop never ends. Everything is read out before the check now.
emacs/test-flan-watch.el covers it, loaded from test-flan-cider.el the way
test-flan-mode.el is, so no build change is needed. 203 checks, 0 failures.
Ported from the author's Clojure watch.el. Three of its decisions survive
unchanged — the program decides what is shown, the request is async, and the
paint is replace-buffer-contents so point survives every tick. The transport
is the part that had to turn round.
An eval here compiles a module and dlopens it, so the planned fix of
compiling the render thunk once and re-invoking it per tick was still a poll,
and a poll has a defect caching does not touch: a thunk runs at a frame
boundary and a stopped program has no more of those. So the program writes
into a table in flan_dev.c from its own loop and Emacs reads the table, which
is memory. The values are then as fresh as the last frame whatever the
repaint interval is, and they are still there while the program is stopped.
The frame thread's constraints decide the storage: no allocation, so names
are fixed arrays rather than strdup'd; no lock, because the reader is the
listener thread; and not the result buffer, which is written once per C-x C-e
and would be overwritten sixty times a second by watch traffic. One seqlock
per slot rather than one for the table, so a reader retries one slot instead
of having to catch the gap between two frames' writes; a snapshot from
adjacent frames is what a HUD looks like anyway.
Sixty-four slots, and past that a name is dropped rather than fatal — killing
the program because somebody watched a 65th value is the diagnostic shooting
the patient. Reported as a flag and not a count: the only number the write
path could keep is of write attempts, which at frame rate says "3847 names"
about one name.
Nothing writes the table until a watch buffer is open, so a watch call in a
program nobody is debugging is a load and a branch that is not taken — the
same number in a release build, since flan_dev.c is linked into both.
Scalars work today through declare-c against four runtime entry points, which
is why this needed no compiler change. A struct or a slice needs a walk over
its type, which is one arm in check.ml beside print; BUILT.md writes it out
rather than reaching into a file another lane holds. Ghost text turns out to
be gated on the same arm, for a different reason: nothing in the table
carries a source location, and a hand-written declare-c call cannot supply
one that does not drift when the line moves.
The mechanical half, ahead of the parser change that needs it. tools/unit-return.py
fills the empty slot with () and rewrites Unit as () wherever a type is spelled --
(Fn [i32] Unit), (Map i32 Unit), a return type written out.
Deciding whether a defn already had a return type is the whole difficulty, and
the script does it the way parse.ml did: is_type_form is transcribed rather than
improved, because being identical to the parser it replaces is what makes the
sweep meaning-preserving. It is re-runnable, so the lanes that branched before
this can have the same pass at merge:
python3 tools/unit-return.py .
python3 tools/unit-return.py --in-strings test/test_flan.ml test/test_acceptance.ml \
test/test_session.ml emacs/test-flan-dev.el emacs/test-flan-mode.el
python3 tools/unit-return.py --raw-ml lib/prelude.ml
python3 tools/unit-return.py --in-html web/index.html
-v logs every defn it saw and what it decided, which is how a sweep of 440 sites
gets reviewed at all. Embedded modes pool a file's type declarations across all
its fragments, because a snippet split across concatenation -- decls ^ "(defn f
[s [u8]] Cursor ...)" -- cannot see the names the other half declared; pooled
names count only in bare-symbol position, for the same reason the prelude's do.
A fragment that cuts off mid-form is skipped rather than guessed at. Five sites
in test_flan.ml still needed a hand, and they are in this commit.
Two things ride along because the sweep needs them: parse.ml reads a lone () as
the return type of a function with no body, which was not a shape the old
optional slot could produce; and the map refusals name () rather than Unit, since
that is now the spelling a caller wrote.
The capability lists were written before the code held the line they claim.
Under an expression root, RET on a field of a union built `(.at s)' and sent
it, and the checker refused it — "a union's fields belong to a case ... they
are reached by (match ...)". A refusal from the far end of a socket is exactly
what this buffer's own comment says not to do: every refusal is by name, here,
with the reason, because RET working on some lines and erroring on others
teaches nothing about the language.
It is a refusal of the *parent* and not of the value at point, which is why it
is not in `flan-inspect-refusal': a struct field that merely holds a union is
an ordinary accessor and has to stay enterable. It is a field of the union
itself that cannot be written. The two cases are one test each.
The slot root steps into it by offset and is unaffected, which is the
difference the manual now claims and the tests now show.
`lib/dev.ml' cited DISCUSS.md item 1 as a hole; item 1 is the answer now, so
it cites BUILT.md instead. And the item 1 stub is two sentences and a pointer
— everything else in it is in BUILT.md verbatim, and DISCUSS.md's own header
says nothing in it is a decision.
The daemon side and the Emacs side both landed with nothing written down. Four
files owed something.
`BUILT.md` gets the whole of it: why rooting at an address alone was rejected
and why that rejection was half wrong, what a path step is and how a union's
case travels with it, why the slot goes by index and not by name, and the two
capability lists side by side — the expression root works on a running program
and cannot name a frame; the slot root names one frame and one slot and reaches
an option's payload and a union case's fields, and needs the program stopped.
Neither contains the other, which is the reason there are two.
`emacs/MANUAL.md` says the same thing in the register that file uses, under the
inspector, because the person pressing `i` is the one who needs to know which
root they got and what it cannot do. The globals section's claim that `i` works
on a global "exactly as it does on a local" was true and is now the interesting
difference, so it says what the difference is.
`NEXT.md`'s decided item is struck with what actually shipped: a frame and a
slot index rather than an address and a type, and `l` crossing between the modes
was predicted as a cost and turned out not to be one.
`DISCUSS.md` item 1 is no longer an open question. The number stays — cimport.ml
and NEXT.md cite these by number — and what stays with it is the one correction
worth keeping: an address is not an expression, but a step does not have to be
one either.
And BUILT.md's last paragraph still said `render.ml` prints `(V {:x 1.5})` and
that the printer would move when its reader did. They moved together some time
ago.
`i' on a local sent the local's *name* to be evaluated, and an expression is
evaluated where the evaluator stands. That is the right frame only when the
frame is the innermost one; on any other it may resolve to a global, to
another binding of the same name, or to nothing, with the locals listing right
above it showing the frame's own storage and nothing saying the two disagree.
The daemon verb for the fix landed already. What was missing was the state
layer under it: `flan-inspect--expr' held a bare expression, so there was
nowhere to put a frame. It is `flan-inspect--root' and `flan-inspect--path'
now — `(:expr E)' or `(:slot FRAME SLOT NAME)', plus the steps walked from it
— and a stack entry is `(ROOT PATH . POINT)'. RET appends a step, `l' restores
a pair it pushed. Every step is still a fresh request, so the view is never
stale.
`l' cannot cross between the two roots, and that is structural rather than a
rule someone has to keep: RET only ever extends the path under the root the
buffer already has, and `flan-inspect' and `flan-inspect-slot' both start with
an empty stack, so a mixed stack cannot be built at all. It stays true if a
third rooting mode is added.
The break buffer hands over the frame and the slot *index*, which is the
fourth element `locals' now puts on each line. A name does not identify a
slot: two slots of one frame can share one, and a refused slot is not in the
listing, so its position is not an identifier either. A global still goes in
by name, because a global's name really is an expression that means the same
thing wherever it is evaluated — the loaded thunk binds to the program's own
storage through the dynamic linker.
Two smaller things the wire needed. A field step carries the type it was read
out of, because a union's payload is at an offset that depends on the case and
only the renderer knows which case the value is in — so `Union.case.field',
which is the head the renderer wrote with the field appended. And an empty
path is sent by omission: Emacs prints an empty list as `nil', which is a
symbol on the wire, so the daemon now reads that as no path rather than
refusing it as a step.
OCaml side is done and builds. Emacs side is mid-edit and INCOMPLETE — see
the handoff below. `dune build --root . @check` is green. `dune test --root .`
was NOT run. The .el files were not byte-compiled and flan-inspect.el will not
work as it stands: the state layer still speaks the old single-expression
shape while the helpers above it have been rewritten for roots and paths.
WHAT WORKS (daemon, lib/, in the parent commit and unchanged here)
- `(:op "inspect" :frame N :slot I :path (...))` renders one value rooted at a
stopped frame's slot address. `Session.render_slot` is `render_locals` with a
path applied to the root before the walk and one line out instead of one per
slot; no second walk was written and no backend change was needed.
- A path step is a string for a struct field, an integer for an array or slice
element, and the symbol `some` for an option's payload. A union case field is
spelled `Union.case.field`, because the payload's offset depends on the case
and only the renderer knows which case the value is in.
- Every step that does not fit the type in hand is refused by name with the
reason: a field the type does not have, an index past a fixed array's end,
`some` on something that is not an option, a union field without its case.
- The frame's identity IS checked, and not by a second copy: `Dev.stopped_frame`
is one function now and `locals` and `inspect` both go through it — alive,
stopped, frame exists, the frame is the program's and not a thunk's, the body
is one this session holds, the slot count matches, and `Emit.slot_fingerprint`
matches. `inspect` additionally refuses an unbound slot, for the listing's
reason: a null address would fault on the stopped game thread.
- The slot travels by INDEX, not by name. Two slots can share a name
(`fresh_slot` only allocates) and a refused slot is not in the listing, so
neither the name nor the position identifies one. `locals` now puts the slot
index as a fourth element on each `:locals` entry.
- `Dev.run_render_thunk` is one function; `locals`, `globals` and `inspect`
share the build/deliver/wait/read tail.
- `layout`'s "union values are milestone 6" is corrected.
WHAT IS HALF-BUILT, AND EXACTLY WHERE IT STOPS
`emacs/flan-inspect.el`. Done: the header comment explaining the two roots;
`flan-inspect-step-expr` taking a 3-element `:field` step; `flan-inspect-wire-step`;
`flan-inspect--root-label`; `flan-inspect-refusal` taking an optional ROOT and
allowing an option's payload under a `:slot` root.
NOT done, and this is the whole of what is left:
1. `flan-inspect--expr` / `flan-inspect--stack` still hold a bare expression.
They must become `flan-inspect--root` (`(:expr EXPR)` or
`(:slot FRAME SLOT NAME)`) plus `flan-inspect--path`, with stack entries of
`(ROOT PATH . POINT)`.
2. `flan-inspect--value` must branch on the root: `eval-expr` with
`(flan-inspect--root-label root path)` for `:expr`; for `:slot`, send
`(:op "inspect" :frame F :slot S :path P)` with P built by
`flan-inspect-wire-step` over the path, and take `:value` from the reply.
3. `flan-inspect--show`, `-into`, `-pop`, `-refresh` rewired to (ROOT PATH).
`-into` must build a `:some` step when the node's kind is `option`, and put
the parent node's `:type` as the third element of a `:field` step.
4. New entry point `flan-inspect-slot (frame slot name)`, kept separate from
`flan-inspect (expr)` — `emacs/flan-mode.el` autoloads and binds the latter
and that file is out of this lane.
5. `emacs/flan-cnr.el`: the `flan-cnr-inspect` text property must carry
`(:slot FRAME SLOT NAME)` on a local line — the slot index is `(nth 3 l)`
now — and `(:expr NAME)` on a global line, with `flan-cnr-inspect`
dispatching to the right entry point.
6. `emacs/test-flan-cider.el`: the fixture at "`i' on a local inspects it by
name" asserts the old behaviour and must be rewritten; the locals fixtures
need a fourth element.
7. `test/test_dev.ml`: no coverage of the new op yet. The discriminating test
to write first is a stack whose OUTER frame has a local whose name is also a
global with a different value, asserting `inspect` answers the frame's value.
A new `test/programs/dev-inspect.flan` is picked up by the existing glob.
8. `BUILT.md`, `emacs/MANUAL.md`, and striking the item from `NEXT.md`'s
"Decided in discussion" and `DISCUSS.md` item 1 — none done.
THE THREE ANSWERS THE TASK ASKED FOR
- Navigation in the new mode: the daemon supports it fully — RET extends the
path, `l` shortens it, and both are a fresh request, so the view is never
stale. The Emacs half of that is item 3 above and is not wired.
- `l` does not cross between the modes, and that is structural rather than a
rule: a stack entry carries its own root, RET only ever extends the path
under the root it already has, and every new root starts with an empty
stack. A mixed stack cannot be constructed, so the question does not arise —
and it stays answered if a third rooting mode is added.
- What each mode cannot do that the other can. The expression root works on a
RUNNING program and roots at anything you can write, a call included; it
cannot name a frame, so it is the bug. The slot root names one frame and one
slot and is exact; it reaches an option's payload and a union case's fields,
which have offsets but no accessor in the surface language; it needs a
stopped program, it is refused when the frame's body was redefined since it
was entered, and it cannot root at an expression at all.
A global is program state a frame happened to touch, not part of it, so
nesting it under one implies an ownership that is not there and repeats the
name once per frame that reads it. One section instead, holding the union of
the globals every frame on the stack references — the compiler does the
choosing, since Reach.expr_refs already answers a body's reference set, and
listing every global a program has would bury the one that matters under the
prelude's PRNG state.
Each entry says which frames touch it, by the index the stack section already
numbers them with, which recovers what per-frame nesting would have told you
at no cost in duplication. Ordered by the innermost frame that touches it:
a deep stack makes the union large and proximity to the error is what puts
the likely culprit on top.
Simpler than locals, because a global is reached by name rather than by
address. Emit.redefinition writes a global the host has as external, so the
thunk binds to the program's own storage and nothing is asked of the stopped
thread — no dev-slot round trip and no not-yet-bound case to refuse.
A frame that cannot be attributed contributes nothing and is named in
:skipped; the union being incomplete and the union being complete are
different answers. The hole in that is stated rather than papered over:
slot_fingerprint hashes a body's slots, which is the right cut for locals and
not for this, so a body that names different globals while binding the same
locals is not caught. The test drives the case that is.
MANUAL.md also loses a stale paragraph claiming the fingerprint check never
fires with a failing test pinned to it. It fires, and test_dev covers it.
The client already knew the moment: flan-dev--absorb reads :stopped off every
reply and the poll covers the case where no reply is coming. This is a hook at
that point, not new plumbing.
Deferred through a zero-delay timer, which is the part that is not optional.
absorb notices the stop in the middle of reading a reply on the socket, with
flan-dev--busy bound, and showing the buffer asks the daemon three more
questions — break, layout, backtrace. Issuing those from inside the read they
were triggered by would interleave two conversations on one connection. The
deferred call re-checks the state rather than trusting the edge that scheduled
it, because by then the edge has been consumed and the program may have been
resumed.
Three decisions, settled and written down beside the code.
It displays and does not select. A program stops on its own clock, not the
editor's, and the likeliest moment is a frame of its own game loop while
someone is typing somewhere else. Taking the window would send the next
keystrokes where they were not aimed. `focus' is there for anyone who
disagrees, and nil goes back to the mode line alone.
(pause) is not a special case, though it was worth asking: it is deliberate at
the moment it is *written*, and the frame it fires on still arrives whenever the
program gets there, which is no more expected than an error. What it does get is
an honest headline. (pause) is `error' under a `restart-case', so nothing in the
compiler knows a breakpoint from a failure and this buffer is the first place
that can — calling it unhandled is a small lie at the top of the one buffer that
exists to say what happened.
A stop mid-edit disturbs nothing, which falls out of displaying rather than
selecting. Two guards go past that: nothing happens under an active minibuffer,
because a prompt is modal and rearranging windows under one is hostile; and
nothing happens inside a keyboard macro, because a macro that behaves
differently depending on whether the program stopped cannot be trusted. In both
cases the mode line still says stopped and C-c C-b still works.
render.ml's output and emacs/flan-inspect.el's parser are the two ends of one
wire format, which is why the printer was left on the colon when the rest of
the corpus moved: shifting it alone would have broken inspection in the dev
loop without breaking a test that said so. They move together here.
The field list in the inspector is labelled with the dot too, which is the
spelling flan-inspect-step-expr already used to build `(.x b)' — the label and
the expression it stands for now read the same.
One case needed a guard the colon never did: `...' also begins with a dot and
is the renderer saying it stopped, not a field called `..'. A field name never
starts with a second dot, so one character of lookahead separates them.
The colon is not gone from the rendered grammar. An enum member is `:green' and
is a *value*, so the two are now told apart by the character alone, which is
the only thing that distinguishes them.
Also font lock, handed over with the same change: `:name' was the rule that
drew field labels, and with the colon belonging to keywords every label in the
corpus was left unfontified. `.name' is drawn as a constant, in both the places
it appears — the label in `{.x 1.0}' and the accessor in `(.x v)', which are
the same name.
Decimal is what the value is and stays first; hex and binary go beside it. It
is the wrong base for about half the numbers anyone opens this buffer for — a
colour is 0x303030FF, a gesture is an OR of flags, a mask is read a bit at a
time — and reading those out of a decimal is arithmetic done by hand.
In two places: under the header of a value opened on its own, and on each
numeric row of a field list. The second is the one that matters, because a leaf
cannot be stepped into, so the field list is the only place most numbers are
ever seen.
Nothing is asked of the program. It is arithmetic on text the renderer already
wrote, so it works on a stopped program and costs no round trip. Binary is
grouped in nibbles because a mask is read in nibbles. A negative is shown as
the 64-bit two's complement it is in memory and says the width out loud: the
rendered value carries none, and every Flan integer comes back through i64.
A float is left alone rather than answered wrongly — its bits are an IEEE
layout, reinterpreting them is a different question, and the rendered text does
not carry the width to answer it.
The pointer half of this is not done and the refusal now says why. Render.render
writes the bare word <ptr> for every pointer on purpose: it is the same renderer
print uses, an address is not stable across runs, and test_acceptance pins the
current text for that reason. Showing one is a decision about the language's
printer, not about this buffer.
The indentation rules were written and tested but never described anywhere a
user would look. MANUAL.md had no section on editing at all — it starts at
`C-c C-c' and assumes the file is already written — so the rule that cost the
friction, a binding vector lining up name under name, was only visible by
trying it.
What is written down is what was checked, not what the port was aimed at: the
call fallback, the `handler-bind' clause vector, `defn' parameter alignment
with a return type after it, and `restart-case' clause bodies were each
reindented from scratch and the manual quotes the result.
NEXT.md keeps the half of the field-label handover that is still open. The
printer in render.ml has to move in the same commit as the inspector that
parses it, and that is the inspector lane's; the font-lock half is done here,
so only that half is struck.
The reported bug — the second and later bindings of a let one column too far —
was never one missing rule. `flan-indent-function' checked the head of the
enclosing form, and inside a binding vector the enclosing open is `[' and the
symbol after it is the first binding's name, so it fell through to Emacs's
`lisp-indent-function', which treats the vector as a call and aligns under the
first argument instead of the first binding.
Emacs Lisp is the wrong reference. It has no vectors-as-bindings, no maps and
no bracket variety, so every rule Flan needs has to be added by hand and the
binding vector is simply the first one hit. The indenter is rewritten from
clojure-mode's source instead: `clojure-mode' is neither an ancestor nor a
dependency — flan-mode still needs nothing beyond stock Emacs — it is the file
whose rules were read and written out again.
A bracket aligns under its first element, and that one rule fixes the binding
vector, `defn' parameter lists, `restart-case' and `handler-bind' clause
parameters and both spellings of a struct literal at once. `{:x 1}' and
`{.x 1}' indent identically because nothing here looks at the key, which is
what the colon-to-dot lane needs of it.
Where Flan diverges it is handled on purpose. `defn' is `:defn' rather than a
count because the return type between the parameters and the body is optional.
A clause — `(Name [params] body)' — is recognised by its shape, since its head
is a condition class or a restart name and can never be in a table; clojure-mode
reaches the same clauses by backtracking out to the enclosing form, which buys
generality this language has no other use for. Special arguments indent by one
body rather than Clojure's two, and a call whose head is alone on its line
indents its arguments by a body rather than aligning them under the head,
because that is how the whole corpus is written.
Checked by reindenting every .flan file in the tree: the only lines that move
are sand.flan's reported bug, raylib.flan's hand-wrapped parameter vectors —
which is the fix — and lone-`;' comment continuations, which stock
`lisp-indent-line' has always moved.
test-flan-mode.el is loaded from test-flan-cider.el rather than given a stanza
of its own, because emacs/*.el is already a dependency of that test.
Two failures in test-flan-cider.el that predate this: fixture frames lacked
`:fetched', so folding one open went looking for a daemon, and `layout' was
identified by being the last request when `flan-cnr-show' now makes three.
render.ml's output is parsed by emacs/flan-inspect.el, which hard-codes
the colon when it reads a field out of a rendered struct. Moving the
printer on its own would break inspection in the dev loop without
breaking a test that says so, so the printer waits and moves with its
reader, in the Emacs lane.
The sweep could not tell a rendered *expectation* from a Flan *source*
snippet -- both are strings in a test -- so it converted both. The suite
named every one it got wrong, and those are back.
emacs/test-flan-dev.el:415 is the one edit inside emacs/: Flan source sent
to the daemon for eval, which the parser now refuses in the old spelling.
One label, in a fixture.
C-c C-b asks layout with the condition's own name and draws the fields under
it. The values stay refused, by name, because the shape of a condition is a
fact about the build and its contents are a fact about the stopped frame — and
only one of those is knowable today. A layout the daemon refuses is nil rather
than an error: the buffer already draws a section saying why one is empty, and
failing the whole command would take away the restarts over an annotation.
The manual described the signature-change refusal as though it were the
design. It is not, and session.ml already said so at the refusal: a
signature change should make a new version, leave old callers on the old
one, and warn at the stale sites. plan.org calls it signature generations
and stale-caller warnings, and it is milestone 7's unfinished half.
The struct-layout rejection is the decided one and stays. Conflating them
made a placeholder look like a rule.
The site's Emacs section is a key table and four paragraphs -- a
reference for someone who already knows the shape. This is the other
document: how to set it up, what the loop actually is, what each buffer's
own keys do, and what to do when something is refused.
The three buffer keymaps were never written down anywhere a user would
look; they were only in the define-key calls.
print-str, print-i64, print-f64, print-bytes, print-line and newline leave
the prelude. print and println are the whole printing surface now, and print
is the better call at every one of the sites that used them: it is the same
structural walk without the newline, so the no-newline case the family was
kept for is covered, and it takes the value as it is. The old print-i64
forced an explicit (i64 x) at every call site, because this language widens
nothing implicitly; that cast is gone from 127 places.
Dropping it moves one answer. hash-grid returns u64, and the cast through
the signed printer showed sand-headless's hash as -2851001042534928384.
print routes a u64 through flan_u64_to_bytes, so it now prints
15595743031174623232 — the same 64 bits, read as the unsigned number they
are. The pinned expectation follows the correction.
test-flan-dev.el and test_session.ml both reached for print-line as "a name
the prelude has"; they reach for rand-seed instead.
A let-bound local is its own name under lldb now, and a redefinition
module carries DWARF when the daemon was asked for it.
Resolved against the println track in session.ml: the thunk keeps the
render walk's appended slots and gains the names beside them, the walk's
own scratch having none to keep.
Emit.redefinition has taken ~debug since it was written and was tested
with it; Session.eval never passed it, so every body installed by C-c C-c
lost its debug info in the running process.
Passing it alone would have been half a fix. Build.shared is what forces
-O0, and dev.ml built modules at -O2, so the llvm.dbg.declares would have
been emitted and then deleted by mem2reg: a line table, and no locals.
And a module with DWARF loaded into a host without it lines up against
nothing. So it is one flag — flan dev --debug and flan reload --debug —
and it sets the host build, the module builds and the emitted metadata
together. Off by default: a debug build is an -O0 build, and quietly
making every reloaded body -O0 changes the frame time of the one function
you are iterating on, in the loop whose point is watching that number.
What a dlopen'd module does to a breakpoint, measured against the reload
fixture rather than reasoned about:
- lldb reads the new module's DWARF on the dlopen and says so: "1
location added to breakpoint 3".
- A breakpoint set by NAME gains a second location either way, so
dlopen was never the difficulty. What the line table buys is that it
stops with source instead of disassembly.
- A FILE AND LINE breakpoint on the new body resolves only with it;
without, it sits at locations = 0 (pending) forever.
- A FILE AND LINE breakpoint on the HOST's copy stays pinned at
locations = 1. That is correct, not stale: the old body is still
mapped and every call site that has not gone through its cell again
still reaches it.
- The stack crosses intact — a frame in the reloaded .so and the one
below it in the host each name their own .flan file.
(lldb) frame variable
(long) step = 10
(long) prior = 11
The transcripts are in flan-dape.el, replacing the note that said the
module carries no DWARF yet.
flan-cnr.el's stack pane was refusing for the wrong reason. DWARF was
never its gap; nothing is attached to the stopped program, and a socket
cannot read another process's frames. Reworded to say that.
Source interleaving in the disassembly buffer is unblocked and not done:
objdump -dS interleaves a --debug module's Flan source correctly, so
Dev.asm_of needs the -S and a parse_listing that tolerates source lines.
A restart the innermost frame shadows could be seen and not taken;
it is taken by position now, off a snapshot that stopped moving under
the break loop. The editor half this was briefed as building already
existed — the stale line that said otherwise is fixed.
The snapshot made the listing stand still; it did not make the handoff safe.
A choice is validated against the snapshot on top when the request lands and
resolved against the snapshot on top when the game thread next looks, and
between those the loop runs evaluations — one that errors pushes a break of
its own, whose loop reaches the flag first and takes its own index 2 for the
one somebody picked off the outer list. That is the failure this change exists
to end, arriving by a different door.
So each snapshot carries a generation, a choice carries the generation it was
validated against, and a loop claims only what is addressed to it. A mismatch
is left set rather than dropped: the listener already answered ok, so the break
it was meant for must still be able to take it. Depth could not do this — an
outer break resuming and a new one starting reuses the number. The snapshot is
also popped before the depth comes down now, so the two never describe
different breaks.
The client's own new path gets tested too: the candidate table is pure, so the
shapes a real daemon will not easily produce are checked directly, and the
break-and-resume test now goes through restart-at rather than by name.
flan-dev-unreachable-restarts was dead on arrival — flan-break reads
:unreachable off its own reply — and is gone.
now lists the restarts by position and sends the position, with the
name alongside as the receipt the program checks. A restart below the
evaluation the break is inside is shown marked rather than hidden: someone who
can see a restart in their own source and not on this list has been told
nothing, and the refusal carries the reason.
session.ml already had this: a compile-time walk over a Tast type that
emits the calls to print a value of it, handling every concrete type the
language has. It was dev-build-only and went to flan_dev_emit, and
prelude.ml justified the per-type print-* functions by saying a real
println had to wait for milestone 5 and generics. It did not. plan.org
specifies println as compiler-provided and per concrete type, which is
not overloading: there is nothing to dispatch on at run time and no
user-supplied printer to choose between, so no type variables appear.
The walk moves to render.ml, parameterised on an emitter and a slot
allocator. The emitter is five functions rather than five extern names
because the two sides are not both extern calls -- the REPL's are, and
stdout's compose a conversion with a write. The slot allocator differs
too: the REPL builds a thunk's frame, println takes slots from the
enclosing function being checked, once per call site.
Two runtime shims, both only reachable from the walk. flan_u64_to_bytes,
because routing u64 through the signed printer makes 0xFFFF...F read as
-1, which is the one way println could disagree with the REPL about a
value both can hold. flan_escape_bytes, so a string nested in a printed
structure is quoted and escaped -- same table as flan_dev_emit_str, noted
in both, because the REPL and println must not disagree about what a
struct looks like.
A string at top level prints raw and nested prints quoted. Not a conflict:
(println "hello") has to print hello, and a struct's string field has to
be distinguishable from the punctuation around it. The split is top-level
vs nested, so it lives in check.ml and not in the walk.
Found on the way: a field of an Option had no gep in emit.ml, so the
walk's Option arm had never run -- the REPL would have failed on one too.
Option is { i8, T } with no declared name, so its layout is now spelled
out. Nothing in the surface language reaches a field of an Option; the
printer does, to read the tag without unwrapping a None.
The print-* functions stay. They print without a newline, which println
cannot express -- slices.flan's show prints elements separated by spaces
-- and they are raw where print is structural.
println.flan covers every arm at -O0 and -O2: the u64, the raw/quoted
split, both Option arms, the depth and span caps, and the slice arm's
loop twice over plus once inside a dotimes, which is where per-call-site
slot allocation would show if it were per-iteration.
Two CIDER-shaped buffers: C-c C-i navigates a value, C-c C-b shows the
condition, the restarts and the stack. C-c C-M-b keeps the old one-key prompt.
The inspector needs no protocol change at all. eval-expr already answers a
rendered string, and Session.render writes exactly seven shapes, so that string
is a grammar. Navigation is a stack of *expressions* rather than of handles:
going into :pos sends (.pos b), into element 2 sends (at (.tags b) 2) - both
ordinary Flan a person could type. CIDER keeps its stack on the server because
a JVM value can be retained; nothing here can, since a Flan value has no header
and the render thunk is dlclosed the moment it returns. The view is therefore
never stale, where CIDER's shows the object as it was when you pushed. What it
buys over C-x C-e is the depth-4 span-8 bound: a field past it comes back as
... and nothing recovers it from the echo area, and re-rooting renders it from
depth 0.
SBCL decided the order - condition, restarts, then stack - because invoke-debugger
prints the condition and show-restarts and then stops; the backtrace is a command
you type. The restarts are the decision and the stack is the explanation for it.
And SBCL found a live bug. show-restarts omits the bracket on a name already
used further in, which is not decoration: §4 takes the first frame offering a
name, so a second frame offering retry is real, is on the list, and cannot be
chosen. The old prompt showed retry twice and sent the string either way, and
the inner frame took it silently. restarts.flan's own nested function has been
that counterexample since the transfer landed. The buffer draws the shadowed row
unbracketed and refuses RET on it by name, with a test asserting nothing was
sent - which stops the lie without restoring the choice. Taking a restart by
index is the fix and is recorded as such.
Sections that cannot be filled are drawn saying why rather than omitted: a
missing section cannot be told from an empty one, and only one of those is a
fact about the program.
Saying "the fields are not available" was under-claiming. A condition is a
struct, and Tast.structs holds every struct's field names and types in the
daemon, which owns the build — no running program is involved in answering
what a Missing is made of. Only the values need the pointer the break loop
was handed, and break_loop currently discards that pointer, so they are two
different gaps with two different fixes on two different sides of the socket.
The buffer now draws a field named and typed with its value refused, which is
what tells you whether the field you were about to blame is even a field of
this condition.
Navigation backwards was not the same list walked the other way. Forward
wrapped and backward stopped, and from the middle of a line the two disagreed
about where a field begins — a field line carries the property on all of it,
so a property-change walk from mid-line finds the end of the field you are
already in. Both now go through one list of field starts. The mid-line case
went red on the first try and the expectation was the thing that was wrong:
landing on the current field's start is what CIDER does and is the less
surprising of the two.
C-c C-b is a completing-read over restart names, which is the whole UI for
the one moment the dev loop exists to make survivable. It shows the names and
nothing else, and it will let you pick one that cannot be taken.
That last part is a bug, not a gap. §4 says restart lookup takes the first
frame offering a name, and flan_find_restart does exactly that; so a second
frame offering "retry" is real, is on the list, and is unreachable — picking
it sends the string "retry" and the inner frame runs, silently. SBCL has
shown this since forever by numbering the restarts and omitting the bracket
on a name already used. Taken as is, and the shadowed row now refuses by name
and says what would fix it: an index verb, which does not exist.
SBCL also decides the order. invoke-debugger prints the condition, then
show-restarts, and stops; the backtrace is a command you type. The restarts
are the decision and the stack is the explanation for it, and a debugger that
opens with forty frames has buried one under the other.
What CIDER's stacktrace buffer gives is the behaviour — frames that fold in
place, everything on the keyboard. Not its cause chain: a JVM exception wraps
another one and a Flan condition wraps nothing.
The fields, the stack and the locals are drawn as sections that say why they
are empty and what each would take. A section left out cannot be told from
one that happened to have nothing in it, and only one of those is a fact
about the program.
C-x C-e renders once and stops at depth 4 and span 8. A field past either
comes back as "..." and nothing recovers it from the echo area. Re-rooting
the walk at that field renders it from depth 0, so the bound moves with you
— that, and not tidiness, is why an inspector is worth having beside the
expression evaluator.
CIDER keeps its inspector stack on the server because a JVM value can be
retained. Nothing here can: a Flan value has no header and the thunk that
rendered it is dlclosed the moment it returns. So the stack is a stack of
expressions on this side, and going into a field means sending a different
one — (.pos b) where the last one was b. It costs a re-evaluation per step,
which buys a view that is never stale and is why refresh is a key someone
presses rather than a timer.
Driven from fixtures, which is also the only way the cases a live program
will not hold still for get tested at all.
The overlay half is checked as far as --batch allows and no further:
execute-kbd-macro runs no pre-command-hook there, so the hook is proved
installed in the right buffer and in nobody else's, and proved to clear and
uninstall when run — which is what the command loop does with it. That Emacs
runs it is Emacs' contract, and a test pretending to check it would be
checking nothing.
No DAP implementation here, and there should not be one. `flan build --debug'
puts DWARF in the executable, lldb reads it, lldb-dap speaks the protocol — so
what was actually missing was a dape-configs entry that knows to build a .flan
file first and where the binary lands.
The build goes through dape's own `compile' key rather than a shell-out, so a
rejected program lands in a compilation buffer and next-error walks it. Flan's
diagnostics are already file:line:col.
`flan-debug' goes through `dape--config-eval' and not `alist-get'. `dape'
takes a config whose forms are already evaluated — that is what M-x dape does
after reading one — and handing it the stored entry would pass the list
(flan-dape--binary (flan-dape--source)) to lldb as a program name. Driven
headlessly to prove it: a breakpoint set by line in the .flan buffer, hit,
reported as flan.tick at debug.flan:19 with c and n in scope.
The keybinding is registered from here rather than in flan-mode.el, so this
file is the only thing anyone has to load to get it and flan-mode keeps
working for someone who never installs dape.
The two frictions are written down at the bottom of flan-dape.el from lldb
transcripts, not from reasoning about what ought to happen, because the guess
I started from was wrong. Across a reload a breakpoint set by *name* gains a
second location and both stay live — the old body is still mapped and still
what old call sites reach. One set by *file and line* stops firing, and not
because dape pinned it to an address: the redefinition module has no line
table to resolve against. Given one, lldb does re-resolve on dlopen.
Which names the gap: Emit.redefinition takes ~debug and Session.eval does not
pass it, so `flan reload' and the `flan dev' daemon build modules without
DWARF. lib/session.ml is the dev loop's file, not this lane's.
test-flan-dape.el is not in dune test. It wants Emacs, dape, lldb-dap and a
built flan at once, and wiring four optional things into the acceptance table
would make that table's failures mean less, not more.
An overlay that lasted until the next accepted evaluation was a durable
annotation on the source, which is not what it is: it is feedback about the
action that just failed, and the moment you move, type or evaluate it is
describing a program state nobody is in any more.
pre-command-hook rather than post-command-hook, which fires at the end of
the failing command and would take the overlay down before redisplay ever
drew it. Buffer-local and installed only while an overlay exists, so a
session of twenty buffers does not end up running this on every keystroke
in all of them.
C-c C-a on a name, C-u for the IR. The header is SBCL's habit of saying
which function and out of what before a line of code, with one line it does
not have: what the answer claims. A reader looking at a listing will assume
it is what the program is running, and for a delivered-but-not-yet-installed
body that is exactly the thing that is not known — so the daemon's own
account of it sits above the first instruction rather than nowhere.
NEXT.md rewrapped to a wider column - a reflow, not a rewrite. The three TODO
entries in it are the substance: live disassembly of what is actually installed
in a cell, error overlays that vanish on the next thing you do rather than
surviving until an evaluation is accepted, and CL-style interactive recovery
where a stopped program offers a typed restart and the editor asks for the
value before invoking it.
flan-mode's declare-functions become real autoloads. A declare-function only
quiets the byte compiler; it does not load anything, so a user who had loaded
only flan-mode could not invoke M-x flan-dev at all.
