flan/emacs/flan-watch.el
Joseph Ferano 635f4d30a6 A stopped program does not sample, so the editor stops asking for a new window
The watch accumulator clears a slot lazily, on its next sample, and the reader
never compares epochs. A previous lane read that as a defect and left
watch_render_num's n=0 arm behind as dead code against the day the reader
became epoch-aware.

It should not. A stopped program takes no samples, so an epoch-aware reader
would report every slot as empty for as long as the program sat in a break
loop — and reading the numbers from the moment you stopped is the whole point
of stopping. The lazy clear is the right answer there.

What was wrong was narrower and lives in the editor: flan-watch--tick sent
`:reset t` five times a second at a program that could not answer it. The read
still goes out every tick; only the reset field drops, guarded on
flan-dev--stopped, which flan-dev.el's background poll already keeps current.

The n=0 arm is deleted rather than commented, since the only way to reach it is
the epoch check just rejected and dead code is an invitation to add one. n is
at least 1 whenever watch_render_num runs, so sum/n cannot divide by zero.

test-flan-watch.el asserts both halves with no daemon: a running tick carries
:reset, a stopped one does not, both still read the table and both leave a
reply in flight. Verified by mutation. What it cannot reach is the daemon
honouring the absent field; test_dev.ml drives a real program for that.
2026-09-13 13:57:52 +07:00

540 lines
26 KiB
EmacsLisp

;;; flan-watch.el --- Watched values, in a pinned buffer or inline -*- lexical-binding: t; -*-
;; A HUD for a running Flan program: a buffer that always shows the current
;; frame's values and nothing else. Ported from the author's Clojure
;; `clj-watch', with one thing kept and one thing inverted.
;;
;; There are two pictures of one table. The buffer is below; ghost text —
;; `flan-watch-ghost-mode', the same values shown inline at the calls that wrote
;; them — is further down, with its own rationale. Everything between here and
;; there is common to both.
;;
;; KEPT, and it is the good idea in the original: **the program decides what is
;; shown**. There is no watch-expression machinery here, no per-variable
;; registration, no UI for building a query. The program says what it wants
;; seen, from inside its own loop, and this paints it. Everything a watch list
;; would need — where it lives, whether it survives a restart, whether it gets
;; committed by accident — stops being a question when the list is the code.
;;
;; INVERTED: the original polls. Emacs calls `(watch/render)' on a timer and
;; paints the string that comes back, which in Clojure costs an eval and an eval
;; is cheap. Here it is not. An evaluation *compiles a module and dlopens it*
;; — tens of milliseconds and a new .so each time, in a directory nothing
;; sweeps — so polling at 5Hz would produce hundreds of shared objects a minute
;; to read a number that was already in a register.
;;
;; So the program pushes. It calls into the runtime's watch table from its own
;; loop; this reads the table, which is memory rather than an evaluation. Both
;; halves are cheap for opposite reasons, and two things fall out that a poll
;; could not have given:
;;
;; the values update at *frame rate* rather than at the timer's rate — the
;; timer only decides how often the picture is repainted, not how fresh it
;; is;
;;
;; and the last frame's values are still there while the program is
;; *stopped*. A break loop is precisely when no thunk can run at a frame
;; boundary, because there are no more frames, and precisely when you want to
;; see what the last one held.
;;
;; Also kept from the original, and for its stated reasons: the request is
;; **async**, because a synchronous call on a timer blocks Emacs's UI every
;; tick; and the paint is `replace-buffer-contents' rather than erase-and-
;; insert, because it diffs, so point and scroll survive a repaint instead of
;; being yanked to the top five times a second.
;;
;; What a program writes today, with no compiler change:
;;
;; (declare-c watch-i64 [name string x i64] i32 "flan_dev_watch_i64")
;; (declare-c watch-f64 [name string x f64] i32 "flan_dev_watch_f64")
;;
;; (defn step [] i64
;; (set ticks (+ ticks 1))
;; (watch-i64 "ticks" ticks)
;; ticks)
;;
;; Scalars only, so far. A struct or a slice needs a compile-time walk over
;; its type — a `(watch "hp" hp)' form in the checker — and that is a file this
;; change does not own. See BUILT.md.
;;; Code:
(require 'flan-dev)
(require 'subr-x)
(defgroup flan-watch nil
"A pinned watch buffer for a running Flan program."
:group 'flan
:prefix "flan-watch-")
(defcustom flan-watch-buffer "*flan-watch*"
"Buffer the watch table is painted into."
:type 'string)
(defcustom flan-watch-interval 0.2
"Seconds between repaints.
This is the *repaint* rate, not the watch rate. The program writes its values
every frame whatever this is; all this decides is how often the picture is
refreshed, which is why a slow value here costs freshness and nothing else."
:type 'number)
(defvar flan-watch--timer nil)
(defvar flan-watch--pending nil
"Non-nil while a watch request is out and its reply has not been read.")
(defvar flan-watch--rows nil
"The last table painted, as a list of (NAME . VALUE).")
(defvar flan-watch--consumers nil
"Which pictures of the table are currently wanted: `buffer', `ghost', or both.
There is one table, one arming message and one timer, and more than one way to
look at what they produce. Holding the subscription here rather than in the
watch buffer is what lets ghost text outlive that buffer being closed — the
original design made the buffer *be* the subscription, which was right when it
was the only consumer and wrong the moment it was not.")
;;; Painting
(defun flan-watch--format (rows overflow)
"The buffer's text for ROWS. OVERFLOW means some name found no slot."
(if (null rows)
(concat "nothing is being watched\n\n"
"The program decides what is shown. Call into the watch table\n"
"from your own loop:\n\n"
" (declare-c watch-i64 [name string x i64] i32 \"flan_dev_watch_i64\")\n"
" ...\n"
" (watch-i64 \"ticks\" ticks)\n")
(let ((w (apply #'max (mapcar (lambda (r) (length (car r))) rows))))
(concat
(mapconcat (lambda (r)
;; Padded before it is propertised, not with a width in the
;; format string: `format' has no `%-*s', and a face on the
;; padding would underline trailing space.
(concat (propertize (car r) 'face
'font-lock-variable-name-face)
(make-string (+ 2 (- w (length (car r)))) ?\s)
(cdr r)))
rows "\n")
"\n"
;; Reported rather than swallowed. A name past the table's limit is a
;; value that simply never appears, and a buffer that said nothing about
;; it would be lying by omission — the reader would go looking for a bug
;; in the program. A flag and not a number: the only count the runtime
;; could keep is of write *attempts* that missed, and those happen every
;; frame, so one name too many would read as thousands.
(if overflow
(concat "\nthe table is full: some names found no slot,"
" and their values are not shown.\n"
"It holds 64.\n")
"")))))
(defun flan-watch--paint (text)
"Replace the watch buffer's contents with TEXT."
(when-let* ((buf (get-buffer flan-watch-buffer)))
(let ((tmp (get-buffer-create " *flan-watch-src*")))
(with-current-buffer tmp
(erase-buffer)
(insert text))
(with-current-buffer buf
(let ((inhibit-read-only t))
;; `replace-buffer-contents' diffs rather than erasing, so point and
;; the window's scroll position survive every tick. `erase-buffer'
;; and insert would yank the cursor to the top five times a second,
;; which makes the buffer unusable for the one thing you want to do
;; in it — look at a particular line while the program runs.
(replace-buffer-contents tmp))))))
;;; Ghost text — the same table, painted at the code instead of beside it
;; The buffer shows everything at once, which is what you want while the game
;; runs. Ghost text shows one value where it was written, which is what you
;; want while you read one function. They are two pictures of one table, so
;; this is an addition and not a replacement, and neither one polls separately:
;; both are painted from the same reply, in `flan-watch--absorb'.
;;
;; WHERE A VALUE ATTACHES. The table carries a name and a rendered string and
;; no source location, and it is not going to grow one here — that needs a
;; `(watch ...)' form in the checker, which is a file this does not own. But a
;; location is not actually missing: `(watch-i64 "hp" hp)' is *in the buffer*,
;; and the name in the table is the string literal in it. So the anchor is
;; found by searching the text rather than by being told, which costs one
;; regexp scan per displayed buffer per repaint and needs nothing new from the
;; daemon.
;;
;; The head of that call cannot be hardcoded. `watch-i64' is a name the
;; program's author chose in their own `declare-c'; only the C symbol behind it
;; is fixed, and the editor never sees that. Hence
;; `flan-watch-ghost-call-regexp', whose default matches the shape the manual
;; suggests and which anyone who named theirs differently can change. Matching
;; the string literal and not the head is what makes that safe: a name in the
;; table came from one of these call sites by construction.
;;
;; WHEN IT UPDATES. On the same timer, from the same reply. The two pictures
;; therefore cannot disagree — they are one table read, painted twice — and
;; there is no second `:op "watch"' in flight, which is the invariant
;; `flan-dev-settle-hook' exists to keep. As with the buffer, the timer decides
;; how often the picture is repainted and not how fresh it is: the program
;; writes every frame regardless.
;;
;; Overlays are deleted and re-placed from scratch on every repaint rather than
;; being tracked across edits. That is the whole answer to the invalidation
;; problem the earlier sketch of this called the work: a line that moved does
;; not leave an overlay behind, because no overlay outlives a tick. The cost is
;; bounded by only scanning buffers that are *shown in a window* — an overlay in
;; a buffer nobody is looking at is invisible anyway, and a project's other
;; hundred files are not scanned. It also means a file you scroll to gets its
;; ghost text within one tick with nothing to hook.
;;
;; TWO SITES, ONE NAME. Both get the overlay and both show the same value, and
;; the text says so. That is the honest picture rather than a failure to pick:
;; the table has one slot per name and the last writer in the frame wins, so
;; the value at both sites really is the same value, and it is whichever call
;; ran last. Silently showing it at one site would suggest the other is not
;; running; showing it at both without a word would read as a coincidence.
;;
;; A LOOP. Inline shows the last value written, exactly as the buffer does.
;; "The last of 4000 iterations" is usually not the interesting one, but every
;; better answer is a UI for building a query, which is the thing this design
;; exists not to have. Settled, not open.
;;
;; STOPPED. The values survive a break — that is a property of pushing rather
;; than polling, and half the reason for it. But they are the *last frame's*,
;; and inline they sit in the middle of live-looking code with no modeline next
;; to them, so they say "last frame" in words and change face. The watch buffer
;; does not need to: you opened it deliberately and `flan:stopped(...)' is
;; already in view.
(defcustom flan-watch-ghost-call-regexp "watch\\(?:-[[:alnum:]]+\\)*"
"Regexp matching the head of a call that writes the watch table.
Matched against the symbol after an open paren, with the watched name as a
string literal after it. The default covers `watch', `watch-i64', `watch-f64',
`watch-str' and the two-segment accumulator names `watch-num-i64' and
`watch-num-f64' — hence the `*\=' rather than a `?\=', which stopped at one
hyphenated segment and so found no site for a numeric watch at all. It is a setting because the Flan name is the program author's
own `declare-c' binding — only the C symbol behind it is fixed, and the editor
never sees that."
:type 'regexp)
(defface flan-watch-ghost-face '((t :inherit shadow))
"Face for an inline watched value while the program is running.")
(defface flan-watch-ghost-stale-face '((t :inherit warning))
"Face for an inline watched value while the program is stopped.
Distinct from the running face because the value is the last frame's, and
inline it has no modeline beside it to say so.")
(defvar flan-watch--ghost-overlays nil
"Every overlay this has placed, across every buffer.")
(defun flan-watch--ghost-clear ()
"Remove every overlay this has placed."
(mapc #'delete-overlay flan-watch--ghost-overlays)
(setq flan-watch--ghost-overlays nil))
(defun flan-watch--ghost-buffers ()
"Flan buffers currently shown in some window, on any frame."
(let (bufs)
(dolist (w (window-list-1 nil 'nomini t))
(let ((b (window-buffer w)))
(when (and (eq (buffer-local-value 'major-mode b) 'flan-mode)
(not (memq b bufs)))
(push b bufs))))
bufs))
(defun flan-watch--ghost-sites ()
"Watch call sites in the current buffer, as a list of (NAME . END-OF-LINE)."
(let ((re (concat "(\\s-*" flan-watch-ghost-call-regexp
"\\s-+\"\\([^\"\n]*\\)\""))
(sites nil))
(save-excursion
(goto-char (point-min))
(while (re-search-forward re nil t)
;; Everything wanted from this match is read out *before* the check.
;; `syntax-ppss' moves point and clobbers the match data, so reading
;; either afterwards puts the scan back where it started and the loop
;; never ends. Found the hard way.
(let ((beg (match-beginning 0))
(end (match-end 0))
(name (match-string-no-properties 1))
(eol (line-end-position)))
;; A call inside a string or a comment is text that looks like code.
(unless (nth 8 (syntax-ppss beg))
(push (cons name eol) sites))
(goto-char end))))
(nreverse sites)))
(defun flan-watch--ghost-row (name rows)
"The row in ROWS for the site name NAME, or nil.
Falls back to a prefix match against a name the runtime clipped: it holds 31
bytes of a name, so a longer one in the source will never match it exactly.
Guarded on the row's length so a short name cannot be claimed by accident."
(or (assoc name rows)
(let (hit)
(dolist (r rows hit)
(when (and (not hit)
(>= (string-bytes (car r)) 31)
(string-prefix-p (car r) name))
(setq hit r))))))
(defun flan-watch--ghost-text (value sites stale)
"The after-string for a value: VALUE, seen at SITES places, STALE if stopped."
(propertize
(concat " => " value
(when (> sites 1) (format " one slot, %d sites" sites))
(when stale " last frame"))
'face (if stale 'flan-watch-ghost-stale-face 'flan-watch-ghost-face)
;; Without this, point at end of line lands on the ghost text rather than
;; on the buffer's own last column.
'cursor t))
(defun flan-watch--ghost-paint (rows overflow)
"Place inline overlays for ROWS. OVERFLOW means some name found no slot."
(flan-watch--ghost-clear)
(let ((stale (and flan-dev--stopped t)))
(dolist (buf (flan-watch--ghost-buffers))
(with-current-buffer buf
(let* ((sites (flan-watch--ghost-sites))
(counts (make-hash-table :test #'equal)))
(dolist (s sites)
(puthash (car s) (1+ (gethash (car s) counts 0)) counts))
(dolist (s sites)
(let* ((row (flan-watch--ghost-row (car s) rows))
(text
(cond
(row (flan-watch--ghost-text
(cdr row) (gethash (car s) counts 1) stale))
;; A site with no row and a full table is a value that
;; exists and was dropped, which is the one case worth
;; saying out loud — the buffer's overflow line says the
;; same thing without being able to say *which*. A site
;; with no row and room to spare has simply not run yet,
;; and annotating every one of those at startup would be
;; noise.
(overflow (flan-watch--ghost-text
"no slot: the table is full" 1 stale)))))
(when text
(let ((ov (make-overlay (cdr s) (cdr s))))
(overlay-put ov 'after-string text)
(overlay-put ov 'flan-watch-ghost t)
(push ov flan-watch--ghost-overlays))))))))))
;;; The tick
(defun flan-watch--absorb (reply)
"Paint REPLY, a watch answer from the daemon."
(pcase (plist-get reply :status)
("ok"
(setq flan-watch--rows
(mapcar (lambda (r) (cons (nth 0 r) (nth 1 r)))
(plist-get reply :watch)))
(flan-watch--paint
(flan-watch--format flan-watch--rows (plist-get reply :overflow)))
(when (memq 'ghost flan-watch--consumers)
(flan-watch--ghost-paint flan-watch--rows (plist-get reply :overflow))))
(_
;; The overlays go rather than being left at their last values. Inline
;; there is nowhere to print the error, so a stale number beside live code
;; would be the whole picture and it would be wrong; the watch buffer, if
;; it is open, says what happened.
(flan-watch--ghost-clear)
(flan-watch--paint
(format "error:\n%s\n" (or (plist-get reply :message) "refused"))))))
(defun flan-watch--settle ()
"Collect an outstanding watch reply, blocking if it has not arrived.
Hung on `flan-dev-settle-hook', so an ordinary request never reads the watch
timer's reply as its own. Blocking here is fine and blocking in the tick is
not: this runs inside something a person asked for, which already waits, and
what it waits for is a table read with nothing compiled behind it."
(when flan-watch--pending
(setq flan-watch--pending nil)
(when-let* ((proc flan-dev--connection))
(when (process-live-p proc)
(ignore-errors (flan-watch--absorb (flan-dev--read-reply proc)))))))
(defun flan-watch--tick ()
"Collect the last reply if it has come, then ask again. Never blocks.
Deliberately not `flan-dev--request', which waits for its answer: a
synchronous call on a 0.2s timer stalls Emacs's UI every tick, and a timer is
the one caller that must not. So this takes whatever has already arrived and
sends the next question, leaving at most one request in flight — the invariant
`flan-dev-settle-hook' exists to keep."
(cond
;; Killing the buffer cancels the buffer's half of the subscription, and
;; only that. When it was the only consumer this stops the timer and
;; disarms the table, so a program nobody is watching is back to paying a
;; load and a branch per watch call; when ghost text is also on, the table
;; stays armed and the ticks carry on feeding it.
((and (memq 'buffer flan-watch--consumers)
(not (get-buffer flan-watch-buffer)))
(flan-watch--drop 'buffer))
((not (process-live-p flan-dev--connection))
(flan-watch--paint "error:\nnot connected to a running program\n")
(flan-watch-stop))
;; Something else owns the connection this instant — an evaluation is
;; mid-flight. Skipping is right: its `flan-dev-settle-hook' has already
;; taken any reply of ours, and the next tick is 0.2s away.
(flan-dev--busy nil)
(t
(when flan-watch--pending
(when-let* ((reply (flan-dev--take-reply flan-dev--connection)))
(setq flan-watch--pending nil)
(flan-watch--absorb reply)))
(unless flan-watch--pending
(condition-case nil
;; `:reset t' opens a new accumulation window for the numeric
;; slots, so their count, range and mean are "since the last tick"
;; rather than since the program started. That is the whole of the
;; accumulator decision as the editor sees it: a min and a max over
;; a session go dead within seconds of play, and this tool exists to
;; show a number while you drag the mouse. The reset happens after
;; the read, on the daemon's side, so this tick's numbers are the
;; last tick's window and nothing is lost between the two.
;;
;; Except while the program is stopped, when the read still happens
;; and the reset does not. A stopped program takes no samples, so
;; there is no window for a reset to close and none for it to open:
;; the runtime clears a slot lazily, on its next sample, which is
;; what keeps a paused program showing the numbers from the moment
;; you paused it — the whole reason to pause. Resetting anyway
;; would ask for a new window five times a second that nothing can
;; fill. The guard lives here rather than in the runtime because
;; "since you last looked" is the editor's policy, not the table's.
;; `flan-dev--stopped' is the one place that state is tracked, and
;; `flan-dev.el''s background poll keeps it current whether or not
;; anyone is evaluating.
(progn (flan-dev--send flan-dev--connection
(if flan-dev--stopped
'(:op "watch")
'(:op "watch" :reset t)))
(setq flan-watch--pending t))
(error (flan-watch-stop)))))))
;;; Commands
(define-derived-mode flan-watch-mode special-mode "flan-watch"
"Major mode for the pinned watch buffer."
(setq-local truncate-lines t))
(defun flan-watch--subscribe (consumer)
"Arm the table for CONSUMER and make sure the shared timer is running.
Arming is a message, not something the daemon infers. The program is the
writer, so it has to be told somebody is looking — and while nobody is, nothing
writes the table at all, which is what makes a watch call in a program nobody is
debugging a load and a not-taken branch. It is sent once for the first
consumer: two of them looking at one table is still one table."
(flan-dev--live-connection)
(unless flan-watch--consumers
(let ((r (flan-dev--request '(:op "watch-enable" :on t))))
(unless (equal (plist-get r :status) "ok")
(user-error "flan: %s" (or (plist-get r :message) "watch refused")))))
(unless (memq consumer flan-watch--consumers)
(push consumer flan-watch--consumers))
(add-hook 'flan-dev-settle-hook #'flan-watch--settle)
(when flan-watch--timer (cancel-timer flan-watch--timer))
(setq flan-watch--timer
(run-with-timer 0 flan-watch-interval #'flan-watch--tick)))
(defun flan-watch--drop (consumer)
"Stop painting CONSUMER, and tear everything down if it was the last one."
(setq flan-watch--consumers (delq consumer flan-watch--consumers))
(when (eq consumer 'ghost) (flan-watch--ghost-clear))
(unless flan-watch--consumers (flan-watch-stop)))
;;;###autoload
(defun flan-watch ()
"Open the watch buffer and start painting the running program's values."
(interactive)
(flan-watch--subscribe 'buffer)
(with-current-buffer (get-buffer-create flan-watch-buffer)
(unless (eq major-mode 'flan-watch-mode) (flan-watch-mode)))
;; Painted before the first reply, rather than left blank until one arrives.
;; An empty buffer is the same picture as a broken one, and the likeliest
;; reason for it here is the honest one — the program is not calling into the
;; table — which is worth saying in words rather than by showing nothing.
(flan-watch--paint (flan-watch--format nil 0))
(display-buffer flan-watch-buffer))
;;;###autoload
(define-minor-mode flan-watch-ghost-mode
"Show watched values inline, beside the calls that wrote them.
An addition to `flan-watch' rather than a replacement: the buffer is what you
want when you want everything at once, and this is what you want when you are
reading one function. Both are painted from the same table read, so they
cannot disagree, and turning either off leaves the other running."
:global t
:lighter " flan-ghost"
(if flan-watch-ghost-mode
;; A refused arming leaves the mode off rather than on-and-doing-nothing.
(condition-case err
(flan-watch--subscribe 'ghost)
(error (setq flan-watch-ghost-mode nil) (signal (car err) (cdr err))))
(flan-watch--drop 'ghost)))
(defun flan-watch-stop ()
"Stop painting, and tell the program to stop writing the table.
Tears down both consumers. `flan-watch--drop' is the way to stop one of them."
(interactive)
(setq flan-watch--consumers nil)
;; Directly, not through the mode function: that would call back into
;; `flan-watch--drop' and back into here.
(setq flan-watch-ghost-mode nil)
(flan-watch--ghost-clear)
(when flan-watch--timer
(cancel-timer flan-watch--timer)
(setq flan-watch--timer nil))
;; Settle before disarming, or the disarm request reads the tick's reply.
(flan-watch--settle)
(remove-hook 'flan-dev-settle-hook #'flan-watch--settle)
;; Only on a connection that is already live, and this is the important half.
;; `flan-dev--request' *reconnects* — which is right for something a person
;; did and wrong here, because this is also called from the tick, and the
;; reason the tick calls it is that the connection has gone. Reconnecting
;; from a timer would quietly erase the `lost' state that exists to be seen,
;; which `flan-dev.el' already forbids for its own poll timer. And there is
;; nothing to disarm anyway: the table went with the program.
(when (process-live-p flan-dev--connection)
(ignore-errors (flan-dev--request '(:op "watch-enable" :on nil)))))
;;; What ghost text still cannot show
;; Built, above, on a buffer search rather than on a location from the runtime
;; — see the rationale there. An earlier note in this file said ghost text was
;; gated on a `(watch ...)' form in the checker, because nothing in the table
;; carries a file:line:col. That is still true of the *table* and it turned out
;; not to be the blocker: the call site is in the buffer, and the name in the
;; table is the string literal in it, so the anchor is searched for instead of
;; being told. Nothing new is asked of the daemon.
;;
;; What the search cannot do, and what it costs:
;;
;; - A watch call written by a macro has no literal to find, so it gets no
;; ghost text. The watch buffer still shows it. This is the honest limit
;; of anchoring on text: the editor is reading source, not debug info, and
;; a form that only exists after expansion is not in the source.
;;
;; - A name assembled at run time — anything that is not a literal at the call
;; site — is the same case for the same reason.
;;
;; - The module in the running program can be older than the buffer. A site
;; you have typed but not yet installed with `C-c C-c' has no row, and one
;; you deleted but not yet installed still writes. Deliberately unreported:
;; the stale-caller machinery in `flan-dev.el' already tracks what needs
;; re-evaluating, and a second opinion about it here would be a worse one.
;;
;; Still genuinely blocked on the checker, and unchanged by any of this: a
;; **struct or a slice**. Rendering one is a compile-time walk over its type,
;; the runtime has scalar entry points only, and no amount of searching the
;; buffer produces a value the table does not hold. See BUILT.md.
(provide 'flan-watch)
;;; flan-watch.el ends here