From 3dd9f61b7d7e37e373b861802ecdbdfaad8f0dc6 Mon Sep 17 00:00:00 2001 From: Joseph Ferano Date: Sun, 13 Sep 2026 21:06:58 +0700 Subject: [PATCH] A macro call says what it expands to, and a Form learns to print itself C-c C-m. One step on the bare key, the fixpoint under C-u: a macro may quasiquote a call to another macro, and Loc.from_macro is outermost-wins, so by the time a full expansion settles the intermediate name is gone. One step is the only thing that can say which macro produced what. The expansion runs against the macros the *session* holds -- the prelude's, its imports', and every defmacro evaluated since it started -- and writes nothing back: a defmacro handed to C-c C-m does not join the session by having been looked at. Both non-termination refusals stay refusals, and only where they are needed. One step makes one call and does not look at the answer, so (s/spin) one- stepped answers with itself; all the way hits the fuel and names the macro, inside Dev.serve's guard, so the daemon replies rather than hanging. Macro's module handling is a Fun.protect now -- a build that raised was a process about to exit, and the daemon is not that process. No printer for a Form existed. Form.to_string is an error-message renderer and is what Macro.key digests, so it is untouched; Form.to_source round-trips floats, strings and bytes through the reader, and Form.pretty decides where the line breaks go and leaves the columns to flan-mode. The answer is a read-only flan-mode buffer shaped like the disassembly one, with cnr's idea in it: m expands the form at point one more step in place. Three inherited keys refuse by name -- an expansion is in no file. The text is sent padded onto its own line and its own column, unlike C-x C-e, so the refusal lands on the call and not at the start of its line. --- BUILT.md | 102 ++++++++++++++ emacs/MANUAL.md | 43 +++++- emacs/flan-dev.el | 280 +++++++++++++++++++++++++++++++++++++++ emacs/flan-mode.el | 11 ++ emacs/test-flan-cider.el | 78 +++++++++++ emacs/test-flan-dev.el | 136 +++++++++++++++++++ lib/dev.ml | 65 +++++++++ lib/form.ml | 145 ++++++++++++++++++++ lib/macro.ml | 95 +++++++++++-- lib/session.ml | 69 ++++++++++ test/test_repl.ml | 86 ++++++++++++ test/test_session.ml | 133 +++++++++++++++++++ web/index.html | 1 + 13 files changed, 1234 insertions(+), 10 deletions(-) diff --git a/BUILT.md b/BUILT.md index bedf164..ec6c5f0 100644 --- a/BUILT.md +++ b/BUILT.md @@ -5505,3 +5505,105 @@ Both editor paths work and they are different wraps — `Parse.expr`'s for `C-x plus editing that macro and calling it again; the assertions are on the IR and not on the absence of an exception, because an expression that did not expand raises while one that expanded to the *wrong* thing does not. + +## `C-c C-m`: what a macro call expands to, and a printer for `Form` + +Macros grew far enough today that the editor had to be able to ask. The argument is one line: **a macro is +importable from a package, arriving qualified**, so `(rl/with-drawing …)` is a call whose `defmacro` lives in +another directory. There is nothing beside the call site to read — and there would be nothing to read even if the +`defmacro` were in the buffer, because a macro answers a `Form` and nobody ever wrote that down. + +### One step on the key, all the way under `C-u` + +Both, because the difference is real here and the second is nearly free once the first exists. A macro may +quasiquote a call to another macro — `mac/quad` answers `(mac/twice (mac/twice n))` — so one step and the fixpoint +are different text. + +One step is the *bare* key, and the reason is `Loc.from_macro`. It is outermost-wins by design: the macro the +author actually wrote is the one worth naming. So by the time a full expansion settles, every node of it is stamped +`mac/quad` and `mac/twice` is unnameable. All the way is the answer the compiler acts on; one step is the only way +to find out which macro produced what. `C-u` for the other half rather than a second key is the rule `C-c C-a` +already follows: it is the same question asked of the same form. + +One step is **outermost-only**, and that is a deliberate divergence from `Macro.expand_form`, which expands a +call's arguments before calling it. `(mac/twice (mac/twice 1))` one-stepped here is `(+ (mac/twice 1) (mac/twice +1))`; the compiler's own first move is `(mac/twice (+ 1 1))`. Different intermediates, the same fixpoint. Written +down because someone will otherwise discover it and file it. + +### The session's macros, not a fresh read + +`Session.macroexpand` reads `t.macros` and writes nothing at all. Re-reading the file would answer with macros the +session was never told about, and with whatever is *saved* rather than what is typed — the unsaved-versus-saved +skew "A session expands the buffer's own macros" already refused, and it is worse here than anywhere, because an +expansion that disagrees with what an evaluation does is a lie about what the code *is*. + +Writing nothing is a property and it is pinned: a `defmacro` handed to `C-c C-m` must not join the session by +having been looked at, so `test_session` and `test_repl` both expand one and then require the name to still be +unknown. `C-c C-c` is where a declaration goes. + +### The refusals, and the one that must not be reachable + +The verb sits inside the guard the robustness lane put round the whole of `handle` in `Dev.serve` — confirmed +rather than assumed, and it is why `Dev.macroexpand` has no `Loc.Error` arm of its own. Both non-termination +refusals raise `Loc.Error` and come back as replies with the call site on them. + +**The bound has to be reached, and only where it is needed.** A hang in the daemon wedges the editor with the +program still on screen. So: all the way runs `Macro.expand_form`, hits `Macro.fuel` and names the macro; one step +makes exactly one call and does not look at what comes back, so `(s/spin)` one-stepped *answers*, with `(s/spin)` +and the name `s/spin`. A command that refused to show anybody the thing they were trying to see would be the wrong +kind of safe. The ring is refused a level up, while the package holding it is parsed, so no session over one can +exist and nothing on this path can reach it. + +`Macro`'s module handling is now a `Fun.protect`. It used to be two statements after the call, which was enough +while a build was the only caller — a build that raised was a process about to exit. The daemon is not that +process: a macro that does not settle raises, the editor is told, and the next `C-c C-m` does it again, so +`Dynload.owned` must not be a list that only grows over a session's lifetime. + +### `Form.to_source` and `Form.pretty`, because no printer existed + +`Form.to_string` is an error-message renderer — one line, no width, `%S` and `%g`. It is also what `Macro.key` +digests, so every cached macro module on disk is keyed by what it prints; it is untouched. What an expansion needs +is text a *reader reads back*, and three of its spellings are not round trips, all reachable because a macro may +build any literal: `%g` prints 1.0 as `1` and truncates at six digits, `%S` is OCaml's escaping where the reader +takes exactly six escapes, and `Byte` falls through to `\` where the reader names `\nul` and `\return`. + +`Form.pretty` decides **where the line breaks go and not where the columns do**. Structure is a printer's job; +indentation is `flan-mode`'s, which is where this project's rules already live — the buffer runs `indent-region` +over what it is shown, so nothing in OCaml has to know that a `let` aligns its bindings under the bracket. A client +with no Emacs still gets something readable rather than one very long line. + +### Where the answer goes + +`*flan-macroexpansion*`, a read-only `flan-mode` buffer — what is in it is Flan source and reading it is the whole +point, so the font-lock and the indentation are the ones already there. It follows `flan-disassemble`'s shape, +header comment lines and all, rather than `flan-cnr`'s: cnr's folding is frame-and-locals machinery with its own +text properties and there is nothing here to fold. What *was* taken from cnr is the idea its header names — a key +that expands in place. `m` (or `C-c C-m`) takes the form at point one more step and puts the result where it was, +which is what makes one-step-by-default usable rather than a thing you press once and lose. `a` goes to the +fixpoint, `g` asks again — useful after re-evaluating the `defmacro` — and `q` closes. + +Three keys inherited from `flan-mode-map` are shadowed by name, not unbound, so pressing one says why: `C-c C-c`, +`C-c C-k` and `C-x C-e` over an expansion would install a body nobody wrote under a name somebody did. + +**What `Loc.from_macro` means for what is printed** is a header line, because there is nowhere else it could be +said. Every node below it carries the *call site's* file, line and column with the macro's name stamped on it — +which is how an error inside expanded code points at the call you wrote — and therefore none of it has a location +of its own, nothing in it is the text of any file, and there is nothing for `M-.` to jump to. + +### What gets sent, and the padding that is not `C-x C-e`'s + +The region is `C-x C-e`'s, plus one case it has no need of: the form *at* point when point is on its opening +delimiter. A macro call is a form you put point on, and `backward-sexp` from an open paren takes the previous +sibling, which is never what was meant. + +The text is padded onto its own line **and its own column**, which `flan-dev--text` does not do and says why it +does not: a top-level form starts at column 1, so the columns already agreed. A macro call does not — it is +written well inside a `defn` — and the refusal this path can get carries a column measured from the start of the +snippet, so an unpadded send would draw "did not settle" at the start of the line. Leading newlines and leading +spaces are both whitespace the reader skips. `test-flan-dev.el` checks the arithmetic the only way that proves it: +it evaluates a spinning `defmacro` into a live session, asks for the fixpoint of a call to it from a known buffer +position, and requires the error overlay to start at exactly that position. + +The in-buffer `m` sends its text *un*padded, which is the honest shape there: the expansion is in no file, so +there is no line or column for a refusal to be drawn at. The `:file` still goes on the wire, because that is what +says which session's macros to expand against. diff --git a/emacs/MANUAL.md b/emacs/MANUAL.md index 695a990..325bd72 100644 --- a/emacs/MANUAL.md +++ b/emacs/MANUAL.md @@ -108,6 +108,41 @@ hit it as many times as you like; an ordinary `C-c C-c` over the same form (or expression instead of printing its value. That one does not stick, because there is no definition for it to stick to. +### `C-c C-m` — what a macro call expands to + +Put point on a macro call and press it. `*flan-macroexpansion*` opens with the +code that call turns into, as Flan source, indented and coloured like any other. + +**One step by default, `C-u C-c C-m` for all the way.** The two genuinely +differ: a macro may produce a call to another macro, so `(mac/quad 3)` is +`(mac/twice (mac/twice 3))` after one step and `(+ (+ 3 3) (+ 3 3))` at the end. +One step is the default because it is the one that can say *which* macro +produced what — the full expansion is tagged with the name you wrote and +nothing else. + +Inside the buffer, `m` expands the form at point one more step **in place**, +`a` takes it to the end, `g` asks again — useful after you have re-evaluated +the `defmacro` — and `q` closes it. + +This matters most for a macro you cannot read beside the call: a package's +macros arrive qualified, so `rl/with-drawing` is defined in another directory +and there is nothing next to the call site to look at. + +It expands against the macros **this session holds** — the prelude's, the ones +its imports brought in, and every `defmacro` you have evaluated since it +started — not against a fresh read of the file. So it agrees with what an +evaluation does, and it sees the macro you have typed rather than the one you +last saved. + +Two things the buffer says out loud. Nothing in it has a location of its own: +every node carries the *call site's* file, line and column with the macro's +name stamped on it, which is how an error inside expanded code points at the +call you wrote. And it is not the text of any file, so `C-c C-c`, `C-c C-k` and +`C-x C-e` in that buffer refuse rather than installing code nobody wrote. + +A macro that never settles is refused at a bound and named, with the refusal +drawn on the call — it does not hang. + ### `C-c C-k` — the whole buffer The whole buffer, sent as **one** module rather than as a form at a time. That @@ -554,6 +589,7 @@ mode's, so they work in a file you have only opened. | `C-c C-d` | what the running program currently defines | | `C-c C-v` | help on the name at point | | `C-c C-a` | disassemble a function; `C-u` first for its LLVM IR | +| `C-c C-m` | what the macro call at point expands to; `C-u` for all the way | Completion, eldoc and `M-.` all read one cached answer rather than asking the program per keystroke. It refreshes at the two moments the answer can have @@ -599,6 +635,8 @@ Use `C-c C-g` if you need frames. | `C-c C-b` | a stopped program: condition, restarts, stack | | `C-c C-M-b` | the same restarts, as a one-key prompt | | `C-c C-i` | inspect a value | +| `C-c C-m` | what the macro call at point expands to | +| `C-u C-c C-m` | ...all the way, rather than one step | | `C-c C-a` | disassemble; `C-u` first for LLVM IR | | `C-c C-g` | debug under lldb, through dape | | `C-c C-d` | what the running program defines | @@ -609,8 +647,9 @@ Use `C-c C-g` if you need frames. Commands with no key: `M-x flan-dev` (start a program), `M-x flan-dev-quit` (stop it), `M-x flan-watch` (the watch buffer), `M-x flan-watch-stop`, `M-x flan-watch-ghost-mode` (the same values inline), -`M-x flan-inspect-address` (what is at an address), and `M-x flan-allocations` -/ `M-x flan-leaks` (where the memory went, and what is still held). +`M-x flan-inspect-address` (what is at an address), `M-x flan-macroexpand-all` +(the `C-u` half of `C-c C-m`, by name), and `M-x flan-allocations` / +`M-x flan-leaks` (where the memory went, and what is still held). --- diff --git a/emacs/flan-dev.el b/emacs/flan-dev.el index d128a05..312621c 100644 --- a/emacs/flan-dev.el +++ b/emacs/flan-dev.el @@ -32,6 +32,11 @@ (require 'cl-lib) (require 'xref) (require 'eldoc) +;; The macroexpansion buffer is a `flan-mode' buffer — what is in it is Flan +;; source and reading it is the point — so the mode has to exist by the time +;; this file defines one derived from it. Not a cycle: flan-mode.el autoloads +;; the commands here and requires nothing back. +(require 'flan-mode) (defgroup flan-dev nil "Talking to a running Flan program." @@ -1444,6 +1449,28 @@ columns already were, because a top-level form starts at column 1." (concat (make-string (1- (line-number-at-pos start)) ?\n) (buffer-substring-no-properties start end))) +(defun flan-dev--text-at (start end) + "The buffer text from START to END, on the line AND column it is written at. + +`flan-dev--text' pads lines only, and says why it needs nothing more: a +top-level form starts at column 1, so the columns already agreed. A macro +call does not. It is written somewhere inside a `defn', and a refusal the +daemon reports against it — a macro that never settles is the one that +happens — carries a column that would otherwise be measured from the start of +the snippet and drawn at the start of the line. + +Leading newlines and leading spaces are both whitespace the reader skips, so +padding with each is the whole fix. Byte columns, for the reason +`flan-dev--wire-position' gives: the reader walks the source a byte at a time, +and a space is one byte, so a byte count is exactly how many to write." + (save-excursion + (goto-char start) + (concat (make-string (1- (line-number-at-pos start)) ?\n) + (make-string (- (position-bytes start) + (position-bytes (line-beginning-position))) + ?\s) + (buffer-substring-no-properties start end)))) + (defun flan-dev--defun-bounds () "Bounds of the top-level form containing or preceding point, as (START . END)." (save-excursion @@ -1657,6 +1684,259 @@ that the IR half is findable by name rather than only by a modifier." nil t)))) (flan-disassemble name t)) +;;; What a macro call expands to + +;; CIDER's `C-c C-m', and the reason it arrives now rather than with macros +;; themselves: a macro is importable from a package, so `(rl/with-drawing ...)' +;; is a call whose `defmacro' lives in another directory. Until the editor can +;; ask, there is nothing to read beside the call site at all — and the answer +;; is not in the file either way, because a macro answers a `Form' and nobody +;; ever wrote that down. +;; +;; Two commands off one key, the way `C-c C-a' already does it: `C-c C-m' is +;; one step and `C-u C-c C-m' is all the way. One step is the bare key because +;; it is the one that can name the macro that ran. A macro may quasiquote a +;; call to another macro — `mac/quad' answers `(mac/twice (mac/twice n))' — and +;; `Loc.from_macro' is outermost-wins, so by the time a full expansion settles +;; every node of it is stamped with the name the author wrote and the +;; intermediate ones are gone. All the way is the answer the compiler acts on; +;; one step is how you find out which macro produced what. +;; +;; The buffer is `flan-mode', because what it holds is Flan source and the +;; whole point is to read it: the indentation and the font-lock are this +;; project's own. Three keys are shadowed for exactly that reason — the text +;; in here is *not* the text of any file, it has no locations of its own, and +;; `C-c C-c' over it would install a body nobody wrote under a name somebody +;; did. +;; +;; From cnr, the idea rather than the code: a key that expands in place. `C-c +;; C-m' inside the buffer takes the form at point one more step and puts the +;; result where it was, which is what makes one-step-by-default usable rather +;; than a thing you press twice and lose. cnr's own folding is frame-and-locals +;; machinery with its own text properties and there is nothing here to fold. + +(defcustom flan-macroexpansion-buffer "*flan-macroexpansion*" + "Where `flan-macroexpand' draws." + :type 'string) + +(defvar flan-macroexpand-request-function #'flan-dev--request + "How the macroexpansion buffer reaches the daemon. +One plist in, the reply plist out. A variable for `flan-cnr-request-function''s +reason: so the renderer can be driven from a fixture.") + +(defvar-local flan-macroexpand--origin nil + "What this buffer is showing, as a plist: :code, :file and :all. +`g' re-asks it, so an expansion can be refreshed after the `defmacro' behind +it has been re-evaluated.") + +(defun flan-macroexpand--bounds () + "Bounds of the form to expand, as (START . END). + +The form before point, which is the region `C-x C-e' chooses and the one +CIDER's own macroexpand uses — and the form *at* point when point sits on its +opening delimiter, which `C-x C-e' has no need of and this does. A macro call +is a form you put point on; `backward-sexp' from an open paren takes the +previous sibling, which is never what was meant." + (save-excursion + (skip-chars-forward " \t") + (if (looking-at-p "[([{]") + (cons (point) (save-excursion (forward-sexp) (point))) + (let ((end (point))) + (backward-sexp) + (cons (point) end))))) + +(defun flan-macroexpand--ask (code file all) + "Ask the daemon what CODE, written in FILE, expands to. +ALL asks for the fixpoint rather than one step. Answers the reply plist, or +signals — having drawn the refusal where it happened, which is why the caller +sends padded text." + (let ((r (funcall flan-macroexpand-request-function + (list :op "macroexpand" :code code :file file + :all (if all t nil))))) + (unless (equal (plist-get r :status) "ok") + (let ((loc (plist-get r :loc)) + (msg (plist-get r :message))) + ;; The two refusals that reach here are a macro that never settles and + ;; a ring, and both name a macro at a location — so mark it, exactly as + ;; a refused evaluation is marked. This must not itself signal: the + ;; error the caller is owed is the daemon's. + (ignore-errors (flan-dev--show-error loc (or msg "refused"))) + (user-error "flan: %s%s" (or msg "refused") + (if loc (format " (%s)" loc) "")))) + r)) + +(defun flan-macroexpand--insert-code (text) + "Insert TEXT as code and let `flan-mode' decide its columns. +`Form.pretty' puts the line breaks in — that is structure, and a printer has +to choose it — and stops there. Where the columns go is this project's +indentation, which lives in `flan-mode' and not in the compiler." + (let ((start (point)) + ;; `indent-region' reports its progress, which is noise in the echo + ;; area over a form and a hundred lines of it in a batch run. + (inhibit-message t)) + (insert text "\n") + (indent-region start (point)))) + +(defun flan-macroexpand--render (reply code file all) + "Draw REPLY, the expansion of CODE from FILE, into the macroexpansion buffer." + (with-current-buffer (get-buffer-create flan-macroexpansion-buffer) + (let ((inhibit-read-only t)) + (erase-buffer) + (flan-macroexpansion-mode) + (setq flan-macroexpand--origin (list :code code :file file :all all)) + (let ((start (point))) + (insert (format "; macroexpansion, %s\n" + (if all "all the way" "one step"))) + (put-text-property start (point) 'face 'font-lock-comment-face)) + (flan-disassemble--header "of" (string-trim code)) + (flan-disassemble--header + "macro" (or (plist-get reply :macro) + "none — the head of this form is not a macro")) + ;; What `Loc.from_macro' means for what is printed, said here because + ;; there is nowhere else it could be said. Every node below carries the + ;; *call site's* file, line and column with the macro's name stamped on + ;; it, so an error in expanded code points at the call you wrote. None of + ;; it has a location of its own, nothing in it is the text of any file, + ;; and there is nothing here for `M-.' to jump to. + (flan-disassemble--header + "locations" + (format "every node below carries %s's own %s, tagged with the macro \ +that produced it; the text has no locations of its own and is not in any file" + (file-name-nondirectory file) + (if (plist-get reply :macro) "line and column" "position"))) + (when (plist-get reply :note) + (flan-disassemble--header "note" (plist-get reply :note))) + (insert "\n") + (flan-macroexpand--insert-code (plist-get reply :text)) + ;; Point on the code rather than on the header. Found by the blank line + ;; that separates them and not by counting: a header line is filled, so + ;; a long one is two lines and a count would drift onto it. + (goto-char (point-min)) + (if (re-search-forward "^$" nil t) (forward-line 1) (goto-char (point-min))))) + (display-buffer flan-macroexpansion-buffer)) + +;;;###autoload +(defun flan-macroexpand (&optional all) + "Show what the macro call before point expands to. + +One step, which is the one that can name the macro that ran. With a prefix +argument, or non-nil ALL, expand to the fixpoint instead — a macro may +quasiquote a call to another macro, so the two genuinely differ. + +The form is expanded against the macros this *session* holds: the prelude's, +the ones its imports brought in, and every `defmacro' evaluated since it +started. Not a fresh read of the file, which would answer with what is saved +rather than with what is typed." + (interactive "P") + (let* ((b (flan-macroexpand--bounds)) + (file (or buffer-file-name "")) + ;; Padded onto its own line *and column*, unlike `C-x C-e', because + ;; the refusals this path can get name a location inside the snippet + ;; and a macro call is written well inside a line. + (code (flan-dev--text-at (car b) (cdr b))) + (r (flan-macroexpand--ask code file all))) + (flan-macroexpand--render r (buffer-substring-no-properties (car b) (cdr b)) + file all) + (pulse-momentary-highlight-region (car b) (cdr b)) + (unless (plist-get r :expanded) + (message "flan: %s" (or (plist-get r :note) "nothing expanded"))) + r)) + +;;;###autoload +(defun flan-macroexpand-all () + "Expand the macro call before point to its fixpoint. +`flan-macroexpand' with a prefix argument does the same thing; this exists so +the other half is findable by name and not only by a modifier." + (interactive) + (flan-macroexpand t)) + +(defun flan-macroexpand-again (&optional all) + "Expand the form at point one more step, in place. + +The buffer holds code with no file behind it, so the expansion replaces the +text it came from rather than opening anything. With a prefix argument, or +non-nil ALL, take it all the way instead." + (interactive "P") + (unless flan-macroexpand--origin + (user-error "flan: this is not a macroexpansion buffer")) + (let* ((b (flan-macroexpand--bounds)) + (code (buffer-substring-no-properties (car b) (cdr b))) + ;; Unpadded, and that is the honest shape here: this text is in no + ;; file, so there is no line or column for a refusal to be drawn at. + ;; The file still goes on the wire — it is what tells the daemon which + ;; session's macros to expand against. + (r (flan-macroexpand--ask code (plist-get flan-macroexpand--origin :file) + all))) + (if (not (plist-get r :expanded)) + (message "flan: %s" (or (plist-get r :note) "nothing expanded")) + (let ((inhibit-read-only t) + (start (car b))) + (delete-region (car b) (cdr b)) + (goto-char start) + ;; No trailing newline from `--insert-code': this is a form inside a + ;; line, not a section. + (let ((from (point))) + (insert (plist-get r :text)) + (indent-region from (point)) + (pulse-momentary-highlight-region from (point))) + (goto-char start)) + (message "flan: %s" (or (plist-get r :macro) "expanded"))))) + +(defun flan-macroexpand-all-again () + "Expand the form at point to its fixpoint, in place." + (interactive) + (flan-macroexpand-again t)) + +(defun flan-macroexpand-refresh () + "Ask again for the expansion this buffer is showing. +The `defmacro' behind it may have been re-evaluated since, and an expansion +drawn from the old body is exactly the staleness this whole feature exists to +remove." + (interactive) + (unless flan-macroexpand--origin + (user-error "flan: this is not a macroexpansion buffer")) + (let* ((o flan-macroexpand--origin) + (r (flan-macroexpand--ask (plist-get o :code) (plist-get o :file) + (plist-get o :all)))) + (flan-macroexpand--render r (plist-get o :code) (plist-get o :file) + (plist-get o :all)))) + +(defun flan-macroexpand--not-source () + "Refuse to evaluate the contents of the macroexpansion buffer." + (interactive) + (user-error + "flan: this is an expansion, not source — it is in no file, and installing \ +it would redefine a name with code nobody wrote")) + +(defvar flan-macroexpansion-mode-map + (let ((map (make-sparse-keymap))) + ;; The three that would send this buffer's text to the daemon as though it + ;; were a file. Inherited from `flan-mode-map' and shadowed by name rather + ;; than by unbinding, so pressing one says why instead of doing nothing. + (define-key map (kbd "C-c C-c") #'flan-macroexpand--not-source) + (define-key map (kbd "C-c C-k") #'flan-macroexpand--not-source) + (define-key map (kbd "C-x C-e") #'flan-macroexpand--not-source) + (define-key map (kbd "C-c C-m") #'flan-macroexpand-again) + ;; One-key aliases, as the break buffer has: this is a buffer you read with + ;; one hand. + (define-key map (kbd "m") #'flan-macroexpand-again) + (define-key map (kbd "a") #'flan-macroexpand-all-again) + (define-key map (kbd "g") #'flan-macroexpand-refresh) + (define-key map (kbd "q") #'quit-window) + map) + "Keymap for `flan-macroexpansion-mode'.") + +(define-derived-mode flan-macroexpansion-mode flan-mode "Flan-Macro" + "Mode for the buffer `flan-macroexpand' writes. + +Derived from `flan-mode' because what is in it is Flan source and reading it +is the whole point: the indentation and the font-lock are the ones this +project already has. Read-only, and the three keys that would send its text +back to the daemon say why they will not. + +\\{flan-macroexpansion-mode-map}" + (setq buffer-read-only t)) + ;;; What the program is made of, and what it is still holding ;; Two readings of one table. A dev build records the type at every diff --git a/emacs/flan-mode.el b/emacs/flan-mode.el index f954c70..747aa2a 100644 --- a/emacs/flan-mode.el +++ b/emacs/flan-mode.el @@ -73,6 +73,11 @@ (autoload 'flan-dev-restart-program "flan-dev" nil t) ;; Bound below, like the rest, and it was the one missing an autoload. (autoload 'flan-disassemble "flan-dev" nil t) +;; C-c C-m and the half of it that is findable by name rather than by a +;; modifier. Real autoloads for the reason stated above: a `declare-function' +;; would leave M-x with nothing to load. +(autoload 'flan-macroexpand "flan-dev" nil t) +(autoload 'flan-macroexpand-all "flan-dev" nil t) (defgroup flan nil "Editing and evaluating Flan." @@ -190,6 +195,12 @@ line is off screen." ;; IR it was built from. C-u for the IR rather than a second key: it is ;; the same question asked of the same body. (define-key map (kbd "C-c C-a") #'flan-disassemble) + ;; What the macro call before point expands to. CIDER's key, and `C-u' for + ;; the fixpoint rather than a second one — the same question asked of the + ;; same form, which is the rule `C-c C-a' above already follows. One step + ;; is the bare key because it is the one that can name the macro that ran: + ;; a full expansion is stamped with the outermost name only. + (define-key map (kbd "C-c C-m") #'flan-macroexpand) ;; The way out when a reload is refused: rebuild, relaunch, reconnect. (define-key map (kbd "C-c C-x") #'flan-dev-restart-program) map) diff --git a/emacs/test-flan-cider.el b/emacs/test-flan-cider.el index 7e487d2..767ef19 100644 --- a/emacs/test-flan-cider.el +++ b/emacs/test-flan-cider.el @@ -1115,6 +1115,84 @@ stopped program, which is the case where it should fire." (string-match-p "unhandled" text))) +;;; The macroexpansion buffer + +;; The same kind of thing again: a reply in, a buffer out. What the live test +;; in test-flan-dev.el cannot easily reach is the shape where *nothing* +;; expanded — a head that is not a macro, and a macro whose answer is its own +;; call — because both have to be arranged in a program. Here they are a +;; plist. + +(message "\nthe macroexpansion buffer") + +(let ((flan-macroexpand-request-function + (lambda (_) '(:status "ok" :text "(if (not false)\n (do 1 2))" + :flat "(if (not false) (do 1 2))" + :expanded t :macro "unless")))) + (with-temp-buffer + (flan-mode) + (insert "(unless false 1 2)") + (goto-char (point-min)) + (flan-macroexpand)) + (let ((text (with-current-buffer flan-macroexpansion-buffer (buffer-string)))) + (test-flan--check "the expansion is drawn" + (string-match-p (regexp-quote "(if (not false)") text)) + (test-flan--check "with the macro that produced it named" + (string-match-p "macro +unless" text)) + (test-flan--check "and the form it came from" + (string-match-p (regexp-quote "(unless false 1 2)") text)) + ;; The one thing a reader of this buffer has to be told, because there is + ;; nowhere else it could be said: nothing in it has a location of its own. + (test-flan--check "and what the locations in it mean" + (string-match-p "no locations of its own" text)) + (test-flan--check "one step says so, rather than implying a fixpoint" + (string-match-p "one step" text)))) + +;; A head that is not a macro. The form comes back as itself, and the buffer +;; has to say which kind of nothing that is rather than looking like a very +;; short expansion. +(let ((flan-macroexpand-request-function + (lambda (_) '(:status "ok" :text "(+ 1 2)" :flat "(+ 1 2)" + :expanded nil + :note "the head of this form is not a macro")))) + (with-temp-buffer + (flan-mode) + (insert "(+ 1 2)") + (goto-char (point-min)) + (flan-macroexpand)) + (let ((text (with-current-buffer flan-macroexpansion-buffer (buffer-string)))) + (test-flan--check "a head that is not a macro says so" + (string-match-p "not a macro" text)) + (test-flan--check "and the macro line says none rather than being absent" + (string-match-p "macro +none" text)))) + +;; The refusals reach here as an ordinary error reply, and the command has to +;; signal rather than draw an empty buffer: a macro that never settles is the +;; one that gets here. +(let ((flan-macroexpand-request-function + (lambda (_) '(:status "error" + :message "expanding s/spin did not settle after 200 rounds" + :loc "buf.flan:3:3")))) + (with-temp-buffer + (flan-mode) + (insert "(s/spin)") + (goto-char (point-min)) + (test-flan--check + "a refused expansion signals, carrying the daemon's words" + (let ((said (test-flan--caught #'flan-macroexpand))) + (and said (string-match-p "did not settle" said)))))) + +;; And the buffer refuses to be treated as source, which is the one way it +;; could do damage: the text in it is in no file and installing it would +;; redefine a name with code nobody wrote. +(with-current-buffer (get-buffer-create flan-macroexpansion-buffer) + (test-flan--check + "the expansion buffer refuses C-c C-c, C-c C-k and C-x C-e" + (seq-every-p + (lambda (k) + (eq (key-binding (kbd k)) #'flan-macroexpand--not-source)) + '("C-c C-c" "C-c C-k" "C-x C-e")))) + ;; `flan-mode' itself — indentation, which is a function from text to text and ;; so belongs with the other fixture-driven checks rather than with anything ;; that needs a daemon. Loaded rather than run separately because diff --git a/emacs/test-flan-dev.el b/emacs/test-flan-dev.el index 5ee03f5..684fdf8 100644 --- a/emacs/test-flan-dev.el +++ b/emacs/test-flan-dev.el @@ -947,6 +947,142 @@ is written instead — the real `message' call the real command makes." (flan-dev-quit) (ignore-errors (delete-file socket4))) + ;; ── C-c C-m: what a macro call expands to ───────────────────────────── + ;; + ;; test_session.ml drives the expansion itself and is where the language + ;; cases live. What this adds is the half that is only true in an editor: + ;; which region the command picks, that the expansion arrives as readable + ;; source in a buffer, that expanding again in place works, and — the one + ;; worth a real daemon — that a macro which never settles comes back as a + ;; refusal drawn on the call, at its own line *and its own column*. + ;; + ;; That last one is what `flan-dev--text-at' exists for. `C-x C-e' sends a + ;; raw substring and `flan-dev--text' pads lines only, because a top-level + ;; form starts at column 1; a macro call is written well inside a line, and + ;; a refusal against it would otherwise be drawn at the start of that line. + (let ((socket5 (concat socket "-macro"))) + (ignore-errors (delete-file socket5)) + (flan-dev program socket5) + (test-flan--check "a daemon to expand macros against" + (process-live-p flan-dev--connection)) + (with-current-buffer (get-file-buffer file) + (goto-char (point-max)) + (insert "\n(defn expandable [] i32\n (unless false 1 2)\n (clamp 9 0 3))\n") + ;; One step, on a prelude macro. Point on the opening paren, which is + ;; where a reader puts it and which `C-x C-e' could not use. + (goto-char (point-max)) + (search-backward "(unless false 1 2)") + (flan-macroexpand) + (test-flan--check + "C-c C-m expands the form point is on" + (with-current-buffer flan-macroexpansion-buffer + (string-match-p "(if (not false)" (buffer-string)))) + (test-flan--check + "and says which macro ran" + (with-current-buffer flan-macroexpansion-buffer + (string-match-p "macro +unless" (buffer-string)))) + (test-flan--check + "and the buffer is Flan source, not a dump" + (with-current-buffer flan-macroexpansion-buffer + (and (derived-mode-p 'flan-mode) buffer-read-only))) + ;; The three keys that would send an expansion back as though it were a + ;; file refuse by name rather than doing nothing. + (test-flan--check + "C-c C-c in the expansion buffer refuses, and says why" + (with-current-buffer flan-macroexpansion-buffer + (condition-case e (progn (call-interactively + (key-binding (kbd "C-c C-c"))) + nil) + (user-error (string-match-p "not source" (error-message-string e)))))) + ;; All the way, off the same key with a prefix. + (goto-char (point-max)) + (search-backward "(clamp 9 0 3)") + (flan-macroexpand t) + (test-flan--check + "C-u C-c C-m expands all the way" + (with-current-buffer flan-macroexpansion-buffer + (and (string-match-p "all the way" (buffer-string)) + (string-match-p (regexp-quote "(min 3 (max 0 9))") + (buffer-string))))) + ;; A macro that never settles, evaluated into the session and then asked + ;; about. One step answers — it makes one call and does not look at what + ;; comes back — and all the way is refused at the bound rather than + ;; hanging the daemon, which is the failure that would wedge the editor + ;; with the program still on screen. + (flan-dev--request + (list :op "eval" :file file + :code "(defmacro spinner [args] `(spinner ~@args))")) + (goto-char (point-max)) + (insert "\n(defn spun [] i32\n (spinner 1))\n") + (goto-char (point-max)) + (search-backward "(spinner 1)") + (let ((call (point)) + (r (flan-macroexpand))) + (test-flan--check + "one step of a macro that does not settle answers" + (and (null (plist-get r :expanded)) + (equal (plist-get r :macro) "spinner"))) + (flan-dev-clear-errors) + (goto-char call) + (let ((said (condition-case e (progn (flan-macroexpand t) nil) + (user-error (error-message-string e))))) + (test-flan--check + "and all the way is refused rather than hanging" + (and said (string-match-p "did not settle" said))) + ;; The whole of `flan-dev--text-at': the refusal's location is the + ;; call's own line and column, so the overlay lands on the call and + ;; not at the start of the line it is written on. + (let ((ovs (flan-dev--error-overlays))) + (test-flan--check + "and the refusal is drawn on the call, at its own column" + (and (= 1 (length ovs)) + (= (overlay-start (car ovs)) call))))) + (flan-dev-clear-errors)) + + ;; Expanding again in place, which is what makes one-step-by-default + ;; usable rather than a thing you press once and lose. Two macros into + ;; the session, the outer one quasiquoting a call to the inner: that is + ;; the shape where one step leaves a macro call standing, and it is the + ;; only shape where expanding in place has anything to do. + (flan-dev--request + (list :op "eval" :file file + :code "(defmacro m-inner [args] `(+ ~(at args 0) 1))")) + (flan-dev--request + (list :op "eval" :file file + :code "(defmacro m-outer [args] `(m-inner ~(at args 0)))")) + (goto-char (point-max)) + (insert "\n(defn outered [] i32 (m-outer 5))\n") + (goto-char (point-max)) + (search-backward "(m-outer 5)") + (flan-macroexpand) + (test-flan--check + "one step leaves the call to the macro it quasiquoted" + (with-current-buffer flan-macroexpansion-buffer + (string-match-p (regexp-quote "(m-inner 5)") (buffer-string)))) + (with-current-buffer flan-macroexpansion-buffer + (goto-char (point-min)) + (search-forward "(m-inner 5)") + (goto-char (match-beginning 0)) + (flan-macroexpand-again) + (test-flan--check + "m in the expansion buffer expands the form at point in place" + (and (string-match-p (regexp-quote "(+ 5 1)") (buffer-string)) + (not (string-match-p (regexp-quote "(m-inner 5)") + (buffer-string)))))) + ;; And the same call taken all the way in one go, which is the half the + ;; prefix argument is for: no intermediate at all. + (goto-char (point-max)) + (search-backward "(m-outer 5)") + (flan-macroexpand t) + (test-flan--check + "C-u goes straight to the fixpoint" + (with-current-buffer flan-macroexpansion-buffer + (and (string-match-p (regexp-quote "(+ 5 1)") (buffer-string)) + (not (string-match-p (regexp-quote "(m-inner 5)") + (buffer-string))))))) + (flan-dev-quit) + (ignore-errors (delete-file socket5))) + (if (zerop test-flan--failures) (message "flan-dev.el: all tests passed") (message "\n%d failure(s)" test-flan--failures) diff --git a/lib/dev.ml b/lib/dev.ml index 82ac227..1f76977 100644 --- a/lib/dev.ml +++ b/lib/dev.ml @@ -546,6 +546,52 @@ let eval_expr t ~code ~origin ~pause = | exception Failure m -> error m) | exception Loc.Error { Loc.dloc = l; dmsg = msg; _ } -> error ~loc:(Loc.to_string l) msg +(* What a macro call expands to — [C-c C-m], and the one verb here that never + touches the program. + + Deliberately not gated on [alive t]. Every other verb in this file is a + question about a running process and says so when there is not one; + expansion is a question about the *compiler*, answered out of the macros the + session holds, and it is still answerable after the program has exited. That + is worth having rather than tidying away: the moment you most want to know + what a macro produced is often just after the code it produced crashed. + + No [Loc.Error] arm either, and that is not an omission. The guard round + [handle] in [serve] catches everything non-fatal and answers it with + [reply_of_exn] — which is where the two non-termination refusals land, with + the call site's location on them, rather than in a hang that would leave the + editor waiting on a daemon with the program still on screen. *) +let macroexpand t ~code ~origin ~all = + let x = Session.macroexpand ~origin ~all t.session code in + ok + ([ (* Line breaks in, columns left to the editor: see [Form.pretty]. *) + ":text " ^ Wire.quote (Form.pretty x.Session.xafter); + (* And the same thing on one line, for a client with no indenter and for + the echo area. *) + ":flat " ^ Wire.quote (Form.to_source x.Session.xafter); + ":source " ^ Wire.quote (Form.to_source x.Session.xbefore); + (if all then ":all t" else ":all nil"); + (if x.Session.xchanged then ":expanded t" else ":expanded nil") ] + @ (match x.Session.xmacro with + | Some n -> [ ":macro " ^ Wire.quote n ] + | None -> []) + @ + (* The two ways of coming back unchanged are different facts and the + editor should not have to guess which it has. Only the one where a + macro *did* run is a statement about that macro. *) + match (x.Session.xchanged, x.Session.xmacro) with + | true, _ -> [] + | false, None -> + [ ":note " + ^ Wire.quote + "the head of this form is not a macro this session holds — the \ + prelude's, an import's, and every defmacro evaluated since it \ + started are what it can expand" ] + | false, Some n -> + [ ":note " + ^ Wire.quote + (n ^ " expanded to the call it was given, unchanged") ]) + let describe t = ok [ ":fns " @@ -2102,6 +2148,25 @@ let handle t req = in eval_expr t ~code ~origin ~pause | None -> error "eval-expr needs :code") + (* [:all], absent or [nil] being false and anything else true — the spelling + [:pause], [:on] and [:reset] already use. One step is the default because + it is the one that can name the macro that ran: a full expansion of a + macro that quasiquotes a call to another is stamped with the outermost + name only, [Loc.from_macro] being outermost-wins, so the intermediate is + unnameable by the time it settles. *) + | Some "macroexpand" -> + (match Wire.string_field req "code" with + | Some code -> + let origin = + match Wire.string_field req "file" with Some f -> f | None -> "" + in + let all = + match Wire.field req "all" with + | Some { Form.v = Form.Sym "nil"; _ } | None -> false + | Some _ -> true + in + macroexpand t ~code ~origin ~all + | None -> error "macroexpand needs :code") | Some "describe" -> describe t | Some "defs" -> defs t | Some "break" -> break t diff --git a/lib/form.ml b/lib/form.ml index 8ee4239..121ee07 100644 --- a/lib/form.ml +++ b/lib/form.ml @@ -39,3 +39,148 @@ let rec to_string f = | List l -> "(" ^ seq l ^ ")" | Vec l -> "[" ^ seq l ^ "]" | Map l -> "{" ^ seq l ^ "}" + +(* ── Printing a Form back as source ───────────────────────────────── + + [to_string] above is an error-message renderer: one line, no width, and + [%S] and [%g] where a reader's own spelling was never needed to say + "expected a name, found this". It is also what [Macro.key] digests, so it + is left exactly as it is — every cached macro module on disk is keyed by + what it prints today. + + What follows is the other job, and it arrived with [C-c C-m]: text a person + reads and a reader reads back. An expansion is *only* text. A macro answers + a [Form] and nothing in the language ever wrote it down, so unlike every + other thing the editor shows there is no file to point at and no source to + fall back on — whatever this prints is the whole of what anybody sees. + + Three places where [to_string] is not a round trip, all of them reachable + from an expansion because a macro may build any literal at all: + + - [%g] prints 1.0 as "1", which reads back as an [Int], and it truncates at + six significant digits. Shortest-round-trip here, then a ".0" when + nothing in the text says "float". + - [%S] is OCaml's escaping. The reader takes exactly six escapes — newline, + tab, return, backslash, quote and nul — and every other byte literally, + so the three-digit decimal escape [%S] writes would not read back. + - [Byte] falls through to \, which spells 0 and 13 as a NUL and a + carriage return sitting in the middle of the source. The reader has names + for those and this uses them. *) + +let escape s = + let b = Buffer.create (String.length s + 2) in + String.iter + (fun c -> + match c with + | '\n' -> Buffer.add_string b "\\n" + | '\t' -> Buffer.add_string b "\\t" + | '\r' -> Buffer.add_string b "\\r" + | '\\' -> Buffer.add_string b "\\\\" + | '"' -> Buffer.add_string b "\\\"" + | '\000' -> Buffer.add_string b "\\0" + | c -> Buffer.add_char b c) + s; + Buffer.contents b + +let float_repr x = + (* The shortest of the three precisions that survives [float_of_string] is + the one the reader parses back to the same bits. 15 covers almost every + literal anyone writes; 17 covers every double there is. *) + let rec shortest = function + | [] -> Printf.sprintf "%.17g" x + | p :: ps -> + let s = Printf.sprintf "%.*g" p x in + if float_of_string s = x then s else shortest ps + in + let s = shortest [ 15; 16; 17 ] in + (* "1" is an integer to the reader, so a float whose text has no point and + no exponent needs one. Guarded on the text and not on the value: nan and + the infinities print as words, and "nan.0" is no improvement on a literal + no reader accepts either way. *) + let plain = + s <> "" + && String.for_all (fun c -> (c >= '0' && c <= '9') || c = '-' || c = '+') s + in + if plain then s ^ ".0" else s + +let byte_repr b = + match b with + | 32 -> "\\space" + | 9 -> "\\tab" + | 10 -> "\\newline" + | 13 -> "\\return" + | 0 -> "\\nul" + (* Printable ASCII is written as itself. Anything else has no spelling in + the reader at all — [read_byte] takes a name or a single character — so + it is written as the decimal the reader would have to grow, rather than + as a byte that would corrupt the line it is on. *) + | b when b > 32 && b < 127 -> Printf.sprintf "\\%c" (Char.chr b) + | b -> Printf.sprintf "\\%d" b + +(** One line, and a reader reads it back. *) +let rec to_source f = + let seq l = String.concat " " (List.map to_source l) in + match f.v with + | Sym s -> s + | Kw s -> ":" ^ s + | Int i -> Int64.to_string i + | Float x -> float_repr x + | Str s -> "\"" ^ escape s ^ "\"" + | Byte b -> byte_repr b + | List l -> "(" ^ seq l ^ ")" + | Vec l -> "[" ^ seq l ^ "]" + | Map l -> "{" ^ seq l ^ "}" + +(** The same text with line breaks in it, for a form too wide to read on one. + + Where the breaks go, and deliberately not where the columns do. A list + that fits is written flat; one that does not keeps its head on the opening + line and puts each remaining element on its own, two columns in. That is + the structural half, which a printer has to decide. The indentation is the + editor's: [flan-mode] re-indents what it is shown, and that is where this + project's indentation rules already live, so nothing here tries to know + that a [let] aligns its bindings under the bracket. A client with no Emacs + still gets something readable rather than one very long line. *) +let pretty ?(width = 72) (f : t) : string = + let b = Buffer.create 256 in + let rec go col f = + let flat = to_source f in + if col + String.length flat <= width then Buffer.add_string b flat + else + let elements ind rest = + List.iter + (fun e -> + Buffer.add_char b '\n'; + Buffer.add_string b (String.make ind ' '); + go ind e) + rest + in + match f.v with + (* A call or a special form: the head names what this is, so it stays on + the opening line whatever the rest of it costs. *) + | List (({ v = Sym _; _ } as h) :: rest) -> + Buffer.add_char b '('; + Buffer.add_string b (to_source h); + elements (col + 2) rest; + Buffer.add_char b ')' + | List (x :: rest) -> + Buffer.add_char b '('; + go (col + 1) x; + elements (col + 1) rest; + Buffer.add_char b ')' + | Vec (x :: rest) -> + Buffer.add_char b '['; + go (col + 1) x; + elements (col + 1) rest; + Buffer.add_char b ']' + | Map (x :: rest) -> + Buffer.add_char b '{'; + go (col + 1) x; + elements (col + 1) rest; + Buffer.add_char b '}' + (* An empty bracket, or a single atom longer than the width. Neither has + a break in it to take. *) + | _ -> Buffer.add_string b flat + in + go 0 f; + Buffer.contents b diff --git a/lib/macro.ml b/lib/macro.ml index ed4b43e..4ffb065 100644 --- a/lib/macro.ml +++ b/lib/macro.ml @@ -291,8 +291,17 @@ let rounds ~(prelude : string list) (pending : (string * Form.t) list) a macro would otherwise re-read the whole of it on every parse. *) let prelude_macros = lazy (macros_in (Prelude.forms ())) -let program (forms : Form.t list) : Form.t list = - if !building then forms +(* The module the forms below will be expanded against, or [None] when there is + nothing to expand them with. + + Split out of [program] rather than copied into the editor's path, because + everything in it is a decision with a paragraph attached — which names are + ambient, which shadow which, and the shortcut that keeps a macro-free build + free of a clang driver. Two copies of that would drift, and the second copy + is the one a reader would not know to distrust. [program] below is the whole + of what used to be here; [expand_step] and [expand_all] are the editor's. *) +let loaded_for (forms : Form.t list) : loaded option = + if !building then None else let prelude = Lazy.force prelude_macros in (* The prelude's own macros are dropped from [mine], and the reason is that @@ -340,14 +349,84 @@ let program (forms : Form.t list) : Form.t list = macro pays nothing: a file that calls none costs one scan and no compiler. Without it every build in the suite would link a macro module for the prelude's macros and pay a clang driver to answer nothing. *) - if all = [] || not (List.exists (names_macro all) forms) then forms + if all = [] || not (List.exists (names_macro all) forms) then None else begin let extra = rounds ~prelude mine in - let l = compile all (List.map snd extra) in - let out = List.map (expand_form l) forms in - Dynload.dl_close l.handle; - Dynload.release (); - out + Some (compile all (List.map snd extra)) end +(* Closing the handle and freeing the marshaller's scratch, on the way out of + an expansion however it ends. + + [Fun.protect] and not two statements after the call, which is what this was + while a build was the only caller: a build that raised was a compile that + failed and a process about to exit, so a leaked handle and a few unfreed + argument buffers cost nothing anybody could measure. The daemon is not that + process. A macro that does not settle raises out of here, the editor is told + and stays connected, and the next [C-c C-m] does it again — so the one thing + [Dynload.owned] must not become is a list that only ever grows over a + session's lifetime. *) +let with_module (l : loaded) (f : unit -> 'a) : 'a = + Fun.protect + ~finally:(fun () -> + Dynload.dl_close l.handle; + Dynload.release ()) + f + +let program (forms : Form.t list) : Form.t list = + match loaded_for forms with + | None -> forms + | Some l -> with_module l (fun () -> List.map (expand_form l) forms) + let () = Parse.expander := program + +(* ── What the editor asks ────────────────────────────────────────── + [C-c C-m]. Two questions and not one, because a macro that quasiquotes a + call to another macro makes the difference real: [mac/quad] answers + [(mac/twice (mac/twice n))], and the fixpoint of that says nothing about + which macro produced what. + + Both answer the name at the head when it is a macro, so the editor can say + *which* macro it just ran rather than only that something changed. That name + is the one thing the printed text cannot carry: [Loc.from_macro] is + outermost-wins, so every node of a full expansion is stamped with the macro + the author wrote and the intermediate names are gone by the time it settles. + + One step is outermost-only, and that is a deliberate difference from + [expand_form], which expands a call's arguments *before* calling it. So + [(mac/twice (mac/twice 1))] one-stepped here is [(+ (mac/twice 1) + (mac/twice 1))], where the compiler's own first move is [(mac/twice (+ 1 + 1))]. Different intermediates, the same fixpoint. One step is a view of what + this macro did; all the way is the answer the compiler acts on. *) + +let head_macro (l : loaded) (f : Form.t) : string option = + match f.Form.v with + | Form.List ({ Form.v = Form.Sym n; _ } :: _) when List.mem_assoc n l.fns -> + Some n + | _ -> None + +(** One round of the outermost call, or the form unchanged when its head does + not name a macro. Nothing here needs the fuel [settle] carries: one call is + one call, and what it answers is not looked at again. *) +let expand_step (f : Form.t) : Form.t * string option = + match loaded_for [ f ] with + | None -> (f, None) + | Some l -> + with_module l (fun () -> + match f.Form.v with + | Form.List ({ Form.v = Form.Sym n; _ } :: args) + when List.mem_assoc n l.fns -> + ( Expand.call ~loc:(Loc.from_macro n f.Form.loc) (List.assoc n l.fns) + args, + Some n ) + | _ -> (f, None)) + +(** To the fixpoint, through exactly the walk a build goes through — so the + text this answers is the text the checker is about to be handed, and the + refusals are the build's own. A macro that does not settle raises + [Loc.Error] out of [settle] at the bound, and a ring was refused a level up + in [rounds] before anything was compiled at all. *) +let expand_all (f : Form.t) : Form.t * string option = + match loaded_for [ f ] with + | None -> (f, None) + | Some l -> with_module l (fun () -> (expand_form l f, head_macro l f)) diff --git a/lib/session.ml b/lib/session.ml index 2516e17..8026181 100644 --- a/lib/session.ml +++ b/lib/session.ml @@ -1248,3 +1248,72 @@ let eval_expr ?(origin = "") ?(pause = false) t src : change = lost track of what the program contains is worse than one that died. *) t.program <- { t.program with Tast.fns = t.program.Tast.fns @ fresh }; { ir; names = []; fns = []; installs = true } + +(* ── What a macro call expands to ──────────────────────────────────── *) + +(* [C-c C-m]. The one thing the editor can ask that compiles nothing, sends + nothing to the program, and leaves the session exactly as it found it. + + It is a session operation rather than a [Reader] plus a [Macro] call for the + reason the whole of this file exists: what a call expands to is decided by + *which macros this session holds* — the prelude's, the ones its imports + brought in, and the [defmacro]s the buffer has evaluated since it started — + and that set lives in [t.macros] and nowhere else. Expanding against a fresh + read of the file would answer with macros the session was never told about + and with a version of the ones it was told about that is whatever happens to + be *saved*. An expansion that disagreed with what an evaluation does would + be worse than no expansion at all: a macro decides what the code is. + + Nothing here is assigned. [eval] commits [t.macros] and [eval_expr] bumps + [t.thunks] and [t.program]; this reads and writes nothing, so a form that + only *looks* like a declaration — a [defmacro] handed to [C-c C-m] — does + not join the session by having been looked at. [test_session] pins that. *) + +type expansion = { + xbefore : Form.t; (* what was sent, quasiquote already desugared *) + xafter : Form.t; (* what it expands to *) + xmacro : string option; (* the macro at the head, when the head is one *) + (* False when the expansion is the form itself. Reported rather than left for + the editor to diff, because the two ways of being unchanged are different + facts: a head that names no macro, and a macro whose answer is its own + call. Only [xmacro] tells them apart. *) + xchanged : bool; +} + +(* [all] is the difference between CIDER's two commands, and it is real here + rather than inherited: a macro may quasiquote a call to another macro, so + [(mac/quad 3)] one-stepped is [(mac/twice (mac/twice 3))] and all the way is + [(+ (+ 3 3) (+ 3 3))]. + + The refusals both live on the [all] path and both are [Loc.Error], which the + daemon answers as a reply: a macro that never settles is stopped by + [Macro.fuel] and named, and a ring of macros was refused while the package + holding it was parsed, which is before any session over it could exist. One + step cannot reach either — it makes exactly one call and does not look at + the answer — so [(s/spin)] one-stepped is a fact about the macro and only + [C-u] refuses. *) +let macroexpand ?(origin = "") ~(all : bool) t (src : string) : expansion = + let form = + match Reader.read_all ~file:origin src with + | [ f ] -> f + | [] -> fail Loc.unknown "nothing to expand" + | _ :: f :: _ -> fail f.Form.loc "one form at a time" + in + (* Desugared first, exactly as [Parse.parse_forms] does it and for its + reason: a quasiquote that has not been rewritten still looks like a call + to something named [quasiquote], and the [Form.Sym] inside it looks like a + head anything could mistake for one. *) + let before = Expand.quasiquote form in + (* And the session's macros in front of it, as [eval] and [eval_expr] both + put them: [Macro.program] reads [Parse.imported_macros] directly. *) + Parse.with_imported t.macros @@ fun () -> + let after, name = + if all then Macro.expand_all before else Macro.expand_step before + in + { xbefore = before; xafter = after; xmacro = name; + (* Structural, through the printer both halves are shown with, so "changed" + means "would print differently" — which is the claim the buffer makes. + There is no equality on [Form.t] to use instead: it carries a [Loc.t], + and [Loc.from_macro] stamps a name onto every node a macro answers, so + even an identity expansion compares unequal. *) + xchanged = Form.to_source after <> Form.to_source before } diff --git a/test/test_repl.ml b/test/test_repl.ml index 829b503..e5c9a03 100644 --- a/test/test_repl.ml +++ b/test/test_repl.ml @@ -232,6 +232,92 @@ let () = (Option.value ~default:(status r) (field r "message"))); value "a macro defined at the editor" "(thrice 14)" "42"; + (* ── C-c C-m over the socket ──────────────────────────────────── + [test_session] drives the expansion itself and is where the cases + live; this is the same question asked the way a person asks it, and + what it adds is the daemon: the reply's fields, the pretty text + surviving the framing, and — the part worth a socket — that a request + which compiles a macro module and refuses it comes back as a reply + rather than as a daemon that stopped answering. + + The macro set here is [printers.flan]'s, so it covers the prelude's + and the buffer's own; the imported-package half is [test_session]'s, + over [pkg-macro.flan]. *) + let expands ?(all = false) name code want = + let r = + request c + (Printf.sprintf + "(:op \"macroexpand\" :code %s :file \"/tmp/buf.flan\" :all %s)" + (quote code) (if all then "t" else "nil")) + in + match field r "flat" with + | Some v when v = want -> () + | Some v -> fail "%s\n got: %S\n wanted: %S" name v want + | None -> + fail "%s: %s" name (Option.value ~default:(status r) (field r "message")) + in + expands "a prelude macro, expanded" "(clamp 9 0 3)" "(min 3 (max 0 9))"; + expands "the file's own macro, expanded" "(tenfold 7)" "(* 7 10)"; + (* And the one typed at the editor a moment ago, which is the session's + set rather than the file's: nothing on disk declares [thrice]. *) + expands "a macro defined at the editor, expanded" "(thrice 14)" "(* 14 3)"; + (* A form with no macro in it comes back as itself, and says so rather + than echoing and leaving the editor to diff. *) + (let r = + request c + (Printf.sprintf + "(:op \"macroexpand\" :code %s :file \"/tmp/buf.flan\")" + (quote "(+ 1 2)")) + in + (* [:expanded] is a flag and therefore a symbol, not a string — the + spelling [:stopped] and [:overflow] already use — so it is read out + of the form rather than through [Wire.string_field]. *) + let flag k = + match Wire.field r k with + | Some f -> Form.to_source f + | None -> "" + in + if flag "expanded" <> "nil" then + fail "a form that is not a macro call reported :expanded %s" + (flag "expanded"); + if field r "note" = None then + fail "a form that is not a macro call gave no reason"); + (* The name of the macro that ran rides on the reply, because the text + cannot carry it: [Loc.from_macro] is outermost-wins. *) + (let r = + request c + (Printf.sprintf + "(:op \"macroexpand\" :code %s :file \"/tmp/buf.flan\")" + (quote "(tenfold 7)")) + in + if field r "macro" <> Some "tenfold" then + fail "the reply named %s as the macro that ran" + (Option.value ~default:"" (field r "macro"))); + (* :text is the same expansion with the line breaks in it. Nothing here + asserts where they go — that is [Form.pretty]'s and the editor's — only + that the field is there and reads back as the same form. *) + (let r = + request c + (Printf.sprintf + "(:op \"macroexpand\" :code %s :file \"/tmp/buf.flan\")" + (quote "(tenfold 7)")) + in + match field r "text" with + | Some t when t = "(* 7 10)" -> () + | Some t -> fail ":text was %S" t + | None -> fail ":text was missing"); + (* And the session is untouched by having been asked: a [defmacro] handed + to C-c C-m does not join it. C-c C-c is where a declaration goes, and + the round trip below is the only way to check the aftermath. *) + ignore + (request c + (Printf.sprintf + "(:op \"macroexpand\" :code %s :file \"/tmp/buf.flan\")" + (quote "(defmacro looked-at [args] `(* ~(at args 0) 5))"))); + (let r = evals "(looked-at 3)" in + if status r = "ok" then + fail "a defmacro joined the session by being macroexpanded"); + ignore (request c "(:op \"close\")"); Unix.close c end; diff --git a/test/test_session.ml b/test/test_session.ml index 0df37ad..4d03c9c 100644 --- a/test/test_session.ml +++ b/test/test_session.ml @@ -529,6 +529,139 @@ let () = | exception Loc.Error { Loc.dmsg = m; _ } -> fail "a session that refused a macro could not evaluate afterwards: %s" m); + (* ── C-c C-m: what a macro call expands to ───────────────────────── + + The verb that compiles nothing and sends nothing. What it has to get right + is the set it expands against: whatever *this session* holds, which is the + prelude's macros, the ones its imports brought in, and every [defmacro] + the buffer has evaluated since it started. A fresh read of the file would + answer with a different set and with whatever is saved rather than what is + typed, and an expansion that disagreed with an evaluation is the quietest + wrongness there is — a macro decides what the code *is*. + + Asserted on the printed text, because the printed text is the whole of + what a person is shown. There is no IR to read here and nothing to compare + structurally: a [Form] carries a [Loc.t] and [Loc.from_macro] stamps a + name onto every node a macro answers, so even an identity expansion is + structurally unequal to its input. *) + let expands ?(all = false) name src want = + match Session.macroexpand ~origin:"programs/pkg-macro.flan" ~all tm src with + | x -> + let got = Form.to_source x.Session.xafter in + if got <> want then + fail "%s\n got: %S\n wanted: %S" name got want + | exception Loc.Error { Loc.dmsg = m; _ } -> fail "%s: %s" name m + in + (* A prelude macro: ambient, in every session, and the set that worked before + any of this existed. *) + expands "a prelude macro, one step" "(unless false 1 2)" + "(if (not false) (do 1 2))"; + (* An imported package's, arriving qualified. This is most of the argument + for the feature: the definition is in another directory and cannot be read + beside the call site any more. *) + expands "an imported package macro, one step" "(mac/twice 4)" "(+ 4 4)"; + (* The buffer's own, which is the set a session had to start holding before + an editor could ask anything about it. *) + expands "the file's own macro, one step" "(tenfold 7)" "(* 7 10)"; + (* And the one that makes the two commands different rather than one with a + flag: [quad] quasiquotes a call to [twice], so a single step stops with a + macro call still in it and the full expansion does not. Both are true and + they answer different questions. *) + expands "a macro that expands to a call to another macro, one step" + "(mac/quad 3)" "(mac/twice (mac/twice 3))"; + expands ~all:true "a macro that expands to a call to another macro, all the way" + "(mac/quad 3)" "(+ (+ 3 3) (+ 3 3))"; + (* Outermost-only, which is where one step and the compiler's own first move + deliberately differ: [expand_form] expands a call's *arguments* before + calling it, so the compiler's first move here is [(mac/twice (+ 3 3))]. + Same fixpoint, different intermediate, and the intermediate is the whole + of what one step is for. *) + expands "one step does not expand the arguments first" + "(mac/twice (mac/twice 3))" "(+ (mac/twice 3) (mac/twice 3))"; + (* Not a macro call at all. The form comes back as it was, and the answer + that matters is [xmacro]: nothing ran. *) + (match Session.macroexpand ~origin:"programs/pkg-macro.flan" ~all:false tm + "(+ 1 2)" + with + | x -> + if x.Session.xchanged || x.Session.xmacro <> None then + fail "a form whose head is not a macro reported a macro" + | exception Loc.Error { Loc.dmsg = m; _ } -> + fail "expanding a form that is not a macro call: %s" m); + (* And the name of the macro that ran, which is the one thing the text cannot + carry: [Loc.from_macro] is outermost-wins, so a full expansion is stamped + with the macro the author wrote and every intermediate name is gone. *) + (match Session.macroexpand ~origin:"programs/pkg-macro.flan" ~all:false tm + "(mac/quad 3)" + with + | x when x.Session.xmacro = Some "mac/quad" -> () + | x -> + fail "expanding (mac/quad 3) named %s" + (match x.Session.xmacro with None -> "nothing" | Some n -> n) + | exception Loc.Error { Loc.dmsg = m; _ } -> fail "naming the macro: %s" m); + + (* The session is not touched by having been asked. [eval] commits + [t.macros], [eval_expr] bumps [t.thunks] and [t.program]; this writes + nothing, so a [defmacro] handed to C-c C-m must not join the session by + having been looked at. C-c C-c is where a declaration goes. + + The head is not a macro, so nothing expands; the claim is about the + *aftermath*, and it is checked the only way it can be — by calling the + name and requiring it to still be unknown. *) + (match Session.macroexpand ~origin:"programs/pkg-macro.flan" ~all:false tm + "(defmacro looked-at [args] `(* ~(at args 0) 5))" + with + | _ -> () + | exception Loc.Error { Loc.dmsg = m; _ } -> + fail "expanding a defmacro: %s" m); + (match Session.eval_expr ~origin:"programs/pkg-macro.flan" tm "(looked-at 3)" with + | _ -> fail "a defmacro joined the session by being macroexpanded" + | exception Loc.Error _ -> ()); + + (* Non-termination, on this path, in both directions. + + One step makes exactly one call and does not look at what comes back, so + [(s/spin)] one-stepped is a fact about the macro and terminates — the + bound must be where it is needed and nowhere else, or the command would + refuse to show anybody the thing they are trying to see. + + All the way is the path with the fuel on it, and what has to be true is + that the bound is *reached* rather than the daemon hanging: a hang here + wedges the editor with the program still running and no way to say so. + [Loc.Error] out of this call is what [Dev.serve]'s guard answers as a + reply. *) + (match Session.macroexpand ~origin:"programs/pkg-macro-idle.flan" ~all:false ti + "(s/spin)" + with + | x -> + if Form.to_source x.Session.xafter <> "(s/spin)" then + fail "one step of a macro that does not settle answered %S" + (Form.to_source x.Session.xafter) + else if x.Session.xchanged then + fail "one step of a macro that expands to itself reported a change" + else if x.Session.xmacro <> Some "s/spin" then + fail "one step of a macro that does not settle did not name it" + | exception Loc.Error { Loc.dmsg = m; _ } -> + fail "one step of a macro that does not settle was refused: %s" m); + (match Session.macroexpand ~origin:"programs/pkg-macro-idle.flan" ~all:true ti + "(s/spin)" + with + | _ -> fail "a macro that does not settle was expanded all the way" + | exception Loc.Error { Loc.dmsg = m; _ } -> + if not (has m "did not settle") then + fail "expanding a macro that does not settle all the way said %S" m); + (* And the session survives the refusal, as it does after an evaluation that + was refused. *) + (match Session.macroexpand ~origin:"programs/pkg-macro-idle.flan" ~all:true ti + "(mac/twice 21)" + with + | x -> + if Form.to_source x.Session.xafter <> "(+ 21 21)" then + fail "after a refused expansion, (mac/twice 21) expanded to %S" + (Form.to_source x.Session.xafter) + | exception Loc.Error { Loc.dmsg = m; _ } -> + fail "a session that refused an expansion could not expand afterwards: %s" m); + (* A form typed into a file that is *imported as a package* has to be qualified the way the import qualified it, or it splices as a brand-new unrelated name: the evaluation reports success and the running program diff --git a/web/index.html b/web/index.html index 9416cea..a8233c2 100644 --- a/web/index.html +++ b/web/index.html @@ -1679,6 +1679,7 @@ something surprising.

C-c C-ba stopped program: the condition, the restarts, the stack C-c C-M-bthe same restarts, as a one-key prompt C-c C-iinspect a value, navigating into its fields +C-c C-mwhat the macro call at point expands to, one step; C-u first for all the way C-c C-adisassemble a function; C-u first for its LLVM IR C-c C-gdebug under lldb, through dape — bound only once flan-dape.el is loaded, so flan-mode works without dape installed C-c C-dwhat the running program currently defines