An indented surface beside s-expressions is specified as a second reader chosen by file extension

TODO.org

@@ -608,12 +608,12 @@ commitment restarts the process. SBCL's push-through-and-invalidate is cheap onl
 because its instances carry headers. =docs/SBCL-REDEFINITION-NOTES.md= is the
 reading behind it.
 
-** WAIT An F#-ish indentation surface beside s-expressions
+** NEXT An indented surface beside s-expressions
-Deferred 2026-09-19 with no spike queued, blocked on evidence: the author will write
+Decided 2026-09-25 as a test drive: a second reader, chosen by file extension,
-imperative Flan as it stands and see whether the parens still grate. If it is ever
+producing the same forms. =spec-syntax.md= holds the decisions, the proposals
-built, it is one AST with the existing forms unchanged and a second reader in
+awaiting confirmation, and the build order. Rules out Parinfer, wisp and
-front. Rules out Parinfer, wisp and sweet-expressions, and a simplified in-paren
+sweet-expressions, and a simplified in-paren syntax — all thin the parens
-syntax — all thin the parens without removing them.
+without removing them.
 
 ** TODO The shims in sand.flan can go
 =sand.flan= defines =dyn->f64= and =dyn->u32=, one-line functions whose only job

spec-syntax.md

@@ -0,0 +1,367 @@
+# Spec 3 — The indented surface
+
+Status: **draft**. Sections 1 and 3 are decided by the author. Section 2 is
+proposed and needs the author's confirmation before the matching piece is
+built. Nothing here changes what a program means: the indented syntax is a
+second reader that produces the same `Form.t` tree as `lib/reader.ml`.
+
+Evidence for every file:line below was read on 2026-09-25. Re-read before
+building on it.
+
+## 1. Decided
+
+**One tree, two readers.** A new reader turns indented text into `Form.t`,
+exactly the nine node kinds in `lib/form.ml:11-27`, with locations filled the
+way `lib/reader.ml:47` fills them. Everything after the reader (`Expand`,
+`Parse`, `Check`, both backends) is untouched. This is the plan the old WAIT
+entry in `TODO.org` described, and it rules out Parinfer, wisp and
+sweet-expressions the same way.
+
+**Both syntaxes live side by side.** File extension picks the reader. A program
+may mix files of both kinds, and imports cross freely. The paren syntax is not
+deprecated; this is a test drive.
+
+**Blocks are indentation.** A header line (`fn …`, `if …`, `while …`, `else`,
+…) followed by deeper-indented lines opens a block. Headers need no `:`,
+`then`, `do` or `end`. (An ordinary call takes a block only with a trailing
+`:`, section 3.)
+
+**Calls and indexing are adjacency.** `f(a, b)` with no space before `(` is a
+call. `x[i]` with no space before `[` is indexing; `x[i, j]` is `(at x i j)`.
+Arguments are separated by commas.
+
+**Lisp names stay.** `key-pressed?`, `dyn->f64`, `bytes=?`, `swap!`, `*earmuffs*`
+are one token each. The rule that makes this work: **binary operators need
+spaces around them.** `a-b` is a name; `a - b` is subtraction. No
+snake_case mapping.
+
+**Package qualifier stays `/`.** `rl/draw-fps(20, 20)`. Division is always
+spaced, `a / b`.
+
+**Keywords stay.** `:key-r`, `:else`. A colon glued to the front of a word is a
+keyword; `x: T` (colon glued to the end, space after) is an annotation.
+
+**Collection literals.** `[a b]` or `[a, b]` is a vector, `{k v}` a map or struct
+literal, `'(a b c)` a list. `'name` stays the quoted symbol that
+`invoke-restart` takes (60 uses, all restart names).
+
+**Types are marked, and omitted means dyn.** `x: i32` gives a type. A parameter,
+field or global with no `: T` is `dyn`. The reader **writes `dyn` explicitly**
+into the parameter vector: it emits `[x dyn y dyn]`, never `[x y]`, because
+`Check.pair_params` (check.ml:1589) reads `[x y]` as one parameter of type `y`
+when some type is named `y`. That silent misparse is what `parse.ml:1413-1418`
+warns about; the new syntax must not inherit it.
+
+**The return type is inferred when omitted.** Body-local only, as
+`docs/SPIKE-INFERENCE.md` ("The cheap first step" and "Verdict") scopes it:
+- the return type is the body's type; a `dyn` body gives `dyn`; `return`s of
+  different types give `dyn`; no value gives `()`.
+- it reads only the function's own body, never a call site.
+- a self-recursive or mutually recursive function must write its return type.
+  Refuse by name, naming the whole cycle. The corpus has 17 self-recursive
+  functions and 5 mutual groups (all in `calc-me.flan`, `vendor/edn`,
+  `vendor/json`).
+- nothing is ever defaulted in signature position. Parameters are never
+  inferred; an omitted parameter type is `dyn`, which is a rule, not inference.
+
+The paren syntax spells "infer the return" as `_` in the return slot
+(section 3), which is what the indented reader emits when `->` is omitted.
+
+**A changed signature installs, and the warning names the cause.** The dev loop
+already accepts a signature change and stops stale callers with `StaleCall`
+(`lib/session.ml:353-363`, `TODO.org` "Signature generations and stale-caller
+warnings"). When the change came from inference, the stale-caller warning says
+so and points at the line: *"speed now returns f64, not i32, because of line
+12"*.
+
+**Macros stay.** They run on `Form.t`, so they work in both syntaxes. A call
+can take an indented block as its last arguments, so `rl/with-drawing` and
+`comment` take a body (with a trailing `:`, section 3):
+
+```
+rl/with-drawing():
+  rl/clear-background(rl/black)
+  game-draw()
+```
+
+reads as `(rl/with-drawing (rl/clear-background rl/black) (game-draw))`.
+Replacing macros with built-in constructs is **not** part of this work.
+
+**Diagnostics may print paren syntax** during the test drive. `Form.to_string`,
+`Types.to_string`, the usage strings in `parse.ml` and `check.ml`, and
+`Render` all print parens today (inventory in section 5). Fixing that waits on
+the author deciding to switch.
+
+## 2. Proposed (confirm before building the piece it governs)
+
+Each item: the proposal, then the reason in one line.
+
+### Lexical
+
+- **Extension `.fln`.** Short; `.flan` keeps meaning parens, so
+  `generated.flan` and every existing path stay valid.
+- **Comments stay `;`.** Nothing else wants the character.
+- **Spaces only.** A tab in indentation is an error. The corpus has no tabs.
+- **Indentation is measured in columns, any width.** A dedent must land on a
+  column already on the stack (GDScript `gdscript_tokenizer.cpp:1291-1296`).
+- **Blank and comment-only lines never open or close a block** (GDScript
+  1170-1239).
+- **Inside `(` `[` `{`, newlines and indentation are ignored** except where a
+  trailing block is allowed. Make it parser-driven, the way GDScript's
+  `push_multiline` is (`gdscript_parser.cpp` 658-672, 3695-3770), not a paren
+  counter in the lexer, or a block inside a call can't work.
+- **Continuation outside brackets:** a line that starts with a spaced infix
+  operator (`+`, `and`, `==`, …) continues the previous line; so does a line
+  after one that ends in a spaced infix operator. (F# `LexFilter.fs` 360-380,
+  1850-1870, 2345-2360.) No `\` continuation.
+- **Minus.** `-` glued to a digit is a negative literal (`-1`; 269 in the
+  corpus). `-` glued to a name is negation (`-x` becomes `(- x)`; no name starts
+  with `-` except two prelude sentinels, `lib/prelude.ml:2280,2285`, which
+  rename). `a - b` is subtraction. `a -1` is an error: "separate with a comma or
+  space the minus".
+- **`->` needs spaces as the return arrow.** `dyn->f64` stays a name.
+- **Character literals stay `\c`**, lexed before brackets and operators:
+  `\(`, `\,`, `\space`. 277 uses, many of them delimiters of the new syntax.
+
+### Collections and separators
+
+- **Commas separate elements. With no commas, whitespace does, but only
+  between single terms.** `[1 2 3]`, `[i n]`, `{.x 1 .y 2}` and `[4 f32]` read
+  as today. `[a - 1 b]` is refused: "separate elements with commas". This keeps
+  the Lisp look for data and is refusable by shape.
+- **Struct literal:** `Vector2{.x 1, .y 2}` (brace glued to the name) reads
+  `(Vector2 {.x 1 .y 2})`. A bare `{.x 1}` is today's bare literal. `{:a 1}` is a
+  dyn map.
+- **No set literal.** Flan has none today: `#{1 2}` reads as the symbol `#` and
+  a map. Adding sets is a language change, not a syntax one.
+
+### Expressions
+
+- **Precedence**, low to high: `or` < `and` < `not` < comparisons
+  (`== != < <= > >=`) < `<< >>` < `+ -` < `* / %` < unary `-` < postfix (call,
+  index, field).
+- **`==` is `=`; `=` is assignment.** `x = v` reads `(set x v)`, `a[i] = v`
+  reads `(set (at a i) v)`, `p.x = v` reads `(set (.x p) v)`. `x += v` reads
+  `(set x (+ x v))`; like `++` today, the place is evaluated twice.
+- **A run of the same operator flattens** (variadics, section 3):
+  `a + b + c` reads `(+ a b c)`, `a < b < c` reads `(< a b c)` (Flan's chain
+  semantics, `test/programs/chain.flan`). This keeps the converter round trip
+  exact (section 4).
+- **Field access is postfix:** `camera.target.x` reads `(.x (.target camera))`.
+  A capitalised left side is a qualified case, not a field: `Shape.Rect` stays
+  one symbol. `test/programs/dev-rerun.flan:65` names a global
+  `.init-once.counter`; rename it.
+- **`and`, `or`, `not` are words**, since they are Flan's own names.
+- **Casts and type-taking builtins are calls:** `i32(x)`, `vec-new(u8)`,
+  `max-value(u8)`, `the([3 f32], [1 2 3.5])`.
+
+### Statements and blocks
+
+- **`let x = v`** scopes to the end of its block and reads as
+  `(let [x v] rest…)`. Consecutive `let`s merge into one binding vector.
+  `let x = v` followed by a deeper-indented block scopes to that block only,
+  which is how the printer writes a `let` that has siblings after it.
+  Destructuring: `let {.x .y} = p`, `let [head & tail] = xs`. (`defer` is
+  function-scoped, not let-scoped, `TODO.org` "defer may be written in a let",
+  so merging never moves a cleanup.)
+- **`if`/`elif`/`else`.** `else` and `elif` sit at the `if`'s column. No `elif`
+  reads as `if` (with else) or `when` (without); with `elif` it reads as `cond`.
+  One-line form: `if c then a else b`, for use in a `let`.
+- **`while c`, `until c`**, optional label first: `while :outer c`.
+- **`for i in range(n)`**, `range(a, b)`, `range(a, b, step)` read as
+  `dotimes`. `range` here is syntax, not a function. `..` is avoided because
+  `a..b` would lex as one name.
+- **`return v`, `break`, `break :outer`, `continue`, `defer expr`** (or `defer`
+  plus a block).
+- **`match`:**
+
+  ```
+  match shape
+    Circle(r) -> 3.14 * r * r
+    Rect(w, h) -> w * h
+    :north -> 0
+    _ -> 0
+  ```
+  An arm's body can be an indented block, which reads as `(do …)`.
+- **Conditions**, clauses at the header's column:
+
+  ```
+  handler-case
+    edn/read-file("game-data.edn")
+  on FileError(c)
+    nil
+
+  restart-case
+    agent/poll()
+  restart continue()
+    ()
+  restart use-value(v: i32)
+    v * 2
+  ```
+  `handler-bind` takes the same `on` clauses; the reader moves them in front of
+  the body, where the form wants them.
+- **Unit:** `()` as a statement reads `(do)`; in a type it is `()`.
+- **Lambda:** `fn(i, j) = i * 10 + j`, or `fn(i, j)` plus a block.
+
+### Definitions
+
+- `fn name(a: i32, b) -> R` plus a block; `fn name(a) = expr` for one
+  expression. Reads `(defn name [a i32 b dyn] R …)`. A `{:where …}` constraint
+  becomes `where ordered?($t)` after the return type.
+- `def x = v`, `def x: T = v`, `once x: T`, `once x = v`, `const n = 3`,
+  `def scratch: [4 u8] = uninit`.
+- `struct Cell` with a `name: Type` line per field. `data Shape` with a line per
+  case: `Circle(r: f32)`, `Empty`. `enum K` with `lo = -1`, `mid`. `union U` like
+  `struct`.
+- `import rl "vendor:raylib"`.
+- **Every other form uses the fallback** (next item) until someone asks for
+  sugar: `defclass`, `defgeneric`, `defmulti`, `defmethod`, `declare`,
+  `declare-c`, `defalias`, `defmacro`, `loop`/`recur`, `array-fill`.
+
+### The fallback
+
+**Any form can be written as a call:** `head(arg, …)`, or `head(arg, …):`
+plus an indented block, reads as `(head arg … block…)`. Commas vanish into the form.
+`defmethod(describe, :square, [s]):` plus a block is
+`(defmethod describe :square [s] …)`. So every form is reachable on day one,
+the printer has something to fall back on, and the sugar above can land one
+piece at a time.
+
+### Types
+
+After `:` and `->`, a small type grammar that reads to today's type forms:
+`i32`, `$t`, `()`, `[T]`, `[const T]`, `[n T]`, `Vec(T)`, `Map(K, V)`,
+`Option(T)`, `Ptr(T)`, `Ptr(const T)`, `Fn(A, B) -> R`, `CFn(A) -> R`,
+`rl/Vector2`.
+
+### Macro templates
+
+```
+defmacro(with-mode-2d, [camera & body]):
+  quote
+    begin-mode-2d(~camera)
+    ~@body
+    end-mode-2d()
+```
+
+`quote` plus a block is a quasiquote; `~x` and `~@xs` are unquote and splice,
+the Clojure spellings the reader already has. (An earlier sketch used `$x`;
+that collides with type variables such as `$t`.)
+
+## 3. Settled after review (2026-09-25)
+
+1. **A call takes a block only with a trailing `:`.** `rl/with-drawing():`
+   then the indented body. A deeper-indented line after an ordinary call is an
+   error, never an extra argument, so a stray indent can't be silently absorbed
+   into the call above it. This is F#'s rule too: a block opens only after a
+   specific token (`=`, `->`, `then`, `do`, …; `LexFilter.fs` 2236-2526), and
+   F#'s own `with-drawing` would need `fun () ->`. Headers (`fn`, `if`,
+   `while`, …) are the openers here and need no `:`. The fallback form takes
+   its block the same way: `defmethod(describe, :square, [s]):`.
+2. **Variadics stay.** Three spellings, all reading to the same variadic form:
+   - a run of one operator flattens, `a + b + c` → `(+ a b c)`, `a < b < c` →
+     `(< a b c)`, `x and y and z` → `(and x y z)`;
+   - an operator glued to `(` is a call: `+(a, b, c)`, `!=(a, b, c)`,
+     `and(p, q, r)`;
+   - ordinary variadic functions are just calls: `println(a, b, c)`.
+   The one exception: **`a != b != c` is refused**, with a message pointing at
+   `!=(a, b, c)`, because Flan's `!=` means "all distinct", not what the chain
+   suggests.
+3. **`'(a b)` is quoted**: a list of symbols, as in Lisp, the same `'` as
+   `'name`. `list(a, b)` builds a list of values. Revisit if lists get common.
+4. **Private functions are `fn- name(…)`.**
+5. **"Infer the return type" is `_` in the return slot** in the paren syntax:
+   `(defn f [x i32] _ (+ x 1))`. The indented reader emits `_` when `->` is
+   omitted. `_` in type position means "fill this in" in Rust and OCaml too,
+   and no type can be named `_`.
+
+## 4. Build order
+
+Each step lands on its own, with `dune test --root .` green.
+
+1. **The reader** (`lib/indent_reader.ml` or similar): tokenizer with the
+   indent stack, then a parser to `Form.t`. Start with what
+   `sand.flan` and `algorithms.flan` need, then the fallback, then the sugar
+   in section 2 in order of corpus frequency (`set`, `let`, `+`, `at`, `=`,
+   `if`, `dotimes`, …). Until step 6 lands, a `.fln` function must write
+   `-> T`; omitting it is refused with a message saying inference is coming.
+   **Test:** hand-convert `algorithms.flan` and `sand.flan` to `.fln`; the
+   forms read from each pair must be equal, ignoring locations.
+2. **Switch readers by extension** at every program-source entry point:
+   `Front.load` (`lib/front.ml:16`), `Load.import` (`lib/load.ml:1211`, plus
+   `entries` at 137-142 and `is_package_file` at 103-105), `Session.create`
+   (`lib/session.ml:260`), and `bin/main.ml` 327, 334, 393-399, 557-560. The
+   prelude (`lib/prelude.ml:2441`), the wire protocol (`lib/wire.ml:103`) and the
+   registry spelling (`lib/dev.ml:2725`) stay paren syntax.
+3. **The printer**, `Form.t` → indented text, and a `flan convert` command.
+   **Test:** for every corpus file, read with parens, print indented, read
+   indented; the forms must be equal to the first read, after one normalisation:
+   a `let` whose whole body is another `let` counts as equal to the merged
+   `let`. That covers 394 files and runs on readers alone, so it's fast.
+4. **The dev loop.** Code-carrying wire ops (`eval`, `eval-expr`,
+   `macroexpand`, `set`) get an explicit `:syntax` field instead of guessing
+   from `:file`. The `:file` guess breaks for `<repl>`/`<inspect>` origins and
+   for `flan-macroexpand-again` (`emacs/flan.el:3279-3309`), which sends
+   paren-syntax expansion text under the original file's name. Replace the
+   space-padding in `flan--text-at` (`emacs/flan.el:2602-2622`), which breaks
+   significant indentation, with `:line`/`:col` fields; the reader seeds its
+   indent stack with that column.
+5. **Emacs mode** for `.fln`:
+   - A top-level form runs from a column-0 line that isn't `else`, `elif`,
+     `on` or `restart` to just before the next one, minus trailing blank and
+     comment lines (python.el `python-nav-end-of-defun`,
+     `~/Repositories/emacs/lisp/progmodes/python.el:2175-2195`).
+   - An inner block is its header line plus every deeper-indented line after
+     it. Send the text unchanged, with its start line and column.
+   - Replace the `flan-mode`-only checks: `flan-watch--ghost-buffers` uses
+     `(eq major-mode 'flan-mode)` (`emacs/flan-watch.el:247-255`), and there
+     are `derived-mode-p 'flan-mode` checks at `flan.el:1140, 2095, 2408` and
+     in `flan-dape.el:96-129`.
+   - The breakpoint position Emacs sends (`:pause (LINE COL)`,
+     `flan--pause-bounds` at `flan.el:2637-2660`) must equal the start
+     location the reader gave that form. `Ast.mark_pause` matches exactly
+     (`ast.ml:491-492`).
+6. **Return-type inference** in `Check`, with the recursion refusal and the
+   stale-caller cause. This is independent of steps 1-5 once the marker exists.
+
+Out of scope: dropping macros, built-in replacements for `with-*`/`defedn`,
+printing diagnostics in the new syntax, converting the prelude or vendor
+packages, website and docs.
+
+## 5. Reference
+
+**Where the grammar gets hard**, from a survey of the whole corpus (394 files):
+- 513 `(Type {…})` constructions, all with `.field` keys.
+- 1304 `(.f x)` accesses; chains of up to 5 `(.a (.b x))` deep.
+- 65 `cond` and 185 `match`, written as flat pairs.
+- 1229 `set`s, all with 2 arguments. The target is a symbol 809 times, `at` 217,
+  a field 180, `get` 10 and `deref` 9.
+- 22 chained comparisons and 16 variadic `!=`.
+- `and`/`or` with up to 15 arguments (`vendor/edn/edn.flan:251`).
+- 332 `()`.
+- 1048 lines of code-generating macros in `vendor/edn/provide.flan` and
+  `vendor/json/provide.flan`, with 139 quasiquotes and 188 unquotes.
+
+**What prints paren syntax to a user:**
+- `Form.to_string` in `parse.ml` and `expand.ml` diagnostics.
+- About 70 hard-coded usage strings and "write (…)" hints in `parse.ml` and
+  `check.ml`.
+- `Types.to_string`, which is in every type error and in the eldoc, `defs`,
+  `layout` and locals replies.
+- The macroexpand `:text`, `:flat` and `:source` fields (`dev.ml:1416-1430`).
+- Macro signatures in `defs` (`dev.ml:1616-1629`).
+- `Render`'s value notation (`println`, the REPL, the inspector), which
+  `emacs/flan-inspect.el:122-272` parses and `flan-inspect--literal` (991)
+  writes back as code.
+
+**Offside-rule references:**
+- F#: `~/Repositories/Fable/src/fcs-fable/src/Compiler/SyntaxTree/LexFilter.fs`.
+  Copy the adjacency rule (2663-2670), continuation operators (360-380,
+  1850-1870), and `else`/`elif` alignment (2028-2032, 202-218). Skip the
+  context stack; it exists for keyword-opened blocks, which this syntax
+  doesn't have.
+- GDScript: `~/Repositories/godot/modules/gdscript/gdscript_tokenizer.cpp`
+  (`check_indent`, 1143-1304) and `gdscript_parser.cpp` (multiline flags and
+  `push_expression_indented_block`).
+- tree-sitter-python:
+  `~/Repositories/tree-sitter/languages/tree-sitter-python/src/scanner.c`.