diff --git a/CLAUDE.md b/CLAUDE.md index 8859b5d..f8a5a3a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,7 +36,7 @@ These invariants are NOT SUGGESTIONS. These MUST be followed. - **We do not handle all possible OOXML elements: so make in-place XML mutations, not AST/Markdown round-trips.** The AST or Markdown are just views. Mutate `XmlNode` refs, only emit fresh XML for inserted nodes. This way, elements we cannot parse will not be broken. See [src/core](src/core/CLAUDE.md). - **Most things, even those that are not easily representable, should be in Markdown.** For these, we (1) build comments into the markdown to give folks a better sense of what is going on, (2) make sure that these comment do not break the markdown and survive the roundtrip, and (3) provide intuitive verbs to deal with concepts. If some structure is truly not representable in Markdown, we nudge agents to render the pages and look. We should guarantee that the things that we're not doing the above for have no possible representation. - **Authoring has two channels: parsed and literal.** Markdown is the DEFAULT: `edit`/`insert` take their content as a positional argument (`docx edit FILE --at p3 "Net **30** days"`), which is exactly `--markdown` (kept as an alias). New markdown content must look like the document it lands in: a `- item` beside a list joins THAT list (`continueList` in `core/lists/continue-list.tsx`), and whatever the replaced paragraph / neighbor doesn't format falls back FIRST to the look that was AT that position (for `edit`, the replaced paragraph's rendered face + explicit size), THEN to the document's DOMINANT face/size — the `docx:base` `read` shows (`matchDocumentLook` in `core/dominant-formatting.ts`) — never the new style chain's default. `--markdown`/`--from` parse GFM (+ math + CriticMarkup + inline HTML); `--text-file` (on `insert` and `create`, `-` = stdin) is the PARSER-FREE channel — every character lands verbatim, each newline starts a new paragraph. It exists because GFM corruption of literal prose isn't always escapable: a bare URL autolinks with NO escape sequence at all, and CriticMarkup eats `{++…++}` regardless of backslashes (`*x*`, `3.`, `[t](u)` ARE backslash-escapable, but the agent — often Haiku — must get the whole dialect right, and the field evidence is they don't). So "put this exact text in, untouched" gets its own flag rather than an escaping burden on the weakest actor. `--text` stays inline single-paragraph; both it and the bulk-literal `--text-file` insert markdown-looking content VERBATIM (`--text "**bold**"` writes literal `**` — use `--markdown` to parse). The only value `--text` still refuses is a shell-gutted currency amount (`rejectShellMangledValue`: `"$300"` → `.00`), which is never intentional. **Every inline `--text`/`--markdown` argv value gets its `\n`/`\t`/`\r`(`\r\n`) decoded at the CLI ingress** (`decodeInlineEscapes` in `cli/parse-helpers.ts`) — a weak agent writes `--text "a\nb"` for a line break, but bash double-quotes hand the CLI the literal backslash-n, which used to land verbatim in the run. It's applied by EVERY inline authoring surface, not just the body: `edit`/`insert`/`create`, `footnotes`/`endnotes add`+`edit` (`--text` and inline `--markdown`), `comments add`+`reply` (`--text`), `headers`/`footers set` (`--text`), and the `find`/`replace` positionals (QUERY/PATTERN/REPLACEMENT) — where a decoded `\n` is ALSO the gate that routes the command to the cross-paragraph matcher (a `\n` in a find/replace pattern matches an in-paragraph line break OR the boundary between consecutive paragraphs — in the body or within one table cell, never across a cell wall; a `\n` in a replace REPLACEMENT means a paragraph mark, editor-style — single-line replacement across a boundary MERGES paragraphs, replacement `\n` SPLITS; untracked only, refuses under tracking). Decoded whitespace flows through the same real-character handling everything uses — the shared **`textToRunElements`** emitter in `core/blocks.tsx` (`textToRuns` + `RunElement`, nulls filtered) turns a real `\n`/`\t` into ``/``, so the note/comment/marginal emitters (which previously emitted a single raw `` that swallowed a newline) now render breaks just like a body paragraph; the markdown block splitter handles the `--markdown` side. Plain single-line text still collapses to one `` run, byte-identical to the old shape. ONLY those whitespace escapes decode: a `\`-before-punctuation (markdown's own `\*`/`\[`/`\.`, and a literal `\\`) passes through UNTOUCHED, so decoding can't corrupt markdown. The decode is INLINE-argv-only — `--text-file`/`--markdown-file`/`--from` read real characters from a file and `--batch` JSONL is already `JSON.parse`-decoded, so none are re-decoded; `--text-file` therefore stays the one channel where a bare `\n` really means backslash-n. -- **Comments are never anything but hints.** Every `` comment `read` writes into Markdown — locators (``) and the structural visibility annotations (`docx:section` — rendered at the section's START with an `applies-to="pX..pY (below)"` scope on deviating sections so the "which side of the boundary?" off-by-one is explicit, `docx:page`, `docx:table` (widths/borders + the `tables format` properties align/style/cell-margins/repeat-header/row-heights), `docx:cell` merge/shading/vAlign/halign/borders/margins, `docx:track-changes on|off pending=N` ALWAYS at the head (the one hint that states its default too — weak agents can't reliably read "no hint" as "off," and a wrong tracking guess is high-cost, so it's stated outright; `pending=N` likewise always, `pending=0` included, since the default accepted view hides a pending deletion outright, plus a compact `(hdr0: 1, fn2: 1)` when some pending changes live outside the body), `docx:header`/`docx:footer` for page headers/footers (content in a `text` attr, fields as `{page}`/`{date}`/… tokens, a `pending="tcN …"` attr naming the part's tracked changes in every view, and the text following the view — CriticMarkup under `--current`, safe inside the comment because `htmlAttr` escapes every `>` so a deletion's `--` can never form `-->` — authored via `docx headers`/`docx footers`; a UNIFORM marginal rides the head, a marginal that DIFFERS by section renders at that section's START, next to the content it governs), `docx:textbox tbxN anchor="pM"` … `docx:textbox-end tbxN` bracketing a text box's story, printed right after the paragraph that anchors it (the story's own paragraphs carry bare `tbxN:pK` locators), `docx:layout` flagging tab-aligned content that wraps in render — inside a multi-column section, OR a line whose trailing content rides a right-edge LEFT tab so a long value overflows the margin (the résumé `San`/`Francisco` split); the latter is also consolidated into ONE top-of-read `fix-all` summary naming the single-call cure `edit --at pN-pM --tabs right` (a render-only break Markdown can't show), and future image/style hints, all via `formatNote` in `cli/read/annotations.ts`) — is a READ-TIME VISIBILITY HINT. **Naming rule: a BARE comment is a locator (an address you pass to `--at`: ``, ``; a plain empty cell uses `` so it can be filled directly); a `docx:TYPE` comment is docx-cli METADATA** (structure/formatting). Metadata never rides a bare locator — it's its own `docx:` annotation, which may carry the relevant locator as a bare leading token (``). So `docx:` is greppable as "everything docx-cli added beyond addressing," and locators stay short. The Markdown importer DROPS them all (`walker.tsx`'s `case "html"` returns `[]`; the inline walker drops inline comments); they NEVER drive reconstruction or carry data through `read → create`. A comment can't silently corrupt the round-trip or smuggle structure back — what an agent sees is a hint, nothing more. The lossless view is `read --ast`; in-place `edit` preserves structure; the authoring verbs (`docx sections`, `docx tables …`) change it. Emit hints **deviation-only** (only what differs from the document default — even columns, default geometry, `single` borders, `align=left` emit nothing), with TWO deliberate exceptions that always emit even at their default: `docx:section` (every section gets a marker so the `sN` is always addressable) and `docx:track-changes` (states `off` and `pending=0` too, so a weak agent never infers state from a missing hint). Run FORMATTING is the one thing that's Markdown-representable and DOES round-trip, but it rides HTML ELEMENTS (``/``/``/``, via `gatherHtmlSpans`), NOT comments. Even `` (the dominant font/size note) is a hint: `read` omits the baseline from runs and shows the note so an agent can match it, but the importer drops it — a full `read → create` rebuild falls back to the template docDefaults (Calibri 11pt) for the dominant font/size. The footnote-shaped read annotations are hints too: the `[^tcN]` (`--current`) and `[^cN]` (`--comments`) labels and their definition lines are DROPPED on import (`remarkReadHintLabels` in `core/markdown/read-hint-labels.ts`; real notes always read as `[^fnN]`/`[^enN]`), so a pasted `--current` line behaves exactly like its label-free CriticMarkup. No comment, anywhere, is parse-back. +- **Comments are never anything but hints.** Every `` comment `read` writes into Markdown — locators (``) and the structural visibility annotations (`docx:section` — rendered at the section's START with an `applies-to="pX..pY (below)"` scope on deviating sections so the "which side of the boundary?" off-by-one is explicit, `docx:page`, `docx:table` (widths/borders + the `tables format` properties align/style/cell-margins/repeat-header/row-heights), `docx:cell` merge/shading/vAlign/halign/borders/margins, `docx:track-changes on|off pending=N` ALWAYS at the head (the one hint that states its default too — weak agents can't reliably read "no hint" as "off," and a wrong tracking guess is high-cost, so it's stated outright; `pending=N` likewise always, `pending=0` included, since the default accepted view hides a pending deletion outright, plus a compact `(hdr0: 1, fn2: 1)` when some pending changes live outside the body), `docx:header`/`docx:footer` for page headers/footers (content in a `text` attr, fields as `{page}`/`{date}`/… tokens, a `pending="tcN …"` attr naming the part's tracked changes in every view, and the text following the view — CriticMarkup under `--current`, safe inside the comment because `htmlAttr` escapes every `>` so a deletion's `--` can never form `-->` — authored via `docx headers`/`docx footers`; a UNIFORM marginal rides the head, a marginal that DIFFERS by section renders at that section's START, next to the content it governs), `docx:textbox tbxN anchor="pM"` … `docx:textbox-end tbxN` bracketing a text box's story, printed right after the paragraph that anchors it (the story's own paragraphs carry bare `tbxN:pK` locators), `docx:layout` flagging tab-aligned content that wraps in render — inside a multi-column section, OR a line whose trailing content rides a right-edge LEFT tab so a long value overflows the margin (the résumé `San`/`Francisco` split); the latter is also consolidated into ONE top-of-read `fix-all` summary naming the single-call cure `edit --at pN-pM --tabs right` (a render-only break Markdown can't show), `docx:chart`/`docx:smartart`/`docx:shape`/`docx:drawing` INLINE where a non-picture drawing sits (zero-width, like the drawing in the offset model; a line it LEADS is re-parsed past it by `reparseHintLedLines` so CommonMark's HTML-block rule can't swallow the text after it), and future image/style hints, all via `formatNote` in `cli/read/annotations.ts`) — is a READ-TIME VISIBILITY HINT. **Naming rule: a BARE comment is a locator (an address you pass to `--at`: ``, ``; a plain empty cell uses `` so it can be filled directly); a `docx:TYPE` comment is docx-cli METADATA** (structure/formatting). Metadata never rides a bare locator — it's its own `docx:` annotation, which may carry the relevant locator as a bare leading token (``). So `docx:` is greppable as "everything docx-cli added beyond addressing," and locators stay short. The Markdown importer DROPS them all (`walker.tsx`'s `case "html"` returns `[]`; the inline walker drops inline comments); they NEVER drive reconstruction or carry data through `read → create`. A comment can't silently corrupt the round-trip or smuggle structure back — what an agent sees is a hint, nothing more. The lossless view is `read --ast`; in-place `edit` preserves structure; the authoring verbs (`docx sections`, `docx tables …`) change it. Emit hints **deviation-only** (only what differs from the document default — even columns, default geometry, `single` borders, `align=left` emit nothing), with TWO deliberate exceptions that always emit even at their default: `docx:section` (every section gets a marker so the `sN` is always addressable) and `docx:track-changes` (states `off` and `pending=0` too, so a weak agent never infers state from a missing hint). Run FORMATTING is the one thing that's Markdown-representable and DOES round-trip, but it rides HTML ELEMENTS (``/``/``/``, via `gatherHtmlSpans`), NOT comments. Even `` (the dominant font/size note) is a hint: `read` omits the baseline from runs and shows the note so an agent can match it, but the importer drops it — a full `read → create` rebuild falls back to the template docDefaults (Calibri 11pt) for the dominant font/size. The footnote-shaped read annotations are hints too: the `[^tcN]` (`--current`) and `[^cN]` (`--comments`) labels and their definition lines are DROPPED on import (`remarkReadHintLabels` in `core/markdown/read-hint-labels.ts`; real notes always read as `[^fnN]`/`[^enN]`), so a pasted `--current` line behaves exactly like its label-free CriticMarkup. A real `[^fnN]`/`[^enN]` copied WITHOUT its definition is not a hint — it addresses the document's existing note (`remarkNoteReferenceLabels`: reused when an untracked edit replaces the block that references it, cloned anywhere else, USAGE error when no such note exists), so it never lands as literal text. No comment, anywhere, is parse-back. - **Everything we parse MUST be represented in the AST**. Markdown is always either a subset or equal to what is in the AST, never less than it. `RUN_BEARING_WRAPPER_TAGS` in `src/core/parser/run-ops.ts` is the AST↔XML offset bridge; every offset-aware walker reads it. See [src/core](src/core/CLAUDE.md). - **`` is transparent at EVERY level, and text boxes are first-class.** ECMA-376 Part 3 lets a producer wrap a block, a paragraph's runs, or a run's children in a markup-compatibility wrapper and nest wrappers inside branches; Word writes every modern shape/text box that way (`wps` under ``, VML under ``, SAME story in both). Every walker resolves it through `alternateContentBranch` in `src/core/mc.ts` (first Choice, else Fallback) — the reader at block/paragraph/run level, the offset walkers via `wrapperContent(node)` in `parser/run-ops.ts` (NEVER `node.children` for a wrapper — that is what keeps a multi-Choice wrapper counted exactly once), and the text-box collector while descending a shape. Dropping the wrapper was issue #4: `replace --all` reported 2 and exited 0 while a third "Acme" sat in a text box a human reads first. A `` story is a block container like a cell (`TextBoxRun` on the anchor paragraph, `tbxN:pK` ids, `blockReferences` parents inside the Choice copy), so `find`/`replace`/`edit`/`insert`/`delete`/`comments` reach it by default; `Document.save` (and the raw gate, BEFORE validating) mirrors each Choice story onto its Fallback twin (`syncTextBoxFallbacks`, post-order so nested boxes sync inner-first) — verified against Word for Mac 365: Word regenerates a box's Fallback when IT edits the box and leaves an untouched box's Fallback verbatim (a stale one stays stale forever), so mirroring on our writes is exactly its behavior. `wc` counts what `read` shows — text boxes included, everywhere. **Word discards comments and footnotes/endnotes inside a text box on its next save** (verified: the markers vanish from the story and the comment from comments.xml, regardless of the Fallback), so `comments add` (`--at` AND `--anchor`) and `footnotes`/`endnotes add` REFUSE a `tbxN` target with `UNSUPPORTED` + a pointer to the anchor paragraph; tracked changes and hyperlinks inside a box survive Word and are allowed. The box rides its anchor RUN: EVERY whole-paragraph `edit` lifts object-bearing runs (drawing/pict/object/AlternateContent) out and puts them back — `--text` around the token diff (`extractObjectRuns`), `--markdown`/`--runs` around the node replace (`liftObjectRuns` → `reattachObjectRuns` in `track-changes/replace.tsx`, untracked carriers even under tracking; a `--markdown` that re-states an image via `![alt](sha256.ext)` drops the lifted DRAWING so `img0` isn't duplicated, while a text box always returns), a `replace` on the anchor run's text keeps a trailing zero-width shape (`sliceRun`), and a tracked delete of the anchor wraps the run in `` WITHOUT rewriting the story's `` (`TextBoxRun.trackedChange` hides it in the accepted view; `iterateBlocks({ view })`). An UNTRACKED removal of the anchor paragraph (`delete --at pN`, `edit --at pN --text ""`) re-anchors the box on the neighboring paragraph — previous, else next — via `relocateObjectRuns` in `track-changes/replace.tsx`; deleting the invoice's Notes line used to take the stamp box with it, silently. - **Every element must have stable positional ids for locators** (`p0`, `t0`, `c0`, `img0`, `link0`, `tc0`, `hdr0`/`ftr0` for page headers/footers, `tbx0` for a text box — its story chains like a cell: `tbx0:p1`, `tbx0:t0:r0c0:p0`). Block ids shift after structural edits. Re-read between non-trivial mutations. To not force agents to re-read on every modification, we create batch versions of any editing call such that the already made locators can be used. diff --git a/src/cli/edit/batch.ts b/src/cli/edit/batch.ts index d7872c8..7e84da8 100644 --- a/src/cli/edit/batch.ts +++ b/src/cli/edit/batch.ts @@ -734,7 +734,11 @@ async function buildWholeParagraphContent( const source = requireString(raw.markdown, index, "markdown"); let blocks: XmlNode[]; try { - blocks = await new MarkdownImport(document).blocks(source); + blocks = await new MarkdownImport(document).blocks(source, { + // Untracked: a `[^fnN]` copied from `read` keeps the note this entry's + // paragraph references instead of cloning it. + replacing: opts.track ? [] : [blockRef.node], + }); } catch (error) { if (error instanceof MarkdownImportError) { throw new EntryError( diff --git a/src/cli/edit/index.tsx b/src/cli/edit/index.tsx index 3fef6ef..16f320f 100644 --- a/src/cli/edit/index.tsx +++ b/src/cli/edit/index.tsx @@ -466,7 +466,11 @@ async function commitBlockEdit( }, )); } else if (opts.spec.kind === "markdown") { - const resolved = await resolveMarkdownBlocks(document, opts.spec); + const resolved = await resolveMarkdownBlocks( + document, + opts.spec, + track ? [] : [blockRef.node], + ); if (typeof resolved === "number") return resolved; ({ node: resultNode, resolvedAuthors } = edit.paragraph( blockRef, @@ -575,9 +579,18 @@ async function commitRangeEdit( if (opts.dryRun) return respondDryRun(opts); + const track = resolveTracked(document, opts.trackFlag); let spec: ParagraphContentSpec; if (opts.spec.kind === "markdown") { - const resolved = await resolveMarkdownBlocks(document, opts.spec); + const replacing = rangeRef.parent.slice( + rangeRef.startIndex, + rangeRef.endIndex + 1, + ); + const resolved = await resolveMarkdownBlocks( + document, + opts.spec, + track ? [] : replacing, + ); if (typeof resolved === "number") return resolved; spec = resolved; } else if (opts.spec.kind === "text" || opts.spec.kind === "runs") { @@ -586,7 +599,6 @@ async function commitRangeEdit( return fail("USAGE", "Unsupported edit spec for range locator"); } - const track = resolveTracked(document, opts.trackFlag); let resolvedAuthors: string[]; try { resolvedAuthors = new Edit(document).range(rangeRef, spec, { @@ -1183,15 +1195,20 @@ async function loadMarkdownFile(path: string): Promise { /** Resolve a `markdown` spec into a `markdown-blocks` spec by parsing the * source against the now-open document. The lens registers footnote bodies, * mints image rels, and provisions any styles the source references — all on - * `document` — before returning the splice-ready blocks. */ + * `document` — before returning the splice-ready blocks. `replacing` is the + * block(s) an UNTRACKED edit replaces (empty under tracking), so a `[^fnN]` + * copied from `read` keeps its note instead of cloning it. */ async function resolveMarkdownBlocks( document: Document, spec: Extract, + replacing: readonly XmlNode[], ): Promise< Extract | number > { try { - const blocks = await new MarkdownImport(document).blocks(spec.source); + const blocks = await new MarkdownImport(document).blocks(spec.source, { + replacing, + }); return { kind: "markdown-blocks", blocks, diff --git a/src/cli/read/markdown.ts b/src/cli/read/markdown.ts index d2c1456..ceef780 100644 --- a/src/cli/read/markdown.ts +++ b/src/cli/read/markdown.ts @@ -1608,7 +1608,14 @@ function renderRuns( else ctx.referencedEndnoteIds.add(run.id); out += `[^${run.id}]`; } else if (run.type === "chart") { - out += `\`[${run.kind}]\``; + // A chart / SmartArt / shape has no Markdown form. Its marker is a + // `docx:` HINT, not content: it used to print as inline code + // (`` `[chart]` ``), so a read line copied back into `edit` kept the + // chart (lifted and re-attached) AND wrote the literal text "[chart]" + // — and its 9 characters skewed every span read off the line, since the + // object is zero-width in the offset model. The importer drops the + // comment (a leading one is split off by `reparseHintLedLines`). + out += formatNote(run.kind, []); } // A text box contributes nothing inline: its story renders as its own // block right after the anchor (see `renderTextBoxes`). diff --git a/src/core/ast/document/notes.ts b/src/core/ast/document/notes.ts index 0485496..a9acfb5 100644 --- a/src/core/ast/document/notes.ts +++ b/src/core/ast/document/notes.ts @@ -161,6 +161,30 @@ export class NotesView { return undefined; } + /** A real note (not Word's `w:type` separator boilerplate) by numeric id. */ + findUserNote(numericId: string): XmlNode | undefined { + const found = this.findByNumericId(numericId); + if (!found || found.node.getAttribute("w:type")) return undefined; + return found.node; + } + + /** Copy note `numericId` under a fresh id (placed right after the original) + * and return the new id — what Word does when text holding a note reference + * is pasted. Markers whose ids must stay document-unique (bookmarks, comment + * ranges + references, `w14` paragraph ids) are dropped from the copy, so the + * clone can't collide with the original. The caller has already checked the + * note exists (`findUserNote`). */ + cloneNote(numericId: string): string { + const found = this.findByNumericId(numericId); + if (!found) throw new Error(`Internal: no note ${numericId} to clone`); + const copy = found.node.clone(); + const id = this.nextId(); + copy.setAttribute("w:id", id); + stripUniqueMarkers(copy); + found.parent.splice(found.parent.indexOf(found.node) + 1, 0, copy); + return id; + } + /** Lazily provision the two character/paragraph styles a note relies on. * The baseline catalog defines both — these calls just register the style * nodes in `styles.xml` if not already present, which avoids Word's @@ -178,3 +202,26 @@ export class NotesView { return XmlNode.findRoot(this.tree, config.rootTag); } } + +/** Drop, in place, every marker under `node` whose id must be unique across the + * document — so a cloned note body can't duplicate one. */ +function stripUniqueMarkers(node: XmlNode): void { + delete node.attributes["w14:paraId"]; + delete node.attributes["w14:textId"]; + node.children = node.children.filter((child) => !isUniqueMarker(child)); + for (const child of node.children) stripUniqueMarkers(child); +} + +function isUniqueMarker(node: XmlNode): boolean { + if (UNIQUE_MARKER_TAGS.has(node.tag)) return true; + return ( + node.tag === "w:r" && node.findChild("w:commentReference") !== undefined + ); +} + +const UNIQUE_MARKER_TAGS: ReadonlySet = new Set([ + "w:bookmarkStart", + "w:bookmarkEnd", + "w:commentRangeStart", + "w:commentRangeEnd", +]); diff --git a/src/core/markdown/CLAUDE.md b/src/core/markdown/CLAUDE.md index d7572a4..2f17e93 100644 --- a/src/core/markdown/CLAUDE.md +++ b/src/core/markdown/CLAUDE.md @@ -7,6 +7,8 @@ Files behind the `@core/markdown` barrel ([index.ts](index.ts)): - [inline.tsx](inline.tsx) — phrasing walker (`walkInline`). Returns ``-child siblings: a mix of ``, ``, ``, ``, ``. Threads an immutable `InlineFormat` through nested strong / em / delete / inlineCode / bracketed-span wrappers. - [inline-surgery.ts](inline-surgery.ts) — `remarkInlineSurgery` plugin. Post-parse, rewrites each phrasing-children array with two reducers: text-marker surgery gathers CriticMarkup (`{++…++}` / `{--…--}`) and legacy Pandoc spans (`[text]{.class key="value"}`); **HTML-element surgery (`gatherHtmlSpans`)** re-pairs the `` / `` / `` / `` / `` tags `read` now emits (remark leaves inline HTML as FLAT, unpaired `html` tokens — open tag, mdast content, close tag as separate siblings — so we re-pair them, the same stack-reduce as the text markers). Both feed `bracketedSpan` (etc.) parent nodes the inline walker dispatches on. Each tag's attributes are parsed with the project's `XmlNode.parse` (fast-xml-parser), not a bespoke regex. Scanning whole sibling arrays (not single `text` nodes) is what lets a marker straddle inline formatting. - [read-hint-labels.ts](read-hint-labels.ts) — `remarkReadHintLabels` plugin. Drops the footnote-shaped READ annotations (`[^tcN]` / `[^cN]` + their definitions) on import — see "Footnote-shaped read annotations" below. +- [note-reference-labels.ts](note-reference-labels.ts) — `remarkNoteReferenceLabels` plugin. A `[^fnN]`/`[^enN]` with no definition in the source addresses the document's EXISTING note — see "Read-copied note references" below. +- [hint-led-lines.ts](hint-led-lines.ts) — `reparseHintLedLines`. Re-parses the text after a LEADING `docx:` hint comment so CommonMark's HTML-block rule can't swallow the line — see "Inline hints that lead a line" below. - [escape.ts](escape.ts) — `inlineEscapeMask`, the parser-driven escape mask `read` applies (see "Metacharacters are escaped" below). - [display-math.ts](display-math.ts) — `promoteDisplayMath`. Post-parse, turns a top-level paragraph that is ONLY a one-line `$$…$$` (whitespace and `` comments aside) into a display `math` block. `read` prints a display equation exactly that way, and remark-math alone parses it as INLINE math, so pasting a read line back into `edit`/`insert`/`create` demoted the block to inline (a multi-line `\begin{gathered}` block collapsed into one inline ``). A `$$…$$` beside other text stays inline. A paragraph made ONLY of CriticMarkup each wrapping one `$$…$$` — how `read --current` prints a tracked display-equation replace, `{--$$old$$--}{++$$new$$++}` — can't become a block (a block can't sit inside ``), so `markTrackedDisplayMath` flags each `inlineMath` `data.display` and the inline walker emits `` inside its ``/``, the shape a tracked display replace writes. - [errors.ts](errors.ts) — `MarkdownImportError` (code is a strict subset of `cli/respond.ts`'s `ErrorCode`). @@ -37,6 +39,14 @@ When `` is on, `criticInsert` wraps its (now possibly multi-r `remarkReadHintLabels` ([read-hint-labels.ts](read-hint-labels.ts)) has two halves, because GFM only parses `[^x]` as a reference when a `[^x]:` definition exists: a micromark TEXT construct (`add: "before"`, so it wins over GFM's footnote call) claims every `[^tcN]`/`[^cN]` reference — defined or not — as a `readHintLabel` node; a transformer, ordered BEFORE `remarkInlineSurgery`, removes those nodes and every hint-named `footnoteDefinition`, re-merging the text around a removed label so the surgery tokenizes `{++45++}[^tc1]` as an insertion. Being a parser construct it is escape- and code-aware for free (`` `[^tc0]` `` and `\[^tc0]` stay literal), and `escape.ts` registers the plugin too, so `read` escapes a LITERAL `[^tc0]` in document text (`\[\^tc0\]`) and it round-trips instead of vanishing. Matching is case-insensitive (GFM normalizes identifiers) and exact-shape only: `[^note]`, `[^fn1]`, `[^cat]` stay real footnotes. A user's own footnote literally named `[^c1]`/`[^tc1]` is dropped — name it anything else. `[^revN]` needs no rule: `track-changes list` emits `revN` group ids, but `read` never prints them as labels. +## Read-copied note references (`[^fnN]` / `[^enN]`) address the existing note + +`read` prints a real note reference as `[^fnN]` inline and its body as a `[^fnN]: …` definition at the END of the read. An agent copying one line back into `edit`/`insert` carries the label without the definition, and GFM only parses `[^x]` when a definition exists — so it used to land as the literal characters `[^fn1]` (and the note was lost on the next accept). `remarkNoteReferenceLabels` claims every `[^fn]`/`[^en]` with a micromark text construct (escape/code-aware; `escape.ts` registers it, so a LITERAL `[^fn1]` in document text reads as `\[\^fn1\]` and round-trips), then resolves each: a label the source also DEFINES goes back to an ordinary `footnoteReference` (a whole read pasted into `create` mints notes exactly as before); otherwise it becomes an `existingNoteReference` the inline walker resolves against the target document. **Reuse vs clone:** OOXML allows exactly one reference per note (Word "repairs" N:1 as corruption), so the walker REUSES note N only when `blocks(source, { replacing })` names the block(s) an UNTRACKED edit replaces and one of them references N (the old reference leaves with the old block); everywhere else — an insert, a second citation, any tracked edit (whose old reference stays behind in its ``) — it clones the note (`NotesView.cloneNote`, dropping bookmarks/comment markers/`w14` ids from the copy), which is Word's own paste behavior. Accept then keeps the clone and reject the original, each with one live reference. A label naming no note is a USAGE error, never literal text. `edit` (single, range, batch) is the only caller that passes `replacing`. + +## Inline hints that lead a line + +A `docx:` hint can sit inline, at the position of something Markdown can't show — `` (also `docx:smartart`/`docx:shape`/`docx:drawing`) marks a non-picture drawing, zero-width like the drawing itself is in the offset model. Mid-line it's an inline html node the walker drops. But CommonMark opens an HTML BLOCK at any line that begins with `Figure 2 ` parsed as one dropped block and took the caption with it. `reparseHintLedLines` (called at the end of `parseToMdast`) replaces an html block that starts with `docx:` comments and still holds non-comment text by the parse of that remainder; a comments-only line (``) stays a dropped block, and an author's own non-`docx:` comment keeps CommonMark's meaning. The drawing itself is never in the Markdown — `edit` lifts and re-attaches it — so copying the line back keeps it once. + ## Run-formatting encoding: hybrid HTML (the read↔import contract) Run-level formatting with no native markdown syntax is emitted as **HTML a markdown reader actually renders** — not Pandoc `[text]{…}` spans, which show literal brackets in GitHub / VS Code / Obsidian. `read --markdown` emits it (via `htmlFormattingWrappers` in [cli/read/markdown.ts](../../cli/read/markdown.ts)) and the import walker parses it back (via `gatherHtmlSpans` in [inline-surgery.ts](inline-surgery.ts), which converts each tag to a `bracketedSpan` the inline walker already overlays). **The tag → property mapping is the contract; keep `htmlFormattingWrappers` and `gatherHtmlSpans`/`applyCssStyle` in sync.** `read --ast` is the lossless format; markdown is the human/agent comprehension format. diff --git a/src/core/markdown/escape.ts b/src/core/markdown/escape.ts index 87686d7..dfffc0e 100644 --- a/src/core/markdown/escape.ts +++ b/src/core/markdown/escape.ts @@ -3,6 +3,7 @@ import remarkGfm from "remark-gfm"; import remarkMath from "remark-math"; import remarkParse from "remark-parse"; import { unified } from "unified"; +import { remarkNoteReferenceLabels } from "./note-reference-labels"; import { remarkReadHintLabels } from "./read-hint-labels"; // A node's source-offset span. Every mdast node carries this, but the union also @@ -20,11 +21,14 @@ type Positioned = { // `remarkReadHintLabels` IS included: its parser construct claims a literal // `[^tcN]`/`[^cN]` in run text (the importer would drop it as a read // annotation), so the mask escapes it and the text survives the round-trip. +// `remarkNoteReferenceLabels` likewise claims a literal `[^fn1]`/`[^en1]` (the +// importer would resolve it to the document's note), so it's escaped too. const rawParser = unified() .use(remarkParse) .use(remarkGfm) .use(remarkMath) - .use(remarkReadHintLabels); + .use(remarkReadHintLabels) + .use(remarkNoteReferenceLabels); // Every char CommonMark lets a backslash escape. A `\` before one of these is an // escape; escaping anything else (a letter, a space) is a no-op the parser drops. diff --git a/src/core/markdown/hint-led-lines.ts b/src/core/markdown/hint-led-lines.ts new file mode 100644 index 0000000..ab70a70 --- /dev/null +++ b/src/core/markdown/hint-led-lines.ts @@ -0,0 +1,66 @@ +import type { Root, RootContent } from "mdast"; + +/** Re-parse the content after a LEADING `docx:` hint comment. + * + * CommonMark opens an HTML block at any line that begins with ``, so a line `read` prints with an + * inline zero-width hint first — `Figure 2: sales ` + * — parsed as ONE html block. The importer drops html blocks (every comment is + * a hint, never content), which silently took the text after the hint with it: + * a copied line handed back to `edit` replaced the paragraph with nothing. + * + * So an html block that STARTS with one or more `docx:` comments and still + * holds text after them is replaced by the parse of that remainder — the hint + * goes, the content stays, exactly as if the hint were mid-line (where it is + * already an inline html node the walker drops). Only `docx:` comments qualify: + * an author's own `` keeps CommonMark's meaning. A line that is ONLY + * comments (``) is left as the dropped html block. */ +export function reparseHintLedLines( + tree: Root, + parse: (source: string) => Root, +): void { + reparseContainer(tree, parse); +} + +function reparseContainer( + node: { children?: unknown[] }, + parse: (source: string) => Root, +): void { + if (!Array.isArray(node.children)) return; + const children = node.children as RootContent[]; + const out: RootContent[] = []; + for (const child of children) { + const remainder = + child.type === "html" ? hintLedRemainder(child.value) : null; + if (remainder === null) { + if (BLOCK_CONTAINERS.has(child.type)) { + reparseContainer(child as { children?: unknown[] }, parse); + } + out.push(child); + continue; + } + out.push(...parse(remainder).children); + } + node.children = out; +} + +/** Block-level nodes whose children are blocks (where an html BLOCK can sit). */ +const BLOCK_CONTAINERS: ReadonlySet = new Set([ + "blockquote", + "listItem", + "footnoteDefinition", + "list", +]); + +const LEADING_HINTS = /^\s*(?:[ \t]*)+/; +const ANY_COMMENT = //g; + +/** The source after the leading `docx:` comments, or null when the block + * doesn't start with one or holds nothing but comments after it. */ +function hintLedRemainder(value: string): string | null { + const match = value.match(LEADING_HINTS); + if (!match) return null; + const remainder = value.slice(match[0].length); + if (remainder.replace(ANY_COMMENT, "").trim() === "") return null; + return remainder; +} diff --git a/src/core/markdown/import.tsx b/src/core/markdown/import.tsx index c9c7845..52710d2 100644 --- a/src/core/markdown/import.tsx +++ b/src/core/markdown/import.tsx @@ -29,8 +29,10 @@ import { getPageContentWidthEmu } from "../sections"; import { resolveAuthor, resolveDate, TrackChanges } from "../track-changes"; import { promoteDisplayMath } from "./display-math"; import { MarkdownImportError } from "./errors"; +import { reparseHintLedLines } from "./hint-led-lines"; import { type ResolvedImage, type WalkContext, walkInline } from "./inline"; import { remarkInlineSurgery } from "./inline-surgery"; +import { remarkNoteReferenceLabels } from "./note-reference-labels"; import { remarkReadHintLabels } from "./read-hint-labels"; import { walkRoot } from "./walker"; @@ -55,13 +57,23 @@ export class MarkdownImport { * `word/_rels/footnotes.xml.rels`, not the document's (a dangling rId * otherwise — Word reports "unreadable content"). `options.stripImages` * drops every `image`/`imageReference` before the walk so a note body - * (text + links only) can't mint a media rel in the wrong part. */ + * (text + links only) can't mint a media rel in the wrong part. + * + * `options.replacing` names the blocks an UNTRACKED edit is about to replace + * with this source. A `[^fnN]`/`[^enN]` copied from `read` whose reference + * lives in one of them keeps that note (the old reference leaves with the old + * block); every other copied reference — an insert, a tracked edit, a second + * citation — clones the note, so no two references ever share one (Word + * "repairs" that as corruption). Omit it under tracking: the old reference + * stays behind in its ``, and accept/reject must each end with + * exactly one live reference. */ async blocks( source: string, options: { authorFlag?: string; relationships?: RelationshipsView; stripImages?: boolean; + replacing?: readonly XmlNode[]; } = {}, ): Promise { // The leading `` note (emitted by `read`) is a @@ -85,6 +97,7 @@ export class MarkdownImport { imageCache: new Map(), definitions: collectDefinitions(tree), relationships: options.relationships ?? this.document.relationships, + reusableNoteIds: noteReferenceKeys(options.replacing ?? []), }; if (tracked) { @@ -118,6 +131,10 @@ function parseToMdast(source: string): Root { // Before inline surgery: the `[^tcN]`/`[^cN]` read-annotation labels are // dropped first, so CriticMarkup beside one tokenizes as if it weren't there. .use(remarkReadHintLabels) + // After the hint labels (which own `[^tcN]`/`[^cN]`), before surgery: + // a `[^fnN]` copied from `read` without its definition addresses the + // document's existing note instead of landing as literal text. + .use(remarkNoteReferenceLabels) .use(remarkInlineSurgery); const tree = processor.parse(source); // `runSync` applies the transformer plugins (remarkReadHintLabels's drop, @@ -127,9 +144,29 @@ function parseToMdast(source: string): Root { // it generic. processor.runSync(tree); promoteDisplayMath(tree as Root, source); + reparseHintLedLines(tree as Root, parseToMdast); return tree as Root; } +/** `footnote:N` / `endnote:N` for every note reference inside `blocks` — the + * notes an untracked edit replacing those blocks may keep rather than clone. */ +function noteReferenceKeys(blocks: readonly XmlNode[]): Set { + const keys = new Set(); + const visit = (node: XmlNode): void => { + const kind = + node.tag === "w:footnoteReference" + ? "footnote" + : node.tag === "w:endnoteReference" + ? "endnote" + : null; + const id = node.getAttribute("w:id"); + if (kind && id) keys.add(`${kind}:${id}`); + for (const child of node.children) visit(child); + }; + for (const block of blocks) visit(block); + return keys; +} + /** Walk the mdast tree once and pull out every `footnoteDefinition`. We * don't need to remove them — the block walker treats them as no-ops since * we've already registered the bodies into footnotes.xml. */ diff --git a/src/core/markdown/inline.tsx b/src/core/markdown/inline.tsx index 77d6600..d75ad2d 100644 --- a/src/core/markdown/inline.tsx +++ b/src/core/markdown/inline.tsx @@ -12,12 +12,14 @@ import type { RelationshipsView } from "../ast/document/relationships"; import { latexToOmml } from "../equation"; import { Image } from "../image"; import { w } from "../jsx"; +import { NoteReferenceRun, noteConfig } from "../notes"; import type { NullableXmlNode, XmlNode } from "../parser"; import { firstInvalidRunFormat, type RunFormatEnums } from "../run-formatting"; import type { TrackedMeta } from "../track-changes"; import { Del, Ins } from "../track-changes/emit"; import { MarkdownImportError } from "./errors"; import type { CriticDelete, CriticInsert } from "./inline-surgery"; +import type { ExistingNoteReference } from "./note-reference-labels"; /** Pre-resolved image entry the lens populates by walking the mdast tree * once before the synchronous OOXML emit. Inline image runs read from this @@ -66,6 +68,11 @@ export type WalkContext = { * The lens preallocates an author/date pair so all tracked nodes from * one import share metadata. */ mintTrackedMeta?: () => TrackedMeta; + /** `footnote:N` / `endnote:N` keys of the notes this import may REUSE: the + * ones referenced from the blocks an untracked edit replaces. Each is + * consumed by the first `existingNoteReference` to it; any later reference + * to the same note (or one with no key) clones the note instead. */ + reusableNoteIds: Set; }; /** Walk a sequence of phrasing nodes into ``-child siblings: ``, @@ -165,6 +172,8 @@ function walkPhrasing( return []; case "footnoteReference": return [footnoteReferenceRun(node.identifier, ctx)]; + case "existingNoteReference": + return [existingNoteReferenceRun(node, ctx)]; case "inlineMath": return [inlineMathElement(node)]; case "criticInsert": @@ -432,6 +441,34 @@ function footnoteReferenceRun(identifier: string, ctx: WalkContext): XmlNode { ); } +/** A `[^fnN]`/`[^enN]` copied from `read` without its definition: a reference to + * the document's EXISTING note N. It keeps that note when the edit is replacing + * the very block that references it (`ctx.reusableNoteIds`); otherwise it gets + * a clone of the note — Word's own copy/paste behavior, and the only valid one, + * since OOXML allows exactly one reference per note. A label naming no note in + * this document is refused rather than written as literal `[^fn9]` text. */ +function existingNoteReferenceRun( + node: ExistingNoteReference, + ctx: WalkContext, +): XmlNode { + const config = noteConfig(node.kind); + const notes = + node.kind === "footnote" ? ctx.document.footnotes : ctx.document.endnotes; + if (!notes?.findUserNote(node.numericId)) { + throw new MarkdownImportError( + "USAGE", + `[^${node.label}] names no ${node.kind} in this document`, + `Copy the reference from this document's read view, or write a new ${node.kind} with a definition (\`[^${node.label}]: text\`) or \`docx ${node.kind}s add\`. To keep the characters literally, escape them: \\[^${node.label}].`, + ); + } + const key = `${node.kind}:${node.numericId}`; + const id = ctx.reusableNoteIds.delete(key) + ? node.numericId + : notes.cloneNote(node.numericId); + notes.ensureNoteStyles(ctx.document.ensureStyles()); + return ; +} + function inlineMathElement(node: InlineMath): XmlNode { try { // `data.display`: a lone `$$…$$` inside CriticMarkup (`promoteDisplayMath`). diff --git a/src/core/markdown/note-reference-labels.ts b/src/core/markdown/note-reference-labels.ts new file mode 100644 index 0000000..ad138cb --- /dev/null +++ b/src/core/markdown/note-reference-labels.ts @@ -0,0 +1,235 @@ +import type { Root } from "mdast"; +import type { Data, Plugin } from "unified"; +import type { NoteKind } from "../notes"; + +/** Remark plugin: a `[^fnN]` / `[^enN]` copied from `read` addresses the + * document's EXISTING note N. + * + * `read` prints every real note reference as `[^fnN]`/`[^enN]` inline and the + * note's body as a `[^fnN]: …` definition at the end of the read. An agent + * following "copy a line from the read view, change it, pass it back" copies + * the line WITHOUT that definition — and GFM only parses `[^x]` as a reference + * when a `[^x]:` definition exists, so the label used to land as the literal + * characters `[^fn1]` (and the note itself was lost on the next accept). + * + * Two halves, like `remarkReadHintLabels`: + * - a micromark TEXT construct, tried before GFM's own footnote call, claims + * every `[^fn]` / `[^en]`. Being a parser construct it is + * escape- and code-aware for free (`\[^fn1]` and `` `[^fn1]` `` stay + * literal), and `escape.ts` registers the plugin too, so `read` escapes a + * LITERAL `[^fn1]` in document text and it round-trips as text; + * - a transformer resolves each claimed label: when the same source DEFINES + * it (`[^fn1]: body` — a whole read pasted into `create`), it goes back to + * being an ordinary GFM `footnoteReference`, so the definition mints a note + * exactly as before; otherwise it becomes an `existingNoteReference` the + * inline walker resolves against the target document's notes part. */ +export const remarkNoteReferenceLabels: Plugin<[], Root> = function () { + const data = this.data(); + data.micromarkExtensions = [ + ...(data.micromarkExtensions ?? []), + noteLabelSyntax, + ]; + data.fromMarkdownExtensions = [ + ...(data.fromMarkdownExtensions ?? []), + noteLabelFromMarkdown, + ]; + return (tree) => { + const root = tree as unknown as MutableNode; + resolveNoteLabels(root, definedIdentifiers(root)); + }; +}; + +/** An inline reference to note `numericId` of `kind` that already exists in the + * target document (`[^fn1]` with no `[^fn1]:` definition in the source). */ +export type ExistingNoteReference = { + type: "existingNoteReference"; + kind: NoteKind; + numericId: string; + /** The label as written (`fn1`), for error messages. */ + label: string; +}; + +const NOTE_LABEL_TOKEN = "noteReferenceLabel"; + +const CODE = { + leftBracket: 91, + rightBracket: 93, + caret: 94, + zero: 48, + nine: 57, + upperE: 69, + upperF: 70, + upperN: 78, + lowerE: 101, + lowerF: 102, + lowerN: 110, +} as const; + +/** Tokenize `[^fn]` / `[^en]` at a `[`. Anything else is `nok`, + * so the `[` falls through to the hint-label construct, GFM's footnote call, or + * a link label untouched. */ +function tokenizeNoteLabel( + effects: Effects, + ok: State, + nok: State, +): State | undefined { + let digits = 0; + return start; + + function start(code: Code): State | undefined { + effects.enter(NOTE_LABEL_TOKEN); + effects.consume(code); + return caret; + } + + function caret(code: Code): State | undefined { + if (code !== CODE.caret) return nok(code); + effects.consume(code); + return prefix; + } + + function prefix(code: Code): State | undefined { + const isPrefix = + code === CODE.lowerF || + code === CODE.upperF || + code === CODE.lowerE || + code === CODE.upperE; + if (!isPrefix) return nok(code); + effects.consume(code); + return afterPrefix; + } + + function afterPrefix(code: Code): State | undefined { + if (code !== CODE.lowerN && code !== CODE.upperN) return nok(code); + effects.consume(code); + return number; + } + + function number(code: Code): State | undefined { + if (code !== null && code >= CODE.zero && code <= CODE.nine) { + digits++; + effects.consume(code); + return number; + } + if (code !== CODE.rightBracket || digits === 0) return nok(code); + effects.consume(code); + effects.exit(NOTE_LABEL_TOKEN); + return ok; + } +} + +/** Every footnote definition identifier in the source, lowercased the way GFM + * normalizes them. */ +function definedIdentifiers(root: MutableNode): Set { + const out = new Set(); + visit(root, (node) => { + if (node.type === "footnoteDefinition" && node.identifier) { + out.add(node.identifier.toLowerCase()); + } + }); + return out; +} + +function resolveNoteLabels(node: MutableNode, defined: Set): void { + if (!Array.isArray(node.children)) return; + node.children = node.children.map((child) => + child.type === NOTE_LABEL_TOKEN ? resolveNoteLabel(child, defined) : child, + ); + for (const child of node.children) resolveNoteLabels(child, defined); +} + +/** `[^fn1]` → a GFM reference when the source defines it, else a reference to + * the document's existing note. */ +function resolveNoteLabel( + node: MutableNode, + defined: Set, +): MutableNode { + const label = (node.label ?? "").toLowerCase(); + if (defined.has(label)) { + return { + type: "footnoteReference", + identifier: label, + label, + position: node.position, + }; + } + const reference: ExistingNoteReference = { + type: "existingNoteReference", + kind: label.startsWith("fn") ? "footnote" : "endnote", + numericId: String(Number(label.slice(2))), + label, + }; + return { ...reference, position: node.position }; +} + +function visit(node: MutableNode, visitor: (node: MutableNode) => void): void { + visitor(node); + if (!Array.isArray(node.children)) return; + for (const child of node.children) visit(child, visitor); +} + +// The micromark/from-markdown extension types come from packages we don't +// depend on directly; derive them from remark-parse's `Data` augmentation and +// state the tokenizer's tiny structural surface locally. +type MicromarkExtension = NonNullable[number]; +type FromMarkdownExtension = Exclude< + NonNullable[number], + unknown[] +>; +type Code = number | null; +type State = (code: Code) => State | undefined; +type Effects = { + enter(type: string): unknown; + exit(type: string): unknown; + consume(code: Code): void; +}; + +const noteLabelSyntax = { + text: { + [CODE.leftBracket]: { + name: NOTE_LABEL_TOKEN, + add: "before", + tokenize: tokenizeNoteLabel, + }, + }, +} as unknown as MicromarkExtension; + +const noteLabelFromMarkdown = { + enter: { + [NOTE_LABEL_TOKEN](this: CompileContext, token: unknown) { + this.enter({ type: NOTE_LABEL_TOKEN }, token); + }, + }, + exit: { + [NOTE_LABEL_TOKEN](this: CompileContext, token: unknown) { + const node = this.stack[this.stack.length - 1]; + // `[^fn1]` → `fn1`. + if (node) node.label = this.sliceSerialize(token).slice(2, -1); + this.exit(token); + }, + }, +} as unknown as FromMarkdownExtension; + +type CompileContext = { + stack: MutableNode[]; + enter(node: { type: string }, token: unknown): void; + exit(token: unknown): void; + sliceSerialize(token: unknown): string; +}; + +type MutableNode = { + type: string; + label?: string; + identifier?: string; + children?: MutableNode[]; + position?: unknown; +}; + +declare module "mdast" { + interface PhrasingContentMap { + existingNoteReference: ExistingNoteReference; + } + interface RootContentMap { + existingNoteReference: ExistingNoteReference; + } +} diff --git a/tests/cli/footnotes.test.ts b/tests/cli/footnotes.test.ts index baccd89..9ae90cc 100644 --- a/tests/cli/footnotes.test.ts +++ b/tests/cli/footnotes.test.ts @@ -2,7 +2,7 @@ import { describe, expect, test } from "bun:test"; import { join } from "node:path"; import { Pkg } from "../../src/core/ast/document/package"; import { runCli, tempWorkspace } from "./harness"; -import { trackedKinds } from "./helpers"; +import { readDocumentXml, trackedKinds } from "./helpers"; type FootnoteRefRun = { type: string; @@ -1003,3 +1003,127 @@ describe("docx footnotes/endnotes — inline escape decoding (--text)", () => { expect(xml).toMatch(/[\s\S]*<\/w:ins>/); }); }); + +// A `[^fnN]` copied from `read` WITHOUT its `[^fnN]: …` definition (one line +// handed back to edit/insert) used to land as the literal characters "[^fn1]" — +// GFM only parses a reference that has a definition — and the note was lost on +// the next accept. It now addresses the document's existing note. +describe("a read-copied [^fnN] addresses the existing note", () => { + async function docWithFootnote(label: string): Promise { + const docPath = await freshDoc(label, [ + "Governing law is Delaware.", + "Notices go to the address above.", + ]); + await runCli( + "footnotes", + "add", + docPath, + "--anchor", + "Delaware", + "--text", + "See Del. Code.", + ); + return docPath; + } + + async function readLine(docPath: string, id: string): Promise { + const result = await runCli("read", docPath, "--from", id, "--to", id); + const line = result.stdout + .split("\n") + .find((candidate) => candidate.includes(``)); + return (line ?? "").replace(` `, ""); + } + + test("an untracked edit of the referencing paragraph keeps the note", async () => { + const docPath = await docWithFootnote("ref-edit-keep"); + const line = await readLine(docPath, "p0"); + expect(line).toBe("Governing law is Delaware[^fn1]."); + + const result = await runCli( + "edit", + docPath, + "--at", + "p0", + line.replace("Delaware", "New York"), + ); + expect(result.exitCode).toBe(0); + expect(await readLine(docPath, "p0")).toBe( + "Governing law is New York[^fn1].", + ); + expect(await listFootnotes(docPath)).toEqual([ + { id: "fn1", text: "See Del. Code." }, + ]); + const xml = await readDocumentXml(docPath); + expect(xml).not.toContain("[^fn1]"); + }); + + test("inserting a copy elsewhere clones the note (one reference per note)", async () => { + const docPath = await docWithFootnote("ref-insert-clone"); + const line = await readLine(docPath, "p0"); + await runCli("insert", docPath, "--after", "p1", line); + await runCli("delete", docPath, "--at", "p0"); + + const refs = await refRuns(docPath); + expect(refs.map((run) => run.id)).toEqual(["fn2"]); + const notes = await listFootnotes(docPath); + expect(notes.find((note) => note.id === "fn2")?.text).toBe( + "See Del. Code.", + ); + }); + + test("under tracking the copy clones, so accept and reject each keep one note", async () => { + const docPath = await docWithFootnote("ref-tracked"); + const line = await readLine(docPath, "p0"); + await runCli("track-changes", "on", docPath); + await runCli( + "edit", + docPath, + "--at", + "p0", + line.replace("Delaware", "New York"), + ); + + const accepted = `${docPath}.accepted.docx`; + await runCli("track-changes", "accept", docPath, "--all", "-o", accepted); + expect(await readLine(accepted, "p0")).toBe( + "Governing law is New York[^fn2].", + ); + expect((await refRuns(accepted)).map((run) => run.id)).toEqual(["fn2"]); + + const rejected = `${docPath}.rejected.docx`; + await runCli("track-changes", "reject", docPath, "--all", "-o", rejected); + expect(await readLine(rejected, "p0")).toBe( + "Governing law is Delaware[^fn1].", + ); + expect((await refRuns(rejected)).map((run) => run.id)).toEqual(["fn1"]); + }); + + test("a label naming no note is refused, not written literally", async () => { + const docPath = await docWithFootnote("ref-unknown"); + const result = await runCli("edit", docPath, "--at", "p1", "Text[^fn9]."); + expect(result.exitCode).toBe(2); + expect(result.stdout).toContain("[^fn9] names no footnote"); + expect(await readLine(docPath, "p1")).toBe( + "Notices go to the address above.", + ); + }); + + test("literal [^fn1] text is escaped by read and round-trips as text", async () => { + const docPath = await docWithFootnote("ref-literal"); + await runCli( + "edit", + docPath, + "--at", + "p1", + "--text", + "Literal [^fn1] here", + ); + const line = await readLine(docPath, "p1"); + expect(line).toBe("Literal \\[\\^fn1\\] here"); + + await runCli("edit", docPath, "--at", "p1", line); + const xml = await readDocumentXml(docPath); + expect(xml).toContain("Literal [^fn1] here"); + expect(await refRuns(docPath)).toHaveLength(1); + }); +}); diff --git a/tests/cli/read.test.ts b/tests/cli/read.test.ts index 5270a1e..424086d 100644 --- a/tests/cli/read.test.ts +++ b/tests/cli/read.test.ts @@ -272,9 +272,9 @@ describe("docx read (markdown)", () => { test("strict-profile: chart/smartart/drawing placeholders render", async () => { const out = await render(fixture("strict-profile.docx")); - expect(out).toContain("`[chart]`"); - expect(out).toContain("`[smartart]`"); - expect(out).toContain("`[drawing]`"); + expect(out).toContain(""); + expect(out).toContain(""); + expect(out).toContain(""); }); test("locator pins use HTML comments, not ", async () => { @@ -603,15 +603,52 @@ describe("docx read (markdown) color and highlight", () => { }); describe("docx read (markdown) chart / drawing placeholders", () => { - test("chart drawing renders as `[chart]`", async () => { + // A zero-width `docx:` hint, not content: the old inline-code `[chart]` + // placeholder was written back as literal text when a read line was copied + // into `edit`, and its 9 characters skewed every span read off the line. + test("chart drawing renders as a docx:chart hint", async () => { const out = await render(fixture("strict-profile.docx")); - expect(out).toContain("`[chart]`"); + expect(out).toContain(""); + expect(out).not.toContain("`[chart]`"); }); test("smartart and generic drawing placeholders render too", async () => { const out = await render(fixture("strict-profile.docx")); - expect(out).toContain("`[smartart]`"); - expect(out).toContain("`[drawing]`"); + expect(out).toContain(""); + expect(out).toContain(""); + }); + + test("a copied chart line edits back without literal text or a lost chart", async () => { + const docPath = join(tempWorkspace("chart-copy-back"), "chart.docx"); + await Bun.write(docPath, Bun.file(fixture("strict-profile.docx"))); + const before = await render(docPath); + const line = before + .split("\n") + .find((candidate) => candidate.includes("")); + expect(line).toBe(" "); + + // The hint leads the line: CommonMark would swallow the whole line as an + // HTML block, so the importer re-parses the text after it. + const result = await runCli( + "edit", + docPath, + "--at", + "p1", + "Figure 2: sales by region", + ); + expect(result.exitCode).toBe(0); + const after = await render(docPath); + expect(after).toContain( + "Figure 2: sales by region ", + ); + const xml = await readDocumentXml(docPath); + expect(xml).not.toContain("[chart]"); + expect(xml.match(/ match.locator)).toEqual(["p1:10-15"]); }); test("chart placeholders survive in JSON AST", async () => {