diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f15e63a..7c5be76 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -67,10 +67,10 @@ jobs: path: ./artifacts merge-multiple: true - name: Stage attribution files for release - # Per LGPL §4 we ship LGPL-3.0.txt + NOTICES alongside every binary; - # NOTICES is the canonical attribution surface for PDFium (BSD-3), - # jpeg-js (BSD-3), highlight.js (BSD-3), and libheif (LGPL). - run: cp LICENSE LGPL-3.0.txt NOTICES ./artifacts/ + # LICENSE + NOTICES ship alongside every binary; NOTICES is the + # canonical attribution surface the BSD-3 components require: PDFium, + # jpeg-js, and highlight.js. + run: cp LICENSE NOTICES ./artifacts/ - name: Stage install.sh for release # Shipped as a release asset (not served from a branch) so the documented install # command pins to a tag and is covered by SHA256SUMS below. It lives in the skill diff --git a/CLAUDE.md b/CLAUDE.md index 8859b5d..ba5c5ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,7 +35,7 @@ These invariants are NOT SUGGESTIONS. These MUST be followed. - **Changes must survive the write-read loop.** This tool is primarily designed for agent use, and specifically by weaker agents like Haiku. Thus, any change made should be retrievable for observation on the next turn in the AST. Whenever possible, it should be available in Markdown, which is the default read view. - **We do not handle all possible OOXML elements: so make in-place XML mutations, not AST/Markdown round-trips.** The AST or Markdown are just views. Mutate `XmlNode` refs, only emit fresh XML for inserted nodes. This way, elements we cannot parse will not be broken. See [src/core](src/core/CLAUDE.md). - **Most things, even those that are not easily representable, should be in Markdown.** For these, we (1) build comments into the markdown to give folks a better sense of what is going on, (2) make sure that these comment do not break the markdown and survive the roundtrip, and (3) provide intuitive verbs to deal with concepts. If some structure is truly not representable in Markdown, we nudge agents to render the pages and look. We should guarantee that the things that we're not doing the above for have no possible representation. -- **Authoring has two channels: parsed and literal.** Markdown is the DEFAULT: `edit`/`insert` take their content as a positional argument (`docx edit FILE --at p3 "Net **30** days"`), which is exactly `--markdown` (kept as an alias). New markdown content must look like the document it lands in: a `- item` beside a list joins THAT list (`continueList` in `core/lists/continue-list.tsx`), and whatever the replaced paragraph / neighbor doesn't format falls back FIRST to the look that was AT that position (for `edit`, the replaced paragraph's rendered face + explicit size), THEN to the document's DOMINANT face/size — the `docx:base` `read` shows (`matchDocumentLook` in `core/dominant-formatting.ts`) — never the new style chain's default. `--markdown`/`--from` parse GFM (+ math + CriticMarkup + inline HTML); `--text-file` (on `insert` and `create`, `-` = stdin) is the PARSER-FREE channel — every character lands verbatim, each newline starts a new paragraph. It exists because GFM corruption of literal prose isn't always escapable: a bare URL autolinks with NO escape sequence at all, and CriticMarkup eats `{++…++}` regardless of backslashes (`*x*`, `3.`, `[t](u)` ARE backslash-escapable, but the agent — often Haiku — must get the whole dialect right, and the field evidence is they don't). So "put this exact text in, untouched" gets its own flag rather than an escaping burden on the weakest actor. `--text` stays inline single-paragraph; both it and the bulk-literal `--text-file` insert markdown-looking content VERBATIM (`--text "**bold**"` writes literal `**` — use `--markdown` to parse). The only value `--text` still refuses is a shell-gutted currency amount (`rejectShellMangledValue`: `"$300"` → `.00`), which is never intentional. **Every inline `--text`/`--markdown` argv value gets its `\n`/`\t`/`\r`(`\r\n`) decoded at the CLI ingress** (`decodeInlineEscapes` in `cli/parse-helpers.ts`) — a weak agent writes `--text "a\nb"` for a line break, but bash double-quotes hand the CLI the literal backslash-n, which used to land verbatim in the run. It's applied by EVERY inline authoring surface, not just the body: `edit`/`insert`/`create`, `footnotes`/`endnotes add`+`edit` (`--text` and inline `--markdown`), `comments add`+`reply` (`--text`), `headers`/`footers set` (`--text`), and the `find`/`replace` positionals (QUERY/PATTERN/REPLACEMENT) — where a decoded `\n` is ALSO the gate that routes the command to the cross-paragraph matcher (a `\n` in a find/replace pattern matches an in-paragraph line break OR the boundary between consecutive paragraphs — in the body or within one table cell, never across a cell wall; a `\n` in a replace REPLACEMENT means a paragraph mark, editor-style — single-line replacement across a boundary MERGES paragraphs, replacement `\n` SPLITS; untracked only, refuses under tracking). Decoded whitespace flows through the same real-character handling everything uses — the shared **`textToRunElements`** emitter in `core/blocks.tsx` (`textToRuns` + `RunElement`, nulls filtered) turns a real `\n`/`\t` into ``/``, so the note/comment/marginal emitters (which previously emitted a single raw `` that swallowed a newline) now render breaks just like a body paragraph; the markdown block splitter handles the `--markdown` side. Plain single-line text still collapses to one `` run, byte-identical to the old shape. ONLY those whitespace escapes decode: a `\`-before-punctuation (markdown's own `\*`/`\[`/`\.`, and a literal `\\`) passes through UNTOUCHED, so decoding can't corrupt markdown. The decode is INLINE-argv-only — `--text-file`/`--markdown-file`/`--from` read real characters from a file and `--batch` JSONL is already `JSON.parse`-decoded, so none are re-decoded; `--text-file` therefore stays the one channel where a bare `\n` really means backslash-n. +- **Authoring has two channels: parsed and literal.** Markdown is the DEFAULT: `edit`/`insert` take their content as a positional argument (`docx edit FILE --at p3 "Net **30** days"`), which is exactly `--markdown` (kept as an alias). New markdown content must look like the document it lands in: a `- item` beside a list joins THAT list (`continueList` in `core/lists/continue-list.tsx`), and whatever the replaced paragraph / neighbor doesn't format falls back FIRST to the look that was AT that position (for `edit`, the replaced paragraph's rendered face + explicit size), THEN to the document's DOMINANT face/size — the `docx:base` `read` shows (`matchDocumentLook` in `core/dominant-formatting.ts`) — never the new style chain's default. `--markdown`/`--from` parse GFM (+ math + CriticMarkup + inline HTML); `--text-file` (on `insert` and `create`, `-` = stdin) is the PARSER-FREE channel — every character lands verbatim, each newline starts a new paragraph. It exists because GFM corruption of literal prose isn't always escapable: a bare URL autolinks with NO escape sequence at all, and CriticMarkup eats `{++…++}` regardless of backslashes (`*x*`, `3.`, `[t](u)` ARE backslash-escapable, but the agent — often Haiku — must get the whole dialect right, and the field evidence is they don't). So "put this exact text in, untouched" gets its own flag rather than an escaping burden on the weakest actor. `--text` stays inline single-paragraph; both it and the bulk-literal `--text-file` insert markdown-looking content VERBATIM (`--text "**bold**"` writes literal `**` — use `--markdown` to parse). The only value `--text` still refuses is a shell-gutted currency amount (`rejectShellMangledValue`: `"$300"` → `.00`), which is never intentional. **Every inline `--text`/`--markdown` argv value gets its `\n`/`\t`/`\r`(`\r\n`) decoded at the CLI ingress** (`decodeInlineEscapes` in `cli/parse-helpers.ts`) — a weak agent writes `--text "a\nb"` for a line break, but bash double-quotes hand the CLI the literal backslash-n, which used to land verbatim in the run. It's applied by EVERY inline authoring surface, not just the body: `edit`/`insert`/`create`, `footnotes`/`endnotes add`+`edit` (`--text` and inline `--markdown`), `comments add`+`reply` (`--text`), `headers`/`footers set` (`--text`), and the `find`/`replace` positionals (QUERY/PATTERN/REPLACEMENT) — where a decoded `\n` is ALSO the gate that routes the command to the cross-paragraph matcher (a `\n` in a find/replace pattern matches an in-paragraph line break OR the boundary between consecutive paragraphs — in the body or within one table cell, never across a cell wall; a `\n` in a replace REPLACEMENT means a paragraph mark, editor-style — single-line replacement across a boundary MERGES paragraphs, replacement `\n` SPLITS; untracked only, refuses under tracking). Decoded whitespace flows through the same real-character handling everything uses — the shared **`textToRunElements`** emitter in `core/blocks.tsx` (`textToRuns` + `RunElement`, nulls filtered) turns a real `\n`/`\t` into ``/``, so the note/comment/marginal emitters (which previously emitted a single raw `` that swallowed a newline) now render breaks just like a body paragraph; the markdown block splitter handles the `--markdown` side. Plain single-line text still collapses to one `` run, byte-identical to the old shape. ONLY those whitespace escapes decode: a `\`-before-punctuation (markdown's own `\*`/`\[`/`\.`, and a literal `\\`) passes through UNTOUCHED, so decoding can't corrupt markdown. One more carve-out on the markdown side: a link/image DESTINATION (`[t](dest)`, `![a](dest)`) is never decoded (`decodeInlineMarkdownEscapes`, which every inline `--markdown` ingress uses). A destination can't hold whitespace, so a decoded `\t` only breaks it, and Windows paths are full of them: `![Logo](C:\Users\me\tests\logo.png)` used to land as literal text at exit 0. The decode is INLINE-argv-only — `--text-file`/`--markdown-file`/`--from` read real characters from a file and `--batch` JSONL is already `JSON.parse`-decoded, so none are re-decoded; `--text-file` therefore stays the one channel where a bare `\n` really means backslash-n. - **Comments are never anything but hints.** Every `` comment `read` writes into Markdown — locators (``) and the structural visibility annotations (`docx:section` — rendered at the section's START with an `applies-to="pX..pY (below)"` scope on deviating sections so the "which side of the boundary?" off-by-one is explicit, `docx:page`, `docx:table` (widths/borders + the `tables format` properties align/style/cell-margins/repeat-header/row-heights), `docx:cell` merge/shading/vAlign/halign/borders/margins, `docx:track-changes on|off pending=N` ALWAYS at the head (the one hint that states its default too — weak agents can't reliably read "no hint" as "off," and a wrong tracking guess is high-cost, so it's stated outright; `pending=N` likewise always, `pending=0` included, since the default accepted view hides a pending deletion outright, plus a compact `(hdr0: 1, fn2: 1)` when some pending changes live outside the body), `docx:header`/`docx:footer` for page headers/footers (content in a `text` attr, fields as `{page}`/`{date}`/… tokens, a `pending="tcN …"` attr naming the part's tracked changes in every view, and the text following the view — CriticMarkup under `--current`, safe inside the comment because `htmlAttr` escapes every `>` so a deletion's `--` can never form `-->` — authored via `docx headers`/`docx footers`; a UNIFORM marginal rides the head, a marginal that DIFFERS by section renders at that section's START, next to the content it governs), `docx:textbox tbxN anchor="pM"` … `docx:textbox-end tbxN` bracketing a text box's story, printed right after the paragraph that anchors it (the story's own paragraphs carry bare `tbxN:pK` locators), `docx:layout` flagging tab-aligned content that wraps in render — inside a multi-column section, OR a line whose trailing content rides a right-edge LEFT tab so a long value overflows the margin (the résumé `San`/`Francisco` split); the latter is also consolidated into ONE top-of-read `fix-all` summary naming the single-call cure `edit --at pN-pM --tabs right` (a render-only break Markdown can't show), and future image/style hints, all via `formatNote` in `cli/read/annotations.ts`) — is a READ-TIME VISIBILITY HINT. **Naming rule: a BARE comment is a locator (an address you pass to `--at`: ``, ``; a plain empty cell uses `` so it can be filled directly); a `docx:TYPE` comment is docx-cli METADATA** (structure/formatting). Metadata never rides a bare locator — it's its own `docx:` annotation, which may carry the relevant locator as a bare leading token (``). So `docx:` is greppable as "everything docx-cli added beyond addressing," and locators stay short. The Markdown importer DROPS them all (`walker.tsx`'s `case "html"` returns `[]`; the inline walker drops inline comments); they NEVER drive reconstruction or carry data through `read → create`. A comment can't silently corrupt the round-trip or smuggle structure back — what an agent sees is a hint, nothing more. The lossless view is `read --ast`; in-place `edit` preserves structure; the authoring verbs (`docx sections`, `docx tables …`) change it. Emit hints **deviation-only** (only what differs from the document default — even columns, default geometry, `single` borders, `align=left` emit nothing), with TWO deliberate exceptions that always emit even at their default: `docx:section` (every section gets a marker so the `sN` is always addressable) and `docx:track-changes` (states `off` and `pending=0` too, so a weak agent never infers state from a missing hint). Run FORMATTING is the one thing that's Markdown-representable and DOES round-trip, but it rides HTML ELEMENTS (``/``/``/``, via `gatherHtmlSpans`), NOT comments. Even `` (the dominant font/size note) is a hint: `read` omits the baseline from runs and shows the note so an agent can match it, but the importer drops it — a full `read → create` rebuild falls back to the template docDefaults (Calibri 11pt) for the dominant font/size. The footnote-shaped read annotations are hints too: the `[^tcN]` (`--current`) and `[^cN]` (`--comments`) labels and their definition lines are DROPPED on import (`remarkReadHintLabels` in `core/markdown/read-hint-labels.ts`; real notes always read as `[^fnN]`/`[^enN]`), so a pasted `--current` line behaves exactly like its label-free CriticMarkup. No comment, anywhere, is parse-back. - **Everything we parse MUST be represented in the AST**. Markdown is always either a subset or equal to what is in the AST, never less than it. `RUN_BEARING_WRAPPER_TAGS` in `src/core/parser/run-ops.ts` is the AST↔XML offset bridge; every offset-aware walker reads it. See [src/core](src/core/CLAUDE.md). - **`` is transparent at EVERY level, and text boxes are first-class.** ECMA-376 Part 3 lets a producer wrap a block, a paragraph's runs, or a run's children in a markup-compatibility wrapper and nest wrappers inside branches; Word writes every modern shape/text box that way (`wps` under ``, VML under ``, SAME story in both). Every walker resolves it through `alternateContentBranch` in `src/core/mc.ts` (first Choice, else Fallback) — the reader at block/paragraph/run level, the offset walkers via `wrapperContent(node)` in `parser/run-ops.ts` (NEVER `node.children` for a wrapper — that is what keeps a multi-Choice wrapper counted exactly once), and the text-box collector while descending a shape. Dropping the wrapper was issue #4: `replace --all` reported 2 and exited 0 while a third "Acme" sat in a text box a human reads first. A `` story is a block container like a cell (`TextBoxRun` on the anchor paragraph, `tbxN:pK` ids, `blockReferences` parents inside the Choice copy), so `find`/`replace`/`edit`/`insert`/`delete`/`comments` reach it by default; `Document.save` (and the raw gate, BEFORE validating) mirrors each Choice story onto its Fallback twin (`syncTextBoxFallbacks`, post-order so nested boxes sync inner-first) — verified against Word for Mac 365: Word regenerates a box's Fallback when IT edits the box and leaves an untouched box's Fallback verbatim (a stale one stays stale forever), so mirroring on our writes is exactly its behavior. `wc` counts what `read` shows — text boxes included, everywhere. **Word discards comments and footnotes/endnotes inside a text box on its next save** (verified: the markers vanish from the story and the comment from comments.xml, regardless of the Fallback), so `comments add` (`--at` AND `--anchor`) and `footnotes`/`endnotes add` REFUSE a `tbxN` target with `UNSUPPORTED` + a pointer to the anchor paragraph; tracked changes and hyperlinks inside a box survive Word and are allowed. The box rides its anchor RUN: EVERY whole-paragraph `edit` lifts object-bearing runs (drawing/pict/object/AlternateContent) out and puts them back — `--text` around the token diff (`extractObjectRuns`), `--markdown`/`--runs` around the node replace (`liftObjectRuns` → `reattachObjectRuns` in `track-changes/replace.tsx`, untracked carriers even under tracking; a `--markdown` that re-states an image via `![alt](sha256.ext)` drops the lifted DRAWING so `img0` isn't duplicated, while a text box always returns), a `replace` on the anchor run's text keeps a trailing zero-width shape (`sliceRun`), and a tracked delete of the anchor wraps the run in `` WITHOUT rewriting the story's `` (`TextBoxRun.trackedChange` hides it in the accepted view; `iterateBlocks({ view })`). An UNTRACKED removal of the anchor paragraph (`delete --at pN`, `edit --at pN --text ""`) re-anchors the box on the neighboring paragraph — previous, else next — via `relocateObjectRuns` in `track-changes/replace.tsx`; deleting the invoice's Notes line used to take the stamp box with it, silently. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3dd0bfd..89f3d55 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,7 +54,7 @@ src/ jsx/ # Fragment, namespace() + tag namespaces (w, a, wp, pic, cp, dc, …), auto-runtime ast/ # types + DocView + XML→AST reader; text.ts (paragraph helpers), sym.ts ( decode) locators/ # parse "p3:5-20" → Locator + resolve to refs - image/ # drawing.tsx ( + addImagePart + collectImageRuns), source.ts (load path/data:/http + HEIC→JPEG), formats.ts (one mime↔ext table) + image/ # drawing.tsx ( + addImagePart + collectImageRuns), source.ts (load path/data:/http; refuses HEIC with a conversion hint), formats.ts (one mime↔ext table) equation/ # latexToOmml / ommlToLatex bidirectional (temml + own MathML↔OMML adapter) markdown/ # MarkdownImport lens: remark-parse + remark-gfm + remark-math + CriticMarkup plugin → walker → emitters raw/ # Raw lens: fragment gate pipeline (element-order, namespaces+dcx:raw marker, reference audit) + libxml2-wasm XSD engine over bundled ECMA-376 schemas (schemas/) @@ -90,7 +90,6 @@ See [CLAUDE.md](CLAUDE.md) and the nested `CLAUDE.md` files under `src/` for con - **Diff**: [`diff`](https://www.npmjs.com/package/diff) (jsdiff, BSD-3) backs `docx diff` — unified line hunks + HTML-aware word-level refinement of the read view - **Math**: [`temml`](https://www.npmjs.com/package/temml) (MIT) compiles LaTeX → MathML; an in-house MathML → OMML adapter handles the OOXML side bidirectionally - **Render**: [`@hyzyla/pdfium`](https://www.npmjs.com/package/@hyzyla/pdfium) (MIT wrapper; PDFium WASM redistributed under BSD-3-Clause) for the PDF → PNG/JPG step, plus [`pngjs`](https://www.npmjs.com/package/pngjs) / [`jpeg-js`](https://www.npmjs.com/package/jpeg-js) for image encoding -- **Images**: [`heic-convert`](https://www.npmjs.com/package/heic-convert) (wasm libheif, LGPL-3.0) transcodes HEIC/HEIF input to JPEG on insert - **Validation**: [`libxml2-wasm`](https://www.npmjs.com/package/libxml2-wasm) (MIT, libxml2 compiled to WASM) + the bundled ECMA-376 5th-edition transitional XSDs power `docx validate` and the `docx raw` schema gate — in-process, ~55 ms one-time schema compile, single-digit ms per document - **Quality**: Biome + Knip + tsc; LibreOffice headless for round-trip integration tests - **Standard**: ECMA-376 Part 1 §17 (WordprocessingML), Transitional profile diff --git a/LGPL-3.0.txt b/LGPL-3.0.txt deleted file mode 100644 index 0a04128..0000000 --- a/LGPL-3.0.txt +++ /dev/null @@ -1,165 +0,0 @@ - GNU LESSER GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - - This version of the GNU Lesser General Public License incorporates -the terms and conditions of version 3 of the GNU General Public -License, supplemented by the additional permissions listed below. - - 0. Additional Definitions. - - As used herein, "this License" refers to version 3 of the GNU Lesser -General Public License, and the "GNU GPL" refers to version 3 of the GNU -General Public License. - - "The Library" refers to a covered work governed by this License, -other than an Application or a Combined Work as defined below. - - An "Application" is any work that makes use of an interface provided -by the Library, but which is not otherwise based on the Library. -Defining a subclass of a class defined by the Library is deemed a mode -of using an interface provided by the Library. - - A "Combined Work" is a work produced by combining or linking an -Application with the Library. The particular version of the Library -with which the Combined Work was made is also called the "Linked -Version". - - The "Minimal Corresponding Source" for a Combined Work means the -Corresponding Source for the Combined Work, excluding any source code -for portions of the Combined Work that, considered in isolation, are -based on the Application, and not on the Linked Version. - - The "Corresponding Application Code" for a Combined Work means the -object code and/or source code for the Application, including any data -and utility programs needed for reproducing the Combined Work from the -Application, but excluding the System Libraries of the Combined Work. - - 1. Exception to Section 3 of the GNU GPL. - - You may convey a covered work under sections 3 and 4 of this License -without being bound by section 3 of the GNU GPL. - - 2. Conveying Modified Versions. - - If you modify a copy of the Library, and, in your modifications, a -facility refers to a function or data to be supplied by an Application -that uses the facility (other than as an argument passed when the -facility is invoked), then you may convey a copy of the modified -version: - - a) under this License, provided that you make a good faith effort to - ensure that, in the event an Application does not supply the - function or data, the facility still operates, and performs - whatever part of its purpose remains meaningful, or - - b) under the GNU GPL, with none of the additional permissions of - this License applicable to that copy. - - 3. Object Code Incorporating Material from Library Header Files. - - The object code form of an Application may incorporate material from -a header file that is part of the Library. You may convey such object -code under terms of your choice, provided that, if the incorporated -material is not limited to numerical parameters, data structure -layouts and accessors, or small macros, inline functions and templates -(ten or fewer lines in length), you do both of the following: - - a) Give prominent notice with each copy of the object code that the - Library is used in it and that the Library and its use are - covered by this License. - - b) Accompany the object code with a copy of the GNU GPL and this license - document. - - 4. Combined Works. - - You may convey a Combined Work under terms of your choice that, -taken together, effectively do not restrict modification of the -portions of the Library contained in the Combined Work and reverse -engineering for debugging such modifications, if you also do each of -the following: - - a) Give prominent notice with each copy of the Combined Work that - the Library is used in it and that the Library and its use are - covered by this License. - - b) Accompany the Combined Work with a copy of the GNU GPL and this license - document. - - c) For a Combined Work that displays copyright notices during - execution, include the copyright notice for the Library among - these notices, as well as a reference directing the user to the - copies of the GNU GPL and this license document. - - d) Do one of the following: - - 0) Convey the Minimal Corresponding Source under the terms of this - License, and the Corresponding Application Code in a form - suitable for, and under terms that permit, the user to - recombine or relink the Application with a modified version of - the Linked Version to produce a modified Combined Work, in the - manner specified by section 6 of the GNU GPL for conveying - Corresponding Source. - - 1) Use a suitable shared library mechanism for linking with the - Library. A suitable mechanism is one that (a) uses at run time - a copy of the Library already present on the user's computer - system, and (b) will operate properly with a modified version - of the Library that is interface-compatible with the Linked - Version. - - e) Provide Installation Information, but only if you would otherwise - be required to provide such information under section 6 of the - GNU GPL, and only to the extent that such information is - necessary to install and execute a modified version of the - Combined Work produced by recombining or relinking the - Application with a modified version of the Linked Version. (If - you use option 4d0, the Installation Information must accompany - the Minimal Corresponding Source and Corresponding Application - Code. If you use option 4d1, you must provide the Installation - Information in the manner specified by section 6 of the GNU GPL - for conveying Corresponding Source.) - - 5. Combined Libraries. - - You may place library facilities that are a work based on the -Library side by side in a single library together with other library -facilities that are not Applications and are not covered by this -License, and convey such a combined library under terms of your -choice, if you do both of the following: - - a) Accompany the combined library with a copy of the same work based - on the Library, uncombined with any other library facilities, - conveyed under the terms of this License. - - b) Give prominent notice with the combined library that part of it - is a work based on the Library, and explaining where to find the - accompanying uncombined form of the same work. - - 6. Revised Versions of the GNU Lesser General Public License. - - The Free Software Foundation may publish revised and/or new versions -of the GNU Lesser General Public License from time to time. Such new -versions will be similar in spirit to the present version, but may -differ in detail to address new problems or concerns. - - Each version is given a distinguishing version number. If the -Library as you received it specifies that a certain numbered version -of the GNU Lesser General Public License "or any later version" -applies to it, you have the option of following the terms and -conditions either of that published version or of any later version -published by the Free Software Foundation. If the Library as you -received it does not specify a version number of the GNU Lesser -General Public License, you may choose any version of the GNU Lesser -General Public License ever published by the Free Software Foundation. - - If the Library as you received it specifies that a proxy can decide -whether future versions of the GNU Lesser General Public License shall -apply, that proxy's public statement of acceptance of any version is -permanent authorization for you to choose that version for the -Library. diff --git a/NOTICES b/NOTICES index 166fa2c..f520c2a 100644 --- a/NOTICES +++ b/NOTICES @@ -4,13 +4,10 @@ This file lists the third-party software docx-cli redistributes (statically linked into the published binaries via `bun build --compile`, or bundled into `dist/index.js`), along with the attribution each license requires. -docx-cli itself is licensed under the MIT License (see LICENSE). Linking -permissively-licensed libraries (MIT / Apache-2.0 / BSD-3-Clause) leaves -that license unchanged. Linking the LGPL-3.0 libheif-js library does NOT -make docx-cli copyleft either — LGPL §4 permits use by works under other -licenses provided the conditions in this NOTICES file (acknowledgement + -LGPL-3.0 text + the right to swap in a modified libheif by rebuilding -docx-cli from source) are met. +docx-cli itself is licensed under the MIT License (see LICENSE). Every +library it redistributes is permissively licensed (MIT / Apache-2.0 / +BSD-3-Clause, or a dual license under which docx-cli elects one of those), +which leaves that license unchanged. Whenever a runtime dependency is added or changed in package.json, re-audit whether the new entry needs an attribution here. @@ -59,32 +56,6 @@ not included verbatim inside the @hyzyla/pdfium npm tarball; the wrapper's own LICENSE.md covers only the MIT-licensed wrapper code.) -================================================================================ -libheif-js (HEIC/HEIF input transcoding for `docx insert --image`) -================================================================================ - -License: LGPL-3.0-only (see LGPL-3.0.txt at the root of this distribution). -Copyright: libheif authors, https://github.com/strukturag/libheif. - -docx-cli links libheif-js statically into its compiled binaries (`dist/docx` -and the GitHub Release artifacts). LGPL §4 imposes the following obligations -that this notice satisfies: - - (a) Acknowledged here: libheif-js is incorporated into docx-cli's binaries. - (b) The full LGPL-3.0 text is distributed as LGPL-3.0.txt alongside this - file. - (c) docx-cli's own source code is publicly available at - https://github.com/kklimuk/docx-cli — anyone may rebuild the - binaries with a modified libheif by replacing the `libheif-js` / - `heic-convert` dependency in package.json and running - `bun run build:binary`. libheif's source is at - https://github.com/strukturag/libheif. - -Linking LGPL-3.0 code does NOT make docx-cli a derivative work for the -purpose of LGPL §3 — docx-cli remains under the MIT License declared in -LICENSE. - - ================================================================================ BSD-3-Clause components ================================================================================ @@ -174,7 +145,6 @@ Other dependencies (license elections + MIT components) remark-math MIT unist-util-visit MIT mdast-util-math MIT - heic-convert MIT (wrapper around LGPL libheif-js, attributed above) Each package's own LICENSE file is distributed alongside its source inside `node_modules//` for anyone consuming docx-cli via Bun's dependency diff --git a/README.md b/README.md index dc02424..f9ea06b 100644 --- a/README.md +++ b/README.md @@ -168,4 +168,4 @@ Tested on every commit: about 1,900 unit tests, a LibreOffice round-trip of ever ## License -MIT. The standalone binaries statically link libheif (LGPL-3.0) for HEIC image support; see [NOTICES](NOTICES) for the attributions and your rights under the LGPL. +MIT. Every library docx-cli depends on is permissively licensed; [NOTICES](NOTICES) carries the attributions for the ones bundled into the npm package and the binaries. diff --git a/REFERENCE.md b/REFERENCE.md index 263489d..abad4e6 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -570,7 +570,7 @@ docx track-changes apply doc.docx --accept rev0 --reject rev1 # finalize: acce **Text boxes and markup compatibility.** Word writes every modern shape and text box as an `` pair (a `wps` shape under ``, a legacy VML copy under ``), and the markup-compatibility spec lets that wrapper sit at any level — around a block, a paragraph's runs, or a run's children — and nest. The reader resolves the wrapper wherever it appears (first `Choice`, else `Fallback`), so nothing under it is dropped, and every offset walker descends the same branch (a wrapper with several `Choice`s counts exactly the one the reader read); a `` story surfaces as a `textBox` run whose blocks are addressable `tbxN:pK` paragraphs. Edits land in the `Choice` copy (the one Word reads) and a text box's `Fallback` twin is mirrored from it on save, exactly as Word does when it edits the box. A wrapper that is not a text box has no mirrorable story, so an edit inside it changes the `Choice` only — the same two-faced file Word leaves when it preserves a fallback it didn't regenerate. -**Rich content.** Images insert from a path, `data:` URI, or `http(s)` URL (bounded fetch; HEIC→JPEG transcode; SVG sanitized; non-public/metadata addresses refused at every redirect hop). Equations round-trip OOXML `` ↔ LaTeX (reconstructed, not legacy plaintext) — authored via temml (LaTeX→MathML) plus an in-house MathML→OMML adapter (the math path pulls in no LGPL code). Code blocks emit one `CodeBlock`-styled paragraph per line with optional lowlight syntax highlighting (37 bundled languages); they collapse back to a GFM fenced block on read. GFM task lists round-trip Word's checkbox content control (and the Word-for-Web Wingdings-glyph variant), surfacing as `taskState` in the AST. Tables operate on a merge-aware logical grid so `gridSpan`/`vMerge` cells map onto physical ``, and structural edits refuse to bisect an existing merge. +**Rich content.** Images insert from a path, `data:` URI, or `http(s)` URL (bounded fetch; SVG sanitized; non-public/metadata addresses refused at every redirect hop). HEIC/HEIF input is refused, by extension, MIME type, or file header, with an `IMAGE_SOURCE` error whose hint gives the conversion command (`sips` on macOS, `heif-convert` or ImageMagick on Linux): Word can't display HEIC, and docx-cli bundles no HEVC decoder. Equations round-trip OOXML `` ↔ LaTeX (reconstructed, not legacy plaintext) — authored via temml (LaTeX→MathML) plus an in-house MathML→OMML adapter. Code blocks emit one `CodeBlock`-styled paragraph per line with optional lowlight syntax highlighting (37 bundled languages); they collapse back to a GFM fenced block on read. GFM task lists round-trip Word's checkbox content control (and the Word-for-Web Wingdings-glyph variant), surfacing as `taskState` in the AST. Tables operate on a merge-aware logical grid so `gridSpan`/`vMerge` cells map onto physical ``, and structural edits refuse to bisect an existing merge. **Markdown dialect.** `create --from`, `insert/edit --markdown`, and the note bodies all parse the same GFM + math + CriticMarkup + inline-HTML-formatting dialect (remark + remark-gfm + remark-math + an in-house inline-surgery transform), composing the existing OOXML emitters. `read` emits a compatible dialect, so the read → edit → write loop round-trips (lossless for paragraphs, lists, and nested blockquotes; code/tables/math/headings inside a blockquote intentionally escape to top level on import). `read --ast` is the fully lossless JSON form. The `[^tcN]` and `[^cN]` labels `read --current` / `--comments` add (and their definition lines) are read-time hints like the `` comments: the importer drops them, so pasting a `--current` line back applies its CriticMarkup exactly as if the labels weren't there, and never mints a footnote. Real footnotes read as `[^fnN]`/`[^enN]`; name your own anything but `tcN`/`cN`. diff --git a/SECURITY.md b/SECURITY.md index 24c43f0..0bbea9f 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -98,7 +98,7 @@ Offline, bootstrap keeps the working binary and exits 0. **Air-gapped or mirrored installs.** 1. On a connected machine, download `docx-` and `SHA256SUMS` from a tagged release and verify them with `shasum -a 256 -c SHA256SUMS --ignore-missing`. -2. Copy the binary, plus `NOTICES` and `LGPL-3.0.txt` from the same release, to each machine (or your software-distribution tool), mark the binary executable, and put it on `PATH`. +2. Copy the binary, plus `LICENSE` and `NOTICES` from the same release, to each machine (or your software-distribution tool), mark the binary executable, and put it on `PATH`. Nothing in docx-cli phones home after that. @@ -115,4 +115,4 @@ Nothing in docx-cli phones home after that. **What the checksum does and doesn't prove.** `SHA256SUMS` is published in the same release as the binaries. It protects against a corrupted or swapped download, not against a compromised release. For higher assurance, build from source (`bun run build:binary`) at a reviewed tag. -**License.** MIT. The standalone binaries statically link libheif (LGPL-3.0) for HEIC support; [NOTICES](NOTICES) lists the attributions and your LGPL rights, and redistributing the binary internally is fine with those files alongside it. +**License.** MIT. Every library docx-cli depends on is permissively licensed; [NOTICES](NOTICES) lists the attributions for the ones bundled into the binary, and redistributing the binary internally is fine with `LICENSE` and `NOTICES` alongside it. diff --git a/bun.lock b/bun.lock index d368ed7..8ea43e3 100644 --- a/bun.lock +++ b/bun.lock @@ -9,7 +9,6 @@ "diff": "^9.0.0", "fast-xml-builder": "^1.1.6", "fast-xml-parser": "^5.7.2", - "heic-convert": "^2.1.0", "jpeg-js": "^0.4.4", "jszip": "^3.10.1", "libxml2-wasm": "^0.7.1", @@ -27,7 +26,6 @@ "@biomejs/biome": "^2.4.10", "@types/bun": "^1.3.13", "@types/hast": "^3.0.4", - "@types/heic-convert": "^2.1.0", "@types/jpeg-js": "^0.3.7", "@types/mdast": "^4.0.4", "@types/pngjs": "^6.0.5", @@ -158,8 +156,6 @@ "@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], - "@types/heic-convert": ["@types/heic-convert@2.1.0", "", {}, "sha512-Cf5Sdc2Gm2pfZ0uN1zjj35wcf3mF1lJCMIzws5OdJynrdMJRTIRUGa5LegbVg0hatzOPkH2uAf2JRjPYgl9apg=="], - "@types/jpeg-js": ["@types/jpeg-js@0.3.7", "", { "dependencies": { "jpeg-js": "*" } }, "sha512-v3FEzt2TXOOIgvwwOqMoeEhqYqUoK4wcV/K7JbKj8CzwDE9sSuor8EZpSJKaOnuEtyO0W+Nd2uKk8MbtusYpEg=="], "@types/katex": ["@types/katex@0.16.8", "", {}, "sha512-trgaNyfU+Xh2Tc+ABIb44a5AYUpicB3uwirOioeOkNPPbmgRNtcWyDeeFRzjPZENO9Vq8gvVqfhaaXWLlevVwg=="], @@ -212,10 +208,6 @@ "get-tsconfig": ["get-tsconfig@4.14.0", "", { "dependencies": { "resolve-pkg-maps": "^1.0.0" } }, "sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA=="], - "heic-convert": ["heic-convert@2.1.0", "", { "dependencies": { "heic-decode": "^2.0.0", "jpeg-js": "^0.4.4", "pngjs": "^6.0.0" } }, "sha512-1qDuRvEHifTVAj3pFIgkqGgJIr0M3X7cxEPjEp0oG4mo8GFjq99DpCo8Eg3kg17Cy0MTjxpFdoBHOatj7ZVKtg=="], - - "heic-decode": ["heic-decode@2.1.0", "", { "dependencies": { "libheif-js": "^1.19.8" } }, "sha512-0fB3O3WMk38+PScbHLVp66jcNhsZ/ErtQ6u2lMYu/YxXgbBtl+oKOhGQHa4RpvE68k8IzbWkABzHnyAIjR758A=="], - "highlight.js": ["highlight.js@11.11.1", "", {}, "sha512-Xwwo44whKBVCYoliBQwaPvtd/2tYFkRQtXDWj1nackaV2JPXx3L0+Jvd8/qCJ2p+ML0/XVkJ2q+Mr+UVdpJK5w=="], "husky": ["husky@9.1.7", "", { "bin": { "husky": "bin.js" } }, "sha512-5gs5ytaNjBrh5Ow3zrvdUUY+0VxIuWVL4i9irt6friV+BqdCfmV11CQTWMiBYWHbXhco+J1kHfTOUkePhCDvMA=="], @@ -238,8 +230,6 @@ "knip": ["knip@6.11.0", "", { "dependencies": { "fdir": "^6.5.0", "formatly": "^0.3.0", "get-tsconfig": "4.14.0", "jiti": "^2.6.0", "minimist": "^1.2.8", "oxc-parser": "^0.128.0", "oxc-resolver": "^11.19.1", "picomatch": "^4.0.4", "smol-toml": "^1.6.1", "strip-json-comments": "5.0.3", "tinyglobby": "^0.2.16", "unbash": "^3.0.0", "yaml": "^2.8.2", "zod": "^4.1.11" }, "bin": { "knip": "bin/knip.js", "knip-bun": "bin/knip-bun.js" } }, "sha512-84PTlN8Q5smLpTbzs8smTVh8PMbTDXtw0tFksXq/m6auGFC/KSzJykKFmnYh3As38kiWDkoDBvdTTyKk5M1TAQ=="], - "libheif-js": ["libheif-js@1.19.8", "", {}, "sha512-vQJWusIxO7wavpON1dusciL8Go9jsIQ+EUrckauFYAiSTjcmLAsuJh3SszLpvkwPci3JcL41ek2n+LUZGFpPIQ=="], - "libxml2-wasm": ["libxml2-wasm@0.7.1", "", {}, "sha512-aZpJJL/j6T3D+5TmhG4D0ylR3mN6UzmqmBjyb/p+zEAaouG6GpfHiUNUzKR3vKCEoJt/Z2L15XPDCVPuFJIQhg=="], "lie": ["lie@3.3.0", "", { "dependencies": { "immediate": "~3.0.5" } }, "sha512-UaiMJzeWRlEujzAuw5LokY1L5ecNQYZKfmyZ9L7wDHb/p5etKaxXhohBcrw0EYby+G/NA52vRSN4N39dxHAIwQ=="], @@ -413,7 +403,5 @@ "zod": ["zod@4.4.2", "", {}, "sha512-IynmDyxsEsb9RKzO3J9+4SxXnl2FTFSzNBaKKaMV6tsSk0rw9gYw9gs+JFCq/qk2LCZ78KDwyj+Z289TijSkUw=="], "zwitch": ["zwitch@2.0.4", "", {}, "sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A=="], - - "heic-convert/pngjs": ["pngjs@6.0.0", "", {}, "sha512-TRzzuFRRmEoSW/p1KVAmiOgPco2Irlah+bGFCeNfJXxxYGwSw7YwAOAcd7X28K/m5bjBWKsC29KyoMfHbypayg=="], } } diff --git a/package.json b/package.json index b2cbeaf..22117ba 100644 --- a/package.json +++ b/package.json @@ -50,7 +50,6 @@ "skills/", "README.md", "LICENSE", - "LGPL-3.0.txt", "NOTICES" ], "engines": { @@ -72,7 +71,6 @@ "@biomejs/biome": "^2.4.10", "@types/bun": "^1.3.13", "@types/hast": "^3.0.4", - "@types/heic-convert": "^2.1.0", "@types/jpeg-js": "^0.3.7", "@types/mdast": "^4.0.4", "@types/pngjs": "^6.0.5", @@ -85,7 +83,6 @@ "diff": "^9.0.0", "fast-xml-builder": "^1.1.6", "fast-xml-parser": "^5.7.2", - "heic-convert": "^2.1.0", "jpeg-js": "^0.4.4", "jszip": "^3.10.1", "libxml2-wasm": "^0.7.1", diff --git a/src/cli/edit/index.tsx b/src/cli/edit/index.tsx index 3fef6ef..7021e50 100644 --- a/src/cli/edit/index.tsx +++ b/src/cli/edit/index.tsx @@ -26,6 +26,7 @@ import { adoptPositionalMarkdown, batchExampleIntro, decodeInlineEscapes, + decodeInlineMarkdownEscapes, hasRunFormatFlags, parseClearTags, parseRunFormat, @@ -1041,7 +1042,7 @@ async function validateParagraphEdit( const text = decodeInlineEscapes(values.text as string | undefined); const runsJson = values.runs as string | undefined; - const markdownInline = decodeInlineEscapes( + const markdownInline = decodeInlineMarkdownEscapes( values.markdown as string | undefined, ); const markdownFile = values["markdown-file"] as string | undefined; diff --git a/src/cli/footnotes/add.tsx b/src/cli/footnotes/add.tsx index ea7a97f..d577319 100644 --- a/src/cli/footnotes/add.tsx +++ b/src/cli/footnotes/add.tsx @@ -19,7 +19,11 @@ import { type TrackedMeta, } from "@core/track-changes"; import { Ins } from "@core/track-changes/emit"; -import { decodeInlineEscapes, parseRunsArg } from "../parse-helpers"; +import { + decodeInlineEscapes, + decodeInlineMarkdownEscapes, + parseRunsArg, +} from "../parse-helpers"; import { EXIT, fail, @@ -151,7 +155,7 @@ export async function runAddNote( const text = decodeInlineEscapes(parsed.values.text as string | undefined); const runsJson = parsed.values.runs as string | undefined; - const markdown = decodeInlineEscapes( + const markdown = decodeInlineMarkdownEscapes( parsed.values.markdown as string | undefined, ); const markdownFile = parsed.values["markdown-file"] as string | undefined; diff --git a/src/cli/footnotes/edit.tsx b/src/cli/footnotes/edit.tsx index 6cc1dc7..dc1d90d 100644 --- a/src/cli/footnotes/edit.tsx +++ b/src/cli/footnotes/edit.tsx @@ -10,7 +10,11 @@ import { TrackChanges, type TrackedMeta, } from "@core/track-changes"; -import { decodeInlineEscapes, parseRunsArg } from "../parse-helpers"; +import { + decodeInlineEscapes, + decodeInlineMarkdownEscapes, + parseRunsArg, +} from "../parse-helpers"; import { EXIT, fail, @@ -111,7 +115,7 @@ export async function runEditNote( const text = decodeInlineEscapes(parsed.values.text as string | undefined); const runsJson = parsed.values.runs as string | undefined; - const markdown = decodeInlineEscapes( + const markdown = decodeInlineMarkdownEscapes( parsed.values.markdown as string | undefined, ); const markdownFile = parsed.values["markdown-file"] as string | undefined; diff --git a/src/cli/images/add.ts b/src/cli/images/add.ts index d204c0b..52edc9a 100644 --- a/src/cli/images/add.ts +++ b/src/cli/images/add.ts @@ -34,7 +34,8 @@ Placement (exactly one required): Content (required): --image SRC The image source: a local file path, a data: URI, or an http(s) URL (fetched with an SSRF guard; sized to fit the - page width by default). + page width by default). HEIC/HEIF is refused (Word + can't display it): convert it to JPEG or PNG first. --alt TEXT Alt text / description for the image --width LENGTH Display width: bare inches (2) or a unit suffix — in, cm, mm, pt, px (96dpi), tw (e.g. 2in, 5cm, 300px). Default: diff --git a/src/cli/images/replace.ts b/src/cli/images/replace.ts index 0b841ce..5f1b82c 100644 --- a/src/cli/images/replace.ts +++ b/src/cli/images/replace.ts @@ -1,5 +1,6 @@ import { findContainingParagraph } from "@core/comments"; import { + heicExtensionRefusal, ImageSourceError, Images, imageFormatForExtension, @@ -93,6 +94,10 @@ export async function run(args: string[]): Promise { // `image/x-emf`/`image/bmp`, so extension → canonical Word MIME is what lets // emf/wmf/bmp/ico round-trip. const sourceExtension = sourcePath.split(".").pop() ?? ""; + const heicRefusal = heicExtensionRefusal(sourceExtension); + if (heicRefusal) { + return fail("IMAGE_SOURCE", heicRefusal.message, heicRefusal.hint); + } const format = imageFormatForExtension(sourceExtension); if (!format) { return fail( @@ -131,14 +136,15 @@ export async function run(args: string[]): Promise { // Route the replacement through the shared resolver (same as `docx images add`) // so it gets the same safety: SVG sanitization (scripts/handlers/animation - // stripped), HEIC→JPEG transcode, and the SSRF guard on remote srcs. We keep + // stripped), the HEIC header sniff (a `.jpg` that's really HEIC is refused), + // and the SSRF guard on remote srcs. We keep // the extension/MIME resolved above for part naming; only the bytes come back. let bytes: Uint8Array; try { bytes = (await loadImageSource(sourcePath)).bytes; } catch (error) { if (error instanceof ImageSourceError) { - return fail("IMAGE_SOURCE", error.message); + return fail("IMAGE_SOURCE", error.message, error.hint); } throw error; } diff --git a/src/cli/insert/index.tsx b/src/cli/insert/index.tsx index 3672b16..c204f8a 100644 --- a/src/cli/insert/index.tsx +++ b/src/cli/insert/index.tsx @@ -5,6 +5,7 @@ import { adoptPositionalMarkdown, batchExampleIntro, decodeInlineEscapes, + decodeInlineMarkdownEscapes, parseRunsArg, parseSpacingIndentFlags, pickContextualHelp, @@ -505,7 +506,7 @@ async function resolveMarkdownSpec( flag: "markdown" | "markdown-file", ): Promise | number> { if (flag === "markdown") { - const source = decodeInlineEscapes(values.markdown as string); + const source = decodeInlineMarkdownEscapes(values.markdown as string); return { kind: "markdown", source }; } const path = values["markdown-file"] as string; diff --git a/src/cli/parse-helpers.ts b/src/cli/parse-helpers.ts index 7884d2f..edf8e8c 100644 --- a/src/cli/parse-helpers.ts +++ b/src/cli/parse-helpers.ts @@ -113,6 +113,40 @@ const OTHER_CONTENT_FLAGS = [ "column-break", ] as const; +/** `decodeInlineEscapes` for an inline `--markdown` value (or the positional + * content that `adoptPositionalMarkdown` folds into it). Link and image + * DESTINATIONS — the `dest` in `[label](dest)` and `![alt](dest)` — are left + * undecoded. A destination can't contain a tab or a newline, so decoding one + * there only ever breaks the link, and a Windows path is exactly where `\t`, + * `\n` and `\r` turn up: `![Logo](C:\Users\me\tests\logo.png)` decoded `\t` + * into a TAB, so GFM no longer saw an image and the command inserted the + * syntax as literal text with exit 0. The label and an optional `"title"` + * decode as usual. */ +export function decodeInlineMarkdownEscapes(value: string): string; +export function decodeInlineMarkdownEscapes( + value: string | undefined, +): string | undefined; +export function decodeInlineMarkdownEscapes( + value: string | undefined, +): string | undefined { + if (value === undefined) return undefined; + let decoded = ""; + let cursor = 0; + for (const match of value.matchAll(LINK_DESTINATION)) { + const destinationStart = match.index + "](".length; + const destinationEnd = destinationStart + (match[1] ?? "").length; + decoded += decodeInlineEscapes(value.slice(cursor, destinationStart)); + decoded += value.slice(destinationStart, destinationEnd); + cursor = destinationEnd; + } + return decoded + decodeInlineEscapes(value.slice(cursor)); +} + +/** An inline link/image destination right after `](`: CommonMark's `<…>` form + * or a bare run with no whitespace or parentheses (the raw argv still spells + * an escape as two characters, so a path like `C:\new\x.png` matches whole). */ +const LINK_DESTINATION = /\]\((<[^<>\n]*>|[^\s()<]*)/g; + /** Decode the whitespace escape sequences weak agents type LITERALLY into an * inline `--text` / `--markdown` argv value. A model reaching for a line break * writes `--text "a\nb"`, but bash double-quotes don't interpret `\n` — the CLI @@ -123,8 +157,9 @@ const OTHER_CONTENT_FLAGS = [ * the markdown parser's block splitting). ONLY these: a `\` before anything else — * markdown's own escaping (`\*`, `\[`, `\.`) and a literal `\\` — passes through * UNTOUCHED, because `\` before a letter was never a valid markdown escape (so - * decoding `\n`/`\t`/`\r` can't corrupt markdown) while `\`-before-punctuation is - * (so we must not touch it). Applied at the INLINE argv ingress ONLY: `--text-file` + * decoding `\n`/`\t`/`\r` can't corrupt markdown — except inside a link or image + * destination, which markdown callers skip via `decodeInlineMarkdownEscapes`) + * while `\`-before-punctuation is (so we must not touch it). Applied at the INLINE argv ingress ONLY: `--text-file` * / `--markdown-file` / `--from` read real characters from a file, and `--batch` * JSONL is already decoded by `JSON.parse` — none are re-decoded. The * literal-fidelity channel where a bare `\n` really means backslash-n stays diff --git a/src/core/image/CLAUDE.md b/src/core/image/CLAUDE.md index 500b113..a637fcd 100644 --- a/src/core/image/CLAUDE.md +++ b/src/core/image/CLAUDE.md @@ -10,7 +10,7 @@ The `a:`/`pic:` namespaces are declared inline on `a:graphic`/`pic:pic` (matchin ## One format table, keyed by extension -[formats.ts](formats.ts) is the single source of truth for which image formats we extract / insert / replace. Resolution is by **file extension → canonical Word MIME** (`imageFormatForExtension`), never by sniffed MIME: Bun's `Bun.file().type` reports `image/emf`/`image/x-ms-bmp` where the docx and `extract` use `image/x-emf`/`image/bmp`, so extension-keying is what lets emf/wmf/bmp/ico round-trip. `extract` reads the part's stored MIME instead (`extensionForImageMime`). HEIC/HEIF are insert-only *input* (transcoded to JPEG in [source.ts](source.ts)), never an embedded format. **Adding a format:** add the extension→MIME and MIME→extension entries to both maps; insert/extract/replace pick it up automatically (insert needs a `readPixelDimensions` case too, or it requires `--width`/`--height`). +[formats.ts](formats.ts) is the single source of truth for which image formats we extract / insert / replace. Resolution is by **file extension → canonical Word MIME** (`imageFormatForExtension`), never by sniffed MIME: Bun's `Bun.file().type` reports `image/emf`/`image/x-ms-bmp` where the docx and `extract` use `image/x-emf`/`image/bmp`, so extension-keying is what lets emf/wmf/bmp/ico round-trip. `extract` reads the part's stored MIME instead (`extensionForImageMime`). HEIC/HEIF is never an embedded format: Word can't display it and every HEVC decoder is copyleft (we dropped the LGPL `libheif-js` to keep the project fully MIT), so [source.ts](source.ts) recognizes it — by extension, `image/heic`/`image/heif` MIME, or an `ftyp` HEIF brand in the header, so a mislabeled `.jpg` is still caught — only to REFUSE it (`ImageSourceError` → `IMAGE_SOURCE`, with a `hint` naming the `sips`/`heif-convert`/`magick` conversion command). `images replace` vets its `--with` extension before loading, so it calls the same `heicExtensionRefusal`. Do NOT shell out to an OS converter here: only `docx render` may need an external application. **Adding a format:** add the extension→MIME and MIME→extension entries to both maps; insert/extract/replace pick it up automatically (insert needs a `readPixelDimensions` case too, or it requires `--width`/`--height`). ## Threat model — `--image` is attacker-influenced input diff --git a/src/core/image/formats.ts b/src/core/image/formats.ts index fc6b853..1021aae 100644 --- a/src/core/image/formats.ts +++ b/src/core/image/formats.ts @@ -1,9 +1,9 @@ // The image formats docx-cli treats as first-class — the SAME set we extract, // insert, and replace, so a file pulled out by `images extract` drops straight // back in via `docx images add` or `images replace`. Extension spellings are -// Word-canonical (jpeg/tiff, the names Word writes into word/media), and the -// HEIC/HEIF transcode lives in image-source.ts (insert-only input, never an -// embedded format). +// Word-canonical (jpeg/tiff, the names Word writes into word/media). HEIC/HEIF +// is deliberately absent: Word can't display it, so source.ts recognizes it +// (by extension, MIME, or `ftyp` header) only to refuse it with a hint. // SVG is supported but sanitized on the insert path (see `sanitizeSvg` in // `source.ts`) — unlike every other format here, SVG carries arbitrary XML diff --git a/src/core/image/index.ts b/src/core/image/index.ts index 57767b3..33e4211 100644 --- a/src/core/image/index.ts +++ b/src/core/image/index.ts @@ -11,6 +11,7 @@ export { } from "./formats"; export { computeExtentEmu, + heicExtensionRefusal, type ImageSource, ImageSourceError, loadImageSource, diff --git a/src/core/image/source.ts b/src/core/image/source.ts index 1c7af6f..18470d2 100644 --- a/src/core/image/source.ts +++ b/src/core/image/source.ts @@ -5,7 +5,6 @@ import { lookup } from "node:dns/promises"; import { fileURLToPath } from "node:url"; -import convert from "heic-convert"; import { CANONICAL_IMAGE_EXTENSION, EXTENSION_BY_IMAGE_MIME, @@ -30,10 +29,13 @@ export type ImageSource = { }; /** Thrown for any source-resolution failure (unreadable path, non-image - * content-type, fetch timeout, oversize download). The CLI maps it to a - * single error code. */ + * content-type, fetch timeout, oversize download, HEIC input). The CLI maps it + * to a single error code; `hint`, when set, rides along as the error's hint. */ export class ImageSourceError extends Error { - constructor(message: string) { + constructor( + message: string, + public hint?: string, + ) { super(message); this.name = "ImageSourceError"; } @@ -251,23 +253,23 @@ async function loadFile(path: string): Promise { return finalizeSource(bytes, extension); } -/** Normalize the extension, transcode HEIC→JPEG if needed, and read dimensions - * — the shared tail every loader funnels through. */ +/** Normalize the extension, refuse HEIC, sanitize SVG, and read dimensions — + * the shared tail every loader funnels through. */ async function finalizeSource( bytes: Uint8Array, extension: string, ): Promise { const lower = extension.toLowerCase(); - let normalized = CANONICAL_EXTENSION[lower] ?? lower; + const normalized = CANONICAL_EXTENSION[lower] ?? lower; let finalBytes = bytes; - // Word can't render HEIC/HEIF, so transcode to JPEG before embedding. Detect - // by header too (not just the label) so a `.jpg` that's really HEIC, or an - // untyped fetch, is still caught. - if (HEIC_EXTENSIONS.has(normalized) || isHeic(bytes)) { - finalBytes = await transcodeHEICToJPEG(bytes); - normalized = "jpeg"; - } + // Word can't render HEIC/HEIF and we bundle no HEVC decoder (every one is + // copyleft), so refuse rather than embed an image Word shows as broken. + // Detect by header too (not just the label) so a `.jpg` that's really HEIC, + // or an untyped fetch, is still caught. + const refusal = heicExtensionRefusal(normalized); + if (refusal) throw refusal; + if (isHeic(bytes)) throw heicRefusal("the file's header is HEIC/HEIF"); // SVG is the one format here that's executable in downstream renderers, so // scrub it before embedding — fast-xml-parser already rejects XXE, this @@ -296,6 +298,24 @@ async function finalizeSource( }; } +/** The refusal for a HEIC/HEIF extension label, or undefined for any other + * extension. Exported so `images replace` — which vets the `--with` extension + * before loading it — refuses `.heic` with the same message and hint. */ +export function heicExtensionRefusal( + extension: string, +): ImageSourceError | undefined { + const lower = extension.toLowerCase(); + if (!HEIC_EXTENSIONS.has(lower)) return undefined; + return heicRefusal(`.${lower} input`); +} + +function heicRefusal(detectedBy: string): ImageSourceError { + return new ImageSourceError( + `HEIC/HEIF images are not supported (${detectedBy}): Word can't display them. Convert to JPEG or PNG first.`, + HEIC_CONVERSION_HINT, + ); +} + function isHeic(bytes: Uint8Array): boolean { if (bytes.length < 12) return false; const boxType = new TextDecoder().decode(bytes.slice(4, 8)); @@ -304,23 +324,6 @@ function isHeic(bytes: Uint8Array): boolean { return HEIF_BRANDS.has(majorBrand); } -async function transcodeHEICToJPEG(bytes: Uint8Array): Promise { - try { - // heic-decode reads the buffer as a typed array (slice + char spread), so - // pass the Uint8Array directly; @types says ArrayBufferLike but an actual - // ArrayBuffer throws inside the lib. - const jpeg = await convert({ - buffer: bytes as unknown as ArrayBufferLike, - format: "JPEG", - quality: 0.92, - }); - return new Uint8Array(jpeg); - } catch (error) { - const reason = error instanceof Error ? error.message : String(error); - throw new ImageSourceError(`Failed to transcode HEIC to JPEG: ${reason}`); - } -} - /** `file:///C:/x.png` is `C:\x.png` on Windows, not `/C:/x.png`, and a percent-encoded * space is a space. A URL with a host keeps meaning the path after the prefix, as it * always has (`file://relative/x.png`): on Windows, converting it would make a UNC path, @@ -472,9 +475,11 @@ function readJPEGDimensions( const DEFAULT_FETCH_TIMEOUT_MS = 10_000; const MAX_FETCH_BYTES = 25 * 1024 * 1024; -// Accepted *input* = the shared embeddable formats (image-formats.ts) plus the -// `jpg`/`tif` aliases and HEIC/HEIF, which finalizeSource normalizes/transcodes -// before embedding. Everything we extract or replace is therefore insertable. +// Recognized *input* = the shared embeddable formats (formats.ts) plus the +// `jpg`/`tif` aliases, which finalizeSource normalizes, and HEIC/HEIF, which it +// refuses — they're listed only so a `data:image/heic` URI or an `image/heic` +// fetch reaches that refusal and its conversion hint instead of a generic +// "unsupported MIME" error. Everything we extract or replace is insertable. const MIME_BY_EXTENSION: Record = { ...IMAGE_MIME_BY_EXTENSION, heic: "image/heic", @@ -491,6 +496,9 @@ const CANONICAL_EXTENSION = CANONICAL_IMAGE_EXTENSION; const HEIC_EXTENSIONS = new Set(["heic", "heif"]); +const HEIC_CONVERSION_HINT = + "Convert it, then insert the converted file. macOS: `sips -s format jpeg IN.heic --out OUT.jpg`. Linux: `heif-convert IN.heic OUT.jpg` (libheif-examples) or ImageMagick `magick IN.heic OUT.jpg`. Otherwise, open it in an image editor and export as JPEG or PNG."; + // HEIF-family `ftyp` brands (the major brand at bytes 8–12 of the ISO-BMFF box). const HEIF_BRANDS = new Set([ "heic", diff --git a/src/core/insert/index.tsx b/src/core/insert/index.tsx index 257d926..9d9a5a1 100644 --- a/src/core/insert/index.tsx +++ b/src/core/insert/index.tsx @@ -402,7 +402,7 @@ async function buildImageParagraph( source = await loadImageSource(spec.src); } catch (error) { if (error instanceof ImageSourceError) { - throw new InsertError("IMAGE_SOURCE", error.message); + throw new InsertError("IMAGE_SOURCE", error.message, error.hint); } throw error; } diff --git a/src/core/markdown/import.tsx b/src/core/markdown/import.tsx index c9c7845..6b3b034 100644 --- a/src/core/markdown/import.tsx +++ b/src/core/markdown/import.tsx @@ -417,7 +417,7 @@ async function loadAndRegisterImage( source = await loadImageSource(image.url); } catch (error) { if (error instanceof ImageSourceError) { - throw new MarkdownImportError("IMAGE_SOURCE", error.message); + throw new MarkdownImportError("IMAGE_SOURCE", error.message, error.hint); } throw error; } diff --git a/src/core/render/CLAUDE.md b/src/core/render/CLAUDE.md index 98fda52..fbda50c 100644 --- a/src/core/render/CLAUDE.md +++ b/src/core/render/CLAUDE.md @@ -128,7 +128,7 @@ Earlier iterations supported `pdftoppm` (poppler) + `magick` (ImageMagick) as PD PDFium is dual-licensed BSD-3-Clause OR Apache-2.0 (we redistribute under BSD-3). The required notice lives in [`NOTICES`](../../../NOTICES) at the repo root, which the release workflow stages into each artifact bundle and the npm `files` list ships alongside `dist/index.js`. If the bundled WASM binary's source changes (we don't modify it — we consume @hyzyla/pdfium's published build), update NOTICES accordingly. -The same NOTICES file carries the BSD-3 attributions for `jpeg-js` and `highlight.js`, the LGPL-3.0 acknowledgement for `libheif-js` (statically linked into the compiled binary via `heic-convert`), and the MIT election for `jszip` (which is dual MIT-or-GPL). +The same NOTICES file carries the BSD-3 attributions for `jpeg-js` and `highlight.js`, and the MIT election for `jszip` (which is dual MIT-or-GPL). ## CI considerations diff --git a/tests/cli/edit.test.ts b/tests/cli/edit.test.ts index ab57b57..ed2c93f 100644 --- a/tests/cli/edit.test.ts +++ b/tests/cli/edit.test.ts @@ -3837,6 +3837,26 @@ describe("docx edit — inline escape decoding (--text / --markdown)", () => { // decodes `\n`/`\t` at the inline argv ingress so they flow through the same // real-newline handling as `--text-file` / batch. See decodeInlineEscapes. + test("markdown leaves a link/image destination undecoded (Windows paths)", async () => { + // `\t` in `C:\a\tests\…` decoded to a TAB broke the image, and the syntax + // landed as literal text at exit 0. The not-found error proves the + // destination reached the loader verbatim. + const docPath = await freshCopy("esc-md-winpath"); + const windowsPath = "C:\\a\\tests\\new\\logo.png"; + const result = await runCli( + "edit", + docPath, + "--at", + "p0", + `![Logo](${windowsPath})`, + ); + expect(result.exitCode).toBe(1); + expect(result.parsed).toMatchObject({ + code: "IMAGE_SOURCE", + error: `Image file not found: ${windowsPath}`, + }); + }); + test("--text decodes \\n to a line break within the paragraph", async () => { const docPath = await freshCopy("esc-text-nl"); const result = await runCli( diff --git a/tests/cli/images.test.ts b/tests/cli/images.test.ts index e84b8cd..7a9ff94 100644 --- a/tests/cli/images.test.ts +++ b/tests/cli/images.test.ts @@ -200,7 +200,7 @@ describe("docx images", () => { const ASSETS = join(import.meta.dir, "..", "fixtures", "assets"); const PNG_PATH = join(ASSETS, "sample.png"); // 96×64 const JPG_PATH = join(ASSETS, "sample.jpg"); // 120×80 -const HEIC_PATH = join(ASSETS, "sample.heic"); // 96×64, transcoded to JPEG +const HEIC_PATH = join(ASSETS, "sample.heic"); // 96×64; refused, never embedded const EMU_PER_PIXEL = 9525; const EMU_PER_INCH = 914400; @@ -398,49 +398,6 @@ describe("docx images add", () => { expect((await imageRuns(docPath)).length).toBe(2); }); - test("transcodes a HEIC file to JPEG before embedding", async () => { - const docPath = await newDoc("img-heic"); - const result = await runCli( - "images", - "add", - docPath, - "--after", - "p0", - "--image", - HEIC_PATH, - "--alt", - "Student photo", - ); - expect(result.exitCode).toBe(0); - - // The embedded part is a JPEG, not HEIC (Word can't render HEIC). - const pkg = await Pkg.open(docPath); - expect(pkg.listParts()).toContain("word/media/image1.jpeg"); - expect(pkg.listParts().some((part) => part.endsWith(".heic"))).toBe(false); - - const [image] = await imageRuns(docPath); - expect(image?.contentType).toBe("image/jpeg"); - // Dimensions survive the transcode (source HEIC is 96×64). - expect(image?.widthEmu).toBe(96 * EMU_PER_PIXEL); - expect(image?.heightEmu).toBe(64 * EMU_PER_PIXEL); - }); - - test("transcodes HEIC supplied as a data: URI", async () => { - const docPath = await newDoc("img-heic-datauri"); - await runCli( - "images", - "add", - docPath, - "--after", - "p0", - "--image", - await dataUri(HEIC_PATH, "image/heic"), - ); - const pkg = await Pkg.open(docPath); - expect(pkg.listParts()).toContain("word/media/image1.jpeg"); - expect((await imageRuns(docPath))[0]?.contentType).toBe("image/jpeg"); - }); - test("refuses to fetch from a loopback/private address (SSRF gate)", async () => { // An agent steered into `http://localhost:.../` or the cloud-metadata // endpoint must not turn this CLI into an outbound proxy. Verify both the @@ -1159,3 +1116,149 @@ describe("docx images add — size units", () => { expect(height.result.exitCode).toBe(2); }); }); + +// HEIC/HEIF is refused, not transcoded: Word can't display it, and every HEVC +// decoder is copyleft (docx-cli dropped the LGPL libheif-js to stay MIT). Each +// image ingress funnels through the same refusal, exits nonzero with a hint +// naming the conversion command, and leaves the file untouched. +describe("HEIC/HEIF input is refused on every image ingress", () => { + async function expectHeicRefusal( + docPath: string, + before: Uint8Array, + result: { exitCode: number; parsed?: unknown }, + ): Promise { + expect(result.exitCode).toBe(1); + const error = result.parsed as { + code: string; + error: string; + hint: string; + }; + expect(error.code).toBe("IMAGE_SOURCE"); + expect(error.error).toContain("HEIC/HEIF images are not supported"); + expect(error.hint).toContain("sips -s format jpeg IN.heic --out OUT.jpg"); + expect(error.hint).toContain("heif-convert IN.heic OUT.jpg"); + expect(error.hint).toContain("magick IN.heic OUT.jpg"); + expect(error.hint).toContain("export as JPEG or PNG"); + expect(await Bun.file(docPath).bytes()).toEqual(before); + } + + async function heicMislabeledAsJpeg(label: string): Promise { + const path = join(tempWorkspace(label), "really-heic.jpg"); + await Bun.write(path, Bun.file(HEIC_PATH)); + return path; + } + + test("images add refuses a .heic file and writes nothing", async () => { + const docPath = await newDoc("img-heic"); + const before = await Bun.file(docPath).bytes(); + const result = await runCli( + "images", + "add", + docPath, + "--after", + "p0", + "--image", + HEIC_PATH, + ); + await expectHeicRefusal(docPath, before, result); + expect(result.parsed).toMatchObject({ + error: expect.stringContaining(".heic input"), + }); + }); + + test("images add refuses a data:image/heic URI", async () => { + const docPath = await newDoc("img-heic-datauri"); + const before = await Bun.file(docPath).bytes(); + const result = await runCli( + "images", + "add", + docPath, + "--after", + "p0", + "--image", + await dataUri(HEIC_PATH, "image/heic"), + ); + await expectHeicRefusal(docPath, before, result); + }); + + test("images add sniffs HEIC bytes behind a .jpg label", async () => { + // Word can't render HEIC, so embedding a mislabeled file would ship a + // broken image; the ftyp header check catches it. + const docPath = await newDoc("img-heic-mislabeled"); + const before = await Bun.file(docPath).bytes(); + const result = await runCli( + "images", + "add", + docPath, + "--after", + "p0", + "--image", + await heicMislabeledAsJpeg("img-heic-mislabeled-src"), + ); + await expectHeicRefusal(docPath, before, result); + expect(result.parsed).toMatchObject({ + error: expect.stringContaining("header"), + }); + }); + + test("insert refuses a markdown image pointing at a .heic file", async () => { + const docPath = await newDoc("img-heic-md-insert"); + const before = await Bun.file(docPath).bytes(); + const result = await runCli( + "insert", + docPath, + "--at", + "p0", + `![Photo](${HEIC_PATH})`, + ); + await expectHeicRefusal(docPath, before, result); + }); + + test("create --from refuses a markdown image pointing at a .heic file", async () => { + const workspace = tempWorkspace("img-heic-md-create"); + const source = join(workspace, "source.md"); + await Bun.write(source, `# Report\n\n![Photo](${HEIC_PATH})\n`); + const docPath = join(workspace, "out.docx"); + const result = await runCli("create", docPath, "--from", source); + expect(result.exitCode).toBe(1); + expect(result.parsed).toMatchObject({ + code: "IMAGE_SOURCE", + hint: expect.stringContaining("sips -s format jpeg"), + }); + // `create` stages the blank package before its markdown pass (a + // pre-existing trait of every --from failure), so assert no image landed. + const pkg = await Pkg.open(docPath); + expect(pkg.listParts().some((part) => part.startsWith("word/media/"))).toBe( + false, + ); + }); + + test("images replace refuses a .heic file or HEIC behind a .jpg label", async () => { + const docPath = await newDoc("img-heic-replace"); + await runCli( + "images", + "add", + docPath, + "--after", + "p0", + "--image", + PNG_PATH, + ); + const before = await Bun.file(docPath).bytes(); + for (const source of [ + HEIC_PATH, + await heicMislabeledAsJpeg("img-heic-replace-src"), + ]) { + const result = await runCli( + "images", + "replace", + docPath, + "--at", + "img0", + "--with", + source, + ); + await expectHeicRefusal(docPath, before, result); + } + }); +}); diff --git a/tests/cli/insert.test.ts b/tests/cli/insert.test.ts index f401ac2..28f1b3e 100644 --- a/tests/cli/insert.test.ts +++ b/tests/cli/insert.test.ts @@ -1,4 +1,5 @@ import { beforeEach, describe, expect, test } from "bun:test"; +import { mkdirSync } from "node:fs"; import { join } from "node:path"; import { Pkg } from "@core/ast/document/package"; import { runCli, tempWorkspace } from "./harness"; @@ -1344,3 +1345,74 @@ describe("insert — the empty anchor's paragraph mark and spacing bags", () => }); }); }); + +describe("insert — inline markdown keeps backslash image paths intact", () => { + // Inline markdown decodes a typed `\n`/`\t`/`\r` into real whitespace, but a + // link or image DESTINATION can't hold whitespace: decoding the `\t` of a + // Windows path like `C:\Users\me\tests\logo.png` broke the image, and the + // command inserted the syntax as literal text with exit 0. + test("a backslash path with \\t/\\n/\\r segments reaches the image loader verbatim", async () => { + const docPath = join(tempWorkspace("ins-md-winpath"), "doc.docx"); + await runCli("create", docPath, "--text", "Before"); + const windowsPath = "C:\\a\\tests\\new\\reports\\logo.png"; + const result = await runCli( + "insert", + docPath, + "--at", + "p0", + `![Logo](${windowsPath} "Company\\tlogo")`, + ); + // Not on disk anywhere, so the loader's not-found error proves the + // destination arrived undecoded (and that it WAS parsed as an image). + expect(result.exitCode).toBe(1); + expect(result.parsed).toMatchObject({ + code: "IMAGE_SOURCE", + error: `Image file not found: ${windowsPath}`, + }); + }); + + test("a native path under tests/new/reports directories embeds the image", async () => { + // On Windows this path is backslash-separated, so it is the real-world + // case; on POSIX it is the same flow with forward slashes. + const workspace = tempWorkspace("ins-md-nativepath"); + const imageDir = join(workspace, "tests", "new", "reports"); + mkdirSync(imageDir, { recursive: true }); + const imagePath = join(imageDir, "logo.png"); + await Bun.write( + imagePath, + Bun.file(join(import.meta.dir, "..", "fixtures", "assets", "sample.png")), + ); + const docPath = join(workspace, "doc.docx"); + await runCli("create", docPath, "--text", "Before"); + const result = await runCli( + "insert", + docPath, + "--at", + "p0", + `![Logo](${imagePath})`, + ); + expect(result.exitCode).toBe(0); + const pkg = await Pkg.open(docPath); + expect(pkg.listParts()).toContain("word/media/image1.png"); + }); + + test("\\n outside a destination still decodes into a new paragraph", async () => { + const docPath = join(tempWorkspace("ins-md-decode-outside"), "doc.docx"); + await runCli("create", docPath, "--text", "Before"); + const result = await runCli( + "insert", + docPath, + "--at", + "p0", + "First [site](https://example.com/a\\tb)\\n\\nSecond", + ); + expect(result.exitCode).toBe(0); + const xml = await readDocumentXml(docPath); + expect(xml).toContain(">Second<"); + expect(xml).not.toContain("\\n"); + // The link survived: its destination's `\t` was left alone, so no + // literal `[site](` syntax leaked into the text. + expect(xml).toContain("