Skip to content

ReferenceCheck: opt-in --content check of path:line citations (#2221) - #2688

Open
drpaulobernardy wants to merge 2 commits into
danielmiessler:mainfrom
drpaulobernardy:refcheck-content
Open

drpaulobernardy wants to merge 2 commits into
danielmiessler:mainfrom
drpaulobernardy:refcheck-content

Conversation

@drpaulobernardy

Copy link
Copy Markdown

Note

🤖 Paulino (AI assistant) responding on behalf of Dr. Paulo

Carries #2221 as the opt-in --content mode you described there.

What changes

  • LIFEOS/TOOLS/lib/CitationCheck.ts holds the check: checkCitation(root, path, line, snippet) -> { state, foundAt? }, plus findCitations(text), which reads the citations in a markdown text and counts the path:line pointers with no snippet. It has no state of its own, so other tools can call it without the CLI.
  • States: ok, moved (the snippet is on exactly one other line), ambiguous (on several), changed, gone, dir, and outside. Only ok and moved are not rotten.
  • Paths resolve through realpathSync and must stay inside the checked tree. A path that leaves it, through ../, an absolute path or a symlink, comes back outside. That state is new: your condition needed a name for it, and it counts as rotten.
  • ReferenceCheck.ts --content reads the citations in every scanned .md. Fenced blocks are skipped, as the rest of the tool skips them, and so are files marked references: historical. It lists moved and rotten citations, prints a count line, and in --json adds a content key. A rotten citation exits 1; a citation that cannot be read is a scan error and exits 2.
  • path:line pointers with no snippet keep today's behavior and are counted separately (withoutSnippet). The snippet may sit in double backticks when it holds a backtick.
  • The default run and --stale are unchanged: without --content no citation is read, and the content key never appears.

--changed does not narrow --content, because the useful case is a citation whose target changed, and that is not the referring file. The help text says so.

Tests

LIFEOS/TOOLS/lib/CitationCheck.test.ts has 13 tests:

  • One test per state: ok, moved, ambiguous, changed, gone, dir and outside. The outside test covers a symlink, a ../ path and an absolute path.
  • The ~/.claude/ prefix reads as the checked root.
  • findCitations handles single and double backticks, skips fenced blocks, and counts pointers.
  • Three runs of the CLI on a temporary --root: the default run and --stale show no content key and exit 0 even with a rotten citation present, while --content reports it and exits 1.

All 13 pass. On main, the module does not exist, so the file fails to load. With only ReferenceCheck.ts reverted, the --content CLI test fails.

Checked on a real install

  • On one install (4,182 files, 9,698 refs), --json and --stale --json were run with the tool from main and with this branch. The output is byte-identical except elapsedMs.
  • That install keeps its notes outside the scanned tree, so --content found only one citation there. I also ran checkCitation directly over 726 notes that use the convention: 7,738 ok, 9 moved and 294 gone. The 9 moved match what the install's own citation checker reports; the gone ones are paths written relative to another checkout, which this check rightly does not follow.

One limit, the same as in the issue: this checks that a citation still points at the same code, not that the sentence next to it is true.

🤖 Generated with Claude Code

…miessler#2221)

lib/CitationCheck.ts holds the check, checkCitation(root, path, line, snippet) -> state, with the states
ok, moved, ambiguous, changed, gone, dir and outside. Paths resolve through realpathSync and must stay
inside the checked tree. ReferenceCheck --content reads the citations in each scanned .md, reports moved
and rotten ones, counts path:line pointers with no snippet, and exits 1 on a rotten citation. The default
run and --stale are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014YTdW6sTViEVVK8c3DeQVC

@isihussain isihussain left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Architectural & Quality Review

Thank you @drpaulobernardy. Adding content-aware validation to ReferenceCheck via CitationCheck is a huge improvement over simple mtime or line-existence checks. Distinguishing between ok, moved (single unambiguous relocation), and truly rotten citations (ambiguous, changed, gone) gives actionable feedback to doc maintainers.

Here is a review across correctness, performance, and markdown fence parsing:


1. Performance: File Content Caching

In checkCitation():

const lines = readFileSync(real, 'utf-8').split('\n');

When documentation files contain multiple citations targeting the same source file (e.g., citing several functions in Boundary.ts or MemoryRetriever.ts), checkCitation performs a synchronous disk read and string split for each citation individually.

Recommendation: Passing an optional line cache (cache?: Map<string, string[]>) across the audit run will eliminate redundant disk reads and memory allocations when verifying documents with clustered citations:

const lines = cache?.get(real) ?? readFileSync(real, 'utf-8').split('\n');
cache?.set(real, lines);

2. Markdown Parsing: Code Fence Matching

In findCitations():

if (lineText.trimStart().startsWith('```')) { fenced = !fenced; return; }

Two edge cases in markdown code fences:

  1. Tilde fences (~~~): CommonMark supports code fences using tildes. If a doc uses ~~~markdown, the citations inside it won't be recognized as fenced.
  2. Nested fences: If a 4-backtick block (`````) encloses a 3-backtick example block (````), the boolean toggle fenced = !fenced flips prematurely in the middle of the block.
    Matching with a regex or recording fence character/length (like in `IsaSections`) handles nested examples robustly:
const fenceMatch = /^\s*(`{3,}|~{3,})/.exec(lineText);

3. Path Security: Boundary Containment

The check:

const inside = relative(rootReal, real);
if (inside === '..' || inside.startsWith('..' + sep) || isAbsolute(inside)) return { state: 'outside' };

correctly follows symlinks with realpathSync and enforces that citations cannot be used to probe files outside the repository root.


The test suite in CitationCheck.test.ts covers all 7 citation states thoroughly. Solid addition to the ReferenceCheck toolset!

… fences

findCitations toggled on every line starting with three backticks, so a
~~~ fence was read as prose and a 4-backtick fence holding a 3-backtick
example closed at the first inner fence. A fence now follows CommonMark:
three or more backticks or tildes, closed only by the same character, at
least as long, with nothing after it. New test, red before the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014YTdW6sTViEVVK8c3DeQVC
@drpaulobernardy

drpaulobernardy commented Nov 16, 2026 •

Copy link
Copy Markdown
Author

Note

🤖 Paulino (AI assistant) responding on behalf of Dr. Paulo

Thanks @isihussain, the fence point was right. findCitations toggled on every line starting with three backticks, so a ~~~ fence was read as prose and a 4-backtick fence holding a 3-backtick example closed at the inner fence. Fixed in ffc4d8b: a fence is now three or more backticks or tildes and closes only on the same character, at least as long, with nothing after it. A new test in CitationCheck.test.ts covers both cases and the text after the close; it was red before the change, and the file passes 14 of 14.

fenceMap in ReferenceCheck.ts (around line 529, outside this PR) uses the same toggle, so the default run has the same blind spot with tilde and nested fences.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants