Skip to content

feat(catalog): bundle-file-unreferenced, bundled files SKILL.md never names - #559

Open
napetrov wants to merge 62 commits into
intel:mainfrom
napetrov:feat/bundle-file-unreferenced
Open

napetrov wants to merge 62 commits into
intel:mainfrom
napetrov:feat/bundle-file-unreferenced

Conversation

@napetrov

@napetrov napetrov commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

New warn, one per skill: bundle-file-unreferenced. It counts bundled scripts and documents (.py .sh .js .ts .md .txt …) that neither SKILL.md nor a document it names mentions, by path, file name or folder alone, with case ignored. The message gives the count and three example paths. Data, images, licence and readme files, AGENTS.md, _-prefixed helpers, tests, vendored folders and nested skills are not counted, and a skill at the checkout root is not checked. A document in a test or hidden folder that SKILL.md names is still read. A changelog, contributing or notice file supplies no references. A named document the search cannot read (over 256 KiB, not UTF-8, a symlink or a FIFO) gives no claim.

Evidence

  • GitSkills corpus: of 5.9M files bundled next to a SKILL.md, 63% of .py, 45% of .sh and 47% of .md files are never named in it.
  • 24 upstream repos: 120 of 2,372 skills flagged (596 of 2,391 counted files). These numbers are from before the round of review fixes on named documents and unreadable documents, so they are stale: a rerun is needed before quoting them again. The main false-positive class left is package modules imported by a script that is named.
  • tests/test_catalog_bundle_unreferenced.py: 73 pass; the 360-file timing case runs under its 30 s CPU bound. Self-review of the latest round: no blocking finding, nits recorded in the review.

Merge danger

Conflicts with the other PRs in this series on FINDING_CODES and the docs.

🤖 Generated with Claude Code

…ed files SKILL.md never names

An agent opens the files SKILL.md points it at. In the GitSkills corpus 63% of bundled
.py, 45% of .sh and 47% of .md files are never named in their SKILL.md, so the agent only
finds them by browsing. The new check counts scripts and documents (not data or images)
that neither SKILL.md nor a document it names mentions by path, file name or folder,
case ignored, and reports the count with three examples. Licence and readme files,
_-prefixed helpers, AGENTS.md, tests, vendored folders and nested skills are skipped, and
a skill at the checkout root is not checked.

On 24 upstream skill repos (2,372 skills) it flags 120 skills, 596 of 2,391 counted files.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov
napetrov requested a review from a team as a code owner November 9, 2026 22:25
Copilot AI balanced review requested due to automatic review settings November 9, 2026 22:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Named FIFO references can hang reviews, while substring matching misidentifies overlapping filenames, suppressing warnings and reading unintended documents.

2 open issues
What changed in this PR

2 new findings.

Adds bundle-file-unreferenced to catalog validation, warning when bundled scripts or documents are not named by SKILL.md or a document it references.

Changes:

  • Implements case-insensitive reference matching, document traversal, and bundle exclusions.
  • Integrates one warning per skill, with counts and example paths.
  • Adds regression coverage and documents the new warning.
File Description
dokimi/​tests/​test_content_checks.py Updates expected standalone-review findings.
dokimi/​tests/​test_catalog_bundle_unreferenced.py Tests references, exclusions, document traversal, and review scope.
dokimi/​dokimi/​external/​validate.py Registers and emits the new warning.
dokimi/​dokimi/​external/​bundle_refs.py Implements bundle scanning and reference detection.
dokimi/​docs/​external-content.md Documents warning behavior and exclusions.
dokimi/​changelog.d/​20261109_160000_napetrov_bundle-file-unreferenced.md Announces the warning and its ignore option.

Review effort: Balanced

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
@exai-bot

exai-bot commented Nov 9, 2026 •

Copy link
Copy Markdown

OpenCodeReview: Commit 696225a was not reviewed. This PR has reached its review limit (6 reviews).

CI at `696225a`: 5 still running, 12 passing, 2 skipped
Check Status
Linux Python 3.11 still running
Linux Python 3.14 still running
Linux Wheels Python 3.11 still running
Linux Wheels Python 3.14 still running
Static checks (ruff, schema, zizmor, licenses, pre-commit, bandit, secrets, changelog, schema com... still running

To review this commit: re-request review from exai-bot in the Reviewers list.

<details><summary>Last review (cd26489): no findings</summary>

Open threads from the bot: 0

</details>

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
@codecov-commenter

codecov-commenter commented Nov 9, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 98.36066% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.78%. Comparing base (ba74e4a) to head (696225a).
⚠️ Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
dokimi/dokimi/external/bundle_refs.py 97.81% 4 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #559      +/-   ##
==========================================
+ Coverage   95.75%   95.78%   +0.03%     
==========================================
  Files         423      427       +4     
  Lines       56416    56688     +272     
  Branches     9714     9761      +47     
==========================================
+ Hits        54019    54297     +278     
+ Misses       1489     1486       -3     
+ Partials      908      905       -3     
Files with missing lines Coverage Δ
dokimi/dokimi/external/link_files.py 100.00% <100.00%> (ø)
dokimi/dokimi/external/validate.py 97.85% <100.00%> (+0.45%) ⬆️
dokimi/dokimi/external/bundle_refs.py 97.81% <97.81%> (ø)

... and 30 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

A document SKILL.md names was read whatever its type, so a FIFO named notes.md
blocked the scan until a writer opened it. Read only regular files.

Thread: PRRT_kwDOVBg7g861eMIQ
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…ter file

The raw substring check let SKILL.md's rebuild.py name build.py and guide.mdx
name guide.md. Match the path and the file name as whole names.

Thread: PRRT_kwDOVBg7g861eMIw
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The docs say tests are not counted, but only the test folders were skipped:
test_build.py and build.test.js in scripts/ were counted as unreferenced.

Thread: PRRT_kwDOVBg7g861eOr_
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Readme, licence, changelog, AGENTS.md and CONTRIBUTING.md were left out of the
walk, so a script only README.md named was reported as unreferenced. Walk every
document for traversal and count only the countable ones.

Thread: PRRT_kwDOVBg7g861eOsF
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Only SKILL.md's links were parsed, so [fill](scripts/fill%2Epy) in forms.md left
scripts/fill.py unnamed. Parse the links of every document the scan reads, relative
to its own folder, with the same parser.

Thread: PRRT_kwDOVBg7g861eOsJ
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at c72c3d3. The check works and the five earlier bot findings are fixed, but folder references are still matched too narrowly: a folder written with ./ or ${CLAUDE_SKILL_DIR}/ gives false warns. It also needs a rebase: it conflicts with current main in validate.py.

Agreed with @exai-bot: build.py.bak counts as naming build.py; a folder link in a nested document does not cover its files.

What I ran, at c72c3d3:

  • make ci-local --offline: 17,916 passed, 0 failed, 72 skipped. One FAIL, schema_compat on arms.v1.json, comes from main moving since the base, not from this PR (it touches no schema).
  • Repros of _named and unreferenced_message for folder prefixes, dotted suffixes, nested folder links and ${CLAUDE_SKILL_DIR} links in nested documents.

Checked, no issues

  • Earlier fixes: the FIFO, whole-name, test-name, skipped-README and encoded-link fixes hold, each with a test.
  • Docs and changelog: the new code is in FINDING_CODES, the docs and the module docstring; the fragment is under Added and says how to turn it off.

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
The unreadable-SKILL.md guard from main now wraps the bundle check added here.

Thread: PRR_kwDOVBg7g88AAAABXBtM2g#1
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…shorter file

build.py is not named by a mention of build.py.bak; the trailing lookahead
now also rejects a dot followed by a word character.

Thread: PRRT_kwDOVBg7g861kC5m
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
./scripts/, ${CLAUDE_SKILL_DIR}/scripts/ and a folder link to an ancestor
folder now count as naming the folder's files.

Thread: PRRT_kwDOVBg7g861px8Y
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… own folder

The destination of a folder link is checked as its folder in the document's
folder; the same folder name at the skill root is no longer named by it.

Thread: PRRT_kwDOVBg7g861kC5s
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…he skill root

${CLAUDE_SKILL_DIR}/scripts/x.py in references/guide.md names scripts/x.py,
not references/scripts/x.py. The catalog link list keeps the prefix so the
bundle check can tell it apart.

Thread: PRRT_kwDOVBg7g861px8e
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…hecked

The walk returns after the cap without leaving os.walk early, and the
warning is skipped for an over-cap bundle, so no partial count is given.

Thread: PRRT_kwDOVBg7g861px8h
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… files

'The helpers live in scripts/.' ends the folder name with a period that
the folder lookahead refused, so the folder was reported as unnamed.
A period followed by a word character still continues the name.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>


Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by vlad-perevezentsev

folder references are matched too narrowly
Fixed in 4f58c73: ./, ../ and skill-dir folder forms name their files.

Fixed in 93200bc and cc078d5: ./, ../, skill-dir folder forms count.

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…he read cap is not judged

Thread: PRRT_kwDOVBg7g8611-uo
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… names that folder

Thread: PRRT_kwDOVBg7g8611-u5
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
unreferenced_message was 11 complex against the 10 cap of the growth
ratchet (test_growth_ratchet). The search for named documents moves
to _named_closure, which also returns whether it was incomplete.
Behaviour is unchanged.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by antonwolfy

the ../ back-link is still open
Fixed in 7994848 (the thread above has the detail).

@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by antonwolfy

I pushed a commit per nit, up to 00ae79b
Both are in: a96b4d0 (the my\*script.py case) and 00ae79b (the test). Named in 5f8e805.

@napetrov

Copy link
Copy Markdown
Contributor Author

Round on c228b31, main merged in 2250924.

  • FFOD (../ back link in a nested document): 7994848 names only SKILL.md's root link as naming every file, as decided. Tests in f8b3f62 and c228b31.
  • BYBoFDw nits: a96b4d0 and 00ae79b.

@antonwolfy, the back-link point is fixed at c228b31. Could you re-review?

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at c228b31: Changes needed: 1 must-fix, 1 earlier finding still fails, 1 nit.

Blocking

  • Still fails: earlier finding – 1,900 unnamed scripts/*.py with 50 named documents of 208 KB, all named by SKILL.md, takes 18.6 s of CPU in the search against the 10 s bound; each unnamed file is still checked against the whole text of the documents read. For @napetrov.
  • Still holds on this head: thread by @vlad-perevezentsev – unreferenced_message returns no message and records nothing for SKILL.md naming data/words.txt (300 KB) with scripts/unused.py bundled; the docs now list the skip conditions (dokimi/docs/external-content.md:261-267), but the skip stays silent. For @napetrov.

Earlier rounds

  • Fixed: A line saying helpers are in the backslash folder scripts\ with scripts/helper.py bundled now gives no message.
  • Fixed: Removing the byte-cap guard in _read_doc now fails the suite, 1 failed.
  • Fixed: See [helpers](scripts/ "Helpers"). in references/guide.md names references/scripts/, so only scripts/other.py is reported, 1 of 3.
  • Fixed: Read [guide](.guide.md). with .guide.md naming scripts/run.py now gives no message.
  • Fixed: zeta.py and scripts/a.py, both unnamed, now give 2 of 2 with (scripts/a.py, zeta.py) in path order.
  • Fixed: The module docstring of test_catalog_bundle_unreferenced.py no longer cites scratch.

Verified on c228b31: changed tests, tests pin new code, merge, byte-cap guard, skill.md root link, named symlink and oversize document.

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…up in one pass

A ../ folder from the skill root leaves the skill, so it names no bundled folder. A ../ in a nested
document, past the root, does the same. Pin the read cap: an unnamed file must still be reported.
The name lookup scans each text once, not once per candidate: 50 documents of 208 KB with 1900
unnamed files took 32 s of CPU, and the search now takes about 2 s. The perf test uses that input.

Thread: PRRT_kwDOVBg7g863qCEM
Thread: PRRT_kwDOVBg7g863qCEQ
Review: PRR_kwDOVBg7g88AAAABYFKDfA.a
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by vchamarthi

Still fails: the 18.6 s search. Still holds: the silent skip.
a: fixed in bb0a616, one scan per text. b: Waiting on a decision before I change this: keep the skip silent, with its conditions in the docs (external-content.md 261-267), or emit a note? My recommendation: keep it silent. The finding stays open until you decide.

@napetrov

Copy link
Copy Markdown
Contributor Author

Pushed bb0a616d1 (on c228b31, main merged in 2ffea569a). Since the last round:

  • ../ above the skill root names nothing: bb0a616d1 (CEM)
  • the read-cap test fails on the > variant: bb0a616d1 (CEQ)
  • one name lookup per text: the reviewer's input went from 32 s to about 2 s CPU: bb0a616d1 (PRR a)

The silent skip (PRR b) is held for the maintainers: keep it silent with its conditions in the docs, or emit a note. My recommendation is silent. The reviewers' views stand until then.

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Changes needed: 1 must-fix · re-reviewed bb0a616

Next: @napetrov: fix the 1 must-fix finding and answer 1 open thread from vlad-perevezentsev; then push for a re-review.

Blocking

  • Open question, not answered: thread by @vlad-perevezentsev – SKILL.md naming data/words.txt (350 KB) with scripts/unused.py bundled still gives no message; the docs list the skip, and the decision to count a named oversize .txt as data is still open.

Verified on bb0a616: ✅ changed tests: 409 passed, fail on main · ✅ full suite with main merged in: 21870 passed, its only failures are main's too · ✅ merges onto main · 1 mutation tried · 4 edge inputs tried

6 checks behind it, each with its evidence
Area What holds How
Above skill root ../scripts/, ./../scripts/, ${CLAUDE_SKILL_DIR}/../scripts/ and foo/../scripts/ in SKILL.md each leave scripts/unused.py reported as 1 of 1 ran it, edge input (paths)
Read cap test Changing docs_read >= DOCS_READ_MAX to > fails the suite broke the code, tests caught it
Lookup time test_a_bundle_with_many_files_and_large_documents_is_judged_quickly (1900 unnamed scripts, 50 named documents of about 217 KB) passes within its 10 s bound ran it, edge input (collections)
Changed tests The four changed test files pass on the head (409 passed), including the escaped ; name case tests on the head
Merge The PR merges onto main with no conflict merge onto main
Non-ascii, crlf, u+2028 résumé.py and scripts/ß.py named in text with CRLF or U+2028 line breaks count as named; an unmentioned my script.py is reported ran it, edge input (text)

Comment thread dokimi/dokimi/external/bundle_refs.py Resolved
A search that cannot see every name (over 2000 files, more than 50 named documents, a named
document over 256 KiB, not UTF-8, a symlink or a FIFO, a folder that cannot be listed) gave no
finding at all. It now gives a note, bundle-file-unchecked, with its reason, when something is
still unnamed. Pinned by red tests; the docs and the changelog say so.

Thread: PRRT_kwDOVBg7g863qCEM
Review: PRR_kwDOVBg7g88AAAABYFKDfA.b
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by vchamarthi

Still fails: the 18.6 s search. Still holds: the silent skip.
a: fixed in bb0a616, one scan per text. b: fixed in d573ad3: the check now gives a note (bundle-file-unchecked) with its reason.

@napetrov

Copy link
Copy Markdown
Contributor Author

Pushed d573ad350 (on c228b31, main merged in 2ffea569a). Since the last round:

  • ../ above the skill root names nothing: bb0a616d1 (CEM)
  • the read-cap test fails on the > variant: bb0a616d1 (CEQ)
  • one name lookup per text: the reviewer's input went from 32 s to about 2 s CPU: bb0a616d1 (PRR a)
  • a skipped check is no longer silent: it emits a bundle-file-unchecked note with its reason, with a red-then-green test and docs: d573ad350 (PRR b)

A name escaped as `scripts/my\ run.py` or `scripts/a\;b.py` was read
as `scripts/my/ run.py`, so the file was reported as never named. The
escape set now covers a space and `;` as well as `_`, `*` and `~`; the
name is also read with the separator, so either reading counts.

Thread: PRRT_kwDOVBg7g8630Il3
Red tests first: both names failed before the regex change.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Pushed f406fa941 (on d573ad350). Since the last round:

  • ../ above the skill root names nothing: bb0a616d1 (CEM)
  • the read-cap test fails on the > variant: bb0a616d1 (CEQ)
  • one name lookup per text: the reviewer's input went from 32 s to about 2 s CPU: bb0a616d1 (PRR a)
  • a skipped check is no longer silent: it emits a bundle-file-unchecked note with its reason, with a red-then-green test and docs: d573ad350 (PRR b)
  • a backslash before a space or ; escapes its name too (scripts/my\ run.py, scripts/a\;b.py), red-then-green: f406fa941 (PRRT_kwDOVBg7g8630Il3)

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… one is named

The read cap for named documents is raised from 256 KiB to 1 MiB. A document over the cap is not read, so no claim is made, and the note says which file was truncated.

Thread: PRRT_kwDOVBg7g862mN3b.b
Review: PRR_kwDOVBg7g88AAAABYKG0vQ
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Re: review by vchamarthi

Open question: count a named oversize .txt as data?
No. Cap is 1 MiB now; over it the note says truncated (298dd14).

Comment thread dokimi/dokimi/external/validate.py Resolved
Comment thread dokimi/dokimi/external/validate.py Resolved
Comment thread dokimi/dokimi/external/validate.py Resolved
Comment thread dokimi/dokimi/external/validate.py Resolved
Comment thread dokimi/dokimi/external/validate.py Resolved
Comment thread dokimi/dokimi/external/validate.py Resolved
validate.py went to 846 lines with the bundle-file-unreferenced check,
over the 800-line growth cap on main (799 lines), which fails
tests/test_growth_ratchet.py in CI. The link-file helpers (_file_ref,
_link_files and the regexes they use) move verbatim to
dokimi/external/link_files.py; validate.py re-imports them. No behaviour
change.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov
napetrov requested a review from vchamarthi November 16, 2026 20:42
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Thread: PRRT_kwDOVBg7g864HSqM: already fixed in e3f793e
Thread: PRRT_kwDOVBg7g864HSqT: already fixed in e3f793e
Thread: PRRT_kwDOVBg7g864HSqd: already fixed in e3f793e
Thread: PRRT_kwDOVBg7g864HSqi: already fixed in e3f793e
Thread: PRRT_kwDOVBg7g864HSqk: already fixed in e3f793e

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Head e3f793ea8. Since the last round:

  • The open must-fix (is a named oversize .txt data?) is answered above: the read cap is 1 MiB, and a truncated document gets a note (298dd1426).
  • validate.py is back under the 800-line cap (e3f793ea8), which clears the exai-bot growth-ratchet failures on 3.11 and 3.14.
  • Local on e3f793ea8: full suite 22353 passed, 0 failed. CI: 25 green, 3 skipped.

@vchamarthi ready for re-review.

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I checked 92af3d7: every point from my earlier rounds is fixed, and so are the open ones of @antonwolfy and @vchamarthi. The search is fast now, and a skipped check gives a bundle-file-unchecked note with its reason instead of reading as ok.

Merge: safe now: it merges cleanly into current main, 9 commits behind, and its tests pass on the merge (599 passed, the growth ratchet included). Once it is in, #836 and #839 conflict with it in validate.py and need a rebase.

What I ran, at 92af3d7:

  • make ci-local --offline: 22,301 passed, 0 failed, 72 skipped; every step passes.
  • The search time: 1900 unnamed scripts and 50 named 200 KB documents take 1.5 s CPU, down from 24.7 s at e993313.
  • My repros from all earlier rounds again (folder prefixes, dotted suffixes, nested and skill-dir links, test and hidden folders, unreadable documents, backslash folders and escapes, titled and reference folder links): each gives the expected result, and a plain unnamed script still warns.
  • A 350 KB named word list is read whole; a 2 MB one gives the note naming the file.

Checked, no issues

  • One-pass lookup: _Mentions keeps the old rules: rebuild.py, my-build.py, build.py-old and build.py.bak do not name build.py, and build.py. at the end of a sentence does.
  • Root links: only a root link in SKILL.md names every file (7994848); [top](#top) does not; the docs say so.
  • Note and docs: bundle-file-unchecked is in FINDING_CODES, has its own changelog fragment under Added, and the docs list every skip condition.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The root-link rule from my earlier thread is in the code and the docs, and the new bundle-file-unchecked note closes the silent-skip gap without changing the warn. One nit on the changelog.

What I ran, on a Linux review host at 92af3d7:

  • make ci-local: 34 passed, 0 failed, 14 skipped; 22,354 tests; changed-line coverage 98%.
  • test_catalog_bundle_unreferenced.py alone: 112 passed.
  • A differential check of the one-pass _Mentions lookup against the earlier _mentions rules on 200,000 random texts over the characters that matter (., -, _, /, :, non-ASCII, line breaks): no mismatch, so rebuild.py, my-build.py, build.py-old and build.py.bak still do not name build.py, and build.py. does.

Checked, no issues

  • Skip note: the note appears only when something is unnamed and the search is incomplete; each reason (50-document cap, over 1 MiB, symlink, unlistable folder) has a test, and an unreadable document with every file named gives neither warn nor note.
  • Registry and docs: bundle-file-unchecked is in FINDING_CODES; the docs list the skip conditions and the root-link rule.
  • Repo rules: every non-merge commit has a Signed-off-by; no schema or generated files in the diff; validate.py is under the 800-line cap.

Signed-off-by: Anton Volkov <antonwolfy@gmail.com>

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I pushed a commit per nit, up to 696225a, which fix the nit from my review:

  • 696225a: dokimi/changelog.d/20261109_160000_napetrov_bundle-file-unreferenced.md:3: added the sentence that a skill-root link in SKILL.md (./ or ${CLAUDE_SKILL_DIR}/) names every file and silences the check.

make ci-local at 696225a: 34 passed, 0 failed, 14 skipped.

@antonwolfy

Copy link
Copy Markdown
Contributor

I approved this at 696225a, and CI passes. The merge waits for you:

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I checked 696225a: since my approval at 92af3d7 only the changelog changed, and its new sentence matches the code: a link to the skill root in SKILL.md names every file.

Merge: safe now: it merges cleanly into current main, 14 commits behind, and its tests pass on the merge (599 passed, the growth ratchet included). Once it is in, #836 and #839 conflict with it in validate.py and need a rebase.

What I ran, at 696225a:

  • The PR's catalog tests and the growth ratchet on a merge with current main: 599 passed. I did not rerun make ci-local, since only the changelog changed since 92af3d7, where it passed.

@napetrov

Copy link
Copy Markdown
Contributor Author

@vchamarthi - please review to unblock this

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants