Repository navigation
fix(cli): skip runtime by-products when fingerprinting a skill bundle - #5654
Conversation
The skill hash walked every file in a skill directory, so Python __pycache__/*.pyc left by running a skill script (or node_modules, editor swap files) changed it. Locally gen-skills-manifest then wrote a hash CI's clean checkout could not reproduce, and an installed skill that had run a script read as outdated forever. Skip those by-products on both the source and the installed side so the hash stays a function of authored content. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Edit accuracy: accurate 271 (base branch 271), smooth 154 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (1)
|
jrusso1020
left a comment
There was a problem hiding this comment.
Approved at 2aa4fdf92623ca2a38b9d56e4ce256cbf80af011.
- One walker for both sides.
listFilesSortedis the only file walk behindhashSkillBundle, so the published manifest (buildManifest) and the installed-skill check (hashInstalled) skip the same names. The bundle hash has no other implementation inpackages/orscripts/(publishProject.tshas its own ignore set, but for a different purpose). - Published hashes don't move. Nothing tracked under
skills/matches the skip set: I checkedgit ls-files skillsagainst every dir name and suffix. Soskills-manifest.jsonneeds no regeneration, and installed skills that are current stay current. - Tests. The
.pyccase also edits the.pyand asserts the hash changes, so the skip can't hide a real edit. The second case coversnode_modules,.DS_Store,.swpand~. - Scope. Directories are skipped by exact name, files by exact name or suffix. The
~suffix is the broadest rule, but no skill file ends in~.
Required CI is green.
— Rames
miguel-heygen
left a comment
There was a problem hiding this comment.
Reviewed exact head 2aa4fdf92623ca2a38b9d56e4ce256cbf80af011. No blocker found.
packages/cli/src/utils/skillsManifest.ts:179 applies the same traversal to source manifest generation and installed-bundle checks, so running a script no longer makes an unchanged install look stale. Authored files still affect the fingerprint.
Independent verification: 61 manifest tests pass; restoring the base implementation fails exactly the two added tests. A separate filesystem probe through real buildManifest → checkSkills kept an install current after adding every excluded cache directory/file class, then marked a real .py edit outdated. No tracked skill path matches the new skip set; the committed 21-skill manifest passes regeneration check without changes. Actual hosted runtime logs execute all 61 manifest tests; current-head checks have no failures or pending jobs.
Limits: synthetic installed directories, not a real Claude/Cursor/Codex profile installation. No install, paid call, merge or release performed.
— Magi (desktop-review)
Verdict: APPROVE
Reasoning: The shared hashing path removes runtime noise symmetrically while retaining authored-change detection, supported by direct integration and negative-control checks.
What
hashSkillBundle(packages/cli/src/utils/skillsManifest.ts) now skips files a skill directory picks up from being used rather than authored:__pycache__/,.pytest_cache/,.mypy_cache/,.ruff_cache/,node_modules/directories, and.DS_Store,Thumbs.db,*.pyc,*.pyo,*.swp,*.swo,*~files. Previously only.DS_Storewas skipped.Why
The same hash builds the published
skills-manifest.jsonfromskills/and compares an installed skill against it. Every file in the directory went into it, so:media-use/audio/scripts/lib/gemini-auth_test.py, which importsgemini-auth.pyand leaves__pycache__/gemini-auth.cpython-*.pyc) changed the local hash.gen:skills-manifestthen wrote a hash CI's clean checkout could not reproduce, and--checkfailed with "skills-manifest.json is out of date". This happened twice on feat(skills)!: publish seven knowledge skills aligned with Studio and retire the workflow skills #5651.outdatedpermanently, since no update removes the bytecode.Reproduction on
mainbefore this change:With this change the same command prints
✓ skills-manifest.json is in sync (21 skills).Related work
Refs #5651. That branch's commit
3cd764422("regenerate the skills manifest from tracked files only") changes onlyskills-manifest.jsondata (a regeneration after the stray bytecode was removed); it does not change the generator or the hash. This PR does not touchskills-manifest.json, so the two do not conflict, and once this lands a local regeneration on #5651 can no longer pick up bytecode.How
listFilesSorted, which both the source path (buildManifest→gen-skills-manifest) and the installed path (hashInstalled/checkSkills) go through, so the two sides stay symmetric.skills/matches the skip set (checked withgit ls-files skills), so every published hash and file count is unchanged andskills-manifest.jsonneeds no regeneration. Installed skills that are already current stay current.git ls-fileswalk: the installed side has no git, and the hash must be computed identically on both sides.Test plan
packages/cli/src/utils/skillsManifest.test.ts—__pycache__/*.pycand a stray.pycleave the hash and file count unchanged while a real.pyedit changes it;node_modules/,.DS_Store,.swp,~backups leave it unchanged. Both fail without the fix (2 failed / 59 passed) and pass with it.cd packages/cli && bun x vitest run src/utils/skillsManifest.test.ts src/utils/skillsMirror.test.ts src/utils/skillsUpdateCheck.test.ts src/commands/skills.test.ts→ 4 files, 133 tests passedbun packages/cli/scripts/gen-skills-manifest.ts --check→✓ skills-manifest.json is in sync (21 skills), both on a clean tree and with a__pycache__/*.pycplanted inmedia-use(the reproduction above)bun run lint→ 0 warnings, 0 errorsbun x oxfmt --checkon both changed files → correct formatbun x tsc --noEmit -p packages/cli(afterbun run --cwd packages/core build) → exit 0, no errorsnpx -y fallow audit --base origin/main --threads 4 --fail-on-issues→ no issues in 2 changed filesCOMMENT_CHECK_BASE=origin/main node scripts/check-comment-citations.mjs→ OK;node scripts/comment-ratchet.mjs→ at or under base shareNo visible change
CLI internals only: the skill fingerprint ignores runtime by-products. Published hashes are unchanged.
🤖 Generated with Claude Code