[PerryLink]

Catalog audit: advisory ENTRY static checks over npm-backed entries

Problem

The catalog lists thousands of npm-backed entries, but nothing in the repo can tell
whether a listed tarball still resolves and still passes the ecosystem's ENTRY static
checks. Today that is a manual npx invocation per plugin, so packaging regressions,
renamed repositories, and catalog/npm attribution drift stay invisible until someone
installs the plugin.

Change

Three new files. No runtime code, no package.json script, and no dependency changes
(package.json and the lockfile are untouched).

• scripts/audit-catalog.mjs — read-only, dependency-free audit (Node built-ins +
  the npm CLI):
  • reads the catalog from the source the app already proxies
    (https://api.dsh-plugin.org/plugins.zh.json, per src/server/http/catalog.ts);
    --catalog <file|url> switches to a local snapshot or another URL;
  • keeps only entries that carry an npm package name, samples --limit N (default 20,
    hard cap 50) in catalog order after deduplicating names, and records that rule in
    the report;
  • npm pack <name> --ignore-scripts --registry=https://registry.npmjs.org/ into
    os.tmpdir(), extracts with a built-in tar reader (no new dependency), and checks
    that the npm repository matches the catalog repo — a mismatch is recorded as
    not_run with reason repository_mismatch and the checker is not run for it;
  • runs npx -y dsh-standards@1.6.9 check <dir> --scope entry --entry-claim static --no-exec --json and keeps conclusion.status, counts, coverage, duration and
    reason per package;
  • 404s, timeouts and pack/extract/checker failures become not_run with a reason and
    never abort the round; the temporary tree is removed at the end; the process exits
    0 unconditionally.
• .github/workflows/catalog-audit.yml — manual (workflow_dispatch) only,
  permissions: contents: read, Node 24 (the checker's engines require >= 24.9),
  uploads audit-report.json + step-summary.md as artifacts and appends the summary
  to the run page. No schedule, no comments, no commits, no writes to main.
• tests/audit-catalog.test.mjs — unit tests for entry filtering, sampling and
  clamping, checker-status mapping, repository back-link comparison, and tar path
  safety (17 tests; no network, no child processes). It is .mjs because the subject
  is a build-free script and tsconfig.test.json is strict without allowJs.

Boundaries

• Advisory, non-blocking. Manual-only workflow, never a required check; the script
  always exits 0. Nothing here gates merges, admission, or installs.
• Read-only. No catalog, admission, or ordering changes. A run writes only its own
  two artifacts (plus the run-page summary). No package code is executed: npm pack
  never installs, --ignore-scripts is passed, and the checker runs with --no-exec.
• Not a ranking, score, certification, or security conclusion. Coverage is a static
  ENTRY sample and is partial; not_run means "not assessed here", not "bad". Both the
  JSON (disclaimer) and the Markdown summary say this explicitly.

Verification

npm ci
npm run typecheck                 # clean
npm test                          # 276 tests, 0 fail (17 new)
node scripts/audit-catalog.mjs --limit 20 --out /tmp/catalog-audit
# → exit 0; audit-report.json + step-summary.md written from the live catalog

The 404, wrong-repository, timeout and unparsable-report paths were checked locally
(fixtures for the pure mapping; a nonexistent package and a mismatched catalog repo for
the end-to-end path) and all of them land on not_run without aborting the run.

Rollback

Revert the commit, or delete the three files. Nothing else references them: no
package.json script, no existing workflow, no runtime import. No data or config
migration is involved.

Notes

• The checker is pinned to dsh-standards@1.6.9 so repeated runs stay comparable;
  bumping it is one constant in the script plus one assertion in the tests.
• npx fetches that pinned checker from npm on first use and reuses the npm cache
  afterwards. The workflow is manual and read-only, so this remains an explicit,
  maintainer-initiated action.

[yacuo]

Thanks for the thoughtful proposal and for the real work behind it — the problem statement is fair: catalog entries can drift from npm, and today that only surfaces when someone installs.

We are not going to merge this in its current form. With a catalog this large, sampling a few dozen entries cannot give meaningful coverage, and a manually triggered, non-blocking report adds an ongoing maintenance surface — a bespoke tar reader plus a pinned third-party checker — without changing what users actually see. That trade-off is not one we want to take on right now.

The direction we do consider valuable is entry-time verification rather than periodic sampling: checking npm availability and the npm repository back-link when an entry is ingested or a version changes, with ENTRY checks cached per version. That is deterministic, covers everything that enters the catalog, and costs very little. If you would like to pursue that, we would be glad to discuss a much smaller, metadata-only proposal.

Either way, thank you for taking the time. The write-up and the tests are appreciated, and this is a decision about maintenance scope, not about the quality of your work.