[NSchatz]

What

The first part of the shared CI jobs for the testing standard (cosyte/config, documentation/decisions/0004-the-testing-standard.md). Everything is off until a caller opts in.

• ci.yml takes two inputs, run-coverage-ratchet and run-mutation, both false by default. They add two jobs:
  • coverage-ratchet runs pnpm test:coverage, then cosyte-ratchet check --skip mutation, with --base origin/<base branch> on a pull request.
  • mutation runs on pull requests only. It runs cosyte-mutate changed --base origin/<base branch>, then cosyte-ratchet check --skip coverage --base origin/<base branch>, and keeps reports/mutation/ as an artifact.
  • Both run beside verify with no needs:, on ubuntu-latest at the first Node line, with the whole history checked out. Each writes the ratchet's report to the job summary.
• New reusable nightly-properties.yml, job properties. It runs the caller's test-cmd (default pnpm test) with COSYTE_PROPERTY_RUNS set to property-runs (default 10000). It strips colour codes from the log and keeps every Replay exactly: seed and path once, with the test and the property that printed it. The seeds go in the job summary and seeds.txt, and the log and seeds.txt go in the property-run artifact.
• New reusable weekly-mutation.yml, job mutation. It runs the caller's mutate-cmd (default pnpm exec cosyte-mutate full), then cosyte-ratchet check --skip coverage. The job summary shows each package's score against its floor, and reports/mutation/ goes in an artifact. A score below its floor turns the run red. The job has contents: read and never commits.
• self-check.yml runs all four jobs against test/fixtures/ci-caller/, and a new quality-artifacts job checks what the two scheduled runs kept.
• README.md has a new section, "The testing standard's jobs", and CHANGELOG.md Unreleased says how a caller opts in.

Why

The standard asks for these checks: a coverage and mutation floor that only rises, a pull request's own code held to 100, a deep property run every night with its seeds kept, and a full mutation run every week reported against the floor. Each library will run them from these shared workflows rather than writing its own. The tools are on cosyte/config main but not yet on npm. So every input defaults to off, and a merge changes no caller. Callers opt in after the owner approves that release.

Proof

• Tests failed first: 37 of the new tests were red before the workflows existed. Now all 1284 pass with node --test "test/*.test.mjs".
• test/testing-standard-jobs.test.mjs runs every shell script these jobs add against a stub pnpm. It checks:
  • the arguments each step passes to the tools, with and without a base branch;
  • that a step with no base branch refuses rather than mutating nothing;
  • the summary each step writes;
  • that every failure (exit 1 or 2) reaches the job's exit status;
  • that the seed recorder dedupes and strips colour, and copes with a passing suite, a suite that failed for another reason, and a suite that never ran;
  • that a depth that is not a whole number is refused before the suite starts.
• self-check.yml runs all four jobs on a real runner against the fixture:
  • systems turns on both ci.yml jobs;
  • nightly-properties and weekly-mutation run the two reusables;
  • quality-artifacts downloads what they kept. It checks that the planted seed is in seeds.txt once, at depth 10000, and that the weekly artifact holds the full report and the ratchet's report.
• The fixture's cosyte-mutate and cosyte-ratchet are stand-ins in test/fixtures/quality-stubs/, tested in test/quality-stubs.test.mjs. They accept only the command lines the real tools accept and refuse raise. Each one checks that the base ref has a merge base and that every report it would read is on disk. They judge nothing.
• I rehearsed all of this locally in a scratch clone: a real pnpm install --frozen-lockfile of the fixture, every fixture script with the real dash and PHI checkers, and every new step script run with the real pnpm.
• actionlint 1.7.12 with shellcheck 0.11.0 is clean, and so are the dash gate and the package lists.

What we could not prove before a caller adopts:

• whether the real tools pass a caller's tree on a hosted runner;
• whether a changed-files mutation run fits the 15 minute budget for a pull request, since a large changed file is mutated whole;
• whether a full mutation score is steady enough week to week for the ratchet's exact comparison. Decision 0004 measured it moving 0.36 points between two runs of unchanged code, and leaves a tolerance to the owner.

Choices

Taken on the recommendation, for the owner to review:

1. Opt-in inputs on ci.yml for the pull request jobs, new reusables for the scheduled ones. The pull request jobs follow the precedent of run-bun and run-deno, so a caller adds one line and gets ci / coverage-ratchet and ci / mutation. A reusable cannot carry a schedule, so the nightly and weekly runs are their own files, like nightly-fuzz.yml.
2. The coverage ratchet is its own job, not a step in verify. This keeps verify byte for byte as it was, and judges the floor on one system and one Node line, not on every matrix leg. The cost is a second coverage run in parallel.
3. The coverage ratchet also runs on a push, without --base. It then compares the measurements with the floor file as it stands, as decision 0004 says coverage is compared on every push to main. mutation runs on pull requests only, because a changed-files run needs a base.
4. The tools run through pnpm exec, from the caller's own lockfile, not through npx at a version this repository picks. The caller pins the version.
5. The pull request jobs run fixed commands. A repository with several packages and several Stryker configs cannot use the mutation job as it stands. The weekly run takes mutate-cmd for that case.
6. Timeouts: 30 minutes for coverage-ratchet, 60 for mutation (the same bound cosyte/config uses), 60 for the nightly run, 180 for the weekly run.
7. A drop below a floor turns the weekly run red, so it is noticed. Decision 0004 expects that and leaves any tolerance to the owner.
8. The defaults for node-version on the new reusables are "22", matching nightly-fuzz.yml and the first default Node line of ci.yml.
9. Artifacts: 14 days for mutation-changed, 30 days for property-run and mutation-report. Actions are pinned by SHA: actions/upload-artifact v7.0.2 (already used here) and actions/download-artifact v8.0.2 (new, in self-check.yml only).
10. working-directory's description now names every job that reads it. Its default and behavior are unchanged.
11. No needs: in ci.yml. The one new needs: is self-check.yml's quality-artifacts, which is no caller's context. It is added to the pinned set in test/skipped-required-context.test.mjs.

No job id, existing input default, secret or permission changed. The reference note will list two new inputs, two new jobs and two new files, and nothing a caller has to act on.

Kind: code

https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U