Skip to content

ci: run the shared CI on every system, and under Bun and Deno - #116

Merged
NSchatz merged 7 commits into
mainfrom
ci/systems-and-runtimes
Nov 10, 2026
Merged

NSchatz merged 7 commits into
mainfrom
ci/systems-and-runtimes

Conversation

@NSchatz

@NSchatz NSchatz commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

What

  • ci.yml gains two opt-in jobs, bun and deno, each behind its own input (run-bun, run-deno, both false by default) with a version input (bun-version: "1.4.4", deno-version: "2.9.8"). Each installs with pnpm from the committed lockfile, as verify does, then runs the caller's own test script: bun --bun run test, and deno task test with node taken off PATH.
  • ci.yml gains a working-directory input, "." by default, read by verify, bun and deno only, so this repository can run the workflow against a package of its own.
  • self-check.yml gains a systems job that calls ci.yml itself on ubuntu-latest, windows-latest and macos-latest, at every default Node line (22, 24, 26), with bun and deno on, against a new fixture, test/fixtures/ci-caller/. Each fixture script fails on the hazard it stands for (a carriage return, a dash or PHI checker that cannot run or finds nothing, a run on the wrong system, a missing export target), and lint and phi-scan run the real cosyte-no-emdash and cosyte-phi-scan from @cosyte/script-utils against clean and planted input.
  • verify gains a first step, before the checkout, on every system: git config --global core.autocrlf false and git config --global core.eol lf, so text files check out with LF line endings whatever the runner image's git is set to. The fixture gains its own .gitattributes, * text=auto !eol, so this repository's root eol=lf no longer fixes the fixture to LF and the Windows legs measure the step.
  • scripts/docs-content-check.mjs now runs on Windows. Its entry guard compared paths, and on a Windows runner new URL(import.meta.url).pathname is /D:/a/_temp/docs-content-check.mjs, which matched nothing, so main never ran and the docs-content step exited 0 having checked nothing. It compares file URLs now.
  • README.md gains "Every system and runtime"; CHANGELOG.md's Unreleased section says how a caller opts in.

Unchanged: the job ids verify, prepublish and actionlint; every existing input and its default (os is still '["ubuntu-latest"]'); secrets; permissions; prepublish itself. A caller that passes nothing sees two new skipped checks, ci / bun and ci / deno, and the new first step of verify, which on ubuntu-latest sets the line-ending values the run without it already checked out with.

Why

Our libraries are meant to be tested on Linux, Windows and macOS, and under Bun and Deno as well as Node. The shared CI already took an os list, but nothing had ever run it anywhere but Linux, so nobody knew whether its steps worked elsewhere. This makes each system and runtime something a library turns on with one line, and proves the shared steps on all three systems on every pull request here, before any library adopts them.

Proof

Gate, run before every push: actionlint 1.7.12 with shellcheck 0.11.0, clean; node --test "test/*.test.mjs", 1243 of 1243 pass at the head; node scripts/check-no-emdash.mjs, OK over 120 tracked files; node test/package-lists.mjs, OK. PR title, body and every commit message pass --stdin.

CI. The first push was red on all three macOS legs, which is how the macOS finding below was made. The push that unset the fixture's eol (8f2d4af) was red on all three Windows legs, on purpose: it measured the runner's own checkout before the line-ending step existed (below). Every other push is green; at the head, 21 checks pass and 2 are skipped (systems / actionlint and systems / prepublish, off on purpose). From the logs:

  • Windows line endings. Run 42771076825, at 8f2d4af, the fixture's eol unspecified and no line-ending step, image windows-2025-vs2026 with git 2.56.0.windows.4: format-check named all eleven fixture files as holding a carriage return and failed on Node 22, 24 and 26; the ubuntu and macOS legs of the same run passed it. Run 42771642773, at 80d8a26, with the step: each Windows leg logs the two git config lines and format-check: 11 files, LF line endings only. The LF the earlier runs reported came from this repository's root .gitattributes, * text=auto eol=lf, which overrides core.autocrlf, not from the runner.
  • Windows, the rest: the dash checker passed clean text and found a planted dash (it starts bash and finds Git's); the PHI scanner passed the fixture and found a planted, synthetic SSN shape; the docs-content checker printed Checked 2 file(s), 2 link(s), 0 image(s) and 2 document id(s), which before the guard fix it did not print on any Windows leg.
  • macOS: every step passed, the ESM and CJS smoke step printed esm ok and cjs ok, and the dash checker refused, as the fixture now requires there (see Choices).
  • bun printed test: running under bun 1.4.4 on linux; deno printed test: running under deno 2.9.8 on linux after replacing two PATH entries that held a node (the Node setup-node installed, and /usr/local/bin).

Each new test shown failing first. Against main: test/ci-systems.test.mjs fails 10 of 10 (the jobs, inputs and systems job do not exist); test/docs-content-entry.test.mjs fails (no invokedDirectly), and keeps the old guard as a control that it asserts is false under Node's win32 path rules; test/ci-caller-fixture.test.mjs fails (no fixture). Against the head before the line-ending fix: the fixture's line endings are decided by the checkout failed (.gitignore has its eol fixed) until the fixture's .gitattributes existed; verify sets LF line endings before it checks out and the line-ending step turns a CRLF checkout into LF failed (verify has no step named ...) until the step existed. The second runs the step's own script against a git set up as the Windows image's is (core.autocrlf true, CRLF as the native line ending, in a system config file): its control checks out CRLF without the step, and with it the files come out LF, except one whose .gitattributes sets eol=crlf. At the CI level, the macOS dash checker and the silent docs-content step on Windows were each a red or a missing line before the change that handles them.

Coverage. test/fixtures/ci-caller/tool.mjs: 100% lines, branches and functions. The changed lines of docs-content-check.mjs: covered. The deno step's script has no coverage tool; its tests run it against stubs through every branch (a PATH entry with node and without, a rerun in the same RUNNER_TEMP, an inherited kept variable, a node still reachable, a failing suite), and the bun step's script is run against a stub for a pass and a failure.

Mutation score. Measured with a small harness of our own (not committed): it deletes each added code line and applies operator swaps (===/!==, &&/||, true/false, exit codes, -e/-f, default values and more) one at a time, refuses to run when the unmutated file already fails, and runs the relevant tests plus actionlint.

File Mutants Killed
ci.yml, added lines 138 137
self-check.yml, added lines 20 20
test/fixtures/ci-caller/tool.mjs 139 139
docs-content-check.mjs, changed lines 5, plus 3 by hand 8
ci.yml, the line-ending step, by hand 10 10

The line-ending step's mutants: each git config line dropped, autocrlf to true or input, eol to crlf or native, --global to --local, shell or working-directory dropped, and a Windows-only if:. autocrlf input is killed only by the test that pins the step's text: it checks out the same bytes, so no behavior test can tell them apart. In the fixture's .gitattributes, dropping !eol is killed; dropping text=auto is equivalent, since the fixture inherits it from the root. The one survivor in the earlier table, with its reason: in the deno step, [ -e "${entry}/node" ] to [ -f ... ] differs only for a node that is not a regular file (a directory or a device), which nothing installs, and the check after the loop refuses to run the suite if any node is still reachable. Two hand mutants on the guard survived first and showed redundant code (a resolve that pathToFileURL already does, a Boolean that changed nothing); the code was simplified so neither exists. An earlier run of the harness reported 151 of 151 for ci.yml, which was wrong: an unrelated test was failing at the time, so every mutant looked killed. The baseline check was added for that reason, and the numbers above are from after it.

Time against the 15 minute budget, last run:

What Wall clock
Whole pull request, first job start to last job end 224 s (Self CodeQL, unchanged here, is the longest; it took 81 s on the run before)
Self-check (was 21 to 25 s on main) 61 s
verify on ubuntu, 22 / 24 / 26 18 / 16 / 22 s
verify on macOS, 22 / 24 / 26 18 / 23 / 31 s
verify on Windows, 22 / 24 / 26 35 / 46 / 59 s
bun / deno 16 / 14 s

These are the fixture's numbers, a floor: its scripts do almost nothing. Over the earlier three green runs Windows legs took 41 to 57 s against 13 to 22 s on ubuntu, about 2.5 to 3 times as long. We have not measured a real library on Windows or macOS; the first library to pass os measures that. Locally, on 2026-11-10, hl7's suite (4,114 tests) took 48.5 s under Node 26, 33.1 s under Bun 1.4.4 (one test failed, see README.md) and 37.4 s under Deno 2.9.8.

Choices

Each of these is a question for the owner before merge.

  1. How to prove ci.yml on three systems when this repository is not a package. Taken: a fixture under test/fixtures/ci-caller/ and a working-directory input, "." by default, so self-check.yml calls the real workflow. Rejected: a self-check job copying verify's steps (it proves the copy, and drifts); a pull request in a library pointing at this branch (a second pull request, outside this repository); a package.json at this repository's root (it ships no package by design). Why: only calling the workflow itself proves the workflow. Cost: a caller-visible input that prepublish and actionlint ignore, documented as such. Undo: remove the input and the systems job.
  2. Where the runtime jobs run. Taken: ubuntu-latest only, one job each, so each is one stable check context (ci / bun, ci / deno). Rejected: a matrix over os, which multiplies legs and makes the context bun (<os>). Undo: add a matrix; that renames the context, so a caller requiring it must follow.
  3. What "the test suite under Bun" means. Taken: bun --bun run test, the caller's own test script with every node it starts run by Bun (measured: without --bun the vitest workers report Node). Rejected: bun test, Bun's own runner, which runs only what is compatible with it; a caller-named command input. Undo: change the one line.
  4. How the suite is put on Deno. Taken: deno task test with every PATH entry that holds node replaced by a directory of links to everything else in it, refusing to run while any node is reachable (measured: with Node on PATH, a script's literal node runs Node). Rejected: leaving Node on PATH (a library whose script says node would never run on Deno); dropping those PATH entries whole (loses every other tool in /usr/local/bin); asking each library for a separate test:deno script. Undo: drop the PATH rewrite.
  5. How the runtime jobs install. Taken: pnpm and the committed lockfile, so the suite runs against the tree verify tested. Rejected: bun install or deno install, which resolve again. Undo: swap the install step.
  6. Runtime versions. Taken: exact defaults, Bun 1.4.4 and Deno 2.9.8 (released 2026-10-22 and 2026-10-21), overridable per caller. Rejected: latest, which can turn a library red with no change on its side. Cost: moving a default later is a change every opted-in caller must act on. Undo: change the defaults.
  7. The dash checker on macOS. It needs GNU grep's -P; the macOS runner's BSD grep has none, so it refuses every input ("the scanner cannot match a known em dash"). Taken: the fixture's lint requires that refusal on macOS and turns red the day the checker passes there. Rejected: installing GNU grep in verify on every macOS leg of every caller (costs time for a tool verify itself never runs; callers run the dash checker in the gate-no-emdash workflows, on ubuntu); dropping the dash checker from the fixture; leaving the macOS legs red. The real fix belongs in @cosyte/script-utils. Undo: remove the macOS expectation.
  8. Line endings on Windows. An earlier version of this pull request dropped a line-ending step because the Windows legs checked out LF; that LF came from this repository's root .gitattributes (* text=auto eol=lf), not from the runner, so the measurement did not show what it was read to show. Measured since, with the fixture's eol unset: the Windows image checks text files out CRLF. Taken: a first step in verify, before the checkout, on every system, setting core.autocrlf false and core.eol lf in the runner's global git config, so a file with no eol attribute checks out as committed, LF for text, whatever the image's git says; a file whose eol the caller's .gitattributes sets keeps it. On ubuntu and macOS the same values were already in effect (the run without the step checked the fixture out LF there), so no current caller's bytes change. Rejected: no step, leaving it to each caller's .gitattributes (all 15 repositories that call ci.yml, checked on 2026-11-10, have * text=auto eol=lf today, but a new one may not, and its Windows legs would then read different bytes than its Linux legs); a Windows-only if: (the same step on every system is simpler and changes nothing on the other two); core.autocrlf input alone (the same bytes, but it leans on git's rule that input silences core.eol, which is harder to read than the two settings that say LF); a report-only step that prints the bytes without setting anything (it would measure, but every caller without eol=lf would still get CRLF). Undo: delete the step; the fixture's format:check then goes red on Windows, which is the tripwire.
  9. The docs-content guard fix in this pull request. Taken: here, because it is a shared step of verify that checked nothing on Windows, which is what this pull request exists to prove. Rejected: a separate pull request. No caller is affected today: none runs verify on Windows.
  10. Which Node lines the systems job tests. Taken: the default (22, 24, 26), so it proves what a caller gets by passing nothing. Rejected: only 24 and 26. Changing the node-versions default is not part of this pull request. Undo: pass node-versions in systems.
  11. The os default stays '["ubuntu-latest"]'. Every library turns on Windows and macOS by passing os, one at a time. Rejected: changing the default, which would change every caller at once and could exceed the time budget for a library nobody has measured. Undo: change the default, a change every caller must act on.
  12. The fixture's own .gitattributes, * text=auto !eol. Taken: so the Windows legs measure the checkout verify sets up rather than this repository's eol=lf, and format:check can go red. Rejected: removing eol=lf from the root (it protects every other file here). Undo: delete the file; the fixture is then LF on every checkout again and the tripwire cannot fire.
  13. How the runner's default was measured. Taken: one push with the fixture's eol unset and no step, red on the three Windows legs on purpose, then the step. Rejected: a permanent job in self-check.yml checking out with the image's own settings (a new job to keep for one fact); citing the runner image's install script instead of a run. Undo: nothing to undo; the red run (42771076825) stays in the history.

Not done here, offered for later: the docs-content step still accepts a silent exit 0 from the checker. Requiring its summary line would have caught the Windows defect without reading logs.

Kind: code

https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U

…nst a fixture

ci.yml gains two jobs, bun and deno, each behind its own input (run-bun,
run-deno) and off by default, that install with pnpm from the lockfile
and run the caller's own test script under that runtime. A
working-directory input, "." by default, lets this repository run the
workflow against a package under test/.

self-check.yml gains a systems job that calls ci.yml on ubuntu-latest,
windows-latest and macos-latest, with both runtimes on, against
test/fixtures/ci-caller: a package whose scripts each fail on the hazard
they stand for, and which runs the real dash and PHI checkers from
@cosyte/script-utils against clean and planted input.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…very system and runtime

The first systems run showed the dash checker from @cosyte/script-utils
cannot run on macOS: it needs GNU grep's -P, and the runner's BSD grep
has none, so it refuses every input. The fixture's lint now requires
that refusal on macOS and turns red the day the checker passes there.

The same run showed actions/checkout on windows-latest writes LF line
endings, so verify needs no line ending step; the fixture's format
check stays as the tripwire.

README.md gains "Every system and runtime", and CHANGELOG.md's
Unreleased section says what a caller does to opt in.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…never matched

The checker decided whether to run main by comparing resolve(argv[1])
with resolve(new URL(import.meta.url).pathname). On a Windows runner
that pathname is /D:/a/_temp/docs-content-check.mjs, which names no
path argv[1] can, so main never ran and the docs-content step exited 0
having checked nothing. The systems run's Windows legs printed no
summary line, which is how it showed. The guard now compares file URLs.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…nst a stub

Mutation testing over the lines this branch adds to ci.yml found
deletions no test noticed: the input descriptions, the runtime jobs'
persist-credentials and cache settings, the bun step's version line,
and the deno step's cleanup of a previous run. Each is now asserted,
and the bun step is run against a stub as the deno step already was.
The deno step's `export PATH` is dropped: PATH is always exported.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
The root .gitattributes says `* text=auto eol=lf`, and an eol attribute
overrides core.autocrlf, so every checkout wrote the fixture with LF on
every system and its format:check could never see a CRLF checkout. The
fixture's own .gitattributes leaves eol unspecified, so the checkout's
git settings decide, and a Windows leg now measures them.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
With the fixture's eol left unspecified, windows-latest (image
windows-2025-vs2026, git 2.56.0.windows.4) checked all eleven of its
files out with CRLF and format:check failed on Node 22, 24 and 26, while
ubuntu and macOS checked them out LF. verify's first step now sets
core.autocrlf false and core.eol lf before the checkout, so the runner
image does not decide. A file whose eol a repository's .gitattributes
sets keeps it, since the attribute overrides both settings.

The earlier LF result came from this repository's root eol=lf, not from
the runner; README.md and CHANGELOG.md now say so.

Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
@NSchatz
NSchatz merged commit 4522886 into main Nov 10, 2026
26 checks passed
@NSchatz
NSchatz deleted the ci/systems-and-runtimes branch November 10, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant