Repository navigation
ci: run the shared CI on every system, and under Bun and Deno - #116
Merged
Merged
Conversation
…nst a fixture ci.yml gains two jobs, bun and deno, each behind its own input (run-bun, run-deno) and off by default, that install with pnpm from the lockfile and run the caller's own test script under that runtime. A working-directory input, "." by default, lets this repository run the workflow against a package under test/. self-check.yml gains a systems job that calls ci.yml on ubuntu-latest, windows-latest and macos-latest, with both runtimes on, against test/fixtures/ci-caller: a package whose scripts each fail on the hazard they stand for, and which runs the real dash and PHI checkers from @cosyte/script-utils against clean and planted input. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…very system and runtime The first systems run showed the dash checker from @cosyte/script-utils cannot run on macOS: it needs GNU grep's -P, and the runner's BSD grep has none, so it refuses every input. The fixture's lint now requires that refusal on macOS and turns red the day the checker passes there. The same run showed actions/checkout on windows-latest writes LF line endings, so verify needs no line ending step; the fixture's format check stays as the tripwire. README.md gains "Every system and runtime", and CHANGELOG.md's Unreleased section says what a caller does to opt in. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…never matched The checker decided whether to run main by comparing resolve(argv[1]) with resolve(new URL(import.meta.url).pathname). On a Windows runner that pathname is /D:/a/_temp/docs-content-check.mjs, which names no path argv[1] can, so main never ran and the docs-content step exited 0 having checked nothing. The systems run's Windows legs printed no summary line, which is how it showed. The guard now compares file URLs. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
…nst a stub Mutation testing over the lines this branch adds to ci.yml found deletions no test noticed: the input descriptions, the runtime jobs' persist-credentials and cache settings, the bun step's version line, and the deno step's cleanup of a previous run. Each is now asserted, and the bun step is run against a stub as the deno step already was. The deno step's `export PATH` is dropped: PATH is always exported. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
The root .gitattributes says `* text=auto eol=lf`, and an eol attribute overrides core.autocrlf, so every checkout wrote the fixture with LF on every system and its format:check could never see a CRLF checkout. The fixture's own .gitattributes leaves eol unspecified, so the checkout's git settings decide, and a Windows leg now measures them. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
With the fixture's eol left unspecified, windows-latest (image windows-2025-vs2026, git 2.56.0.windows.4) checked all eleven of its files out with CRLF and format:check failed on Node 22, 24 and 26, while ubuntu and macOS checked them out LF. verify's first step now sets core.autocrlf false and core.eol lf before the checkout, so the runner image does not decide. A file whose eol a repository's .gitattributes sets keeps it, since the attribute overrides both settings. The earlier LF result came from this repository's root eol=lf, not from the runner; README.md and CHANGELOG.md now say so. Claude-Session: https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
ci.ymlgains two opt-in jobs,bunanddeno, each behind its own input (run-bun,run-deno, bothfalseby default) with a version input (bun-version: "1.4.4",deno-version: "2.9.8"). Each installs with pnpm from the committed lockfile, asverifydoes, then runs the caller's owntestscript:bun --bun run test, anddeno task testwithnodetaken off PATH.ci.ymlgains aworking-directoryinput,"."by default, read byverify,bunanddenoonly, so this repository can run the workflow against a package of its own.self-check.ymlgains asystemsjob that callsci.ymlitself onubuntu-latest,windows-latestandmacos-latest, at every default Node line (22, 24, 26), withbunanddenoon, against a new fixture,test/fixtures/ci-caller/. Each fixture script fails on the hazard it stands for (a carriage return, a dash or PHI checker that cannot run or finds nothing, a run on the wrong system, a missing export target), andlintandphi-scanrun the realcosyte-no-emdashandcosyte-phi-scanfrom@cosyte/script-utilsagainst clean and planted input.verifygains a first step, before the checkout, on every system:git config --global core.autocrlf falseandgit config --global core.eol lf, so text files check out with LF line endings whatever the runner image's git is set to. The fixture gains its own.gitattributes,* text=auto !eol, so this repository's rooteol=lfno longer fixes the fixture to LF and the Windows legs measure the step.scripts/docs-content-check.mjsnow runs on Windows. Its entry guard compared paths, and on a Windows runnernew URL(import.meta.url).pathnameis/D:/a/_temp/docs-content-check.mjs, which matched nothing, somainnever ran and the docs-content step exited 0 having checked nothing. It compares file URLs now.Unchanged: the job ids
verify,prepublishandactionlint; every existing input and its default (osis still'["ubuntu-latest"]'); secrets; permissions;prepublishitself. A caller that passes nothing sees two new skipped checks,ci / bunandci / deno, and the new first step ofverify, which onubuntu-latestsets the line-ending values the run without it already checked out with.Why
Our libraries are meant to be tested on Linux, Windows and macOS, and under Bun and Deno as well as Node. The shared CI already took an
oslist, but nothing had ever run it anywhere but Linux, so nobody knew whether its steps worked elsewhere. This makes each system and runtime something a library turns on with one line, and proves the shared steps on all three systems on every pull request here, before any library adopts them.Proof
Gate, run before every push: actionlint 1.7.12 with shellcheck 0.11.0, clean;
node --test "test/*.test.mjs", 1243 of 1243 pass at the head;node scripts/check-no-emdash.mjs, OK over 120 tracked files;node test/package-lists.mjs, OK. PR title, body and every commit message pass--stdin.CI. The first push was red on all three macOS legs, which is how the macOS finding below was made. The push that unset the fixture's eol (8f2d4af) was red on all three Windows legs, on purpose: it measured the runner's own checkout before the line-ending step existed (below). Every other push is green; at the head, 21 checks pass and 2 are skipped (
systems / actionlintandsystems / prepublish, off on purpose). From the logs:format-checknamed all eleven fixture files as holding a carriage return and failed on Node 22, 24 and 26; the ubuntu and macOS legs of the same run passed it. Run 42771642773, at 80d8a26, with the step: each Windows leg logs the twogit configlines andformat-check: 11 files, LF line endings only. The LF the earlier runs reported came from this repository's root.gitattributes,* text=auto eol=lf, which overridescore.autocrlf, not from the runner.bashand finds Git's); the PHI scanner passed the fixture and found a planted, synthetic SSN shape; the docs-content checker printedChecked 2 file(s), 2 link(s), 0 image(s) and 2 document id(s), which before the guard fix it did not print on any Windows leg.esm okandcjs ok, and the dash checker refused, as the fixture now requires there (see Choices).bunprintedtest: running under bun 1.4.4 on linux;denoprintedtest: running under deno 2.9.8 on linuxafter replacing two PATH entries that held anode(the Nodesetup-nodeinstalled, and/usr/local/bin).Each new test shown failing first. Against
main:test/ci-systems.test.mjsfails 10 of 10 (the jobs, inputs andsystemsjob do not exist);test/docs-content-entry.test.mjsfails (noinvokedDirectly), and keeps the old guard as a control that it asserts is false under Node's win32 path rules;test/ci-caller-fixture.test.mjsfails (no fixture). Against the head before the line-ending fix:the fixture's line endings are decided by the checkoutfailed (.gitignore has its eol fixed) until the fixture's.gitattributesexisted;verify sets LF line endings before it checks outandthe line-ending step turns a CRLF checkout into LFfailed (verify has no step named ...) until the step existed. The second runs the step's own script against a git set up as the Windows image's is (core.autocrlf true, CRLF as the native line ending, in a system config file): its control checks out CRLF without the step, and with it the files come out LF, except one whose.gitattributessetseol=crlf. At the CI level, the macOS dash checker and the silent docs-content step on Windows were each a red or a missing line before the change that handles them.Coverage.
test/fixtures/ci-caller/tool.mjs: 100% lines, branches and functions. The changed lines ofdocs-content-check.mjs: covered. The deno step's script has no coverage tool; its tests run it against stubs through every branch (a PATH entry withnodeand without, a rerun in the sameRUNNER_TEMP, an inheritedkeptvariable, anodestill reachable, a failing suite), and the bun step's script is run against a stub for a pass and a failure.Mutation score. Measured with a small harness of our own (not committed): it deletes each added code line and applies operator swaps (
===/!==,&&/||,true/false, exit codes,-e/-f, default values and more) one at a time, refuses to run when the unmutated file already fails, and runs the relevant tests plus actionlint.ci.yml, added linesself-check.yml, added linestest/fixtures/ci-caller/tool.mjsdocs-content-check.mjs, changed linesci.yml, the line-ending step, by handThe line-ending step's mutants: each
git configline dropped,autocrlftotrueorinput,eoltocrlfornative,--globalto--local,shellorworking-directorydropped, and a Windows-onlyif:.autocrlf inputis killed only by the test that pins the step's text: it checks out the same bytes, so no behavior test can tell them apart. In the fixture's.gitattributes, dropping!eolis killed; droppingtext=autois equivalent, since the fixture inherits it from the root. The one survivor in the earlier table, with its reason: in the deno step,[ -e "${entry}/node" ]to[ -f ... ]differs only for anodethat is not a regular file (a directory or a device), which nothing installs, and the check after the loop refuses to run the suite if anynodeis still reachable. Two hand mutants on the guard survived first and showed redundant code (aresolvethatpathToFileURLalready does, aBooleanthat changed nothing); the code was simplified so neither exists. An earlier run of the harness reported 151 of 151 forci.yml, which was wrong: an unrelated test was failing at the time, so every mutant looked killed. The baseline check was added for that reason, and the numbers above are from after it.Time against the 15 minute budget, last run:
main)verifyon ubuntu, 22 / 24 / 26verifyon macOS, 22 / 24 / 26verifyon Windows, 22 / 24 / 26bun/denoThese are the fixture's numbers, a floor: its scripts do almost nothing. Over the earlier three green runs Windows legs took 41 to 57 s against 13 to 22 s on ubuntu, about 2.5 to 3 times as long. We have not measured a real library on Windows or macOS; the first library to pass
osmeasures that. Locally, on 2026-11-10,hl7's suite (4,114 tests) took 48.5 s under Node 26, 33.1 s under Bun 1.4.4 (one test failed, see README.md) and 37.4 s under Deno 2.9.8.Choices
Each of these is a question for the owner before merge.
ci.ymlon three systems when this repository is not a package. Taken: a fixture undertest/fixtures/ci-caller/and aworking-directoryinput,"."by default, soself-check.ymlcalls the real workflow. Rejected: a self-check job copyingverify's steps (it proves the copy, and drifts); a pull request in a library pointing at this branch (a second pull request, outside this repository); apackage.jsonat this repository's root (it ships no package by design). Why: only calling the workflow itself proves the workflow. Cost: a caller-visible input thatprepublishandactionlintignore, documented as such. Undo: remove the input and thesystemsjob.ubuntu-latestonly, one job each, so each is one stable check context (ci / bun,ci / deno). Rejected: a matrix overos, which multiplies legs and makes the contextbun (<os>). Undo: add a matrix; that renames the context, so a caller requiring it must follow.bun --bun run test, the caller's owntestscript with everynodeit starts run by Bun (measured: without--bunthe vitest workers report Node). Rejected:bun test, Bun's own runner, which runs only what is compatible with it; a caller-named command input. Undo: change the one line.deno task testwith every PATH entry that holdsnodereplaced by a directory of links to everything else in it, refusing to run while anynodeis reachable (measured: with Node on PATH, a script's literalnoderuns Node). Rejected: leaving Node on PATH (a library whose script saysnodewould never run on Deno); dropping those PATH entries whole (loses every other tool in/usr/local/bin); asking each library for a separatetest:denoscript. Undo: drop the PATH rewrite.verifytested. Rejected:bun installordeno install, which resolve again. Undo: swap the install step.latest, which can turn a library red with no change on its side. Cost: moving a default later is a change every opted-in caller must act on. Undo: change the defaults.-P; the macOS runner's BSD grep has none, so it refuses every input ("the scanner cannot match a known em dash"). Taken: the fixture'slintrequires that refusal on macOS and turns red the day the checker passes there. Rejected: installing GNU grep inverifyon every macOS leg of every caller (costs time for a toolverifyitself never runs; callers run the dash checker in thegate-no-emdashworkflows, on ubuntu); dropping the dash checker from the fixture; leaving the macOS legs red. The real fix belongs in@cosyte/script-utils. Undo: remove the macOS expectation..gitattributes(* text=auto eol=lf), not from the runner, so the measurement did not show what it was read to show. Measured since, with the fixture's eol unset: the Windows image checks text files out CRLF. Taken: a first step inverify, before the checkout, on every system, settingcore.autocrlf falseandcore.eol lfin the runner's global git config, so a file with no eol attribute checks out as committed, LF for text, whatever the image's git says; a file whose eol the caller's.gitattributessets keeps it. On ubuntu and macOS the same values were already in effect (the run without the step checked the fixture out LF there), so no current caller's bytes change. Rejected: no step, leaving it to each caller's.gitattributes(all 15 repositories that callci.yml, checked on 2026-11-10, have* text=auto eol=lftoday, but a new one may not, and its Windows legs would then read different bytes than its Linux legs); a Windows-onlyif:(the same step on every system is simpler and changes nothing on the other two);core.autocrlf inputalone (the same bytes, but it leans on git's rule thatinputsilencescore.eol, which is harder to read than the two settings that say LF); a report-only step that prints the bytes without setting anything (it would measure, but every caller withouteol=lfwould still get CRLF). Undo: delete the step; the fixture'sformat:checkthen goes red on Windows, which is the tripwire.verifythat checked nothing on Windows, which is what this pull request exists to prove. Rejected: a separate pull request. No caller is affected today: none runsverifyon Windows.systemsjob tests. Taken: the default (22, 24, 26), so it proves what a caller gets by passing nothing. Rejected: only 24 and 26. Changing thenode-versionsdefault is not part of this pull request. Undo: passnode-versionsinsystems.osdefault stays'["ubuntu-latest"]'. Every library turns on Windows and macOS by passingos, one at a time. Rejected: changing the default, which would change every caller at once and could exceed the time budget for a library nobody has measured. Undo: change the default, a change every caller must act on..gitattributes,* text=auto !eol. Taken: so the Windows legs measure the checkoutverifysets up rather than this repository'seol=lf, andformat:checkcan go red. Rejected: removingeol=lffrom the root (it protects every other file here). Undo: delete the file; the fixture is then LF on every checkout again and the tripwire cannot fire.self-check.ymlchecking out with the image's own settings (a new job to keep for one fact); citing the runner image's install script instead of a run. Undo: nothing to undo; the red run (42771076825) stays in the history.Not done here, offered for later: the docs-content step still accepts a silent exit 0 from the checker. Requiring its summary line would have caught the Windows defect without reading logs.
Kind: code
https://claude.ai/code/session_016orFiR8mq7oCYo77QDTw2U