Repository navigation
test: add a cross-package integration suite, fast on each PR and full nightly - #26
Merged
Merged
Conversation
… nightly Every starter that chains packages gains integration tests in test/integration/, through the libraries' public APIs only, on synthetic data. Files tagged *.fast.test.js run in npm test, so All starters covers them on every pull request; *.full.test.js hold the seed sweeps and run in npm run test:full, which also holds the shared helpers to 100% coverage. integration.yml runs the full suite nightly, on changes to the integration tests, and through workflow_call with a package and an artifact, installing a release tarball in place of the registry version. integration-self-test.yml proves that override with a marked tarball the registry does not have. Lockfiles are regenerated from scratch. Two library defects the suite found are todo tests.
When a library's release calls integration.yml through workflow_call, github.repository is the caller, so a bare checkout tested the library's folders instead of the starters. Both checkouts now name cosyte/examples; our own runs test their commit and a caller gets the default branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Integration tests in every starter that chains packages, through the libraries' public APIs only,
on synthetic data from
@cosyte/synth(the DICOM starter's own hand-written header, since synth hasno DICOM generator). Each starter's tests live in
test/integration/, tagged by file name:*.fast.test.js, the fast slice:npm testnow runs them, sostarters.ymlruns them on everypull request and its existing
All starterscheck stands for the slice.*.full.test.js, the full suite: seed sweeps and many-process runs.npm run test:fullruns everytest and fails unless the helpers in
test/integration/support.jshave 100% line, branch andfunction coverage.
The pipelines, each tagged fast and full:
@cosyte/hl7, each ACK correlated; the starter's listener on each ADT triggercosyteas a child process:convertpiped intovalidate -andredact -;validateandfmtover 8 formats;redactrefusals;map-codesthrough a ConceptMapWorkflows:
integration.yml: the full suite, every starter on Node 22 and 24, oneIntegration (full)check. Nightly on
main, on a pull request that changes an integration test, on dispatch, andthrough
workflow_callwithpackageandartifact: each starter whose lockfile carries thepackage installs the artifact's tarball as the dependency and as an npm override, checks that every
copy in the tree is the tarball's version resolved from the file, then runs the suite.
integration-self-test.yml: packs the locked@cosyte/hl7under a version the registry does nothave (
<locked>-selftest.<run id>.<attempt>, checked absent withnpm view), uploads it, andcalls
integration.ymlwith it. The gate's install step fails unless that marked version is whatevery starter installed.
Every lockfile was regenerated from scratch, so each starter is on the current
@cosyte/*0.1releases. Five starters gain test-only devDependencies (
@cosyte/deid,@cosyte/terminology, andfor cli-tour
@cosyte/ccda,@cosyte/fhir,@cosyte/x12). Each starter README gains an Integrationtests section that says what its tests assert; AGENTS.md, CLAUDE.md and the root README describe the
tags and the workflows. pathways is not covered: the published
@cosyte/pathways0.0.1 needs Node 24and says it is not ready to run.
Why
The libraries are each tested on their own; this suite tests them together, the way an integration
uses them: a parser's output fed to the transform, the transform's to the validator and the
de-identifier, the CLI's output piped into the CLI. The fast slice runs on every pull request, the
full suite nightly, and a library's release can run it against the tarball it is about to publish.
Proof
npm ciandnpm testin every starter on Node 24 and on Node 22, actionlint).npm run test:fullin every starter on Node 22 and on Node 26: all pass, twotodo, helpers at100% line, branch and function coverage.
Starters(checkAll starters): 30seconds from start to finish, 20 jobs, the longest 14 seconds. The full suite (
Integration,check
Integration (full)): 62 seconds, the longest job cli-tour at 45 seconds.Integration self-test) packed@cosyte/hl7 0.1.8-selftest.42873672033.1and thegate's log shows each starter installed it from the tarball, for example cli-tour on Node 22:
node_modules/@cosyte/hl7 0.1.8-selftest.42873672033.1 file:../../../_temp/override/...tgz.Red first. The tests were written against the base and run red until the scripts, helpers and
expectations were right. The red runs that shaped them: hl7-to-fhir's ORU check failed when the
vital-signs starter profile was applied to lab Observations; the X12 de-identification check failed
on a rendering provider who shares the patient's synthetic name; dicom-metadata's UID check failed on
the assumption that a new map gives new UIDs (the remap derives from the original UID); the
x12-remittance full run failed its coverage threshold at 97.37% branch coverage until a dead branch
went. Two tests stay red as
todo, one per library defect below.Library defects found, each a
todotest that shows it, none worked around:@cosyte/deid0.1.5, DICOM:deidentifyDicomwrites De-identification Method(0012,0063)asone 113-character Value, over the 64 characters PS3.5 allows an
LO, and@cosyte/dicomreportsDICOM_DEIDENT_METHOD_VALUE_OVER_LENGTH. Repro, in dicom-metadata:deidentifyDicom(parseDicom(syntheticCtHeader())).warnings.@cosyte/cli0.1.8: NCPDP is read as SCRIPT only. A Telecom D.0 B1 (generateB1({ seed: 1 }))is undetected (exit 65,
CLI_FORMAT_UNDETECTED), and with--format ncpdpfails withNCPDP_SCRIPT_NOT_XML, although--helplists ncpdp for validate, inspect and fmt.Seams the tests pin down, not defects: synth's ORU^R01 leaves
OBR-25empty, so the transformwithholds the DiagnosticReport; synth's
OBX-6units carry no UCUM coding system, so they never reachQuantity.code;@cosyte/terminologyresolves neither three HL7 v3 code systems the transform writesnor four HL7 v3 OIDs and the NCI Thesaurus OID in the synthetic C-CDA; the CLI does not detect NCPDP
SCRIPT without
--format.Choices
npm test, soAll startersis its one check rather than a new one.Undo: drop
test/integration/*.fast.test.jsfrom eachtestscript and add a fast job tointegration.yml.npm teststays one command. Undo: rename.the tree (the CLI's own
@cosyte/hl7) is replaced too. Undo: drop the.overridesline in theinstall step.
todotests, which report without failing the run, so the defect staysvisible and the suite stays green. Undo: remove the
todooption.Kind: code