Skip to content

test: add a cross-package integration suite, fast on each PR and full nightly - #26

Merged
NSchatz merged 2 commits into
mainfrom
feat/integration-suite
Nov 10, 2026
Merged

NSchatz merged 2 commits into
mainfrom
feat/integration-suite

Conversation

@NSchatz

@NSchatz NSchatz commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

What

Integration tests in every starter that chains packages, through the libraries' public APIs only,
on synthetic data from @cosyte/synth (the DICOM starter's own hand-written header, since synth has
no DICOM generator). Each starter's tests live in test/integration/, tagged by file name:

  • *.fast.test.js, the fast slice: npm test now runs them, so starters.yml runs them on every
    pull request and its existing All starters check stands for the slice.
  • *.full.test.js, the full suite: seed sweeps and many-process runs. npm run test:full runs every
    test and fails unless the helpers in test/integration/support.js have 100% line, branch and
    function coverage.

The pipelines, each tagged fast and full:

Starter Pipeline Fast Full
hl7-mllp-listener 7 HL7 v2 kinds from synth over a real MLLP link on 127.0.0.1 and back through @cosyte/hl7, each ACK correlated; the starter's listener on each ADT trigger 2 seeds per kind 150 seeds per kind on one link; 4 senders at once
hl7-to-fhir hl7, transform, fhir validation, terminology (UCUM, code-system resolution), deid over the FHIR Bundle 2 seeds per kind 250 seeds per kind
deid-hl7 deid over 7 HL7 v2 kinds, the output parsed back 2 seeds per kind 250 seeds per kind
x12-remittance 835, 837P, 837I, 837D round trip; deid over X12; the de-identified 835 reads as the same payment 2 seeds each 250 seeds each
x12-eligibility 271 and the 270 built for it, both round-tripped and read 3 seeds 300 seeds
ccda-summary CCD and Referral Note round trip; terminology on every codeSystem OID; deid over C-CDA 2 seeds each 150 seeds each
astm-lab-results framed upload decoded and re-framed byte for byte; orders round trip 3 seeds 100 seeds, 1 to 8 results
ncpdp-claim B1, B2, B3 and the starter's B1 round trip; deid over NCPDP Telecom 2 seeds each 250 seeds each
dicom-metadata Part 10 round trip; deid, the de-identified bytes round-tripped, UID remapping 1 header 20 maps by 10 passes
cli-tour cosyte as a child process: convert piped into validate - and redact -; validate and fmt over 8 formats; redact refusals; map-codes through a ConceptMap seed 1 12 seeds per HL7 kind and per format

Workflows:

  • integration.yml: the full suite, every starter on Node 22 and 24, one Integration (full)
    check. Nightly on main, on a pull request that changes an integration test, on dispatch, and
    through workflow_call with package and artifact: each starter whose lockfile carries the
    package installs the artifact's tarball as the dependency and as an npm override, checks that every
    copy in the tree is the tarball's version resolved from the file, then runs the suite.
  • integration-self-test.yml: packs the locked @cosyte/hl7 under a version the registry does not
    have (<locked>-selftest.<run id>.<attempt>, checked absent with npm view), uploads it, and
    calls integration.yml with it. The gate's install step fails unless that marked version is what
    every starter installed.

Every lockfile was regenerated from scratch, so each starter is on the current @cosyte/* 0.1
releases. Five starters gain test-only devDependencies (@cosyte/deid, @cosyte/terminology, and
for cli-tour @cosyte/ccda, @cosyte/fhir, @cosyte/x12). Each starter README gains an Integration
tests section that says what its tests assert; AGENTS.md, CLAUDE.md and the root README describe the
tags and the workflows. pathways is not covered: the published @cosyte/pathways 0.0.1 needs Node 24
and says it is not ready to run.

Why

The libraries are each tested on their own; this suite tests them together, the way an integration
uses them: a parser's output fed to the transform, the transform's to the validator and the
de-identifier, the CLI's output piped into the CLI. The fast slice runs on every pull request, the
full suite nightly, and a library's release can run it against the tarball it is about to publish.

Proof

  • Gate: the four steps in AGENTS.md passed, 7 of 7 commands (no-emdash, claude-card, npm ci and
    npm test in every starter on Node 24 and on Node 22, actionlint).
  • npm run test:full in every starter on Node 22 and on Node 26: all pass, two todo, helpers at
    100% line, branch and function coverage.
  • CI on this pull request, all green. The fast slice is Starters (check All starters): 30
    seconds from start to finish, 20 jobs, the longest 14 seconds. The full suite (Integration,
    check Integration (full)): 62 seconds, the longest job cli-tour at 45 seconds.
  • The self-test (Integration self-test) packed @cosyte/hl7 0.1.8-selftest.42873672033.1 and the
    gate's log shows each starter installed it from the tarball, for example cli-tour on Node 22:
    node_modules/@cosyte/hl7 0.1.8-selftest.42873672033.1 file:../../../_temp/override/...tgz.

Red first. The tests were written against the base and run red until the scripts, helpers and
expectations were right. The red runs that shaped them: hl7-to-fhir's ORU check failed when the
vital-signs starter profile was applied to lab Observations; the X12 de-identification check failed
on a rendering provider who shares the patient's synthetic name; dicom-metadata's UID check failed on
the assumption that a new map gives new UIDs (the remap derives from the original UID); the
x12-remittance full run failed its coverage threshold at 97.37% branch coverage until a dead branch
went. Two tests stay red as todo, one per library defect below.

Library defects found, each a todo test that shows it, none worked around:

  1. @cosyte/deid 0.1.5, DICOM: deidentifyDicom writes De-identification Method (0012,0063) as
    one 113-character Value, over the 64 characters PS3.5 allows an LO, and @cosyte/dicom reports
    DICOM_DEIDENT_METHOD_VALUE_OVER_LENGTH. Repro, in dicom-metadata:
    deidentifyDicom(parseDicom(syntheticCtHeader())).warnings.
  2. @cosyte/cli 0.1.8: NCPDP is read as SCRIPT only. A Telecom D.0 B1 (generateB1({ seed: 1 }))
    is undetected (exit 65, CLI_FORMAT_UNDETECTED), and with --format ncpdp fails with
    NCPDP_SCRIPT_NOT_XML, although --help lists ncpdp for validate, inspect and fmt.

Seams the tests pin down, not defects: synth's ORU^R01 leaves OBR-25 empty, so the transform
withholds the DiagnosticReport; synth's OBX-6 units carry no UCUM coding system, so they never reach
Quantity.code; @cosyte/terminology resolves neither three HL7 v3 code systems the transform writes
nor four HL7 v3 OIDs and the NCI Thesaurus OID in the synthetic C-CDA; the CLI does not detect NCPDP
SCRIPT without --format.

Choices

  • The fast slice runs inside npm test, so All starters is its one check rather than a new one.
    Undo: drop test/integration/*.fast.test.js from each test script and add a fast job to
    integration.yml.
  • Tags are file names, not test names, so a copier's npm test stays one command. Undo: rename.
  • The release override sets the tarball as the dependency and as an npm override, so a copy deeper in
    the tree (the CLI's own @cosyte/hl7) is replaced too. Undo: drop the .overrides line in the
    install step.
  • Library defects are todo tests, which report without failing the run, so the defect stays
    visible and the suite stays green. Undo: remove the todo option.

Kind: code

… nightly

Every starter that chains packages gains integration tests in test/integration/, through the
libraries' public APIs only, on synthetic data. Files tagged *.fast.test.js run in npm test, so
All starters covers them on every pull request; *.full.test.js hold the seed sweeps and run in
npm run test:full, which also holds the shared helpers to 100% coverage.

integration.yml runs the full suite nightly, on changes to the integration tests, and through
workflow_call with a package and an artifact, installing a release tarball in place of the
registry version. integration-self-test.yml proves that override with a marked tarball the
registry does not have.

Lockfiles are regenerated from scratch. Two library defects the suite found are todo tests.
When a library's release calls integration.yml through workflow_call, github.repository is the caller, so a bare checkout tested the library's folders instead of the starters. Both checkouts now name cosyte/examples; our own runs test their commit and a caller gets the default branch.
@NSchatz
NSchatz merged commit 5d6f9d7 into main Nov 10, 2026
64 checks passed
@NSchatz
NSchatz deleted the feat/integration-suite branch November 10, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant