From 2a8aea3db10ffc70fe6930cc248f09da741c514b Mon Sep 17 00:00:00 2001 From: foxnne Date: Thu, 8 Oct 2026 08:53:35 -0500 Subject: [PATCH] ci: run the headless integration tests, and gate on them `zig build test-integration` (146 integration tests on dvui's testing backend, plus the SDK, sizing and plugin-loader suites: 292 in all) was run by no CI job, so a layout or drag regression reached main unless someone remembered to run it; on 10-03 two of its tests had been failing unnoticed. It runs now in a Linux job of its own beside the test matrix, and `ci-ok` waits on it. Linux only: the code under test is the same everywhere, and on Windows it needs MSVC. The job notes why its log can say `failed command:` on a pass (a test that logs a warning; the demo-replay test always does). CONTRIBUTING.md and CLAUDE.md stop saying CI skips the suite. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 38 +++++++++++++++++++++++++++++++++++++- CLAUDE.md | 2 +- CONTRIBUTING.md | 4 ++-- 3 files changed, 40 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1fd0fdec4..cec5b7c84 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -151,6 +151,42 @@ jobs: shell: bash run: bash scripts/ci-retry.sh zig build test-sdk-version --summary all + # The headless integration tests: real widgets, layouts and drags driven on dvui's testing + # backend (no window, no GPU), plus the SDK, sizing and plugin-loader suites. The suite most + # likely to catch a layout or drag regression, and until this job nothing ran it. Linux only: + # the code under test is the same on every OS, and on Windows it needs MSVC (`build/app.zig`). + # A job of its own beside the matrix, so it adds a compile to the run but not to its length. + # + # A test that logs a warning makes the build print `failed command: …fizzy-integration-tests` + # while still passing (the demo-replay test always does): the exit code is the verdict, and the + # Build Summary's `N/N steps succeeded` says the same. + integration: + name: Integration tests (Linux) + needs: changes + if: needs.changes.outputs.code == 'true' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # See the test job: the key is fixed before anything is unpacked. + - name: Zig cache key + id: zig-cache-key + shell: bash + run: echo "key=${{ hashFiles('**/build.zig.zon') }}" >> "$GITHUB_OUTPUT" + - uses: mlugg/setup-zig@v2 + with: + version: ${{ env.ZIG_VERSION }} + cache-key: ${{ steps.zig-cache-key.outputs.key }} + cache-size-limit: 5120 + - name: Pre-create Zig cache tmp/ + shell: bash + run: mkdir -p "$ZIG_GLOBAL_CACHE_DIR/tmp" "$ZIG_LOCAL_CACHE_DIR/tmp" + - name: Fetch dependencies (with retries) + shell: bash + run: bash scripts/ci-retry.sh zig build --fetch + - name: Run integration tests + shell: bash + run: bash scripts/ci-retry.sh zig build test-integration --summary all + # Fizzy's own backend on Windows — the default there too now (`build/exe.zig`'s # `defaultNativeBackend`) — which nothing else here builds: the consumer apps above run on Linux only. # Cross-compiled from Linux for the GNU ABI, which needs no MSVC SDK (release.yml's msvc @@ -179,7 +215,7 @@ jobs: ci-ok: name: ci-ok if: always() - needs: [changes, test, windows-fizzy-backend] + needs: [changes, test, integration, windows-fizzy-backend] runs-on: ubuntu-latest steps: - name: Every job passed or was skipped diff --git a/CLAUDE.md b/CLAUDE.md index 9e0d9276f..1ded04008 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -137,7 +137,7 @@ one directory — that is why the `app` module root is `app/root.zig`, not `app/ zig build # native exe zig build check-web # wasm zig build test # unit tests -zig build test-integration # headless integration tests (not run by CI: run it yourself) +zig build test-integration # headless integration tests (CI runs them on Linux) zig build test-sdk-version # CI lock: the recorded fingerprint matches the live plugin boundary ``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0dad60cdb..0b8f1d4c0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -74,8 +74,8 @@ The open PR list is the board of who is working on what. ## Verify, and say how - **Gates:** `zig build`, `zig build test`, `zig build test-integration`, `zig build check-web`, - `zig build test-sdk-version`. CI runs all of them but `test-integration`, so run that one - yourself. + `zig build test-sdk-version`. CI runs all of them, `test-integration` on Linux only, but a + minute locally beats a ten-minute round trip: run them before you push. - **Read the Build Summary, not the test count.** `test-integration` prints `failed command:` whenever a test logs a warning, even on a pass, and can show `N/N tests passed` beside a failed step. Only `N/N steps succeeded` is a pass.