Skip to content

Run router integration suites in 6 jobs, not 41 - #2077

Merged
dangusev merged 1 commit into
mainfrom
ci/integration-groups
Nov 18, 2026
Merged

dangusev merged 1 commit into
mainfrom
ci/integration-groups

Conversation

@dangusev

Copy link
Copy Markdown
Contributor

Why

Each Go package in integration.yml ran in its own job: 41 jobs per run. Each job spends about 35 s on checkout, setup-go and the Postgres and Redis containers before its first test, so a run spent about 23 min on setup alone.

How it works

before: 41 jobs, one package each
  job -> checkout, setup-go, Postgres, Redis -> go test ./internal/x

after: 6 jobs, one group of packages each
  job -> checkout, setup-go, Postgres, Redis
      -> go test -p 1 ./internal/a ./internal/b ...  (one package at a time)
Group Packages Test time (run 44301142796)
agent 1 16.5 min
store 7 15.5 min
api 4 16 min
router 3 16 min
routers 7 15 min
rest 19 15.5 min
Before After
Jobs per run 41 6
Setup per run ~23 min ~3.5 min
Run time ~17 min ~17 min, set by internal/agent (unverified)

Changes

  • The 41 matrix.package entries become 6 include entries with group, packages and, where needed, skip, mem0 and ffmpeg.
  • -p 1 runs a group's packages one at a time, because they share the job's Postgres and Redis.
  • Only the api group has mem0 and ffmpeg. The ffmpeg step now checks matrix.ffmpeg.
  • The two skip patterns do not change. Each one now applies to its whole group.
  • Costs: a failed job shows the failing package only in its log, a rerun repeats the whole group, and the groups need rebalancing as suites grow. The header comment says to put a new package in the lightest group.

Validation

  • The new groups hold the same 41 packages as the old matrix. I parsed both versions of the workflow and compared the lists.
  • No skip pattern matches a test in another package of its group. internal/tts/voices has a TestResolverSuite, but not the subtest that is skipped.
  • Unverified until this PR's run: 6 jobs, each finishes within the 40 min timeout. On CI, this is the first time packages share one Postgres. The suites truncate their tables before each test.

🤖 Generated with Claude Code

Each package ran in its own job, and each job spends about 35 s on
setup before its first test. Group the 41 packages into 6 jobs,
balanced by test time; packages in a group run one at a time on the
job's Postgres and Redis.
@dangusev
dangusev marked this pull request as ready for review November 18, 2026 13:12
@dangusev
dangusev merged commit 2a517d0 into main Nov 18, 2026
16 of 23 checks passed
@dangusev
dangusev deleted the ci/integration-groups branch November 18, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants