Repository navigation
test: pull the gateway tier's published images rather than build them on every shard - #575
Merged
Conversation
… on every shard Each of the five gateway shards built the gateway, service-db, db-web, db-mcp and Playwright MCP images from scratch, about 130s a shard. main publishes each to GHCR under a tag that is a digest of its Dockerfile's build context, so the fixtures now pull that image when the registry serves it, and build only when a change to those sources gives them a tag nothing has published. Co-Authored-By: Claude <noreply@anthropic.com>
andrewpark-scaleai
enabled auto-merge (squash)
November 9, 2026 22:13
earakely-scale
approved these changes
Nov 9, 2026
andrewpark-scaleai
deleted the
andrew-park/ci-gateway-published-images
branch
November 9, 2026 22:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The goal is a faster Local backends workflow that still tests what it tested. Each of the five
integration-local-gatewayshards spends about 205s of its roughly 400s of pytest building nine images in its first test's fixtures. In shard 1 of run 42679706897,playwright-mcptook 75s,db-mcp35s andgateway15s. Every shard repeats the same builds.main already publishes five of those nine images to GHCR, from the same Dockerfile and build context the fixtures build them from. These are gateway, service-db, db-web, db-mcp and playwright-mcp. The tag is
src-plus a digest of every file in that build context (PublishedImage.tag).What changes
The fixtures that build from a
DockerImageConfigshare one helper,_pull_published_else_build.linux/amd64manifest by its digest and tags it under the name the fixture used before. It pulls the platform manifest rather than the index, because the containerd image store thatmirror_base_images.pyturns on keeps a pulled index whole, anddocker saveinDockerImageArtifact.putfails when some of the index's platforms are missing.DockerImageArtifact.putand the push to the runner's registry, is unchanged.Verification
src/agent_env/env/gateway/Dockerfile, it built gateway and still pulled service-db.docker saveof its tag wrote 71MB. With the index pulled instead, the same save failed withcontent digest ... not found, which an independent review reproduced.gateway_test.pycollects all 844 items.The gateway shards' CI run on this PR is the end-to-end check.
Risks
published_pinnedreturns None when the registry can't be asked, so the helper builds. A pull that fails after the registry answered fails the fixture.The PR appears safe to merge, with a non-blocking request for focused helper tests.
Fix with agent prompt
Summary
The gateway fixtures now reuse published images when their source tags exist.
Acknowledged choices:
andrewpark-scaleaistates that reused images do not pick up newer upstream packages until their sources change or release builds run.andrewpark-scaleaistates that failures after a successful registry lookup fail the fixture rather than fall back to a build.Diagram
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Fixture requests image] --> B{Matching published image?} B -->|No| F[Build locally] B -->|Yes| C[Resolve current source tag] C --> D{Published amd64 image available?} D -->|No| F D -->|Yes| E[Pull child digest and apply fixture tag] E --> G[Create existing image artifact] F --> GReviews (1) · Last reviewed commit: "test: pull the gateway tier's published ..." · Reviewed by Greptile