Skip to content

test: pull the gateway tier's published images rather than build them on every shard - #575

Merged
andrewpark-scaleai merged 1 commit into
mainfrom
andrew-park/ci-gateway-published-images
Nov 9, 2026
Merged

andrewpark-scaleai merged 1 commit into
mainfrom
andrew-park/ci-gateway-published-images

Conversation

@andrewpark-scaleai

@andrewpark-scaleai andrewpark-scaleai commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Why

The goal is a faster Local backends workflow that still tests what it tested. Each of the five integration-local-gateway shards spends about 205s of its roughly 400s of pytest building nine images in its first test's fixtures. In shard 1 of run 42679706897, playwright-mcp took 75s, db-mcp 35s and gateway 15s. Every shard repeats the same builds.

main already publishes five of those nine images to GHCR, from the same Dockerfile and build context the fixtures build them from. These are gateway, service-db, db-web, db-mcp and playwright-mcp. The tag is src- plus a digest of every file in that build context (PublishedImage.tag).

What changes

The fixtures that build from a DockerImageConfig share one helper, _pull_published_else_build.

  • Unchanged sources are pulled. When the registry serves the tag for the same Dockerfile and context, the helper pulls that image's linux/amd64 manifest by its digest and tags it under the name the fixture used before. It pulls the platform manifest rather than the index, because the containerd image store that mirror_base_images.py turns on keeps a pulled index whole, and docker save in DockerImageArtifact.put fails when some of the index's platforms are missing.
  • Changed sources are built. A change to any file in an image's build context changes its tag. No image has been published under the new tag, so the helper builds the image as before. The four test-only images (slack, email and the slack website's two) always build.
  • Everything after the tag, such as DockerImageArtifact.put and the push to the runner's registry, is unchanged.

Verification

  • With this branch's sources, the helper pulls the five published images and builds the other four. After I appended a line to src/agent_env/env/gateway/Dockerfile, it built gateway and still pulled service-db.
  • I ran the helper for real against Docker 29 with the containerd image store. The five pulls took 29s together on a laptop. After pulling db-web, docker save of its tag wrote 71MB. With the index pulled instead, the same save failed with content digest ... not found, which an independent review reproduced.
  • gateway_test.py collects all 844 items.

The gateway shards' CI run on this PR is the end-to-end check.

Risks

  • A published image was built when its sources were published. A fresh build today would also take any newer apt or npm packages its Dockerfile fetches. So a break in an upstream package that the sources don't pin shows up only when those sources next change, or in the release workflow's build.
  • The fixtures now depend on anonymous GHCR pulls. published_pinned returns None when the registry can't be asked, so the helper builds. A pull that fails after the registry answered fails the fixture.

RetriggerView in GreptileConfidence Score: 4/5

The PR appears safe to merge, with a non-blocking request for focused helper tests.

Fix All in CursorFindings

  1. P2 Fallback branches lack focused tests ▶
Fix with agent prompt
### Issue 1
tst/integration/env/gateway/gateway_test.py:158-168
The new pull-or-build helpers have no focused automated tests. The live fixtures only exercise whichever images GHCR currently serves, so they cannot reliably check missing tags, an index without `linux/amd64`, or command failures. Add tests with fake registry replies and subprocess results for platform selection, build fallback, retagging, and error propagation. This would catch mistakes without Docker, network access, or a full gateway run.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The gateway fixtures now reuse published images when their source tags exist.

  • Gateway test fixtures reuse published images when their sources match.

Acknowledged choices:

  • andrewpark-scaleai states that reused images do not pick up newer upstream packages until their sources change or release builds run.
  • andrewpark-scaleai states that failures after a successful registry lookup fail the fixture rather than fall back to a build.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Fixture requests image] --> B{Matching published image?}
  B -->|No| F[Build locally]
  B -->|Yes| C[Resolve current source tag]
  C --> D{Published amd64 image available?}
  D -->|No| F
  D -->|Yes| E[Pull child digest and apply fixture tag]
  E --> G[Create existing image artifact]
  F --> G
Loading

Reviews (1) · Last reviewed commit: "test: pull the gateway tier's published ..." · Reviewed by Greptile

… on every shard

Each of the five gateway shards built the gateway, service-db, db-web, db-mcp and Playwright MCP images
from scratch, about 130s a shard. main publishes each to GHCR under a tag that is a digest of its
Dockerfile's build context, so the fixtures now pull that image when the registry serves it, and build
only when a change to those sources gives them a tag nothing has published.

Co-Authored-By: Claude <noreply@anthropic.com>
@andrewpark-scaleai
andrewpark-scaleai requested a review from a team as a code owner November 9, 2026 21:05
@andrewpark-scaleai
andrewpark-scaleai enabled auto-merge (squash) November 9, 2026 22:13
@andrewpark-scaleai
andrewpark-scaleai merged commit 79fda4d into main Nov 9, 2026
34 of 35 checks passed
@andrewpark-scaleai
andrewpark-scaleai deleted the andrew-park/ci-gateway-published-images branch November 9, 2026 22:52
andrewpark-scaleai added a commit that referenced this pull request Nov 10, 2026
…image in the mirror

The slow tier's times are from PR #580's run 42716509774, attempt 2, which built no image; the installer's from
the same run; the gateway's and unit tier's from main's run 42715765775, after #575 and #576.

Co-Authored-By: Claude <noreply@anthropic.com>
andrewpark-scaleai added a commit that referenced this pull request Nov 10, 2026
…image in the mirror

The slow tier's times are from PR #580's run 42716509774, attempt 2, which built no image; the installer's from
the same run; the gateway's and unit tier's from main's run 42715765775, after #575 and #576.

Co-Authored-By: Claude <noreply@anthropic.com>
andrewpark-scaleai added a commit that referenced this pull request Nov 10, 2026
…image in the mirror

The slow tier's times are from PR #580's run 42716509774, attempt 2, which built no image; the installer's from
the same run; the gateway's and unit tier's from main's run 42715765775, after #575 and #576.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants