Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .claude/skills/local-dev/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,9 @@ fails its first step.

## Docker

Each server has a `Dockerfile`, and no gate builds them. The build context
Each server has a `Dockerfile`. The gate's `image-smoke` stage builds and
boots the image of each server a change touches (`npm run image-smoke -- --all`
for all seven; `docs/quality-gate.md`). The build context
differs by language. The TypeScript images build from the **repo root**,
because they copy the root `tsconfig.json`:
`docker build -t mcp/<server> -f src/<server>/Dockerfile .`. The Python images
Expand Down
35 changes: 34 additions & 1 deletion .claude/skills/pre-push-gate/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ of the server's stderr. Re-run a subset with

### `interface-diff`

The last stage diffs the MCP interface (capabilities, tools, prompts,
This stage diffs the MCP interface (capabilities, tools, prompts,
resources, resource templates) of each server the change touches, in this
checkout's build, against the same server built from the merge-base with
`origin/v2/main`, and prints one report per server. Each report says why that
Expand Down Expand Up @@ -308,6 +308,39 @@ it compares against in CI, and why it is a script rather than an action, is
in [`docs/quality-gate.md`](../../../docs/quality-gate.md) and the header of
`scripts/interface-diff.mjs`.

### `image-smoke`

The last stage builds the Docker image of each server the change touches,
boots it with `docker run -i`, lists its tools, and (for `everything` and
`memory`) stops it with `docker stop`. A change that touches no image prints
`nothing to build` and passes without Docker. What selects an image is in
[`docs/quality-gate.md`](../../../docs/quality-gate.md). A failure is
`image-smoke — FAIL <server>` and the reason:

- **"no Docker daemon answered"**: the change touches an image and Docker is
not running. Start it and re-run; the stage never skips.
- **"`docker build …` failed"**, with the tail of the build log: the
Dockerfile no longer builds from this checkout. The usual causes are a file
the Dockerfile copies that moved or is excluded by its
`Dockerfile.dockerignore`, a lockfile out of step with a manifest
(`npm ci`, `uv sync --locked`), or, offline, a layer Docker has not cached.
- **"Connection closed"** or a timeout, with the container's stderr: the
image built but its entrypoint did not start the server. Run it by hand:
`docker run -i --rm mcp-image-smoke/<server>:local`.
- **"tools/list does not include …"**: as for `smoke`; the tool the image is
checked for is in `IMAGES` in `scripts/image-smoke.mjs`.
- **"`docker stop` had to kill it"**: the server did not exit on the image's
stop signal. Check the Dockerfile's `STOPSIGNAL` and the server's signal
handlers.
- **"/data holds […] after the stop"**: `memory` left a lock or temporary
file behind, or lost the write.
- **A new Dockerfile fails `test:scripts`** with "every Dockerfile in the
repo has an image entry": add it to `IMAGES`.

Re-run it on its own with `npm run image-smoke`, `-- --server <name>` for one
image, `-- --all` for all seven, or `-- --list` to see what it would select
and why.

## Waiting on the lease

A gate that starts with
Expand Down
144 changes: 144 additions & 0 deletions .github/workflows/image-smoke.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# Builds and boots the Docker image of each server a change touches (#5598).
#
# Every server ships a Dockerfile, and before this workflow nothing built one:
# an image that no longer built, could not find its server, or had to be killed
# by `docker stop` reached a release unnoticed, and the release verification
# found it by hand. The check is `npm run image-smoke`
# (scripts/image-smoke.mjs, whose header holds what it asserts and why). The
# same script is a `local:gate` stage, as `scripts/lib/workflow-gate.test.mjs`
# requires of every npm script a workflow runs.
#
# - An image is built only when the change touches it: its server's directory,
# a root file it copies, a package it installs from the root lockfile, or
# the image tooling itself (every image). A `paths` filter cannot say "the
# lockfile changed in a way that reaches this server", so the workflow runs
# on every change and the `changes` job asks the script which images to
# build (`--list`, the same selection the gate makes). `image` fans out over
# that list, one matrix leg per image. This is the interface diff's shape
# (everything-mcp-diff.yml), for the same reasons.
# - The base is the PR's base, or the branch as it was before a push.
# - A manual run (`workflow_dispatch`) builds every image, changed or not: the
# way to check all seven on a branch, for a release verification or after a
# base image moves.
# - Untrusted code (the PR's Dockerfiles and the selection script) runs with a
# read-only token and no persisted credentials. No job here holds a
# credential, pushes an image or logs in to a registry.
#
# The version PRs are opened with the workflow token, whose events start no
# `pull_request` run here. They change version metadata only, and nothing
# dispatches this workflow for them: GitHub dispatches only a workflow whose
# file is on the default branch, which this one is not until a milestone merge.
name: Image smoke

on:
push:
branches: [main, v2/main]
# No branch filter: PRs to v2/main and to main both run, and so does a PR
# stacked on another PR's branch. The base is the PR's own base either way.
pull_request:
workflow_dispatch:

permissions:
contents: read

# Superseded runs on the same PR are cancelled. Every other run gets a group
# of its own, so a push's run is never dropped behind two queued ones.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
changes:
name: Choose the images to build
timeout-minutes: 10
runs-on: ubuntu-latest
outputs:
images: ${{ steps.list.outputs.images }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
# The base is found in history, so all of it is needed.
fetch-depth: 0
# The event's own commit (the merge commit for a PR), not a branch
# ref that a later push could move before this step runs.
ref: ${{ github.sha }}
persist-credentials: false

# No install: the selection imports nothing outside Node.
- uses: actions/setup-node@v7
with:
node-version: 24

- name: List the images the change touches
id: list
env:
EVENT: ${{ github.event_name }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
BEFORE: ${{ github.event.before }}
run: |
case "$EVENT" in
# The checkout is the PR's merge commit, so its base tip is
# exactly what the PR changes.
pull_request) base="$PR_BASE" ;;
# The branch as it was before this push; a new branch has none.
push)
if [ -n "$BEFORE" ] && [ "$BEFORE" != 0000000000000000000000000000000000000000 ]; then
base="$BEFORE"
else
base=$(git rev-parse HEAD^)
fi ;;
workflow_dispatch) base="" ;;
*) echo "unexpected event: $EVENT" >&2; exit 1 ;;
esac
# The reasons go to the log on stderr; stdout is the JSON list.
if [ -n "$base" ]; then
echo "Comparing against $base"
images=$(npm run --silent image-smoke -- --base "$base" --list)
else
images=$(npm run --silent image-smoke -- --all --list)
fi
echo "Images to build: $images"
echo "images=$images" >> "$GITHUB_OUTPUT"
{
if [ "$images" = "[]" ]; then
echo "The change touches no server's image; there is nothing to build."
else
echo "Building: $(jq -r 'map("`" + .server + "`") | join(", ")' <<<"$images")"
fi
} >> "$GITHUB_STEP_SUMMARY"

image:
name: Image ${{ matrix.server }}
needs: changes
if: needs.changes.outputs.images != '[]'
# A cold build is about a minute; the slack is for a slow registry pull.
timeout-minutes: 20
runs-on: ubuntu-latest
strategy:
# One image's failure must not hide another's verdict.
fail-fast: false
matrix:
include: ${{ fromJSON(needs.changes.outputs.images) }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
persist-credentials: false

- uses: actions/setup-node@v7
with:
node-version: 24
cache: npm

# Only for the MCP client the smoke drives the container with, a root
# devDependency. The image builds from the checkout's source inside
# Docker, so no workspace is built here.
- name: Install dependencies
run: npm ci --ignore-scripts

- name: Build, boot and stop the image
env:
SERVER: ${{ matrix.server }}
run: npm run image-smoke -- --server "$SERVER"
12 changes: 8 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ servers/
├── .claude/skills/ On-demand procedures (see the Skills index above)
├── .claude/settings.json The SessionStart hook that checks this session's instructions are current
├── .changeset/ Pending changesets for the TypeScript servers, and the changesets config
├── scripts/ The pre-push gate (gate-lease, smoke-servers, validate-py, coverage-py, interface-diff), its guards (verify-*),
├── scripts/ The pre-push gate (gate-lease, smoke-servers, validate-py, coverage-py, interface-diff, image-smoke), its guards (verify-*),
│ the skills tooling, release tooling (npm-publish-guard, prepare-python-release,
│ pack-and-verify, release-manifest, release-notes), and the issue-filing sweeps
│ (dependency-refresh, dependabot-alerts, sdk-watch), and the session-start
Expand All @@ -65,6 +65,7 @@ servers/
│ version-packages.yml (the changesets PR), prepare-python-release.yml (the CalVer PR),
│ everything-mcp-diff.yml (each changed server's interface diff on PRs),
│ mcp-interface-diff.yml (a dispatch-only placeholder for that file's new name, #5463),
│ image-smoke.yml (each changed server's Docker image, built and booted),
│ main-source.yml (a PR into main must bring nothing that is not on v2/main),
│ and the scheduled sweeps that file issues: dependency-refresh.yml, dependabot-alerts.yml, sdk-watch.yml
├── .github/ISSUE_TEMPLATE/ Bug and feature issue forms; config.yml routes security and new servers away
Expand Down Expand Up @@ -139,14 +140,17 @@ TypeScript workspace's per-file coverage gate), `validate:py`
(each Python server's locked sync, `ruff check`, `ruff format --check`,
pyright, pytest and build), `coverage:py` (each Python server's per-file
coverage gate), `verify:skills:cli`, `smoke` (every server booted over each
transport it implements), and `interface-diff` (the MCP interface of each
server the change touches, diffed against the base it branched from). The stage-by-stage
transport it implements), `interface-diff` (the MCP interface of each
server the change touches, diffed against the base it branched from), and
`image-smoke` (the Docker image of each server the change touches, built,
booted and stopped; it needs a running Docker daemon when a change touches
one). The stage-by-stage
reference, and what CI runs where, is [`docs/quality-gate.md`](./docs/quality-gate.md);
diagnosing a red stage is the `pre-push-gate` skill.

- **`npm run validate` and the per-package commands above are the inner loop,
not a substitute.** They skip the other language, the pinned skills validator,
the boot smoke and the interface diff.
the boot smoke, the interface diff and the image smoke.
- **Format before committing**: `npm run format` at the root (TypeScript), and
`uv run --frozen ruff format .` in a Python server you changed. The gate only
checks formatting; it never rewrites files.
Expand Down
32 changes: 28 additions & 4 deletions docs/quality-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ out; everything else points here.
| --- | --- | --- | --- |
| Inner loop | `npm run validate -w src/<server>`, `npm run validate:py -- <server>` | Your machine | While iterating on one server |
| Pre-push gate | `npm run local:gate` | Your machine, under a lease | Before every push |
| CI | `.github/workflows/typescript.yml`, `python.yml`, `everything-mcp-diff.yml`; `dco.yml`; `main-source.yml` | GitHub, on every push and pull request (`everything-mcp-diff.yml` diffs only the servers the change touches); `dco.yml` on pull requests only; `main-source.yml` on pull requests into `main` only, and not mirrored by a gate stage, since no local push targets `main` | Before merge |
| CI | `.github/workflows/typescript.yml`, `python.yml`, `everything-mcp-diff.yml`, `image-smoke.yml`; `dco.yml`; `main-source.yml` | GitHub, on every push and pull request (`everything-mcp-diff.yml` diffs, and `image-smoke.yml` builds, only the servers the change touches); `dco.yml` on pull requests only; `main-source.yml` on pull requests into `main` only, and not mirrored by a gate stage, since no local push targets `main` | Before merge |

**`npm run local:gate` runs every check CI runs.** CI splits the same checks
into parallel jobs on a fresh install; the gate runs them in sequence on your
Expand Down Expand Up @@ -58,6 +58,7 @@ in this order, and the first failure stops the run.
| 8 | `verify:skills:cli` | `claude plugin validate` on `.claude/skills`, at the pinned CLI version | `typescript.yml` → **Root guards** |
| 9 | `smoke` | Every server boots over each transport it implements and answers one tool call | `typescript.yml` → **Boot smoke** |
| 10 | `interface-diff` | For each server the change touches, its MCP interface (capabilities, tools, prompts, resources, resource templates) in this checkout's build, diffed against the same server built from the base. It fails only when a diff cannot run; a changed interface is reported, not failed | `everything-mcp-diff.yml` → **Choose the servers to diff**, then **Diff \<server\>** (one leg each, and a PR comment each) |
| 11 | `image-smoke` | For each server the change touches, its Docker image: it builds, a client lists its tools over `docker run -i`, and `docker stop` ends `everything` and `memory` without killing them, `memory` leaving no lock file behind | `image-smoke.yml` → **Choose the images to build**, then **Image \<server\>** (one leg each) |

Notes on the stages:

Expand Down Expand Up @@ -100,9 +101,10 @@ Notes on the stages:
when a server's environment is missing or behind its lockfile, since
`uv sync --locked` then downloads packages (and the pinned interpreter, if
`uv` does not have it); `coverage:py` begins with the same sync, which is a
no-op once `validate:py` has run; and `interface-diff` needs it for the
base's `npm ci` and `uv sync`. With warm caches those are the only stages that can fail
offline.
no-op once `validate:py` has run; `interface-diff` needs it for the
base's `npm ci` and `uv sync`; and `image-smoke` needs it for a build whose
layers Docker has not cached (base images, `npm ci`, `uv sync`, `apt-get`).
With warm caches those are the only stages that can fail offline.
- **`smoke` launches what a user launches**: the built `dist/index.js` for a
TypeScript server, the console script through `uv run --no-sync` for a Python
one. stdio and Streamable HTTP for all seven; HTTP+SSE as well for
Expand Down Expand Up @@ -133,6 +135,28 @@ Notes on the stages:
`mcp-server-diff` changes. A change that touches no server diffs nothing
and passes. `-- --server <name>` or `-- --all` diffs servers regardless,
and `-- --list` prints the selection without diffing.
- **`image-smoke` needs a running Docker daemon, but only when the change
touches an image** (#5598). It selects as `interface-diff` does
(`selectImages` in `scripts/image-smoke.mjs`): a file under `src/<server>/`;
for a TypeScript server, the root `.npmrc` or `tsconfig.json`, which its
image copies, or a `package-lock.json` edit that changes a package the
server installs; and every image when the image tooling changes. A change
that touches none builds nothing and passes with no Docker at all. One that
does, with no daemon answering, **fails rather than skips**: start Docker
and re-run. It builds from the working tree, with the command each
Dockerfile documents (the repository root as the context for a TypeScript
server, the server's directory for a Python one), under the tag
`mcp-image-smoke/<server>:local`, which each run reuses. A build with warm
layers takes a few seconds per image. `-- --server <name>` or `-- --all`
builds images regardless, and `-- --list` prints the selection without
building. In CI a manual run of `image-smoke.yml` builds all seven.
- **`image-smoke` asserts a clean stop only where an image has one**:
`everything` and `memory`, the two whose Dockerfiles declare a `STOPSIGNAL`.
A container Docker has to kill exits 137, and that fails. For `memory` it
also makes a write first and checks that the graph's directory holds the
graph and nothing else after the stop, so the lock #5485 removes on a
signal is gone. The other five images are killed at the timeout today
(#5611) and are not checked for it.
- `python.yml` also has a **Build \<server\>** job per server that re-runs
pyright and `uv build` and uploads the built distribution. It checks nothing
stage 6 does not.
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,11 @@
"verify:dco": "node scripts/verify-dco.mjs",
"smoke": "node scripts/smoke-servers.mjs",
"interface-diff": "node scripts/interface-diff.mjs",
"image-smoke": "node scripts/image-smoke.mjs",
"pack:verify": "node scripts/pack-and-verify.mjs",
"release:notes": "node scripts/release-notes.mjs",
"local:gate": "node scripts/gate-lease.mjs npm run local:gate:stages",
"local:gate:stages": "npm run verify:install-fresh && npm run verify:dco && npm run validate && npm run coverage && npm run validate:py && npm run coverage:py && npm run verify:skills:cli && npm run smoke && npm run interface-diff"
"local:gate:stages": "npm run verify:install-fresh && npm run verify:dco && npm run validate && npm run coverage && npm run validate:py && npm run coverage:py && npm run verify:skills:cli && npm run smoke && npm run interface-diff && npm run image-smoke"
},
"dependencies": {
"@modelcontextprotocol/server-everything": "*",
Expand Down
Loading
Loading