Skip to content

ci: schedule fast-checks only when the diff needs it - #1702

Merged
mbouaziz merged 1 commit into
mainfrom
ci/fast-checks-only-when-needed
Nov 19, 2026
Merged

mbouaziz merged 1 commit into
mainfrom
ci/fast-checks-only-when-needed

Conversation

@mbouaziz

@mbouaziz mbouaziz commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Single commit (squashed from the earlier iteration). Reworks how the fast-checks CI job is scheduled:

  • Trigger fast-checks only if needed. It previously ran on every pipeline from the static setup workflow in .circleci/config.yml. .circleci/generate_config.sh now emits it (job defined in base.yml) only when the diff against merge-base main HEAD touches a file that make check-fmt (skfmt / clang-format / prettier / black extensions, per the Makefile and .prettierignore) or make check-sh (*.sh) covers, or when base.yml's own fast-checks section changed (so edits to the job are still tested). Doc-only and .gitignore-only changes — including pushes of a new worktree branch that sits at main — now schedule nothing beyond setup and the no-op all-checks.
  • Robust base computation on new worktrees. generate_config.sh falls back to origin/main when there is no local main (single-branch clones, new worktrees) and fails with a clear fetch hint if neither exists — previously an empty base made every git diff exit 128, so every path looked changed and all jobs were scheduled.
  • Quieter prettier. make check-fmt no longer forces PRETTIER_LOG_LEVEL=debug; it runs at the default warn (the pre-commit hook keeps its opt-in PRETTIER_LOG_LEVEL=debug git commit).

The job itself is unchanged: it still runs on skiplabs/skip:latest, which ships every check tool including skfmt, so no alternative image or install step is needed. (An earlier iteration of this PR tried a smaller cimg/base image when the diff had no .sk files, but installing the tools there took longer than the image pull saved, so that idea was dropped.)

fast-checks is added to all-checks' requires, so the format/shell gate is covered by the single required status context (ci/circleci: ci/all-checks); build jobs report unprefixed contexts, so ci/circleci: fast-checks is unchanged. Note the tradeoff: fast-checks now starts after setup completes instead of in parallel with it (the price of making it conditional).

Motivation

Every pipeline — including pushes of new worktree branches with empty or doc-only diffs — paid for a pull of the large skiplabs/skip image plus a whole-tree make check-fmt run, and the job log was flooded with prettier debug output.

Validation

  • bash -n and shellcheck clean on .circleci/generate_config.sh; .circleci/config.yml and .circleci/base.yml parse as YAML.
  • Scratch-clone runs of generate_config.sh: doc-only → no fast-checks; .ts-only → fast-checks scheduled and required by all-checks; bin/*.sh-only → scheduled (check-sh covers it); base.yml fast-checks section edit → scheduled via the section rule; no local main → falls back to origin/main; neither → exit 1 with a clear error.
  • CI green on the squashed head (see statuses).

AI disclosure

This PR was implemented with Mistral Vibe (Mistral Vibe CLI); the commit carries the corresponding Co-Authored-By trailer.

Copy link
Copy Markdown
Contributor Author

First pipeline's fast-checks was red: the small-image path used cimg/base:2025.01, which is Ubuntu 22.04 (jammy) — clang-format-20 from the apt.llvm.org noble suite needs glibc >= 2.38 (jammy has 2.35), so the install step failed. Fixed in 16e47b4: the small image is now ubuntu:24.04 (noble, same base as the skiplabs images; runs as root so no sudo). Reproduced the fixed job end-to-end in a chroot of the ubuntu:24.04 arm64 rootfs: install succeeds and make check-fmt FMT_TARGETS="fmt-c fmt-js fmt-py" + make check-sh pass on a clean checkout.

Copy link
Copy Markdown
Contributor Author

Second fast-checks failure (job 23383, ~5s): ubuntu:24.04 has no git, so custom-checkout — which runs before the install step — failed immediately. Fixed in e8ce1f9: the small image is now cimg/base:2025.12-24.04 (noble, ~416MB, ships git/sudo for the checkout; the plain 2025.01 monthly tag is jammy and breaks the noble llvm packages). Re-verified in a chroot of the cimg/base:2025.12-24.04 arm64 rootfs: install step succeeds (clang-format 20.1.8, node 18, pip 24) and make check-fmt FMT_TARGETS="fmt-c fmt-js fmt-py" + make check-sh pass on a clean checkout.

Copy link
Copy Markdown
Contributor Author

Switched the fast-checks job parameter from the image string to a boolean with_skfmt (adf959f). CircleCI can't select docker.image from a boolean, so the job now always runs on the small cimg/base:2025.12-24.04 and installs the light tools there; generate_config.sh emits - fast-checks: {with_skfmt: false} when the diff has no .sk files. Only when with_skfmt is true (default) does the job also run fmt-sk, in a skiplabs/skip:latest container via setup_remote_docker — so PRs without .sk changes never pull the big image. The step logic is now if << parameters.with_skfmt >> instead of a string compare. Note: this PR's own diff has no .sk files, so CI exercises the with_skfmt: false path; the true path (remote-docker fmt-sk) will be exercised by the next compiler/prelude PR.

Copy link
Copy Markdown
Contributor Author

The boolean with_skfmt parameter version is green on 55e4cb8 — all 8 statuses pass, including ci/circleci: fast-checks (run with with_skfmt: false on cimg/base:2025.12-24.04) and ci/circleci: ci/all-checks. One fix along the way: on the cimg image the job runs as the non-root circleci user, so fmt-py's pip-installed black landed in ~/.local/bin (not on PATH) — the job now installs black system-wide with sudo pip, like bin/apt-install.sh does in image builds.

Copy link
Copy Markdown
Contributor Author

Simplified per review (0b6aec3): dropped the setup_remote_docker + docker run skfmt hop — overkill. Since CircleCI can't select a docker.image from a boolean parameter, the job is back to a string image parameter: default skiplabs/skip:latest (full make check-fmt), or cimg/base:2025.12-24.04 passed by generate_config.sh when the diff has no .sk files. No docker involved. Green on 0b6aec3: all 8 statuses pass including ci/circleci: fast-checks and ci/circleci: ci/all-checks.

@mbouaziz
mbouaziz force-pushed the ci/fast-checks-only-when-needed branch from 0b6aec3 to a9e8585 Compare November 18, 2026 22:14
@mbouaziz
mbouaziz enabled auto-merge November 19, 2026 03:06
@mbouaziz
mbouaziz force-pushed the ci/fast-checks-only-when-needed branch from a9e8585 to 6ae9b17 Compare November 19, 2026 03:07
Move fast-checks out of the static setup workflow in config.yml into the
generated config: generate_config.sh now emits it only when the diff
against main touches a file that make check-fmt or make check-sh covers
(formatter extensions or shell scripts), or when base.yml's fast-checks
section itself changed. Doc-only and .gitignore-only changes therefore
run no job beyond setup and the no-op all-checks. The job still runs on
skiplabs/skip:latest, which ships every check tool including skfmt, so
no alternative image or install step is needed.

The merge-base now falls back to origin/main for checkouts without a
local main branch (e.g. new worktrees) and fails loudly if neither
exists, instead of diffing against an empty base.

Also stop forcing PRETTIER_LOG_LEVEL=debug in make check-fmt.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@mbouaziz
mbouaziz disabled auto-merge November 19, 2026 03:10
@mbouaziz
mbouaziz force-pushed the ci/fast-checks-only-when-needed branch from 6ae9b17 to 6871c3f Compare November 19, 2026 03:10
@mbouaziz
mbouaziz merged commit e3d2ba7 into main Nov 19, 2026
1 of 2 checks passed
@mbouaziz
mbouaziz deleted the ci/fast-checks-only-when-needed branch November 19, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant