Repository navigation
Give the validation runner contract step headroom over its measured run time - #825
Merged
Merged
Conversation
…un time Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: coneilen <coneilen@users.noreply.github.com>
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
windows-spikes Swiftjob's "Verify validation runner contract" step had a 5-minute (300s) timeout. That contract has grown with each new set of runner test cases: about 75s a day ago, 180–276s across recent PRs, and 302s on run 38663273264, which failed CI on a 1–2s overrun. The same contract measured up to 341s in deterministic hardening (run 38659424901). Timeouts that close to normal run time make CI flaky, so this raises the step's timeout to 15 minutes and guards it.Changes
.github/workflows/windows-port-validation.yml: raise the "Verify validation runner contract" step timeout from 5 to 15 minutes. The job's 45-minute timeout is unchanged: the job measured 21–25 minutes, so even a 15-minute contract run stays inside it.Tools/windows/Tests/ValidationRunner.Tests.ps1: a new contract fails if that step's timeout is missing or below 12 minutes. That's about twice the measured 341s worst case, the same measured-headroom rule the file already applies to other timeouts. A comment records the measured runs.Test plan
RED: guard extracted from Tools/windows/Tests/ValidationRunner.Tests.ps1 run against origin/main windows-port-validation.yml -> RED: cap=5, exit 1
GREEN: same guard against the edited workflow -> GREEN: cap=15, exit 0; negative with the step timeout removed -> RED: cap empty, exit 1
REGRESSION: PowerShell Parser.ParseFile on Tools/windows/Tests/ValidationRunner.Tests.ps1 -> 0 errors; git diff --check -> clean; bash Tools/ci/tests/classify-changes.test.sh -> 62 passed, 0 failed; exact-head CI runs the full contract in windows-spikes Swift and both deterministic hardening jobs
Measurements (from
gh run viewstep timings, 23 recent port runs and 24 hardening jobs):windows-spikes Swiftjob: 21–25 minutes.Not verified locally: I didn't run the full contract locally, because it opens hidden native windows. Exact-head CI is the regression gate. macOS
make testandmake checkwere not run, because no Swift changed.Checklist
git commit -s) per the DCOmake test)make check)