Repository navigation
Conversation
Paco ReviewThis PR adds a FailOnOtherTerminalReason option to the test wait helper so that cancel-oriented E2E tests fail fast when a PipelineRun finishes with an unexpected reason instead of polling until timeout. It also reworks the gotestsum install script to use the GitHub REST API for authenticated downloads, download to a temp file with retries, and exposes GH_TOKEN in the E2E workflow. Paco found no new confirmed issues in the code reviewed. Note Take this AI review with a grain of salt and use it alongside tests and human review. Reviewed commit: 779fa51. 0 verified findings; 0 summary-only. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3075 +/- ##
=======================================
Coverage 91.98% 91.99%
=======================================
Files 167 167
Lines 12976 12976
=======================================
+ Hits 11936 11937 +1
+ Misses 1039 1038 -1
Partials 1 1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟢 Approval recommended
The review identifies no active issues with the E2E reliability improvements.
No open issues 🎉
What changed in this PR
This PR improves E2E reliability by hardening gotestsum downloads against interrupted responses and shortening cancellation-test failures when a PipelineRun finishes with an unexpected reason.
Changes:
- Downloads gotestsum to a temporary file with retries and uses authenticated GitHub API downloads when a token and
jqare available. - Adds opt-in fail-fast behavior for unexpected terminal PipelineRun reasons, with unit coverage.
- Enables fail-fast waits in the two single-run cancellation tests while preserving existing behavior elsewhere.
| File | Description |
|---|---|
test/pkg/wait/wait.go |
Adds an optional early error for unexpected terminal reasons. |
test/pkg/wait/wait_test.go |
Covers early failure and continued polling for running PipelineRuns. |
test/github_gitops_commands_prefix_test.go |
Enables fail-fast waiting in the GitHub custom-prefix cancellation test. |
test/gitea_gitops_commands_test.go |
Enables fail-fast waiting in the Gitea cancellation test. |
hack/install-gotestsum.sh |
Adds retryable file downloads and authenticated release-asset lookup. |
.github/workflows/e2e.yaml |
Supplies GH_TOKEN to the E2E test step for tool downloads. |
Review effort: Balanced
b0a39a8 to
ab9f819
Compare
The E2E jobs download the gotestsum test runner from GitHub releases before running any test. During GitHub incidents that throttle anonymous traffic, this download can come back cut short, and the script fed the partial file straight to tar, killing the job before a single test ran. The installer now saves the file first and retries the download when it fails or is incomplete. When a token is available it fetches the file through the GitHub API with that token, since the plain release link ignores tokens and is treated as anonymous traffic. The E2E workflow passes the job's own GitHub token to the test step so this authenticated path is used in CI. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
The cancel E2E tests wait for their PipelineRun to be marked as cancelled. When the cancel request was missed and the run simply finished, the test kept waiting for the full ten minute timeout and then reported only a generic timeout. Waiting for a PipelineRun outcome can now opt in to stopping as soon as a matching run ends with a different result, and the error names the run and how it ended. The two cancel tests that watch a single run use it. Tests that track several runs keep the previous behavior, since other runs there are expected to finish differently. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
The E2E jobs used the setup-ko action to install the ko image builder. That action downloads ko from GitHub releases as an anonymous client, with no retry, and unpacks it while it is still downloading. During GitHub incidents that throttle anonymous traffic, this broke most E2E jobs before any test ran. ko is now installed with the Go toolchain, which fetches it from the Go module proxy. This avoids GitHub release downloads entirely and pins ko to a known version. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
Every E2E job downloaded the same helper tools again, several of them from GitHub releases, which fail when GitHub throttles downloads. The gosmee and snazy installs are now pinned to a release and kept in the Actions cache. ko, minica and gotestsum share one cache keyed on their versions, and minica is pinned instead of following its latest version. Once cached, these tools are restored without contacting GitHub releases at all. Building ko from source adds time to every E2E job, and the shared Go cache does not reliably keep it, since that cache is saved by whichever workflow runs first and never updated afterwards. The built ko binary is now cached on its own, keyed on the ko version, so later runs restore it in seconds and only build it again when the version changes. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
ab9f819 to
997162d
Compare
📝 Description of the Change
The scheduled E2E run 42565470032 failed during a GitHub outage that slowed down and cut off downloads for clients that are not logged in. Most jobs died while installing their tools, before a single test ran. One more test waited ten minutes for something that was never going to happen.
This makes the E2E jobs hold up better when GitHub has a bad day:
The image publishing workflow is unchanged.
🔗 Linked GitHub Issue
N/A
🧪 Testing Strategy
🤖 AI Assistance
AI assistance can be used for various tasks, such as code generation,
documentation, or testing.
Please indicate whether you have used AI assistance
for this PR and provide details if applicable.
Important
Slop will be simply rejected, if you are using AI assistance you need to make sure you
understand the code generated and that it meets the project's standards. you
need at least know how to run the code and deploy it (if needed). See
startpaac to make it easy
to deploy and test your code changes.
If the majority of the code in this PR was generated by an AI, please add a
Co-authored-bytrailer to your commit message.For example:
Co-authored-by: Claude noreply@anthropic.com
✅ Submitter Checklist
fix:,feat:) matches the "Type of Change" I selected above.make testandmake lintlocally to check for and fix anyissues. For an efficient workflow, I have considered installing
pre-commit and running
pre-commit installtoautomate these checks.