Repository navigation
fix(ci): fix breg timer log flake and nightly coverage target dir - #2618
Merged
Merged
Conversation
Readiness refuses as soon as the successor activation is applied, before the parked predecessor worker leaves its checkpoint. The removed-timer test then crashed the replica before the worker wrote its record_timer.worker.iteration_failed line, so the log check saw no fenced attempt. Wait for the line itself before the crash. The operator retry journey has the same ordering: each warning follows the commit that makes its attempt visible. Wait for the expected count before stopping those replicas too. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Cargo runs tests from their crate directory, so the relative CARGO_TARGET_DIR sent the registry-rstack bootstrap tests to a scratch parent that does not exist and failed the developer-tools shard every night. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two CI failures on main.
Flaky breg timer test.
native_removed_timer_fences_a_predecessor_process_after_claimsometimes saw 0record_timer.worker.iteration_failedlines. Readiness refuses as soon as the successor activation applies, while the predecessor worker is still parked in its checkpoint poll; the worker logs the fenced iteration only after it resumes, and the test crashed the replicas about 2 ms after unready. Removing the incidental delay reproduces the CI failure 3 of 3 times. The test now waits (bounded, 10 s) for that log line before the crash; assertions are unchanged. The same ordering exists twice inoperator_retry_after_actual_exhaustion(warning written after the attempt commit that the test waits on), so those now wait for the line too.wait_for_log_codedelegates to a newwait_for_log_code_count.Nightly Rust coverage, failing since 23 Oct. The workflow set a relative
CARGO_TARGET_DIR. Cargo runs tests from their crate directory, so theregistry-rstackbootstrap tests, which create scratch space underCARGO_TARGET_DIR, pointed at a directory that does not exist and five of them failed every night in the developer-tools shard. The target dir is now${{ github.workspace }}/target/nightly-rust-coverage, matchingci.yml; a regression test asserts it stays absolute.Verification:
postgres_record_timer_native(35 passed) andpostgres_record_timer_operator(22 passed) with--test-threads=4cargo fmt --all -- --check,cargo clippy --locked --all-targets -p registry-breg -p registry-bregctl -- -D warningstest_nightly_rust_coverage(new test failed before the workflow change, passes after) andtest_ci_changes; actionlintNot verified: the two operator races did not reproduce in 5 runs (found by reading the code); the nightly fix is proven by the next scheduled run.