Skip to content

test(link): make the ssh runner test guard a hang guard, not a timing race - #8582

Merged
lidge-jun merged 1 commit into
devfrom
fix/link-ssh-runner-deadline-race
Nov 12, 2026
Merged

lidge-jun merged 1 commit into
devfrom
fix/link-ssh-runner-deadline-race

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

The dev batch run 43063783324 (852bc147b6) failed on macos 3/4 at tests/clients/link-ssh-runner.test.ts:152, "late stdin end and exit rejections after timeout are observed". The test expected the runner's timeout rejection and got the helper's own "run did not settle within the test deadline".

The fault is in the test helper. bounded() raced a 250 ms wall-clock guard against the runner's own 25 ms deadline, so it acted as a second timer competing with the one under test. The failing case took 420.73 ms, so the runner VM had stalled for well over 250 ms, and the guard won.

bounded() is now a hang guard and no longer a timing assertion:

  • Before: a 250 ms guard; a host stall of a few hundred ms could flip the result.
  • After: a 2 s guard (80× the runner's deadline, inside Bun's default 5 s test timeout, so a real hang still fails with the same message). Before rejecting, it yields one macrotask, so a runner rejection queued in the same timer batch is seen first.

The runner (src/link/ssh-runner.ts) is unchanged.

No code change from #8551 or #8559 is involved. #8551 left the run() timeout path byte-identical: it only added a stderr line-tap branch that run() never enables. #8559 didn't touch the runner. #8551 did add new link-ssh-capture* test files, which changed which files share this test's CI batch, so a load or timing shift from batch composition can't be ruled out. The fix doesn't depend on that question. The test file hasn't changed since #6955 (October). None of the 20 earlier failed dev batches (2026-10-30 to 2026-11-11) failed any test in this file. This batch is the first time it showed up.

Reproduction. I could not reproduce the failure deterministically on local macOS with Bun 1.4.4, the same version as CI:

  • The exact CI batch (batch 19/74, the same 12 files, bun test --isolate --parallel=1 --timeout 60000): 14 runs, 8 of them under 24 CPU hogs, 0 failures.
  • The file under 32 CPU hogs on 16 cores, --rerun-each 40: 1160 pass, 0 fail.
  • Injected synchronous event-loop stalls of 0–400 ms after both timers were scheduled, with and without --isolate: all pass.
  • An instrumented run showed the runner's rejection settling inside its own 25 ms timer tick, before a same-millisecond timer, the next setImmediate, or a +1 ms timer.

So the runner path is sound on this host. The CI failure needed a stall that local runs don't produce, and the fix removes the helper's dependence on two independent timers firing in order.

Verification

  • bun test tests/clients/link-ssh-runner.test.ts: 29 pass, 0 fail.
  • bun test --isolate --parallel=1 --timeout 60000 tests/clients/link-ssh-runner.test.ts --rerun-each 20: 580 pass, 0 fail.
  • The guard still catches real hangs. I removed cancellation.abort() from the runner as a temporary mutation, then restored it. The 11 bounded() tests that depend on that abort (14 tests use the helper; the 3 output-limit cases reject without it) all failed at about 2 s with "run did not settle within the test deadline".
  • bun check on the test file reports one existing TS2769 at what is now line 214 (line 204 on dev). It isn't introduced here, and test files are outside bun run typecheck.

NOT RUN: full bun run test, a macOS CI rerun (the coordinator runs the dev batch), any deterministic local reproduction of the CI stall.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (Test-only; none needed.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (Test-only.)

Summary by CodeRabbit

  • Tests
    • Adjusted a test timeout guard to better align with the runner deadline and test framework timeout.

… race

bounded() raced a 250 ms wall-clock guard against the runner's own 25 ms
deadline. On a stalled macOS CI host (dev batch run 43063783324, macos 3/4)
the guard won and "late stdin end and exit rejections after timeout are
observed" failed with the guard's message instead of the runner's timeout.

The guard is now 2 s, inside Bun's default 5 s test timeout so a real hang
still fails with the same message, and it yields one macrotask before
rejecting so a settlement queued by the same timer batch wins. The runner is
unchanged.
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner November 12, 2026 03:13
@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T03:15:03.756996Z f21e36f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions github-actions Bot added the chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature). label Nov 12, 2026
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The SSH runner test helper now uses a 2,000 ms guard and schedules rejection with setImmediate after the timer fires.

Changes: SSH runner test guard

Layer / File(s) Summary
Guard timing and rejection
tests/clients/link-ssh-runner.test.ts
The guard increases from 250 ms to 2,000 ms. After the timer fires, the helper schedules rejection with setImmediate. Comments describe the guard timing.

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Other

Merge Risk: Minimal · up to f21e3

This is a test-only change that makes a hang guard less likely to flake on macOS CI. It has no production impact and no remaining merge-blocking concerns.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: the ssh runner test guard is turned into a hang guard instead of a timing race. The raw summary and objectives confirm the change to the bounded() helper in tests/clie…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title names the main change: the ssh runner test guard is turned into a hang guard instead of a timing race. The raw summary and objectives confirm the change to the bounded() helper in tests/clients/link-ssh-runner.test.ts, which raises the guard from 250 ms to 2 seconds and defers rejection with setImmediate. The title is specific and matches the changeset, so it passes.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 92e0916 and f21e36f.

Files selected for processing: 1

  • tests/clients/link-ssh-runner.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

@lidge-jun
lidge-jun merged commit 9de73f8 into dev Nov 12, 2026
34 checks passed
@lidge-jun
lidge-jun deleted the fix/link-ssh-runner-deadline-race branch November 12, 2026 04:31

Copy link
Copy Markdown
Contributor

@lidge-jun Could you run the full cross-platform CI on current dev (workflow_dispatch, lane=all) to validate #8580 and #8582 together, and share the run link and exact SHA?

The PR checks passed, but their macOS and Windows jobs were skipped. In the earlier dev run https://github.com/lidge-jun/opencodex/actions/runs/43063783324, macOS failed on the usage-filter index contention fallback and the SSH runner timeout test. Please confirm those two cases pass on native macOS, along with the overall Linux/Windows/macOS results. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants