Skip to content

fix(messaging): reserve SSH forward ports until handoff and recover lost candidates - #7893

Merged
lidge-jun merged 2 commits into
devfrom
p1/7845-remote-port-reservation
Oct 29, 2026
Merged

lidge-jun merged 2 commits into
devfrom
p1/7845-remote-port-reservation

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Summary

Remote messaging setup picked its local SSH forward ports with remotePortCandidate(), which bound port 0, read the number and released the listener before SSH bound that port. Another process, or a second selection in the same process, could take the port in between. Bun also binds a second listener on an occupied port without EADDRINUSE unless exclusive is set, so the collision was not always visible as a bind error. Setup then failed, or forwarded to a port shared with another listener. Authenticated readiness already prevented talking to the wrong peer, so the impact was availability.

This change:

  • Reserves exclusively until SSH spawns. Each candidate is held by an exclusive local listener until immediately before the SSH helper that takes it over starts. Pending selections in one process are tracked, so two selections never return the same port.
  • Detects a lost candidate explicitly. SSH reports -L/-R bind failures through its fixed debug1 messages (identical in OpenSSH 7.9, current portable OpenSSH and Microsoft's Windows OpenSSH). Only that explicit signal counts as forward_port_lost. stderr is scanned in memory and never printed or retained. If the message is missing or worded differently, setup fails closed: readiness stays pending until the existing deadline.
  • Retries a bounded number of times. Setup reselects and retries at most three times, and only after forward_port_lost. Authentication failures, cancellation, independent helper failures and incomplete cleanup stay terminal. Owner authentication, native initialization and return-route registration are still required before the tunnel is used.
  • Fixes the pair-shutdown race. When one helper exits because its port was taken, it closes its sibling. The sibling's shutdown error no longer wins the readiness race and suppresses the retry; each helper's initiating failure is preserved through diagnostic settlement and joined cleanup.

Closes #7845

Verification

Pinned Bun 1.4.4 on Windows.

  • tests/codex-integration/messaging-remote-forward-failure.test.ts, red → green:
    • Original allocator and helper: 2 pass / 4 fail (no exclusivity, undetected bind failure).
    • After the first commit, with the sibling-shutdown race: 11 pass / 4 fail.
    • This head: 15 pass / 0 fail.
  • New tests/codex-integration/messaging-remote-port-reservation.test.ts covers exclusive reservation and duplicate pending selections.
  • Focused set (tests/codex-integration/messaging-remote-*.test.ts, tests/test-layout.test.ts, tests/test-layout-tooling.test.ts): 85 pass / 15 existing Windows skips / 0 fail.
  • bun run test:changed: 769 pass / 112 skip / 0 fail.
  • bun run typecheck (2,228 files), bun run structure:check, bun run privacy:scan, git diff --check: pass. Docs site build passed on the first commit (593 pages).
  • Not run: live SSH against Linux/macOS hosts, the Windows-skipped POSIX cases (left to CI), the full suite.

An independent review of the first commit found the sibling-shutdown race, which the second commit fixes. A fresh independent review of this head (1ef955759) passed with no blockers. It re-ran every ordering: initiator stderr before and after its exit, sibling exit first, both ports lost, and cancellation during shutdown. Retryable cases stopped at three attempts and terminal cases after one; unresponsive helpers settled as cleanup_incomplete within the cleanup ceiling.

Hosted CI: all 30 checks green on 1ef955759. The first test 2/4 attempt failed in tests/ci-workflows/ci-gui-typecheck-gate.test.ts with "Bun's postinstall script was not run". A deadline-killed bun install attempt had left node_modules/bun without its binary, and the retry reported success (written up on #7870). Only that job was rerun, on the same head.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (docs-site/.../reference/cli/messaging.md, structure/remote-messaging.md.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (Authenticated readiness unchanged; SSH stderr is never logged.)

Summary by CodeRabbit

  • Bug Fixes
    • Remote messaging setup now confirms that both SSH forwards are ready before reporting success and performs a final lease renewal.
    • If SSH reports that a selected forward port was lost, setup closes the affected forwards and retries with fresh ports up to three times within the existing setup deadline.
    • Other setup failures, including cancellation and authentication errors, remain terminal.

…tdown

Keep intrinsic I/O and pre-cleanup exit failure provenance on each helper.
Join diagnostic output and pair cleanup before classifying readiness failures,
so cleanup-induced sibling rejection cannot mask an explicit lost forward port.
Independent failures, cancellation and incomplete cleanup remain terminal.

Add deterministic L/R delayed-stderr recovery and exhaustion regressions plus
terminal-failure controls. Keep reservation handoff, setup deadline and the
three-attempt ceiling unchanged.

Validation: regression 11 pass/4 fail before, 15 pass/0 fail after; requested
eight focused files 85 pass/15 skip/0 fail; test:changed 769 pass/112 skip/0 fail;
typecheck, structure:check, privacy:scan and git diff --check passed.
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 1f84057 and 1ef9557.

Files selected for processing: 12
  • docs-site/src/content/docs/reference/cli/messaging.md
  • scripts/test-layout/layout-codex-collision.json
  • src/messaging/remote-forward-failure.ts
  • src/messaging/remote-ports.ts
  • src/messaging/remote-process.ts
  • src/messaging/remote-tunnels.ts
  • structure/remote-messaging.md
  • tests/codex-integration/messaging-remote-enrollment-gaps.test.ts
  • tests/codex-integration/messaging-remote-forward-failure.test.ts
  • tests/codex-integration/messaging-remote-lifecycle.test.ts
  • tests/codex-integration/messaging-remote-port-reservation.test.ts
  • tests/fixtures/test-layout-expected.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

Walkthrough

Remote messaging now holds local forward-port reservations through SSH handoff, detects forward readiness and bind failures, and retries eligible lost-port failures up to three times within the existing setup deadline. Startup also renews the lease before reporting success.

Changes: Remote forwarding setup

Layer / File(s) Summary
Port reservation and SSH readiness
src/messaging/remote-ports.ts, src/messaging/remote-forward-failure.ts, src/messaging/remote-process.ts, tests/codex-integration/messaging-remote-port-reservation.test.ts, tests/codex-integration/messaging-remote-forward-failure.test.ts
Local port candidates are reserved exclusively until handoff. SSH helper stderr is checked for forwarding failure and direction-specific readiness receipts without returning diagnostic text through helper output. Tests cover reservation, receipt parsing, and failure detection.
Forward-pair setup and recovery
src/messaging/remote-tunnels.ts, tests/codex-integration/messaging-remote-port-reservation.test.ts, tests/codex-integration/messaging-remote-forward-failure.test.ts, structure/remote-messaging.md
Pair setup waits for both readiness receipts before loopback inspection. It retries only forward_port_lost, after closing the failed pair, and stops after three attempts. Tests and documentation cover retry, cleanup, cancellation, and terminal failures.
Remote setup integration
src/messaging/remote-tunnels.ts, docs-site/src/content/docs/reference/cli/messaging.md, tests/codex-integration/messaging-remote-enrollment-gaps.test.ts, tests/codex-integration/messaging-remote-lifecycle.test.ts, scripts/test-layout/layout-codex-collision.json, tests/fixtures/test-layout-expected.json
Remote tunnel startup delegates forward setup to the pair flow. The CLI documentation describes lease renewal and bounded recovery. Existing helper mocks and test-layout mappings are updated for the readiness contract and new tests.

Priority: Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant RemoteMessaging
  participant startRemoteForwardPair
  participant SSHHelpers
  participant LoopbackInspection
  RemoteMessaging->>startRemoteForwardPair: request forward setup
  startRemoteForwardPair->>SSHHelpers: start local and return forwards
  SSHHelpers->>startRemoteForwardPair: provide readiness receipts or forward failure
  startRemoteForwardPair->>LoopbackInspection: inspect after both receipts
  LoopbackInspection->>startRemoteForwardPair: return reachability result
  startRemoteForwardPair->>RemoteMessaging: return pair or setup failure
Loading

Possibly related issues

  • lidge-jun/opencodex issue 7845: The changes reserve local candidates through SSH handoff and reselect ports after recognized lost-port failures, matching the issue’s reported release-before-bind race.

Suggested reviewers: phillipod

Merge Risk: Minimal · up to 1ef95

No actionable issue remains from this review; the change is mergeable after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main changes: reserving SSH forward ports until handoff and recovering lost candidates.
Linked Issues check Passed Issue #7845 requires the product paths to retain a local SSH forward candidate until handoff, or detect loss and reselect within a bound, without weakening authenticated readiness. `src/messaging/remo…
Out of Scope Changes check Passed The changes stay within issue #7845. The source changes implement port reservation, SSH bind-failure detection, bounded recovery, pair cleanup, and readiness ordering. The new tests directly verify th…
Full details: Linked Issues check

Explanation

Issue #7845 requires the product paths to retain a local SSH forward candidate until handoff, or detect loss and reselect within a bound, without weakening authenticated readiness. src/messaging/remote-ports.ts adds exclusive socket reservations and process-local pending-port tracking. src/messaging/remote-tunnels.ts hands off the local reservation immediately before SSH spawn, waits for both forward receipts, retries only forward_port_lost, and stops after three attempts. src/messaging/remote-process.ts and src/messaging/remote-forward-failure.ts classify only recognized SSH bind failures and do not retain stderr. The added tests cover exclusive reservation, duplicate pending selections, -L and -R failure detection, delayed diagnostics, retry exhaustion, terminal failures, cancellation, cleanup, and readiness ordering. The changes preserve the existing authenticated tunnel setup path.

Full details: Out of Scope Changes check

Explanation

The changes stay within issue #7845. The source changes implement port reservation, SSH bind-failure detection, bounded recovery, pair cleanup, and readiness ordering. The new tests directly verify those behaviors. The documentation in docs-site/.../messaging.md and structure/remote-messaging.md, test-layout updates, and mock updates support the same implementation and test contract. No unrelated product behavior or cleanup appears in the reviewed changes.

Full details: Docstring Coverage

Explanation

Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 7 functions across 8 files. (4 skipped: 4 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Oct 28, 2026
@lidge-jun
lidge-jun marked this pull request as ready for review October 28, 2026 10:10
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner October 28, 2026 10:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T10:13:13.523691Z 1ef9557 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@lidge-jun
lidge-jun merged commit 69c1c6e into dev Oct 29, 2026
61 of 64 checks passed
@lidge-jun
lidge-jun deleted the p1/7845-remote-port-reservation branch October 29, 2026 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant