Skip to content

Keep rollouts alive through a pause and adopt their live boxes after a restart - #3037

Open
hallerite wants to merge 7 commits into
mainfrom
pause-safe-rollouts
Open

hallerite wants to merge 7 commits into
mainfrom
pause-safe-rollouts

Conversation

@hallerite

@hallerite hallerite commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Prime can now pause a running VM box with its memory (POST /sandbox/{id}/pause|resume, prime-sandboxes 0.5.2). This uses it to keep agentic rollouts alive when the process running them goes away, without snapshotting anything as turns go. It replaces #2994: there, every turn of every rollout snapshots the box's filesystem, and a restore boots a fresh box, relaunches the harness, tells the model the box was restored, and loses whatever ran in the box. Here the box itself is what survives: processes, shell state, servers it started, and the running harness program.

Survive a pause

  • Wait through a pause: starting and polling the harness's background job waits while the box refuses exec for being paused or resuming onto its node, without a time bound.
  • PrimeBackend.pause() / resume() (RuntimeBackend.SUPPORTS_PAUSE).
  • Reset dead connections on resume: outbound TCP connections from before a pause hang forever after it (no RST, no EOF), so an in-flight model call would wait out its 600 s read timeout. pause() lists the box's outbound connections first; after the resume, reset_sockets.py shuts down those still open (pidfd_getfd, checked against the inode), so the call fails at once and its retry lands on the interception server's replay. After someone else's pause (prime sandbox pause), the job poller resets every outbound one. Loopback and the guest agents' connections are left alone.
  • Retry the tunnel edge: the bash/null chat program retries for up to 15 minutes while the tunnel edge answers for the interception server (its HTML 404 Tunnel Not Found or 5xx pages): frpc reconnecting, or the process behind the tunnel restarting.

Adopt a live box in another process

  • Restore points: an adoptable rollout streams a live restore point once its harness program starts (Trace.live, LiveBox): the sandbox, the program's background job, the model and state secrets, and the tunnel's registration. RestoreRequest(trace, point), RunRequest.restore, EnvClient.run(restore=, assembly=), EpisodeAssembly.restores and EpisodeHook.watch / .restore keep feat: rollout checkpointing #2994's names and wire format.
  • Adopting: a restore with a live point binds the sandbox (resuming it if paused), skips task and harness setup, registers the session under the saved secrets on a server that serves the saved tunnel (one per tunnel, shared by every rollout behind it), and waits on the running program instead of launching one. The trace is never rewound: turns the program took after the client's copy arrive with its next request and are recorded as context (sampled=False), not samples. A box that cannot be taken over starts the run over, unbudgeted; a takeover cancelled mid-way leaves the box for the next resume.
  • Adoptable: bash and null (Harness.SUPPORTS_ADOPT) on Prime, in an owned colocated box behind a prime tunnel, without tool servers, tool VMs, compose, or request/response hooks (they write state mid-run). Anything else starts over, as on main.
  • Suspend instead of delete on shutdown: with VF_SUSPEND_DIR set, a worker that shuts down (SIGTERM, or its parent gone) pauses its adoptable rollouts' boxes instead of deleting them (60 s shared budget), keeps their tunnels registered, and records each box in <dir>/<hostname>-<pid>.jsonl. So does a run whose client is gone (a crashed orchestrator: the pool's broker cancels the runs it cannot deliver, a lone server fails to send a delta), while the server keeps running; until its box is paused, the interception server holds that program's requests instead of refusing them, so the pause freezes them and they retry against the adopting process. A run its client cancels still deletes its box.
  • Routing: restores whose boxes share a tunnel go to one worker (run@<tunnel_id> method frame), which adopts the tunnel once.
  • For orchestrators: EnvClient.detach() (cancelled runs stop aborting server-side, so a shutdown can suspend them), and best-effort pause_boxes / delete_boxes.

prime-rl uses this in PrimeIntellect-ai/prime-rl#4348 (ckpt.rollouts): env servers suspend on shutdown, a resumed run pauses the boxes its checkpoint resumes in as soon as it has read it and deletes the rest, then adopts each one.

Tests

On Prime with a scripted model, bash harness (new CI cells Pause and Readopt):

Case Result
test_pause[tool]: 10 s pause during a 20 s sleep 1 upstream call per turn, 1 branch, reward 1.0 (51 s)
test_pause[model]: pause during a model call 1 upstream call per turn, 1 branch, reward 1.0 (51 s)
test_readopt[tool]: process SIGKILLed during a tool call, another adopts same sandbox and background PID; the two gap turns recorded as context; each turn upstream once (53 s)
test_readopt[model]: SIGKILLed with a model call in flight that turn went upstream twice, the others once, 1 branch (18 s)
test_readopt[box-gone]: box deleted before the restore starts over in a new box, reward 1.0 (72 s)
test_suspend_on_shutdown[shutdown]: detach, cancel, SIGTERM the server both boxes PAUSED, 2 manifest lines, one shared tunnel; a 2-worker pool adopts both on one worker and both finish with their original PIDs (57 s)
test_suspend_on_shutdown[client-gone]: the client's socket closes without a word the server suspends both boxes while it keeps running; both are adopted and finish with their original PIDs

Controls for the pause path: without the socket reset the rollout hangs until the 180 s agent timeout; without the pause wait it fails with sandbox_error; with the tunnel down for 40 s, main's harness dies with openai.NotFoundError: 404.

Not covered

  • claude-code / codex / rlm are not adoptable yet: their process stream to the box gives up while paused (prime-sandboxes _can_reconnect), and re-attaching it from a new process is untested. They start over.
  • Takeover relies on private prime_tunnel attributes (serving an existing tunnel from a new process, keeping its registration on shutdown); prime-tunnel should grow a public API for it. Takeover is verified up to 60 min after the old process died.
  • Pauses longer than 10 min during a model call may record a second branch: the interception server's replay of a finished reply expires after IDEMPOTENCY_CACHE_TTL_SECONDS.
  • The socket reset needs root and a Python in the box; without them dead connections fail on their owners' timeouts. read / write / exec still refuse while paused.

🤖 Generated with Claude Code

Note

Add pause, suspension, and live-box restore for rollouts and sandbox backends

  • Lets a rollout survive a pause and be continued in its live box after a restart. A LiveBox restore point (sandbox, tunnel, job handle, secrets) is recorded on the trace and streamed to clients; a restored run adopts the saved sandbox and attaches to its saved job instead of starting over.
  • Adds a suspend system gated by the VF_SUSPEND_DIR environment variable in suspend.py. On shutdown (or a lost client), claimed backends are paused, kept, and recorded in a JSONL manifest; the tunnel and sandbox are not deleted.
  • EnvClient can run detached, restore from a RestoreRequest, and route requests by affinity so a restored run lands on the worker holding its tunnel. EnvServer restores traces, marks runs lost when deltas cannot be delivered, and treats ID-less cancels as lost-client signals.
  • PrimeBackend gains adopt, attach, pause, and resume, retries provider calls refused during a pause, and resets outbound sockets on recovery via reset_sockets.py. Tunnel and tool-gate requests in core.py retry for up to 15 minutes during pauses.
  • Behavioral Change: PrimeBackend.cleanup and teardown keep the sandbox when suspension is requested; interception requests with held-but-unregistered secrets block on a future instead of returning unauthorized. Adoption is rejected for tasks with a deployment spec, tool servers, tool URLs, or interceptors (see Rollout._adoptable in rollout.py). Raises prime-sandboxes minimum to 0.5.2.

Macroscope summarized 84a5c2d.


Note

High Risk
Changes sandbox lifecycle, interception auth secrets, tunnel adoption, and env-server shutdown semantics—core orchestration paths where bugs could leak boxes, drop rollouts, or leave hung requests.

Overview
Rollouts can keep running in the same Prime sandbox when the host process pauses, shuts down, or is replaced—instead of snapshotting the filesystem each turn or booting a fresh box on restore.

Pause/resume on Prime (prime-sandboxes 0.5.2): PrimeBackend gains pause/resume, exec polling that waits through platform pause refusals, and post-resume TCP socket reset (reset_sockets.py) so stale outbound connections fail fast and model/tool retries reconnect. The bash/null chat program retries tunnel-edge HTML errors for up to 15 minutes while interception comes back.

Live-box adoption: Once a harness program starts, the trace streams a LiveBox restore point (sandbox id, background job, interception secrets, tunnel registration). RunRequest.restore / Agent.pending_restore continue that trace in another process: bind the sandbox, attach to the existing job, adopt the saved tunnel on one shared interception server per tunnel, and record gap turns as context (sampled=False) rather than rewinding. Failed or missing boxes start over unbudgeted. VF_SUSPEND_DIR makes shutdown (or a detached client) pause and manifest adoptable boxes instead of deleting them; in-flight interception requests block until suspend completes.

Serve/routing: Deltas carry restore payloads; EnvClient.run(restore=, assembly=) and pool run@<tunnel_id> affinity pin restores to the worker that can hold the tunnel. Bash and null harnesses set SUPPORTS_ADOPT; adoption is rejected when compose, tool VMs, or mid-run hooks would diverge from the saved state.

New CI integration suites Pause and Readopt cover pause mid-tool/model, SIGKILL + restore, deleted box, and suspend-on-shutdown paths.

Reviewed by Cursor Bugbot for commit 84a5c2d. Bugbot is set up for automated code reviews on this repo. Configure here.

hallerite and others added 2 commits November 1, 2026 01:04
A paused Prime box refuses exec, so the harness's job polling failed the
rollout with a sandbox error. Its processes are only frozen, so wait for
the resume instead, without a time bound.

- PrimeBackend.pause()/resume() freeze and continue the box, memory
  included (RuntimeBackend.SUPPORTS_PAUSE; other backends raise).
- After a resume, shut down the outbound TCP connections the pause left
  dead (reset_sockets.py, run inside the box via pidfd_getfd). Otherwise
  the chat program waits out its 600 s read timeout on a model call
  that was in flight. The job poller does the same reset once when
  someone else paused and resumed the box.
- The bash/null chat program retries a request for up to 15 minutes
  while the tunnel edge answers for the interception server (its HTML
  404 "Tunnel Not Found" or 5xx pages). This covers frpc reconnecting
  or the host restarting behind the tunnel.
- prime-sandboxes >= 0.5.2 for pause/resume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@macroscopeapp

macroscopeapp Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This adds pause/resume and cross-process restoration of live rollouts. The opt-in shutdown path retains billable boxes and tunnels—even when pausing fails—so resource inventory, cleanup ownership, and failure monitoring need operational confirmation.

Needs human approval because:

  • 4 blocking correctness issues found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

A process that dies mid-rollout leaves its box running: the harness program
holds its turns, background processes and shell state, and reaches the model
through a tunnel URL and secrets that outlive the process. Let another
process take the rollout over in that box.

- A rollout another process can take over streams its live box once the
  harness program starts: a `restore` delta carrying `RestorePoint.live`
  (`LiveBox`: sandbox_id, tunnel_id, the tunnel's registration, the
  program's job, the model and state secrets, the typed state), collected
  in `EpisodeAssembly.restores`. `Trace.live` holds it, off the record.
- `EnvClient.run(restore=RestoreRequest(trace, point), assembly=...)`
  continues a trace in another env server (`RunRequest.restore`, packed
  raw). The server rebuilds the trace (`EpisodeHook.restore` gives nodes
  back their arrays, `EpisodeHook.watch` sees it), and the episode's one
  agent continues it (`Agent.pending_restore`).
- The run binds the sandbox (`RuntimeBackend.adopt`, resuming it if
  paused), skips task and harness setup, registers its session under the
  saved secrets on a server that serves the saved tunnel
  (`Interception.adopt`: one server per tunnel, shared by every adopted
  rollout behind it), and waits on the running program
  (`HarnessSession.attach`, `RuntimeBackend.attach`) instead of launching
  one. The trace is not rewound: turns the program took after the
  client's copy arrive with its next request and are recorded as context.
  A box that cannot be taken over starts the run over, unbudgeted.
- Adoptable: bash and null (`Harness.SUPPORTS_ADOPT`) on Prime
  (`RuntimeBackend.SUPPORTS_ADOPT`), in an owned colocated box behind a
  prime tunnel, with no task tool servers or request hooks.
- With VF_SUSPEND_DIR set, an env server that shuts down (SIGTERM, its
  parent gone) pauses the boxes of adoptable rollouts instead of deleting
  them, records each in <dir>/<hostname>-<pid>.jsonl, and keeps their
  tunnels registered (`runtimes.suspend`). A per-run cancel still deletes.
- `EnvClient.detach()`: a cancelled run no longer cancels on the server.
- A pool routes restores behind one tunnel to one worker (`run@<tunnel>`).
- `pause_boxes` / `delete_boxes`: best-effort helpers for an orchestrator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread verifiers/v1/runtimes/backends/prime.py Resolved
Comment thread verifiers/v1/rollout.py Resolved
- Before pausing the box itself, PrimeBackend lists its outbound
  connections (reset_sockets.py list); the recovery after the resume
  shuts down only those still open, never one opened since. After a
  pause someone else made, it still shuts down every outbound one.
- Check the borrowed descriptor is the socket read from /proc before
  shutting it down: its owner may have reused the fd meanwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment on lines +363 to +366
if status == "PAUSED":
await self.resume()
elif status != "RUNNING":
raise SandboxError(f"sandbox {box} is {status}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High backends/prime.py:363

Adopting a box that was already resumed leaves its pre-pause sockets untouched, so in-flight model calls can hang until their timeout and fail the restored rollout. _take_over() only recovers when it sees PAUSED; on the RUNNING path, call _recover(self._recovered) to reset connections left by an earlier pause.

Suggested change
if status == "PAUSED":
await self.resume()
elif status != "RUNNING":
raise SandboxError(f"sandbox {box} is {status}")
if status == "PAUSED":
await self.resume()
elif status == "RUNNING":
await self._recover(self._recovered)
else:
raise SandboxError(f"sandbox {box} is {status}")
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/runtimes/backends/prime.py around lines 363-366:

Adopting a box that was already resumed leaves its pre-pause sockets untouched, so in-flight model calls can hang until their timeout and fail the restored rollout. `_take_over()` only recovers when it sees `PAUSED`; on the `RUNNING` path, call `_recover(self._recovered)` to reset connections left by an earlier pause.

@hallerite hallerite changed the title Let rollouts on Prime survive a pause of their box Keep rollouts alive through a pause and adopt their live boxes after a restart Nov 1, 2026
hallerite and others added 3 commits November 1, 2026 03:16
- A takeover cancelled mid-way (the run shutting down again) no longer
  deletes the box at teardown: the next resume can still adopt it. A
  failed one still deletes it, since a paused box never expires.
- A rollout with response interceptors is not adoptable: they may write
  state mid-run, and the live box carries the state as of the program's
  start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An env server whose client (the orchestrator) died cancelled the runs it
could no longer deliver, and the cancel deleted their boxes before the
server itself was stopped: a crashed orchestrator lost most live boxes.

- A run whose client is gone (a pool's broker cancels it with no request
  id, a lone server fails to send it a delta) is cancelled with its
  suspend scope lost: its teardown pauses and records the box like a
  shutdown does, while the server keeps running.
- Until the box is paused, the interception server holds the requests of
  a suspending rollout's program instead of refusing them (401 ends the
  program); the pause freezes them, and they are retried on resume
  against the process that adopts the box.
- test_suspend_on_shutdown also covers the client going away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment on lines +109 to +111
if not suspending() or tunnel_id is None:
return False
return tunnel_id in _kept_tunnels or any(t == tunnel_id for _, t in _live.values())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High runtimes/suspend.py:109

When one rollout has suspended, keeps_tunnel() returns False for its shared tunnel if the last user exits from a healthy run scope. The not suspending() check prevents _kept_tunnels from being consulted, so Interception.adopt() closes the registration and Tunnel.sync_stop() deletes the endpoint needed to readopt the paused box; check _kept_tunnels independently of the current scope.

Suggested change
if not suspending() or tunnel_id is None:
return False
return tunnel_id in _kept_tunnels or any(t == tunnel_id for _, t in _live.values())
if tunnel_id is None:
return False
return tunnel_id in _kept_tunnels or (
suspending() and any(t == tunnel_id for _, t in _live.values())
)
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/runtimes/suspend.py around lines 109-111:

When one rollout has suspended, `keeps_tunnel()` returns `False` for its shared tunnel if the last user exits from a healthy run scope. The `not suspending()` check prevents `_kept_tunnels` from being consulted, so `Interception.adopt()` closes the registration and `Tunnel.sync_stop()` deletes the endpoint needed to readopt the paused box; check `_kept_tunnels` independently of the current scope.

Comment thread verifiers/v1/rollout.py
await self._harness_session.turn(messages)
job, self._attach = self._attach, None
if job is not None:
await self._harness_session.attach(job)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High v1/rollout.py:713

A restored trace can be marked agent_completed and scored with a stale answer when attach(job) reconnects to a job that already finished. The attach path does not replay the program’s final response into the trace, so a successful exit can appear complete despite the missing turn; recover the final transcript or restart instead of accepting that attach.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/rollout.py around line 713:

A restored trace can be marked `agent_completed` and scored with a stale answer when `attach(job)` reconnects to a job that already finished. The attach path does not replay the program’s final response into the trace, so a successful exit can appear complete despite the missing turn; recover the final transcript or restart instead of accepting that attach.

)
except Exception as e: # a sandbox/API failure is one rollout's problem
raise SandboxError(f"prime exec failed: {e}") from e
started = program_started.get()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High backends/prime.py:439

When launch_chat_program prepares a message-list prompt for an agent user, PrimeBackend.write runs chmod, the copy, and cleanup through run, so program_started publishes a helper job to Trace.live before the harness starts. A checkpoint during those awaits restores by attaching to that helper instead of launching the chat program, preventing the rollout from continuing. Notify program_started only for the harness-program launch, not preparatory runtime operations.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/runtimes/backends/prime.py around line 439:

When `launch_chat_program` prepares a message-list prompt for an agent user, `PrimeBackend.write` runs `chmod`, the copy, and cleanup through `run`, so `program_started` publishes a helper job to `Trace.live` before the harness starts. A checkpoint during those awaits restores by attaching to that helper instead of launching the chat program, preventing the rollout from continuing. Notify `program_started` only for the harness-program launch, not preparatory runtime operations.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e98859a. Configure here.

Comment thread verifiers/v1/runtimes/suspend.py Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant