Repository navigation
feat: restore rollouts from memory snapshots - #3141
Draft
mikasenghaas wants to merge 68 commits into
Draft
mikasenghaas wants to merge 68 commits into
mikasenghaas wants to merge 68 commits into
Conversation
Add an in-box relay (`interception/relay`) that a harness in a remote box reaches the interception server through, at 127.0.0.1, instead of the tunnel URL (`agent.relay`, off by default). The relay retries a model call for up to 5 minutes while the tunnel cannot be reached, and marks each retry so the interception server joins it to the call it already serves. When a tunnel is lost for good, its server still takes no new rollouts, but no longer fails the ones that reach it through a relay: it opens a new tunnel, allows its host in each box's egress policy and reroutes each relay. Rollouts with tool servers or deployment services still fail, since those reach the old URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop `agent.relay`: a harness in a remote box always reaches the interception server through its relay, and a relay that does not start leaves it on the tunnel URL with a warning. - Give the relay only the model secret, which the harness already holds: a harness that reads the relay config gains no capability. - Run one relay per rollout (own dir, ephemeral port) and stop it when the rollout closes, so rollouts that share a box never share one. - Make a rollout movable before the server registers it; a tunnel lost while the relay starts waits for it. - Replace the tunnel again when it is lost during a replacement. - Retry a non-streaming body that breaks before any data, and a body that ends before its Content-Length. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relay added about 4 s to setup (GSM8K, p50 6.4 s -> 10.6 s): eight sandbox calls one after another. Launch it while the harness sets up, with a placeholder upstream, and configure it once the interception slot is acquired: one write and one exec on the critical path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `Relay` object owns the relay's lifecycle (launch alongside the harness setup, connect, reroute, stop), so the rollout keeps one field. The retry window is a constant; tests patch the script source. The relay verified the tunnel's certificate against the system CA store, which a task image may lack: terminal-bench-2's financial-document-processor image has no /etc/ssl/certs, so both its relayed rollouts failed on their first model call while the harness's own HTTP client (certifi) worked on main. Trust certifi's roots as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relay is part of the machinery like the interception server: a relay that does not start fails the rollout with a SandboxError, which the rollout retry rules cover, instead of a silent fallback to the tunnel URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `RelayError` (an `InterceptionError`, next to `TunnelError`). A relay that does not start raises it. A relay that gives up on a request appends the reason to a failures file in its directory; when the harness then dies, the rollout checks that file and whether the relay still runs, and records `RelayError` with the relay's reason instead of the harness's `HarnessError`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scripts prepared in parallel each ran the uv bootstrap under a lock of their own script: in an image without uv, two `pip install --user uv` runs or downloads to one path (or apt's lock) raced, and one failed. The relay prepares its script during the harness setup, so CI hit it: a random network-policy cell failed with zero branches. The first preparation per user now holds a bootstrap lock; once uv is there, scripts prepare in parallel. The integration grid's one-branch assertions show the trace's errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Mark a resend as a retry only after an attempt that may have reached the server; a connect failure or an edge page did not, so a body a later turn repeats is never answered from the replay cache. - Never frame SSE with a fixed length, and close an edge error page unread instead of draining it. - Count a new outage from its start once a response stayed up. - Record "ok" after a request goes through again, so a stale relay failure never names a later, unrelated harness death. - Stop the relay process also after a failed or cancelled launch. - Leave a closing rollout's egress alone on a reroute (shared scoring sets its own), and strip the trailing slash `host_url` may return. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent snapshots its prime box as model turns go upstream, and a rollout retry restores the newest snapshot in a new box and continues the trace from there instead of starting over. On by default (`[env.agent.checkpoint]`: interval, keep, after, boxless). The trace records each snapshot (`Trace.checkpoints`), rewinds to the one a retry restores (`Trace.rewind`, `Trace.generation`), and gives another process what it needs to continue it (`Trace.restorable`, `RestorePoint`, `Env.run_slot(restore=...)`). null and bash relaunch on the transcript; claude-code, codex and rlm load their ACP session from the restored box (rlm pinned to nano-rlm with session/load). A request hook that raises now ends the rollout with its error. This is the filesystem-only state of verifiers#2994 as of ebfd9f8, on today's main. Its env-server parts moved with the env server to prime-rl and are not part of this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scope rollout checkpointing to retries within one Agent.run: drop the cross-process restore (Env and Agent.run restore params, RestorePoint, PinnedResult, Trace.restorable, Agents.only). Call the runtime side a snapshot: backends return a typed Snapshot (id, type) from snapshot(), wait_snapshot() and discard_snapshots() take it, and restore_config() builds the config a new box boots from. Runtimes and harnesses declare SUPPORTS_FILESYSTEM_SNAPSHOTS, and RolloutCheckpointConfig.type takes a SnapshotType, so memory snapshots can slot in behind the same API. Also from review: - snapshot every 10th turn by default - copy state before the first await of a capture, so response hooks that run meanwhile do not leak into the checkpoint - trim the checkpoints after the restored one by position, not node - build the rollout params once per run, so an RLM retry keeps its skill home and the restored session loads - discard snapshots on the shared leased client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A turn cancelled while its reply waits for the snapshot capture now still records its call and abandons its preview. The wait is shielded, so the snapshot finishes in the background, where the rollout awaits it. A capture reads the turn's node and tail before its first await, like the state copies, so everything a checkpoint describes comes from the idle box. Docs: show the checkpoint config as a TOML example, and shorten the harness flag's entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restore a rollout from a snapshot end to end on prime: per harness, at the default interval, for rlm with skills, and for each retention policy. The cases run once per SnapshotType, on every harness and runtime that declare SUPPORTS_<TYPE>_SNAPSHOTS, so a new snapshot type gets its cases without a test change. CI runs it as "Checkpoint Restore". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A box-less checkpoint let a harness that runs no code restore from the trace alone on a runtime without snapshots. It served restores in another process, which this PR no longer does; for retries in one run it only saves the few turns of a short chat rollout. Every checkpoint now has a snapshot, and the boxless setting is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record every snapshot the run's boxes take, across attempts, and check each on the platform once the run is done: deleted, except the ones the retention policy keeps, which stay restorable. Before, only the checkpoints left on the final trace were checked, so a rewound snapshot or one still uploading at the end could leak unnoticed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # verifiers/v1/agent.py # verifiers/v1/rollout.py # verifiers/v1/runtimes/backends/prime.py
When a rollout is cancelled (an agent timeout, shutdown), its harness turn is cancelled and then joined. A second cancellation during that join left the turn running while teardown began, where it could still write the trace or use the box. Join it shielded, then re-raise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review: keep the checkpoint config to what a user decides. - interval counts from turn n (the n-th, 2n-th, ...), so a rollout shorter than n turns takes no snapshot; the turn a restore resumes at is not snapshotted again, so keep = "last" no longer deletes the snapshot that just restored - every snapshot is deleted once the run is final: drop `after` - drop keep = "all", which keep = 1 already expresses - drop `type` from the config and SnapshotType from the API: the runtime takes the best snapshot it supports and records its type on the checkpoint (Snapshot.type) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A harness can continue a rollout in a box restored from a snapshot when a new process of it can take the rollout over. For a harness that resumes on the transcript (SUPPORTS_RESUME), a restore is that same relaunch, so it needs nothing more. An ACP harness keeps its session in a live process and needs its agent to load the session from its files: ACPHarness.LOADS_SESSIONS (claude-code, codex, rlm). This replaces the harness-side SUPPORTS_FILESYSTEM_SNAPSHOTS (Harness.can_restore). browser-use now restores like any harness that resumes on its transcript. A restore whose attempt fails before its first new turn is marked failed like one whose box never booted, so the next retry steps back to an older snapshot or starts over instead of restoring the same snapshot until the retries run out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The condition read `target is not None and A or B`, so B ran for runs that restored nothing and read `restored_turns` on them. Group it as `target is not None and (A or B)`. The new test's fake rollout also called the run's `on_trace` hook, which it does not pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mem-ckpt # Conflicts: # verifiers/v1/session.py
…at/mem-ckpt # Conflicts: # verifiers/v1/interception/server.py
A memory snapshot (a pause of the box) holds the box's processes. A retry boots a new box from it, reroutes the box's relay to the new session and waits for the harness program the box still runs, so no harness has to load a session or be told about the restore. The ACP runner can run as a job of the box with one job per request, because no connection from the host survives a stopped box. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
A snapshot at every 10th turn stops every healthy box for a few seconds. Take the snapshot when it is needed instead: when a model call fails with an error the run retries, before the harness gets the error. The box that the retry restores still waits for the reply, and asks for the call again. `checkpoint.interval` is now off by default. It stays as an option for failures that leave no box to snapshot (the box died, the harness program crashed). A model call that fails again right after a restore no longer counts as a restore that led nowhere: the retry restores the same snapshot. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
A memory snapshot holds every process of the box, so the sub-agents of a harness continue with it. Nothing tested that. The new case saves an rlm rollout while two sub-agents are at work, and checks that a second owner's box continues both without starting either again. Co-Authored-By: Claude <noreply@anthropic.com>
The relay starts alongside the harness setup. Its install puts uv in a box that has none, and the setup of some harnesses (prime-agent, hermes-agent) does the same in a shell of its own. The two installs could overlap, and one then ran the other's half-written binary: "uv: Permission denied" from the relay start, seen once in CI. Co-Authored-By: Claude <noreply@anthropic.com>
The relay starts alongside the harness setup. Its install puts uv in a box that has none, and the setup of some harnesses (prime-agent, hermes-agent) does the same in a shell of its own. The two installs could overlap, and one then ran the other's half-written binary: "uv: Permission denied" from the relay start, seen once in CI. Co-Authored-By: Claude <noreply@anthropic.com>
An owner that saves every running rollout pauses all their boxes at once. At thousands of rollouts that is a burst of pause and resume calls the platform may refuse, and a box whose resume fails never runs its rollout again. snapshots_per_min spreads the pauses out with the bucket that already paces creates. Off by default. Co-Authored-By: Claude <noreply@anthropic.com>
A saved rollout was a RestorePoint: the trace plus fields beside it that only that class carried. Whoever held just the trace could not continue it. save_all now returns the trace as plain data, with what a restore needs on the entry of its checkpoint, and a restore takes that trace. Snapshots on a failed model call are now opt-in (checkpoint.on_error). They help only when the harness would end on the error; a harness that handles the failed call itself never uses the snapshot, and a run without retries never takes one. Co-Authored-By: Claude <noreply@anthropic.com>
Main moved blocking commands on Prime to native sessions, so a task that clears scratch directories cannot delete the job files of a running command. A session ends with its box, and a box that boots from a memory snapshot is a new one. The harness's main program therefore stays a server-owned job, which `attach` can continue to wait on. Every other command takes main's session path. Co-Authored-By: Claude <noreply@anthropic.com>
A box that boots from a memory snapshot keeps the sandboxd session of the harness program, so the host can attach to it by its session id. The program no longer needs a polled background job, and it gets the same protection as every other command against a task that clears the scratch directories. A snapshot also ends the stream into the box that was paused. The wait now attaches to the session again when the box runs. Needs a prime-sandboxes release with attach_process. Co-Authored-By: Claude <noreply@anthropic.com>
A memory snapshot resets the box's open connections, so the relay asks again for the call that just failed. The interception server keeps no failed attempt, so the box gets a new model call and the harness rarely gets the error at all. The snapshot was a slow way to retry a model call, and its outcome depended on when the relay's request arrived. A retry of a model call belongs at the call. Snapshots stay for a box or a host that goes away: the owner's save, and `interval`. A call that the box asks again at a snapshotted turn now takes no second snapshot, or a call that keeps failing would take one per request. Co-Authored-By: Claude <noreply@anthropic.com>
The restore tests continued a rollout from one point and cut the trace back at most one turn. This one saves a rollout at three turns and continues each saved trace: with a scripted model, every continuation must end with the messages of the first run. A counter that a process holds only in memory shows a command that ran twice or was skipped. Co-Authored-By: Claude <noreply@anthropic.com>
A pause that the platform accepted stops the box. When the wait for the paused state then failed, the box was never resumed and its snapshot was dropped from the list that teardown frees. One failed batched status call fails every box that waits in it, so a save of many rollouts could stop them all for good. Resume the box whatever the wait came to, keep the snapshot for teardown, and ask again when a status poll fails. Co-Authored-By: Claude <noreply@anthropic.com>
The relay took every 404, 502, 503 or 504 that was not JSON or SSE for the error page of a tunnel edge and retried it for five minutes. The interception server answers an unknown path with a plain-text 404, so a harness that asked for one waited that long. The server now marks each of its responses with a header, and only a response without it counts as the edge's. Co-Authored-By: Claude <noreply@anthropic.com>
Three cases let a box see a reply that its checkpoint does not record, or left a rollout stopped: - A release (the owner's, or the timer) answered the box while another snapshot had it stopped. The snapshot now ends the hold itself when it is done. - A save that was cancelled or failed kept the hold with no timer and left its snapshot pinned for an owner that never got it. - A snapshot of a turn that waited for another snapshot recorded the trace with that turn's reply, which the box had not got. It is now skipped. A save without a hold no longer ends the hold of an earlier one. Co-Authored-By: Claude <noreply@anthropic.com>
The harness program now runs in a session and a restored box is attached to by session id, which needs `attach_process`. Remove the pin when a release has it. Co-Authored-By: Claude <noreply@anthropic.com>
The fake backend finished its snapshot at once, so the save was never cancelled. Co-Authored-By: Claude <noreply@anthropic.com>
# Conflicts: # verifiers/v1/interception/server.py
The relay took every 404, 502, 503 or 504 that was not JSON or SSE for the error page of a tunnel edge and retried it for five minutes. The interception server answers an unknown path with a plain-text 404, so a harness that asked for one waited that long. The server now marks each of its responses with a header, and only a response without it counts as the edge's. Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit a1b88d8)
# Conflicts: # verifiers/v1/interception/relay/__init__.py # verifiers/v1/rollout.py
A checkpoint is a copy of the host's view of the rollout, and the copy missed things or caught them at the wrong moment: - A reply was stored for replay when it was sent, not when it was committed. One that waited behind a hold was on the trace and could not be sent again, so a restored box got a second sample. A turn now stores its reply inside its commit, and a snapshot records the trace only outside that window. - Tool verdicts and the prepared user messages were not copied. A call that a hook denied ran in the restored box, and request hooks ran twice on one message. - A hold stopped only successful replies. Errors and "rollout stopped" now wait too, and a segment that ends under an owner's hold waits for the release, so a saved rollout does not finish while its owner records it as running. - A save whose harness program ended meanwhile is skipped. Smaller fixes with the same root: - The pace of snapshots is waited for before the box is held. - The interval counts turns from the newest snapshot, so conversations that run side by side cannot step past it. - A restored rollout gets the agent time that is left, not the full timeout again. - A rewind drops the rewrite records of the turns it cuts. - A retry may pick the snapshot its owner offered, and no other snapshot of an owner. Co-Authored-By: Claude <noreply@anthropic.com>
Main's lockfile changed format; the only difference to it is the prime-sandboxes pin. Co-Authored-By: Claude <noreply@anthropic.com>
The harness launch passes the session's program to `Runtime.run`. The fake runtime of this new test on main did not accept it. Co-Authored-By: Claude <noreply@anthropic.com>
Move the prime-sandboxes pin to the attach-process branch merged with prime main (0.5.12), and relock from main's lockfile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main moved the episode agent's override from run() to _run(), which dropped the restore argument, and gave RolloutSession a route. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A running rollout can now be stopped and continued later, in a new sandbox and by another process, without losing work. The harness needs no changes and gets no notice.
save_all), or the rollout snapshots itself at every n-th turn (checkpoint.interval). No snapshot is taken by default.This branch contains #3074 (the relay), which is still open: review the relay there and the rest here. It supersedes #3073 (closed).
Before merge:
prime-sandboxesis pinned to the branch of PrimeIntellect-ai/prime#1147 (attach_process). Replace the pin with a release.How it works
flowchart LR subgraph box["Sandbox (forked on restore)"] H["Harness program"] -->|"127.0.0.1"| R["Relay"] end R -->|"tunnel"| A["Interception server, host A"] R -.->|"after restore"| B["Interception server, host B"]Snapshot
Restore
Runtime.attach).API surface
Config
[env.agent.checkpoint]--no-env.agent.checkpointturns it off.checkpoint.intervalcheckpoint.keep"last"runtime.snapshots_per_minFor the owner of a rollout (
verifiers.v1.restore)save_all(hold=False, skipped=None)release_all()save_all(hold=True), under which a saved rollout waits at its checkpoint. A hold ends by itself after 120 s.RunSlot(task, id=..., restore=trace),Agent.run(task, restore=trace)unsaveable(config)Trace
Trace.checkpointsCheckpointper snapshot:id,snapshot,node,pinned,restored,deleted. A saved trace also carries what a restore needs (Checkpoint.restore).Trace.rewindtiming.agent.checkpointRuntime and harness
Runtime.snapshot,wait_snapshot,restore_config,discardSUPPORTS_MEMORY_SNAPSHOTS).Runtime.run(..., program=),Runtime.attachHarness.SUPPORTS_MEMORY_RESTORERelay(#3074)connect,reroute,installed,state,failure,stop.x-vf-interceptionheaderRobustness on Prime
checkpoint.intervalcounts turns from the newest snapshot, so conversations that run side by side in one box cannot step past it.Checkpoint.pinned) is the owner's to delete.Limits
A rollout takes snapshots only when all of these hold. Otherwise a retry starts over, and
save_allreports why it skipped the rollout:Two more:
Verification
CI on Prime sandboxes, job Checkpoint Restore: 16 passed, 4 skipped. Each test proves a restore with a nonce that exists only in the memory of a process in the box, and checks that no snapshot or sandbox leaks.
test_retry_restores_checkpointtest_another_owner_continues_a_saved_rolloutsave_allsaves a running rollout, the rollout is cancelled, and a second env with its own server, tunnel and secrets continues it from the pickled trace.test_another_owner_continues_running_sub_agentstest_every_saved_turn_continues_to_the_same_resulttest_a_restore_point_that_leads_nowhere_starts_the_rollout_overSkips: pi and openclaw finish their run despite the failed call; mini-swe-agent cannot continue from a memory snapshot; Kimi is rejected by the test endpoint.
Training runs through PrimeIntellect-ai/prime-rl#4456, with traces linked there:
The 3 h fork found a platform behaviour: a box from a snapshot older than the paused box's idle timeout ends as idle about 20 s after it runs. 46 creates were repeated for this. The create retry and the start-over above come from that.
Measured on Prime
🤖 Generated with Claude Code
Note
Restore rollouts from memory snapshots
RolloutCheckpointConfigin agent.py (snapshot typememory/filesystem, turn interval default 10, retention policy), aCheckpointmodel andTrace.rewindin trace.py, and aRestorePointserializer in restore.pyAgent.runin agent.py now retries failed owned-runtime rollouts from the newest eligible checkpoint instead of restarting from scratch; borrowed runtimes are still not retried. Failed restore targets are excluded from later selectionPrimeBackendin prime.py implements memory snapshots by pausing and resuming the sandbox, with an awake event that blocks other remote operations during a snapshot, and background-job polling that retries through pause errorsSUPPORTS_MEMORY_RESTORE,LOADS_SESSIONS) and restore hooks in harness.py; ACP sessions can run as detached socket-backed runners that checkpoint and restore in-flight prompts in runner.py/var/tmp/vf-rlm);RolloutSession.abortis first-wins; agent timing now splits checkpoint hold time out of model time; snapshots left by rewinds, retention, and strays are deleted at cleanup; check forTrace._split_agent_time,Relay._start, andACPHarnessSession._stop_detachedif regressions appearMacroscope summarized 87e35c5. (Automatic summaries will resume when PR exits draft mode or review begins).