Repository navigation
feat(local): a supervisor can terminate the sandboxes of a run it killed - #555
Merged
earakely-scale merged 5 commits intoNov 9, 2026
Merged
Conversation
A run killed outright tears nothing down. A local sandbox's work folder is made before anything it runs, under the root AGENT_ENV_LOCAL_SANDBOX_DIR names, and its teardown rebuilt from that folder reaches its containers, networks, compose stack and processes. So a supervisor that gives a run a root of its own can terminate each sandbox found there afterwards: local_sandbox_ids() lists them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ervisor's cleanup get_sandbox reads a port map a killed run left empty as none, and local_sandbox_ids() lists only folders named as sandboxes' are. The test also checks a container-mode sandbox's container, which is named, not labeled, is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # src/agent_env/providers/sandbox_providers/local_docker/provider.py
…ts 143 The launcher runs under set -e, and its TERM trap killed the pid in qemu.pid, which isn't written until QEMU starts. A stop in between failed that kill and exited 1. test_the_launcher_forwards_each_host_port_from_loopback_here_to_the_host_and_seeds_the_trust_files stops the launcher just after it writes the seed, so on a slow runner it hit that window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale
deleted the
edgararakelyan/fd-5375-local-sandbox-cleanup
branch
November 9, 2026 17:12
Comment on lines
+220
to
+222
| deadline = time.monotonic() + 30 | ||
| while not ready() and time.monotonic() < deadline: | ||
| time.sleep(0.05) |
There was a problem hiding this comment.
The new readiness loop uses inline 30 and 0.05 values. The repository requires descriptive names for magic numbers. Name the readiness timeout and polling interval so their purpose and units are clear. This repository requirement must be satisfied before merging.
Rule Used: Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. (source)
Learned From
scaleapi/scaleapi#126388
Prompt To Fix With AI
This is a comment left during a code review.
Path: tst/integration/cli/run_bundle_test.py
Line: 220-222
Comment:
**Readiness timings lack names**
The new readiness loop uses inline `30` and `0.05` values. The repository requires descriptive names for magic numbers. Name the readiness timeout and polling interval so their purpose and units are clear. This repository requirement must be satisfied before merging.
**Rule Used:** Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. ([source](https://app.greptile.com/-/switch-tenant?tenant=scale-ai&next=%2Fcustom-context%3Fmemory%3D002e0051-41ad-46c1-9098-47433c580150))
**Learned From**
[scaleapi/scaleapi#126388](https://github.com/scaleapi/scaleapi/pull/126388)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of FD-5375. Answers #546.
A process running
run_bundlethat is killed outright, with no chance to handle a signal, tears none of its sandboxes down. An outside supervisor, such as REA's measurement helper in #546, needs a supported way to clean up exactly that run's sandboxes afterwards. This documents and tests one for local Docker, using what the local provider already does. The one addition is a public function that lists the sandboxes under a root.How a supervisor cleans up
AGENT_ENV_LOCAL_SANDBOX_DIR.local_sandbox_ids()lists and remove its folder:This works because a local sandbox's work folder,
<root>/agent-env-<id>-<random>, is made before anything the sandbox runs. A teardown rebuilt from that folder (get_sandbox) reaches everything the sandbox started:agentenv.sandboxlabel and its container's name;docker compose downin the folder;That answers the semantics #546 asks about:
For remote providers, a per-run value in
config.toml's[sandbox.attribution]does the same job: every remote provider records it on each sandbox (a Modal tag, E2B metadata).What changes
local_sandbox_ids(), inproviders/sandbox_providers/local_docker/provider.py, lists the ids of the sandboxes with a folder under the root, so a supervisor needn't parse folder names. It lists only folders namedagent-env-<id>-<suffix>, so a stray folder there can't stop the cleanup.get_sandboxreads a port map that is missing or cut short as none. A run killed while writing that file is then still rebuilt and terminated, instead of raising.run_bundle's docstring says what to do when a run is killed outright..env.exampledocumentsAGENT_ENV_LOCAL_SANDBOX_DIR, which only a docstring mentioned before.Also, a fix for a flaky test from #549:
test_disk_recipe.py::test_the_launcher_forwards_each_host_port_from_loopback_here_to_the_host_and_seeds_the_trust_filesfailed this PR's CodeBuild run twice, and PR #560's once, withassert 1 == 143. The cause is in the disk launcher. It runs underset -e, and its TERM trap kills the pid inqemu.pid, which isn't written until QEMU starts. A stop in between fails thatkill, so the launcher exits 1 instead of 143. The test stops the launcher just after it writes the seed, so a slow runner hits that window. The trap now ignores a failedkill, as the EXIT trap beside it already does.test_a_launcher_stopped_before_its_computer_starts_exits_as_one_stopped_after_doesstops the launcher while it decompresses its disk, before QEMU starts. It failed on the old launcher with the same1 == 143, and with the fix, the whole launcher test file passed 6 runs out of 6 with every CPU of a 16-core box busy. Only the launcher image's tag changes; no composed disk's key does.No teardown command or new teardown API, and sandbox labeling is unchanged. I first added the sandbox label to a container-mode sandbox's container, to cover a kill between its
docker runand the folder recording its name. A check onmainshowed it wasn't needed: a teardown rebuilt from the folder removes the containeragent-<id>by name anyway. It removed the container with its record deleted and no label on it.Tests
test_find_work_dir_only_returns_matching_sandbox_idalso checks thatlocal_sandbox_ids()lists the sandboxes under the root, and skips a stray file and a strayagent-env-notesdirectory.test_a_sandbox_killed_while_writing_its_port_map_is_still_rebuilt_for_teardownrebuilds a sandbox from a folder whose.port-map.jsonis empty. Full unit suite on an EC2 box: 14,072 passed. Ruff is clean.run_bundle_test.pyruns a task with a VM-mode sandbox, a container-mode sandbox (nginx) and a long command, and SIGKILLsagent-env runonce the command is running. It then terminates whatlocal_sandbox_ids()finds, and asserts that no folder, process, labeled container or container namedagent-<id>is left._signalledgains an optionalreadycheck, so the kill waits for the command. The whole file passes (10), and the kill tests passed three runs in a row.agent-env runat ten points. Each case was killed either alone or with its process group:Not exercised: a kill in the middle of the run's own teardown. Here the teardown had finished before the SIGKILL landed.
🤖 Generated with Claude Code
The existing requirement to name the readiness timings must be satisfied before merging; no new blocking issue was found.
Fix with agent prompt
Summary
Adds a supported way for supervisors to find local sandboxes after killing a run.
The launcher also keeps its intended exit code when stopped before its PID file exists.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Give each run its own sandbox root] --> B[Run creates sandbox work folders] B --> C[Supervisor kills the run] C --> D[local_sandbox_ids lists IDs] D --> E[Terminate each sandbox] E --> F[Remove its work folder]Reviews (12) · Last reviewed commit: "fix(computer): a disk launcher stopped b..." · Reviewed by Greptile