Repository navigation
fix(episode): mint an episode's run id in its task's namespace - #572
Merged
Merged
Conversation
An episode's run id was a bare uuid4 hex, while `agent-env run` and the local runner mint theirs with new_instance_id(task_id), which keeps an @Local task's namespace. A step that names what it writes after the run, such as run_container_unit_tests_verifier's stdout and stderr artifacts, got a bare id in an episode of an @Local task, and the local-run write fence refused it. A container grader writing its reward file could grade the task under `agent-env run` but not as an OpenEnv episode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An episode's run id was a bare
uuid4().hex.agent-env runand the local runner mint run ids withnew_instance_id(task_id)instead, which keeps an@localtask's namespace.A step that names what it writes after the run therefore got a bare id in an episode of an
@localtask, and the local-run write fence refused it.run_container_unit_tests_verifieris one such step: it names its stdout and stderr FileArtifacts after the run. So a container grader that writes a reward file (reward_path, Harbor's/logs/verifier/reward.txt) failed withLocalRunWriteErrorin an OpenEnv episode, while the same task graded fine underagent-env run. The workaround wasverify_sandboxwithbash_cmd_succeeds, which turns a fractional reward into pass/fail.Episodes now mint their id with
new_instance_id, asagent-env rundoes. The verifier's fractional score then reaches the OpenEnv reward unchanged (reward_ofpasses it through).Tests
tst/unit/task/local_run_write_fence_test.py::test_an_local_episodes_verifier_outputs_are_named_after_its_runruns the real verifier inside an episode of an@localtask. It runs against configured stores that fail the test on any write, and allows loopback only, for the in-process proxy. It fails before the fix withLocalRunWriteErrorand passes after.make unit-testpassed apart from four tests that also fail onmainon that box: two needunzip, and two are MCP CLI codegen tests whose generated scripts fail an import there. 14,133 passed, 9 skipped.ruff checkis clean.🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; no actionable issues were found.Summary
Episodes now use
new_instance_idto keep their task’s namespace.Reviews (1) · Last reviewed commit: "fix(episode): mint an episode's run id i..." · Reviewed by Greptile