Skip to content

Use required shell arguments in the Multi-Turn fixture - #3217

Open
xeophon wants to merge 1 commit into
codex/3205-stirrup-acpfrom
codex/3205-multi-turn-fixture
Open

xeophon wants to merge 1 commit into
codex/3205-stirrup-acpfrom
codex/3205-multi-turn-fixture

Conversation

@xeophon

@xeophon xeophon commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Ask the model to supply only the arguments required to run the shell command. Claude Code can convert an optional quoted Bash timeout into a number during replay, which genuinely changes the tool call. The fixture still checks the file contents, reward, and branch count.

Local unit suite on this commit reported 299 passed, 19 skipped. Full-stack CI before review fixes

Layer 10 of 13. Builds on #3216. Stack index

Note

Require shell arguments in the Multi-Turn fixture task prompt

Updates the task prompt in EchoAgenticTaskset.load in echo_agentic.py to instruct the agent to use only the arguments required to run the command, alongside the existing write-to-file and finish steps.

Macroscope summarized e62406e.


Note

Low Risk
Test-only prompt wording in an integration fixture; no production logic or reward behavior changes.

Overview
The echo-agentic integration fixture now tells the agent to use only the arguments required to run the shell command when writing the target file, in addition to the existing write-and-finish instructions.

This tightens the Multi-Turn fixture prompt so replay does not diverge when runtimes or clients normalize optional bash parameters (e.g. quoted timeouts) into different tool calls. Scoring and artifacts are unchanged—the reward still checks file contents via wrote_phrase.

Reviewed by Cursor Bugbot for commit e62406e. Bugbot is set up for automated code reviews on this repo. Configure here.

@macroscopeapp

macroscopeapp Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 0b124b9

Macroscope's review found this PR approvable — This updates an integration-test prompt to request only required shell arguments. File-content rewards and multi-turn trace checks remain unchanged, with no production behavior changes.

Notes:

  • No code objects were reviewed. Approvability was decided on eligibility alone.

No code changes detected at e62406e. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@xeophon
xeophon force-pushed the codex/3205-multi-turn-fixture branch from 0b124b9 to e62406e Compare November 19, 2026 01:30
@xeophon
xeophon force-pushed the codex/3205-stirrup-acp branch from 80c1fbc to d9f718a Compare November 19, 2026 01:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant