Skip to content

Wait for the shell probe in the Network Policy fixture - #3218

Open
xeophon wants to merge 1 commit into
codex/3205-multi-turn-fixturefrom
codex/3205-network-fixture
Open

xeophon wants to merge 1 commit into
codex/3205-multi-turn-fixturefrom
codex/3205-network-fixture

Conversation

@xeophon

@xeophon xeophon commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Ask the model to run the network probe immediately and wait for its output before responding. The task continues to score the report produced inside the sandbox under each network policy.

Local unit suite on this commit reported 299 passed, 19 skipped. Full-stack CI before review fixes

Layer 11 of 13. Builds on #3217. Stack index

Note

Update Network Policy task prompt to wait for shell probe

Changes the task prompt in NetworkPolicyTaskset.load in network_policy.py. The agent is now told to run the probe command once, wait for it to finish, and reply with the printed output. The prohibition list no longer bans configuring the network, but now bans inspecting or changing the network or firewall and running any other command.

Macroscope summarized ab7cdbe.


Note

Low Risk
Test fixture prompt text only; no production or scoring logic changes.

Overview
Updates the Network Policy integration fixture prompt so agents use the shell tool immediately, run the probe once, and wait for completion before answering with the printed output.

The wording no longer tells agents not to configure the network—only not to inspect or change network/firewall settings or run extra commands.

Reviewed by Cursor Bugbot for commit ab7cdbe. Bugbot is set up for automated code reviews on this repo. Configure here.

@macroscopeapp

macroscopeapp Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 7e0ed60

Macroscope's review found this PR approvable — The integration fixture now explicitly tells the agent to wait for the shell probe before replying. Network enforcement, probe behavior, and grading remain unchanged, keeping the change limited to test instructions.

Notes:

  • No code objects were reviewed. Approvability was decided on eligibility alone.

No code changes detected at ab7cdbe. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@xeophon
xeophon force-pushed the codex/3205-multi-turn-fixture branch from 0b124b9 to e62406e Compare November 19, 2026 01:30
@xeophon
xeophon force-pushed the codex/3205-network-fixture branch from 7e0ed60 to ab7cdbe Compare November 19, 2026 01:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant