Skip to content

Match tool receipts against the prompt the harness replays - #3210

Open
xeophon wants to merge 1 commit into
codex/3205-tool-issuersfrom
codex/3205-mediated-replay
Open

xeophon wants to merge 1 commit into
codex/3205-tool-issuersfrom
codex/3205-mediated-replay

Conversation

@xeophon

@xeophon xeophon commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Removing a provider tool under a restricted network policy also inserts a notice into the model’s prompt. The harness replays its original prompt. Register each sampled call against that history and carry forward the tool results the interceptor already substituted. A loopback HTTP reproduction with two successive tool calls verified that replaying the older branch delivers both of its original substitutions.

Local unit suite on this commit reported 299 passed, 19 skipped. Full-stack CI

Layer 3 of 13. Builds on #3209. Stack index

Note

Match tool receipts against the prompt the harness replays

  • InterceptionServer now captures the incoming request messages as harness history and passes them to gate_tool_calls after the response commits. When request rewriting changes the request, rewritten ToolMessage entries are substituted into that history so gating sees the prompt actually replayed.
  • RolloutSession.gate_tool_calls accepts an optional harness_history argument and registers the current node as issuer for each call under both the trace-branch history and, when distinct, the supplied harness history. This makes issuer keys match either transcript.
  • Removes the side effect in the tool-issuer lookup method that added a sampled issuer to tool_issuers while deriving a call's key.
  • Reworks the replay test in test_agent.py to derive native and policy-filtered requests through AnthropicDialect and NetworkPolicyConfig, and to cover repeated call IDs, changed tool catalogs, and ambiguous issuing turns.

Macroscope summarized cc9d178.


Note

Medium Risk
Changes core tool-call gating and pinned-result matching on the model interception path; incorrect prefix hashing could mis-attribute issuers or lose rewritten tool results.

Overview
Tool-call issuer tracking now keys off the conversation the harness will replay, not only the trace branch. The interception server keeps harness history after rewrite_request—original messages with pinned ToolMessage substitutions merged in—and passes that transcript into gate_tool_calls.

RolloutSession.gate_tool_calls takes an optional harness_history and registers sampled calls under prefix hashes for both the graph branch and the harness history when they differ (e.g. network policy notices or filtered Anthropic prompts). Issuer registration during tool_result_keys path scanning was removed so mapping happens at gate time instead.

Unit coverage in test_agent.py exercises provider-tool filtering, gating with native vs model-facing messages, pin_tool_results on model receipts, and follow-up tool calls across replays.

Reviewed by Cursor Bugbot for commit cc9d178. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread verifiers/v1/session.py Resolved
@macroscopeapp

macroscopeapp Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at cc9d178

Macroscope's review found this PR approvable — Tool receipts are matched against both the harness-replayed prompt and the model-facing transcript, preserving policy-substituted results across replays. The change remains session-local and retains existing tool-gating and ambiguity safeguards.

You can add or adjust custom eligibility rules. Learn more.

@xeophon
xeophon force-pushed the codex/3205-mediated-replay branch from 97a1b67 to cc9d178 Compare November 19, 2026 01:30
@xeophon
xeophon force-pushed the codex/3205-tool-issuers branch from cdf4ed1 to 23e57cb Compare November 19, 2026 01:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant