Repository navigation
fix: Read Codex custom_tool_call_output items as tool results - #1771
Conversation
Codex declares its tools as custom (freeform) tools: one exec tool whose input is a script. Every tool result it sends back is therefore a custom_tool_call_output input item, not function_call_output. The Responses request parser only read function_call_output, so Cortex recorded each Codex tool call but none of its results: a session with 13 tool calls had 0 results, and the timeline showed no command output. Both item types carry the same call_id and output fields, so read them the same way. Codex sends the output as input_text parts, which flattenResponsesContent already joins. The praxis Rust port gets the same change, and the same body gives the same result in both. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Vinod Muthusamy <vinodmut@users.noreply.github.com>
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
Related to #1714, the tracker for moving the laptop data plane to Praxis. Decision D6 there makes Cortex own the Rust parsers, and the parity gate keeps the Go and Rust copies in step until then. This PR changes both copies together:
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com |
clawgenti
left a comment
There was a problem hiding this comment.
Summary
The fix is correct and complete for both parsers. custom_tool_call_output items now go into ToolResults through the same path as function_call_output, so they get the same treatment from flattenResponsesContent, the Messages exclusion, and downstream consumers (timeline tool ends, session archive, CPEX tool_results custom keys). I checked it locally: the new Go test fails on main (ToolResults = []) and passes on this head. I also ran the Rust case's body through Go's ParseRequest on head, and the output is byte-identical to the Rust expectation ({"model":"m","toolResults":[{"toolUseId":"x","content":"a\n\nb"}],"isAction":true}), so parity holds.
I probed edge cases head vs base: string output, null output, and an object output. None of them fails the decode. The PR adds no new typed decoder fields, so the whole-parse failure risk does not apply. Base drops all of them, and head records a result (empty content for the non-text forms).
One behaviour change worth knowing about: CPEX CEL/APL policies on llm: routes that read custom.tool_results (e.g. the secret-in-tool-result pattern) will now fire on Codex traffic. That is the intended effect, but deployments running such policies against Codex will start seeing denials they did not see before. As with function_call_output, a CPEX rewrite of these results fails closed (tool result rewrite on an inference request is not supported).
Only nits and doc follow-ups are below. Nothing blocks the merge.
Author: vinodmut (FIRST_TIME_CONTRIBUTOR association, but repo permission admin, so maintainer)
Areas reviewed: Go (core/parse, core/model, inferenceparser tests), Rust (praxis parity), downstream consumers (cpex, session/timeline, script SDK)
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: passing (Rust praxis job: fmt, clippy, test and e2e all succeeded)
Local verification: go test ./parse/... ./plugins/inferenceparser/... ./plugins/cpex/... ./session/timeline/... ./model/... passes; gofmt -l clean; go vet clean; the new test fails on base and passes on head.
| // - "custom_tool_call_output": the same thing for a custom (freeform) | ||
| // tool, with the same call_id and output fields. Codex declares its | ||
| // tools this way (one "exec" tool taking a script), so every tool result | ||
| // it sends arrives as this type, with the output as input_text parts. |
There was a problem hiding this comment.
nit: The paragraph this list belongs to (line 41, outside the diff) still says "Three shapes are read, the first two confirmed on live Codex traffic". There are now four, and this one is also confirmed on live Codex traffic (your 0.167 capture). Worth updating that line so the header matches the list.
| // function_call_output items of a Responses API request — what each earlier tool | ||
| // call returned, as the client sent it back. Message Content keeps only text blocks, | ||
| // so without this a tool result left nothing behind but its size in ContentBytes. | ||
| // function_call_output and custom_tool_call_output items of a Responses API request — |
There was a problem hiding this comment.
suggestion (follow-up, as noted in the PR body): core/plugins/cpex/cmf_parts.go:186 (the inferenceToCMFParts doc comment) and core/plugins/cpex/README.md:226 still name only function_call_output. The CPEX path now handles custom_tool_call_output automatically through inf.ToolResults, so those two are doc-only. core/plugins/script/sdk/cortex.py (_walk_body / _messages) is a real behaviour gap, though. A script plugin's redaction walk and tool-output detection will still skip Codex tool results. A follow-up issue or PR for that would be good, if #1770 doesn't already track it.
huang195
left a comment
There was a problem hiding this comment.
Summary
The fix is correct in both parsers. I ran TestInferenceParser_ResponsesRequest* at head with GOWORK=off, and they pass. CI is green.
I found no must-fix items. The PR body defers two places to later issues. Both are small, so I suggest that you put them in this PR:
- Script SDK (
core/plugins/script/sdk/cortex.py). The SDK matches onlyfunction_call_outputat two places:- Line 139,
_walk_body. A redaction script does not see Codex tool output, so a secret fromcat .envgoes to the model unchanged. Afunction_call_outputgets redaction. This is the same gap that this PR fixes in the Go parser. - Line 167,
_messages. The item is skipped, not read as user text, so_latest_user_textstays correct. But scripts do not see the tool output. - The fix is
in ("function_call_output", "custom_tool_call_output")at both lines. Add one case next to thefunction_call_outputcase attests/test_script_sdk.py:668.
- Line 139,
- CPEX docs.
core/plugins/cpex/README.md:226and the comment atcore/plugins/cpex/cmf_parts.go:186name onlyfunction_call_output. After this PR, CPEX also getscustom_tool_call_outputresults throughinf.ToolResults.
Author: vinodmut (MEMBER, maintainer)
Areas reviewed: Go (core/parse, core/model, inferenceparser tests), Rust (praxis parity), other readers of the item type (script SDK, CPEX)
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: passing
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
| } | ||
| } | ||
| if raw.Type == "function_call_output" { | ||
| if raw.Type == "function_call_output" || raw.Type == "custom_tool_call_output" { |
There was a problem hiding this comment.
suggestion: The script SDK has the same match at core/plugins/script/sdk/cortex.py:139 (_walk_body, used for redaction) and :167 (_messages). Without the same change there, a redaction script does not see Codex tool output. See the review summary.
Brings in 539 upstream commits. Two adjustments: - core/parse/inference/responses.go: upstream moved the inference parser here from core/plugins/inferenceparser. The custom_tool_call_output fix is re-applied to the moved file, matching rossoctl#1771. - The evolve plugin's registration moves from cmd/cortex to core/plugins/all/plugins_evolve.go, where upstream now registers every bundled plugin. Signed-off-by: Vinod Muthusamy <vinodmut@users.noreply.github.com>
Fixes #1770
What changed
Codex declares its tools as custom (freeform) tools, so every tool result it sends back is a
custom_tool_call_outputinput item. The Responses request parser only readfunction_call_output, so Cortex recorded each Codex tool call but none of its results.responsesInputItem.UnmarshalJSON(core/parse/inference/responses.go) readscustom_tool_call_outputthe same way asfunction_call_output. Both carrycall_idandoutput. Codex sends the output asinput_textparts, whichflattenResponsesContentalready joins.
praxis/filters/src/parse/inference/responses.rs) gets the sameone-line change, per the parity rule.
responses.goand theToolResultsfield comment incore/model/extensions.goname the new item type.The response side already reads Codex's
custom_tool_callitems, so no change is needed there.Not in this PR: the script plugin's Python SDK (
core/plugins/script/sdk/cortex.py) and theCPEX README also name only
function_call_output. Both are listed in the issue.How it was tested
TestInferenceParser_ResponsesRequest_CustomToolResults(
core/plugins/inferenceparser/responses_test.go), with a body trimmed from a captured Codex0.167 request. It fails without the fix and passes with it. The existing
TestInferenceParser_ResponsesRequest_ToolResultsstill passes.go_rules_match_go. Its expected value is the Go parser's output for thesame body. It fails without the fix and passes with it.
core/, withGOWORK=off:gofmt -lis clean, andgo vetpasses for./parse/... ./plugins/inferenceparser/... ./model/....go testpasses for./parse/... ./plugins/... ./model/... ./session/... ./sessionapi/...exceptsession/archive'sTestRecord_NeverBlocksAndCountsDropsWhenFull. That test fails now andthen on unmodified
maintoo: about 1 in 60 runs on bothmainand this branch.praxis/, in arust:1.96container:cargo fmt --checkandcargo clippy -p cortex-praxis-filters --all-targets -- -D warningsare clean.cargo test -p cortex-praxis-filterspasses for theparse::inference::responsestests andthe
paritysuite.main(before the parser moved tocore/parse/inference). Codex 0.167 was routed through a locally built Cortex. Three sessionsrecorded every tool result: 13 of 13, 12 of 12 and 12 of 12, where they recorded 0 before.
Git hook rejections now show in the timeline under the commit they rejected.
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com
Summary by CodeRabbit