Skip to content

fix: Read Codex custom_tool_call_output items as tool results - #1771

Merged
huang195 merged 1 commit into
rossoctl:mainfrom
vinodmut:fix/codex-custom-tool-results
Nov 10, 2026
Merged

huang195 merged 1 commit into
rossoctl:mainfrom
vinodmut:fix/codex-custom-tool-results

Conversation

@vinodmut

@vinodmut vinodmut commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1770

What changed

Codex declares its tools as custom (freeform) tools, so every tool result it sends back is a
custom_tool_call_output input item. The Responses request parser only read
function_call_output, so Cortex recorded each Codex tool call but none of its results.

  • Fix: responsesInputItem.UnmarshalJSON (core/parse/inference/responses.go) reads
    custom_tool_call_output the same way as function_call_output. Both carry call_id and
    output. Codex sends the output as input_text parts, which flattenResponsesContent
    already joins.
  • Praxis: the Rust port (praxis/filters/src/parse/inference/responses.rs) gets the same
    one-line change, per the parity rule.
  • Docs: the doc comment in responses.go and the ToolResults field comment in
    core/model/extensions.go name the new item type.

The response side already reads Codex's custom_tool_call items, so no change is needed there.

Not in this PR: the script plugin's Python SDK (core/plugins/script/sdk/cortex.py) and the
CPEX README also name only function_call_output. Both are listed in the issue.

How it was tested

  • New Go test: TestInferenceParser_ResponsesRequest_CustomToolResults
    (core/plugins/inferenceparser/responses_test.go), with a body trimmed from a captured Codex
    0.167 request. It fails without the fix and passes with it. The existing
    TestInferenceParser_ResponsesRequest_ToolResults still passes.
  • New Rust case in go_rules_match_go. Its expected value is the Go parser's output for the
    same body. It fails without the fix and passes with it.
  • Go: from core/, with GOWORK=off: gofmt -l is clean, and go vet passes for
    ./parse/... ./plugins/inferenceparser/... ./model/.... go test passes for
    ./parse/... ./plugins/... ./model/... ./session/... ./sessionapi/... except
    session/archive's TestRecord_NeverBlocksAndCountsDropsWhenFull. That test fails now and
    then on unmodified main too: about 1 in 60 runs on both main and this branch.
  • Rust: from praxis/, in a rust:1.96 container: cargo fmt --check and
    cargo clippy -p cortex-praxis-filters --all-targets -- -D warnings are clean.
    cargo test -p cortex-praxis-filters passes for the parse::inference::responses tests and
    the parity suite.
  • Live: the same fix on a branch based on an earlier main (before the parser moved to
    core/parse/inference). Codex 0.167 was routed through a locally built Cortex. Three sessions
    recorded every tool result: 13 of 13, 12 of 12 and 12 of 12, where they recorded 0 before.
    Git hook rejections now show in the timeline under the commit they rejected.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

Summary by CodeRabbit

  • Bug Fixes
    • Responses API custom tool outputs are now recognized as tool results, with their text content associated with the corresponding call.

Codex declares its tools as custom (freeform) tools: one exec tool whose
input is a script. Every tool result it sends back is therefore a
custom_tool_call_output input item, not function_call_output. The
Responses request parser only read function_call_output, so Cortex
recorded each Codex tool call but none of its results: a session with
13 tool calls had 0 results, and the timeline showed no command output.

Both item types carry the same call_id and output fields, so read them
the same way. Codex sends the output as input_text parts, which
flattenResponsesContent already joins. The praxis Rust port gets the same
change, and the same body gives the same result in both.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Vinod Muthusamy <vinodmut@users.noreply.github.com>
@vinodmut
vinodmut requested a review from a team as a code owner November 10, 2026 16:30
@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Go and Praxis Responses parsers now capture custom_tool_call_output items as tool results. Both retain the item’s call_id and flatten its output. Tests cover multiple input_text parts.

Changes: Responses tool-result parsing

Layer / File(s) Summary
Recognize custom tool outputs
core/parse/inference/responses.go, core/model/extensions.go, core/plugins/inferenceparser/responses_test.go, praxis/filters/src/parse/inference/responses.rs
The Go and Praxis parsers capture custom_tool_call_output as a tool result, using its call_id and flattened output. Documentation describes the custom output shape. Tests check joined text parts and call ID association.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related issues

  • Issue 1714 tracks gaps in the Rust inference parser, including tool parsing. This change adds custom tool-result parsing to its Responses parser.
  • Issue 1740 proposes parser-level tool-result counts. This change makes custom tool outputs available as tool results but does not add counts.

Possibly related PRs

  • rossoctl/cortex#1658: This PR adds custom tool outputs to ToolResults, which can then be rendered by the detail view.
  • rossoctl/cortex#1600: This PR records custom tool results with their call_id, providing result data for timeline tool spans.
  • rossoctl/cortex#1741: This PR extends the Praxis Responses parser introduced by that PR.

Suggested reviewers: huang195, clawgenti

Merge Risk: Minimal · up to 21fbb

No merge-blocking risk is established by the supplied review evidence.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly summarizes the main change: reading Codex custom_tool_call_output items as tool results.
Linked Issues check Passed Issue #1770 requires the Go and Praxis Responses parsers to record custom_tool_call_output results. The Go parser now handles this item alongside function_call_output and uses the existing output …
Out of Scope Changes check Passed The parser changes, regression tests, and documentation updates all support issue #1770. The PR does not change the Python SDK or CPEX README, which the issue identifies as possible separate work.
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Full details: Linked Issues check

Explanation

Issue #1770 requires the Go and Praxis Responses parsers to record custom_tool_call_output results. The Go parser now handles this item alongside function_call_output and uses the existing output flattener. Its new test checks the result ID, flattened text, and message handling. The Praxis parser has the equivalent change and a parity test case. The updated comments describe the new item type.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 2 functions across 4 files.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3076f367-8254-48eb-8311-793430654c2a
Commits

Reviewing files that changed from the base of the PR and between 28ab6a7 and 21fbb2c.

Files selected for processing: 4

  • core/model/extensions.go
  • core/parse/inference/responses.go
  • core/plugins/inferenceparser/responses_test.go
  • praxis/filters/src/parse/inference/responses.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@vinodmut

Copy link
Copy Markdown
Contributor Author

Related to #1714, the tracker for moving the laptop data plane to Praxis.

Decision D6 there makes Cortex own the Rust parsers, and the parity gate keeps the Go and Rust copies in step until then. This PR changes both copies together: core/parse/inference/responses.go and praxis/filters/src/parse/inference/responses.rs. So the Rust port stays at parity with Go, and Codex tool results are recorded in praxis mode as well as in the Go pipeline.

  • What it pins: the new Rust case in go_rules_match_go uses the Go parser's output for the same body as its expected value, following the rule in praxis/README.md. The parity suite passes.
  • Gap in the parity suite: core/plugins/inferenceparser/testdata/ has no Codex case, only claude-code/ and synthetic/. Nothing there sends custom_tool_call or custom_tool_call_output, so the suite couldn't have caught this. A captured Codex case with a tool round-trip would cover it. I can add one in a follow-up if that's useful.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

@clawgenti clawgenti left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The fix is correct and complete for both parsers. custom_tool_call_output items now go into ToolResults through the same path as function_call_output, so they get the same treatment from flattenResponsesContent, the Messages exclusion, and downstream consumers (timeline tool ends, session archive, CPEX tool_results custom keys). I checked it locally: the new Go test fails on main (ToolResults = []) and passes on this head. I also ran the Rust case's body through Go's ParseRequest on head, and the output is byte-identical to the Rust expectation ({"model":"m","toolResults":[{"toolUseId":"x","content":"a\n\nb"}],"isAction":true}), so parity holds.

I probed edge cases head vs base: string output, null output, and an object output. None of them fails the decode. The PR adds no new typed decoder fields, so the whole-parse failure risk does not apply. Base drops all of them, and head records a result (empty content for the non-text forms).

One behaviour change worth knowing about: CPEX CEL/APL policies on llm: routes that read custom.tool_results (e.g. the secret-in-tool-result pattern) will now fire on Codex traffic. That is the intended effect, but deployments running such policies against Codex will start seeing denials they did not see before. As with function_call_output, a CPEX rewrite of these results fails closed (tool result rewrite on an inference request is not supported).

Only nits and doc follow-ups are below. Nothing blocks the merge.

Author: vinodmut (FIRST_TIME_CONTRIBUTOR association, but repo permission admin, so maintainer)
Areas reviewed: Go (core/parse, core/model, inferenceparser tests), Rust (praxis parity), downstream consumers (cpex, session/timeline, script SDK)
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: passing (Rust praxis job: fmt, clippy, test and e2e all succeeded)
Local verification: go test ./parse/... ./plugins/inferenceparser/... ./plugins/cpex/... ./session/timeline/... ./model/... passes; gofmt -l clean; go vet clean; the new test fails on base and passes on head.

// - "custom_tool_call_output": the same thing for a custom (freeform)
// tool, with the same call_id and output fields. Codex declares its
// tools this way (one "exec" tool taking a script), so every tool result
// it sends arrives as this type, with the output as input_text parts.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The paragraph this list belongs to (line 41, outside the diff) still says "Three shapes are read, the first two confirmed on live Codex traffic". There are now four, and this one is also confirmed on live Codex traffic (your 0.167 capture). Worth updating that line so the header matches the list.

Comment thread core/model/extensions.go
// function_call_output items of a Responses API request — what each earlier tool
// call returned, as the client sent it back. Message Content keeps only text blocks,
// so without this a tool result left nothing behind but its size in ContentBytes.
// function_call_output and custom_tool_call_output items of a Responses API request —

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (follow-up, as noted in the PR body): core/plugins/cpex/cmf_parts.go:186 (the inferenceToCMFParts doc comment) and core/plugins/cpex/README.md:226 still name only function_call_output. The CPEX path now handles custom_tool_call_output automatically through inf.ToolResults, so those two are doc-only. core/plugins/script/sdk/cortex.py (_walk_body / _messages) is a real behaviour gap, though. A script plugin's redaction walk and tool-output detection will still skip Codex tool results. A follow-up issue or PR for that would be good, if #1770 doesn't already track it.

@huang195 huang195 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The fix is correct in both parsers. I ran TestInferenceParser_ResponsesRequest* at head with GOWORK=off, and they pass. CI is green.

I found no must-fix items. The PR body defers two places to later issues. Both are small, so I suggest that you put them in this PR:

  1. Script SDK (core/plugins/script/sdk/cortex.py). The SDK matches only function_call_output at two places:
    • Line 139, _walk_body. A redaction script does not see Codex tool output, so a secret from cat .env goes to the model unchanged. A function_call_output gets redaction. This is the same gap that this PR fixes in the Go parser.
    • Line 167, _messages. The item is skipped, not read as user text, so _latest_user_text stays correct. But scripts do not see the tool output.
    • The fix is in ("function_call_output", "custom_tool_call_output") at both lines. Add one case next to the function_call_output case at tests/test_script_sdk.py:668.
  2. CPEX docs. core/plugins/cpex/README.md:226 and the comment at core/plugins/cpex/cmf_parts.go:186 name only function_call_output. After this PR, CPEX also gets custom_tool_call_output results through inf.ToolResults.

Author: vinodmut (MEMBER, maintainer)
Areas reviewed: Go (core/parse, core/model, inferenceparser tests), Rust (praxis parity), other readers of the item type (script SDK, CPEX)
Agent/IDE config (.claude/.vscode): none
Commits: 1 commit, all signed-off: yes
CI status: passing

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

}
}
if raw.Type == "function_call_output" {
if raw.Type == "function_call_output" || raw.Type == "custom_tool_call_output" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: The script SDK has the same match at core/plugins/script/sdk/cortex.py:139 (_walk_body, used for redaction) and :167 (_messages). Without the same change there, a redaction script does not see Codex tool output. See the review summary.

vinodmut added a commit to vinodmut/cortex that referenced this pull request Nov 10, 2026
Brings in 539 upstream commits. Two adjustments:

- core/parse/inference/responses.go: upstream moved the inference parser
  here from core/plugins/inferenceparser. The custom_tool_call_output fix
  is re-applied to the moved file, matching rossoctl#1771.
- The evolve plugin's registration moves from cmd/cortex to
  core/plugins/all/plugins_evolve.go, where upstream now registers every
  bundled plugin.

Signed-off-by: Vinod Muthusamy <vinodmut@users.noreply.github.com>
@huang195
huang195 merged commit 2153761 into rossoctl:main Nov 10, 2026
39 checks passed
@github-project-automation github-project-automation Bot moved this from New/ToDo to Done in Rossoctl Issue Prioritization Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Codex tool results are not recorded: the Responses parser ignores custom_tool_call_output

3 participants