Skip to content

fix: preserve Responses message output text - #269

Open
Chen Yufeiyang (feiiiiii5) wants to merge 2 commits into
braintrustdata:mainfrom
feiiiiii5:codex/fix-responses-message-text
Open

Chen Yufeiyang (feiiiiii5) wants to merge 2 commits into
braintrustdata:mainfrom
feiiiiii5:codex/fix-responses-message-text

Conversation

@feiiiiii5

@feiiiiii5 Chen Yufeiyang (feiiiiii5) commented Nov 6, 2026 •

Copy link
Copy Markdown

The Responses API returns ordinary text under output[].message.content[].output_text. Both adapters only inspect top-level output_text/text items, so standard message output becomes choices[0].message.content = None/null, including responses containing both text and function calls.

Read message content blocks and append their text in output order, matching the SDK's output_text behavior. The existing top-level text compatibility and tool conversion branches remain unchanged. Python sync and async clients share the converter; the JavaScript path receives the same fix.

Validation with frozen dependency locks, Python 3.12.8 / OpenAI 2.41.0 and Node 26.13.0 / pnpm 10.33.0:

  • Base: Python 4 failed / 6 passing controls; JS 2 failed / 17 passing controls, reproducing lost text through real SDKs with local HTTP mocks.
  • Fixed: Python conversion module 10 passed; full offline Python selection 84 passed / 16 live cases deselected.
  • Fixed: JS oai.test.ts 19 passed; full offline JS selection 75 passed / 8 live cases skipped.
  • The Python conversion module also passes all 10 cases with OPENAI_BASE_URL=https://gateway.invalid/v1; the test clients explicitly use the mocked endpoint. Before that fixture correction, all ten cases failed on unmatched gateway requests.
  • pnpm build and the configured pre-commit hooks on all four changed files pass. git diff --check passes; dependency locks are unchanged.
  • Standalone tsc --noEmit reports TS2345 in the unchanged js/render-messages.test.ts:227. Substituting the exact base TypeScript files through an in-memory compiler host produces the same single diagnostic.

The regressions cover multiple messages/blocks, mixed function calls, tool-only output and both legacy top-level formats. Python uses OpenAI/AsyncOpenAI with mocked HTTP; JS uses the SDK with MSW. No live model calls were made. The modern Python SDK emits four serialization warnings for the intentionally retained legacy top-level format controls.

Test commands and scope
  • pytest -q py/autoevals/test_responses_conversion.py
  • pnpm exec vitest run js/oai.test.ts
  • pnpm build
  • pre-commit run --files py/autoevals/oai.py py/autoevals/test_responses_conversion.py js/oai.ts js/oai.test.ts

The full offline selections omit only the identified live-provider cases: 16 Python parameter instances and 8 JS cases. Mocked RAGAS tests remain selected. Tool-call regressions use the function-call format.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T16:28:39.057440Z 55ea14f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55ea14fdd8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

}
)
client_class = AsyncOpenAI if is_async else OpenAI
client = client_class(api_key="test-api-key", max_retries=0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pin the Python test clients to the mocked base URL

When OPENAI_BASE_URL points to a custom endpoint, both SDK constructors inherit that endpoint, but the Respx route above only matches https://api.openai.com/v1/responses. This affects the pytest environment configured in .github/workflows/python.yaml:35-39 whenever its endpoint secret selects a gateway: all ten parametrized cases fail on unmatched requests before exercising conversion. Pass base_url="https://api.openai.com/v1" to client_class so these mocked tests remain independent of the supported custom-endpoint configuration.

AGENTS.md reference: AGENTS.md:L100-L100

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant