Repository navigation
feat(gateway): inject prepared context into Responses requests - #2400
Merged
hidb4ai merged 5 commits intoNov 2, 2026
Merged
Conversation
…ter release Codex could not run through the Responses ingress: - Every input item with a role was rebuilt from role and content, so Codex's additional_tools item lost its tool definitions and developer messages became system messages without their type. - Unmodeled top-level fields (reasoning, include, prompt_cache_key, parallel_tool_calls) were silently dropped although Core admits them. - Streaming requests were forwarded as streaming and withheld by the send check, which only supports openai-chat streaming. Forward every item that is not a plain user message verbatim, pass the unmodeled fields through for Core's request allowlist to decide (dropping only the client-local client_metadata), always request a non-streaming response upstream, and replay a released response to streaming clients as Responses SSE events. Nothing is released before the release check. Refs oceanbase#2398 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jasper-yl
requested review from
AlexStocks,
PsiACE and
frostming
as code owners
November 2, 2026 11:43
jasper-yl
force-pushed
the
fix/gateway-responses-injection
branch
from
November 2, 2026 11:56
aee57de to
10f6a91
Compare
…ed streams The SSE replay for streaming Responses clients emitted only item snapshots, always ended with response.completed, and covered only released responses: - Replay each message content part as content_part.added/done around one buffered output_text (or refusal) delta/done, so delta-loop consumers see the answer; message items are announced empty as upstream streams do. - Choose the terminal event from the response status: failed/cancelled or a non-null error yield response.failed, incomplete yields response.incomplete. - Replay a permitted degraded passthrough 200 response as well, keeping the x-pc-degraded header. - Number events with sequence_number. Refs oceanbase#2398 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Responses route never injected: inject_context was Chat-only, the recall query and the injection target only recognized `text` parts (Responses user input uses `input_text`), and the declared location was always rooted at /messages, which does not exist in a Responses body. Add WireRenderer::message_list_key (`messages` for Chat, `input` for Responses) and build the declared JSON pointer from it, accept `input_text` parts for the query and the injection target, and let PC_GATEWAY_INJECTION enable injection on both routes. PC_GATEWAY_INJECTION_RESPONSES=off keeps the Responses route uninjected for deployments that want that. Refs oceanbase#2398 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jasper-yl
force-pushed
the
fix/gateway-responses-injection
branch
from
November 2, 2026 12:27
10f6a91 to
08ba829
Compare
hidb4ai
reviewed
Nov 2, 2026
hidb4ai
left a comment
Contributor
There was a problem hiding this comment.
One correctness issue remains: enabling Responses injection rejects supported string-content messages before they reach the upstream. Reproduction and the required behavior are inline.
1 task done
Closed
1 task done
Open
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue or RFC does this PR close?
Refs #2398 (item 4: prepared context is never injected into Responses requests).
This PR is stacked on #2399, which keeps Responses user messages in the canonical form and forwards every other item verbatim. Until #2399 merges, this PR also shows its commit; the change here is the last commit,
feat(gateway): inject prepared context into Responses requests.Rationale for this change
The Responses route never injects prepared context. Three things were missing:
inject_contextis set only for the Chat route.text. Responses user input usesinput_text, so even with injection enabled, no query would be extracted./messages/{i}/.... A Responses body has nomessages, so Core would check the wrong bytes.What changes are included in this PR?
WireRenderer::message_list_key()names the top-level key of the message list in the rendered body:messagesby default,inputforOpenAiResponsesRenderer.GatewayRendererdelegates. The executor builds the declared JSON pointer from it, so the pointer matches the body that is checked and sent.is_text_partacceptstextandinput_textparts, both for the recall query and for the injection target. The injection itself is unchanged: the block is prepended to the first text part of the last user message, and its byte range is declared.PC_GATEWAY_INJECTION=onnow enables injection on both routes, so a deployment that turns injection on gets it for Responses clients such as Codex too.PC_GATEWAY_INJECTION_RESPONSES=offkeeps the Responses route uninjected, for operators who want to roll it out separately.gateway/README.mddocuments both variables.Are there any user-facing changes?
Yes, for deployments that set
PC_GATEWAY_INJECTION=on: Responses requests are now injected as well. SetPC_GATEWAY_INJECTION_RESPONSES=offto keep the previous behavior. Deployments without injection are unchanged.How was this change tested?
cargo fmt --all -- --check,cargo clippy --all-targets -- -D warningsandcargo testingateway/, all clean.New tests:
responses_injection_targets_input_text_and_declares_input_pointer(executor, with the realOpenAiResponsesRenderer): the recall query comes frominput_textparts, the declared pointer is/input/1/content/0/text, and the declared byte range of the rendered body that goes to the check is exactly the injected block. Other parts and the preceding developer item are unchanged.responses_injection_follows_the_injection_switch_unless_opted_out(ingress): the switch matrix.Reverting the pointer root or the
input_textrecognition makes the executor test fail.End to end against a master Server in enforcing mode, with an
openai-responsestarget pointing at a local recorder, one remembered fact in the scope, and a Codex 0.145 request (stream: true). With Responses injection off, the upstream body equals the request. With it on, the prepared context is prepended to the task text, every other item is unchanged, and Core'sbefore_sendandbefore_releasechecks both accept the declared location.AI usage statement
Built with Claude Code (Claude Opus 5.5), which wrote the change and the tests and ran the checks above.
🤖 Generated with Claude Code