Skip to content

feat(gateway): inject prepared context into Responses requests - #2400

Merged
hidb4ai merged 5 commits into
oceanbase:masterfrom
jasper-yl:fix/gateway-responses-injection
Nov 2, 2026
Merged

hidb4ai merged 5 commits into
oceanbase:masterfrom
jasper-yl:fix/gateway-responses-injection

Conversation

@jasper-yl

@jasper-yl jasper-yl commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Refs #2398 (item 4: prepared context is never injected into Responses requests).

This PR is stacked on #2399, which keeps Responses user messages in the canonical form and forwards every other item verbatim. Until #2399 merges, this PR also shows its commit; the change here is the last commit, feat(gateway): inject prepared context into Responses requests.

Rationale for this change

The Responses route never injects prepared context. Three things were missing:

  • inject_context is set only for the Chat route.
  • The recall query and the injection target only recognize content parts of type text. Responses user input uses input_text, so even with injection enabled, no query would be extracted.
  • The declared location is always rooted at /messages/{i}/.... A Responses body has no messages, so Core would check the wrong bytes.

What changes are included in this PR?

  • WireRenderer::message_list_key() names the top-level key of the message list in the rendered body: messages by default, input for OpenAiResponsesRenderer. GatewayRenderer delegates. The executor builds the declared JSON pointer from it, so the pointer matches the body that is checked and sent.
  • A shared is_text_part accepts text and input_text parts, both for the recall query and for the injection target. The injection itself is unchanged: the block is prepended to the first text part of the last user message, and its byte range is declared.
  • PC_GATEWAY_INJECTION=on now enables injection on both routes, so a deployment that turns injection on gets it for Responses clients such as Codex too. PC_GATEWAY_INJECTION_RESPONSES=off keeps the Responses route uninjected, for operators who want to roll it out separately.
  • gateway/README.md documents both variables.

Are there any user-facing changes?

Yes, for deployments that set PC_GATEWAY_INJECTION=on: Responses requests are now injected as well. Set PC_GATEWAY_INJECTION_RESPONSES=off to keep the previous behavior. Deployments without injection are unchanged.

How was this change tested?

  • cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings and cargo test in gateway/, all clean.

  • New tests:

    • responses_injection_targets_input_text_and_declares_input_pointer (executor, with the real OpenAiResponsesRenderer): the recall query comes from input_text parts, the declared pointer is /input/1/content/0/text, and the declared byte range of the rendered body that goes to the check is exactly the injected block. Other parts and the preceding developer item are unchanged.
    • responses_injection_follows_the_injection_switch_unless_opted_out (ingress): the switch matrix.

    Reverting the pointer root or the input_text recognition makes the executor test fail.

  • End to end against a master Server in enforcing mode, with an openai-responses target pointing at a local recorder, one remembered fact in the scope, and a Codex 0.145 request (stream: true). With Responses injection off, the upstream body equals the request. With it on, the prepared context is prepended to the task text, every other item is unchanged, and Core's before_send and before_release checks both accept the declared location.

AI usage statement

Built with Claude Code (Claude Opus 5.5), which wrote the change and the tests and ran the checks above.

🤖 Generated with Claude Code

…ter release

Codex could not run through the Responses ingress:

- Every input item with a role was rebuilt from role and content, so Codex's
  additional_tools item lost its tool definitions and developer messages
  became system messages without their type.
- Unmodeled top-level fields (reasoning, include, prompt_cache_key,
  parallel_tool_calls) were silently dropped although Core admits them.
- Streaming requests were forwarded as streaming and withheld by the send
  check, which only supports openai-chat streaming.

Forward every item that is not a plain user message verbatim, pass the
unmodeled fields through for Core's request allowlist to decide (dropping
only the client-local client_metadata), always request a non-streaming
response upstream, and replay a released response to streaming clients as
Responses SSE events. Nothing is released before the release check.

Refs oceanbase#2398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jasper-yl
jasper-yl force-pushed the fix/gateway-responses-injection branch from aee57de to 10f6a91 Compare November 2, 2026 11:56
@jasper-yl jasper-yl changed the title feat(gateway): inject prepared context into Responses requests behind a switch feat(gateway): inject prepared context into Responses requests Nov 2, 2026
jasper-yl and others added 2 commits November 2, 2026 04:26
…ed streams

The SSE replay for streaming Responses clients emitted only item snapshots,
always ended with response.completed, and covered only released responses:

- Replay each message content part as content_part.added/done around one
  buffered output_text (or refusal) delta/done, so delta-loop consumers see
  the answer; message items are announced empty as upstream streams do.
- Choose the terminal event from the response status: failed/cancelled or a
  non-null error yield response.failed, incomplete yields response.incomplete.
- Replay a permitted degraded passthrough 200 response as well, keeping the
  x-pc-degraded header.
- Number events with sequence_number.

Refs oceanbase#2398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Responses route never injected: inject_context was Chat-only, the recall
query and the injection target only recognized `text` parts (Responses user
input uses `input_text`), and the declared location was always rooted at
/messages, which does not exist in a Responses body.

Add WireRenderer::message_list_key (`messages` for Chat, `input` for
Responses) and build the declared JSON pointer from it, accept `input_text`
parts for the query and the injection target, and let PC_GATEWAY_INJECTION
enable injection on both routes. PC_GATEWAY_INJECTION_RESPONSES=off keeps
the Responses route uninjected for deployments that want that.

Refs oceanbase#2398

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jasper-yl
jasper-yl force-pushed the fix/gateway-responses-injection branch from 10f6a91 to 08ba829 Compare November 2, 2026 12:27

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One correctness issue remains: enabling Responses injection rejects supported string-content messages before they reach the upstream. Reproduction and the required behavior are inline.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hidb4ai
hidb4ai merged commit 2b5c11b into oceanbase:master Nov 2, 2026
2 checks passed
@hidb4ai hidb4ai added the 1.5.1 v1.5.1 release tracking label Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.5.1 v1.5.1 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants