Skip to content

fix(relay): report an untyped SSE error envelope as response.failed (re-land #8299) - #8415

Draft
lidge-jun wants to merge 11 commits into
devfrom
reland/pr-8299
Draft

lidge-jun wants to merge 11 commits into
devfrom
reland/pr-8299

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

Re-land of #8299 (reverted in #8402) with the fixes its post-merge review required.

Some OpenAI-compatible gateways end a stream with an SSE frame that is only {"error":{...}} (no root type), then [DONE] and EOF. Before this change both bare-error recognizers in src/server/relay.ts ignored it, so EOF synthesized response.incomplete / adapter_eof and the diagnostic was lost. After it, the untyped envelope is reported as response.failed across the tee, eager and WebSocket paths, and the request row and account inspector record the same verdict.

What changed relative to the reverted #8299:

  • Selected-credential masking on the untyped policy terminal. An untyped cyber_policy envelope is rewritten into a terminal like the typed error event. The generated terminal now goes through the selected-credential mask on every path, including the WebSocket pump.

  • One precedence rule for bare evidence (src/server/bare-upstream-error.ts): the newest typed error frame wins, an untyped envelope replaces only untyped evidence, and a genuine response.* terminal supersedes both. The client tail, the account inspector, the request row and the WS row all read the same reduced verdict, so a typed quota followed by an untyped unknown no longer reaches the client as a retryable 502 while the row records 429.

  • Status attribution: an untyped rate limit records 429 instead of 502, and a read failure after a captured refusal keeps the refusal verdict for accounting.

  • Explicit codes outrank copy: policy wording no longer overrides an explicit non-policy code.

  • Provider-stated status kept separate from the synthetic 502. The verdict carries httpStatus only when the provider stated one (explicit refusal, recognized class, Codex WS wrapper 5xx), plus a terminalStatus for the wire. A typed error with no recognized class therefore leaves the row open as dev does, so combo preflight still reads explicit status/http_status/numeric code fields, and an ordinary error followed by a socket reset still reports upstream_reset.

  • Policy frames decide only when they win. A per-stream terminal tracker applies the same reducer before the untyped-policy branch, and the client boundary, the inspector, passthrough inspection and the buffered JSON path all use it. An untyped `cyber_policy` envelope that loses to earlier typed evidence is no longer rewritten into a terminal. A winning policy terminal is final and carries the policy verdict, so a typed frame with a recognized code beside policy copy records 400/`cyber_policy` in the row as well as on the client, as dev's row did.

  • Eager reset attribution matches tee and dev: the eager relay marks a read-failure tail as an upstream verdict only for a refusal or provider-stated class, so an unrecognized error followed by a reset keeps the `streamAborted` transport attribution.

  • A bare frame after a genuine terminal leaves the row alone. Readers that keep inspecting past the terminal (the WebSocket EOF observer and the response-log metadata consumer) no longer let a later bare error restart the verdict. Before this, a `max_output_tokens` or completed turn could record 429 and reach account health as a quota failure. The frame's text keeps dev's guard: untyped text that echoes an outbound credential is recorded as `[REDACTED]`.

In buffered non-stream mode, a winning untyped policy frame after a genuine terminal yields `missing_terminal`. That is a deliberate choice: dev already gives `missing_terminal` for the same typed sequence.

Behavior note for reviewers: an untyped cyber_policy envelope ends the turn immediately and drops a later response.completed. That matches the typed error + cyber_policy behavior already on dev (checked with the same two-frame stream against aa30ea9d: typed → response.failed, no completion). It is parity with an existing rule, not a new one.

Combo preflight is unchanged: it treats the untyped frame as a non-control event and commits the target, so it never resends it; the relay reports the failure. A test pins that boundary next to dev's newer bare-error cases.

Closes #8216

Verification

The head merges current `dev` (`aa30ea9d`) into the re-land chain. Both sides were kept in all three conflicts: two docs/test hunks and the WS test import block. Bun 1.4.4 / macOS, with isolated `HOME`/`OPENCODEX_HOME`/`CODEX_HOME`/`CLAUDE_CONFIG_DIR`.

  • `bun test --isolate` over 52 files: the 12 relay/WS/combo/refusal/reset suites, response-log inspection and request-log suites, the policy-fidelity, stream-aborted, attribution and diagnostics suites, every test that references `buffered-sse-json`, `passthrough-dispatch`, `terminalStatusFromParsed` or `cyber_policy`, and the layout, ratchet, lab-boundary, import-cycle and structure guards. Result: 3,617 pass / 0 fail across 60 files.
  • New regressions in `tests/responses/sse-failed-tail.test.ts`: "a policy frame decides client, account and row together" (5 orderings × tee/eager) and "a reset after … keeps tee and eager attribution aligned" (3 cases). With the source changes reverted, 5 of the 8 fail; with them applied, all pass. Another case, "a bare frame after a genuine terminal leaves the row alone" (5 tests), fails 4 of 5 without its fix. Its redaction test fails if the post-terminal guard is removed.
  • `bun run typecheck` (2,434 files), `bun run privacy:scan`, `bun run structure:check`, `bun scripts/file-size-ratchet.ts` and `git diff --check`: passed.
  • Exact-head CI: see the checks on this head.
  • NOT RUN: full `bun run test`; native Windows/macOS lanes (no per-PR `lane=all`, by policy).

Two test assertions from the precedence commit were corrected rather than loosened. The mixed-order matrix now asserts the status the failed row records (`httpStatusForRequestLogTerminal`), which still rejects a stale losing status. The "reset after an ordinary untyped error" case now asserts the `upstream_reset` that its own title and comment describe, matching dev's typed rule in `tests/server/relay-eager.test.ts`. The mixed-order harness now passes the status argument from `onTerminal` through, as the production recorder does.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. Independent correctness and security reviews run on the final head before merge.

Co-authored-by: Ingwannu ingwannu@users.noreply.github.com

…8299)

* fix(relay): report an untyped SSE error envelope as response.failed

Some OpenAI-compatible gateways end a stream with an SSE data frame that
is only `{"error":{...}}`, with no root `type`, followed by `[DONE]` and
EOF. Both bare-error recognizers in src/server/relay.ts required a root
`type: "error"`, so the frame was ignored and EOF synthesized
`response.incomplete` with `adapter_eof`, dropping the diagnostic.

Accept a frame with no root `type` that carries an `error` object in
both `boundedBareUpstreamErrorMessage` (tee inspection) and
`boundedBareUpstreamError` (client relay, tee and eager). A typed event
of any other kind is still never treated as a bare error. Redaction,
message bounds, class/code mapping and terminal precedence are unchanged.

The combo preflight is left as is: it treats the untyped frame as a
non-control event and commits the target, so it never resends it; the
relay now reports the failure. A test pins that boundary.

Closes #8216

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(relay): finish untyped error envelope handling across readers

- Treat an untyped {"error":{...}} cyber_policy refusal as the policy
  terminal in both rewrite selection and terminal detection, like the
  typed error event, so later bytes are dropped.
- Tee inspection uses the same structured extraction as the client
  boundary, so a refusal code without a message reports failed.
- A code or error class without a message keeps a bounded fallback
  diagnostic and ends in response.failed instead of adapter_eof.
- The SSE-to-WebSocket pump skips the untyped envelope instead of
  failing it as invalid JSON, so the boundary's terminal reaches WS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9aeffab)
@coderabbitai

coderabbitai Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

lidge-jun and others added 2 commits November 9, 2026 10:08
Keep upstream error classification consistent through streaming and
accounting, with bounded client diagnostics and terminal precedence.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
@github-actions github-actions Bot added the bug Something isn't working label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

lidge-jun and others added 5 commits November 9, 2026 13:59
Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Typed and untyped bare error frames could disagree: the client tail
remembered the last bare frame while the request row kept a typed-first
provisional status, so a typed quota followed by an untyped unknown
shipped a generic retryable 502 to the client and a 429 row.

Reduce bare evidence once in bare-upstream-error.ts: the newest typed
frame wins, an untyped frame replaces only untyped evidence, and a
genuine terminal supersedes both. The winning verdict derives identity
and status together, and the tee/eager client boundary, account
inspector, request row and WebSocket row all read it. Explicit refusal
copy is recognized beside an unrelated code; losing frames cannot leave
one reader's code beside another's status or copy.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
The unified bare-error verdict always carried a numeric status, falling back
to 502. Two readers depended on that status being absent for an unrecognized
class:

- the request row wrote the 502 into terminalHttpStatus, so combo preflight
  could no longer read a typed error's explicit status/http_status/code fields
  (429, 400, 410 became 502 and a retry decision changed);
- the synthetic read-failure tail treated every bare verdict as recognized,
  so an ordinary error followed by a socket reset reported the provider's
  generic code instead of upstream_reset.

The verdict now exposes httpStatus only when the provider stated one (explicit
refusal, recognized class, Codex WS wrapper 5xx) and terminalStatus for the
wire. A typed unknown leaves the row open as before; an untyped envelope keeps
its provisional 502. The precedence matrix asserts the status the failed row
records, and the reset case asserts the transport failure its own title
describes.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
@lidge-jun lidge-jun changed the title Re-land #8299 (draft: blocked by review) fix(relay): report an untyped SSE error envelope as response.failed (re-land #8299) Nov 12, 2026
jun and others added 3 commits November 12, 2026 19:09
Final-head review found two divergences from dev:

- An untyped cyber_policy envelope rewrote the client tail and forced the
  account status to 400 even when an earlier typed frame owned the verdict,
  while the row kept the typed status (typed quota -> untyped policy gave
  client cyber_policy, account 400, row 429).
- Eager tagged every bare verdict as upstream_error on a read failure, so an
  unrecognized error followed by a reset lost the streamAborted transport
  attribution the tee path and dev both record.

A per-stream terminal tracker now applies the bare-evidence reducer before the
untyped policy branch, and the client boundary, inspector, passthrough
inspection and buffered JSON all use it. A winning policy terminal is final in
the reducer and carries the policy verdict, so a typed frame with a recognized
code beside policy copy records 400/cyber_policy everywhere, as dev's row did.
Eager states upstream_error only for a refusal or provider-stated class, the
same predicate its tail uses.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
Readers that keep inspecting past the terminal (the WebSocket EOF observer
and the response-log metadata consumer) fed a later bare error into the row
reducer after the terminal had retired its evidence. The reducer started a
fresh verdict, so an incomplete max_output_tokens or completed turn followed
by a bare quota frame recorded 429 on the row and reached account health as
quota, while the client had already ended on the genuine terminal.

Once a genuine terminal is recorded, a bare frame no longer touches status,
code or type. Its copy keeps dev's guard: untyped text that fails the
outbound-credential check is recorded as [REDACTED]. Drop the unused
untypedProvisionalStatuses set.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>
A bound applied before the outbound mask can keep a prefix of a long selected
credential that the full-value mask no longer recognizes. The bare verdict
sliced message (512), code and type (128) first, and the eager policy rewrite
sliced its message before the block rewrite ran, so a long unpatterned
credential reached eager and WebSocket clients as a prefix and the request
context kept the same prefix.

Bare copy now goes through the relay's per-request mask and the outbound
credential registry before it is bounded; the client boundary receives the
mask from both relays, and the policy rewrite masks before slicing. A code or
type longer than 128 characters is not a class and is dropped rather than
truncated, as dev's diagnostic token check already did.

Co-authored-by: Ingwannu <ingwannu@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Untyped SSE error envelope is reported as adapter_eof by native Responses relay

2 participants