From e9215f280b7a5689e73dad022d30a71dc2a9b275 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Mon, 19 Oct 2026 18:08:43 +0800 Subject: [PATCH 01/23] feat(combo): preserve native Messages replay Reuse the native Messages executor for eligible canonical Anthropic failover children without replacing combo routing or accounting owners. Inspect only the bounded pre-output Messages prelude before failover; text, tool starts and unknown events commit the selected child. Refs #7350 Co-Authored-By: Claude Code --- .../src/content/docs/guides/claude-code.md | 1 + .../src/content/docs/guides/protocol-paths.md | 17 +- .../docs/reference/configuration/server.md | 2 +- scripts/test-layout/layout.json | 1 + src/server/claude-messages.ts | 32 +- src/server/messages-native-preflight.ts | 98 +++++++ src/server/messages-native.ts | 139 +++++++-- src/server/messages-shared.ts | 18 +- src/server/responses/core-combo-native.ts | 82 ++++-- src/server/responses/core-combo.ts | 4 +- src/server/responses/core-options.ts | 2 +- structure/data-planes/protocol-paths.md | 6 +- structure/providers-and-adapters.md | 2 +- .../transports/responses-combo-boundaries.md | 10 + structure/transports/responses-spend.md | 4 +- structure/transports/responses.md | 4 +- .../messages-native-combo.test.ts | 277 ++++++++++++++++++ tests/fixtures/test-layout-expected.json | 1 + 18 files changed, 623 insertions(+), 77 deletions(-) create mode 100644 src/server/messages-native-preflight.ts create mode 100644 tests/claude-integration/messages-native-combo.test.ts diff --git a/docs-site/src/content/docs/guides/claude-code.md b/docs-site/src/content/docs/guides/claude-code.md index 176723e7533..1b0d60d6f25 100644 --- a/docs-site/src/content/docs/guides/claude-code.md +++ b/docs-site/src/content/docs/guides/claude-code.md @@ -144,6 +144,7 @@ thinking fallback. Adaptive thinking without an explicit effort stays blank. The request controls; they do not confirm what effort the upstream applied or change the wire body. An enabled Anthropic account pool prefers native Messages for eligible direct Anthropic routes +and canonical Anthropic children of strict failover combos without a default effort when neither native rollout flag explicitly disables that path. In Providers → Anthropic → Account pooling → **How account selection works**, **Preserve native Claude requests** stores `anthropicAccountPool.nativeMessages` (default true). Turning it off selects the legacy bridge diff --git a/docs-site/src/content/docs/guides/protocol-paths.md b/docs-site/src/content/docs/guides/protocol-paths.md index 2e5ac6de19c..76917c4ecb8 100644 --- a/docs-site/src/content/docs/guides/protocol-paths.md +++ b/docs-site/src/content/docs/guides/protocol-paths.md @@ -99,8 +99,8 @@ These switches stage the new paths. Each defaults to off except `antigravityDire | Switch | When on | | --- | --- | | `nativeChatCombos` | An eligible Chat candidate inside a combo is sent natively from its own copy of the client body; the other candidates keep the bridge. | -| `managedMessagesNative` | A Messages request whose route is a direct, key-authenticated Anthropic provider is sent as Messages instead of through the bridge. Routes that need bridge-only behaviour (a pinned effort, blocked-skill elision, the web-search sidecar, vision preprocessing, synthetic rows) stay on the bridge. | -| `managedMessagesNativeOAuth` | Native Messages for the unpooled `anthropic` OAuth provider on `api.anthropic.com`. Effective only together with `managedMessagesNative`; a pooled Anthropic OAuth account set stays on the bridge. | +| `managedMessagesNative` | Send eligible direct Anthropic routes and canonical Anthropic children of strict failover combos as Messages. Combo defaults/effort rewrites and bridge-only behaviour (pinned effort, blocked-skill elision, web-search sidecar, vision preprocessing, synthetic rows) retain the bridge. | +| `managedMessagesNativeOAuth` | Extend native Messages to configured Anthropic OAuth instances on `api.anthropic.com`, including stored account pools and eligible failover children. Effective only together with `managedMessagesNative`. | | `directEncoders` | For a non-Responses upstream, the answer to a Chat or Messages client is encoded directly from the adapter's events instead of through the internal Responses stream. The request side is unchanged. | | `antigravityDirect` | **On by default**; set it to `false` to turn it off. A Chat or Messages request whose route is one Google Antigravity provider skips the internal Responses body in both directions: the conversation is decoded from the client body straight into what the Google adapter compiles from, and the answer is encoded directly as with `directEncoders`. The path reads `chat -> ir -> other` or `messages -> ir -> other`. Request settings (model, effort, limits, tool choice) still pass through the usual policies, so account rotation, effort rules and usage are unchanged. Combos, policies, a hosted web-search tool, images for a model marked text-only, a model the shadow-call intercept could rewrite, and text the proxy rewrites (a skills catalog, a generated-image link) stay on the bridge, and the log row names the reason. On a Chat request the decoded conversation counts toward the shared translation budget, so a request whose retained IR exceeds it is refused with HTTP 413 and `translation_buffer_limit` before anything goes upstream. | | `shadowPlan` | At the end of each Chat or Messages request, the plan a preview would have predicted is compared with the path the request took; a disagreement adds `planMismatch: true` to the log row's path record. No second request is sent. | @@ -135,8 +135,13 @@ These paths still use the internal Responses bridge or are not covered: - Chat and Messages requests are still decoded into a Responses-shaped body before the IR, including on a translated path. The exception is a Chat or Messages request to a single Google Antigravity route, unless `antigravityDirect` is set to `false`. -- Routing-policy candidates, combos reached through an effort row, combo candidates while - `nativeChatCombos` is off, and Messages combos. +- Routing-policy candidates, combos reached through an effort row, and Chat combo candidates while + `nativeChatCombos` is off. Messages combos retain the bridge except eligible canonical Anthropic + children of strict failover combos without a default effort. Each eligible child starts from the + source Messages envelope, preserving system/messages/tools/cache breakpoints under the existing + native allowlist and credential rules. Without explicit operator retention, TTL-less combo + breakpoints stay TTL-less. Pre-output retryable errors may fall back; any content/tool block + commits the child and forbids replay. Preserved prefixes do not prove live cache hits. - Direct encoding (`directEncoders`) covers only a concrete, non-Responses route in streaming delivery. Combo and policy children, routed compaction, run-turn adapters (Cursor, Devin, coding-agent CLIs, CodeBuddy), and sidecar turns keep the bridge. @@ -153,8 +158,8 @@ These paths still use the internal Responses bridge or are not covered: their current behavior. Headers alone do not authorize requests or establish official support. The TypeScript Agent SDK's `sdk-ts` user-agent entrypoint is recognized with the same required identity headers, preserving its allowlisted native feature betas on first-party Anthropic requests. -- Native Chat over OAuth is not planned. Native Messages over Anthropic OAuth covers only an - unpooled account; a pooled account set stays on the bridge. +- Native Chat over OAuth is not planned. Native Messages over Anthropic OAuth includes pooled + accounts under the same native eligibility and credential-binding rules. - The managed native Messages path forwards a caller's `anthropic-beta` values only from a short allowlist and only to `api.anthropic.com`, drops top-level fields outside its allowlist without a feature effect, and does not apply diff --git a/docs-site/src/content/docs/reference/configuration/server.md b/docs-site/src/content/docs/reference/configuration/server.md index 8c9f113168b..77aa47db68d 100644 --- a/docs-site/src/content/docs/reference/configuration/server.md +++ b/docs-site/src/content/docs/reference/configuration/server.md @@ -639,7 +639,7 @@ modes, the preview, and the per-request trace. | --- | --- | --- | --- | | `protocols.unrepresentable?` | `"legacy" \| "reject"` | `"legacy"` | `legacy` sends a request whose path drops a feature and records the loss in the trace. `reject` refuses it with HTTP 400 before any send, naming only the feature keys. | | `protocols.rollout.nativeChatCombos?` | `boolean` | `false` | Send an eligible Chat candidate inside a combo natively from its own copy of the client body. | -| `protocols.rollout.managedMessagesNative?` | `boolean` | `false` | Send Messages natively to a direct, key-authenticated Anthropic provider instead of through the internal Responses bridge. | +| `protocols.rollout.managedMessagesNative?` | `boolean` | `false` | Send Messages natively to an eligible direct Anthropic provider or canonical Anthropic child in a strict failover combo without a default effort. Other combo strategies, policy routes and bridge-only rewrites retain the bridge. | | `protocols.rollout.managedMessagesNativeOAuth?` | `boolean` | `false` | Native Messages for the `anthropic` OAuth provider on `api.anthropic.com`, including stored account pools. Requires `managedMessagesNative`. For the settled `anthropic` provider, an enabled pool supplies true for absent native flags when `anthropicAccountPool.nativeMessages` is absent or true. Explicit false/malformed flags and a false/malformed pool preference remain off. Other providers and pool-off keep explicit rollout behavior. Eligible pooled requests retain shared selection, affinity and bounded pre-output recovery. Recognized JSON-string `metadata.user_id` account UUIDs are aligned with the serving OAuth credential; device and session fields are preserved. | | `protocols.rollout.directEncoders?` | `boolean` | `false` | Encode Chat and Messages answers from a non-Responses upstream directly from adapter events. | | `protocols.rollout.antigravityDirect?` | `boolean` | `true` | Send a Chat or Messages request to a single Google Antigravity route without the internal Responses body in either direction. Request settings still pass through the usual policies, and a request the lane cannot carry uses the bridge. Set `false` to turn it off. | diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index a61f3c0d32b..d496dc16d38 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -451,6 +451,7 @@ "claude-messages-endpoint.test.ts": "claude-integration", "claude-messages-thread.test.ts": "claude-integration", "messages-native.test.ts": "claude-integration", + "messages-native-combo.test.ts": "claude-integration", "messages-native-decline-trace.test.ts": "claude-integration", "messages-native-oauth.test.ts": "claude-integration", "messages-native-opaque-state.test.ts": "claude-integration", diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index c38cbbe2064..1ad749dc8ac 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -83,6 +83,7 @@ import { clientWireOf } from "./inference/client-wire"; import { directEncodersApply } from "./inference/client-encoder-delivery"; import type { ClientEncoderOption } from "./responses/core-options"; import { handleResponses } from "./responses"; +import { resolveComboId } from "../combos"; import { previewXaiOauthWireModel } from "./responses/core-normalize"; import { upstreamWireForAdapter } from "../protocols/contract"; import { createProtocolEnvelope, type ProtocolEnvelope } from "../protocols/envelope"; @@ -599,6 +600,7 @@ async function handleClaudeMessagesWithBudget( } // Model resolution reads this view; every other claudeCode setting keeps reading `config`. const cc = claudeCodeForIngress(config.claudeCode, ingress.claudeIntercept === true); + let nativeComboSource = false; let anthropicBody: unknown; let internalBody: Rec; @@ -687,8 +689,13 @@ async function handleClaudeMessagesWithBudget( // proxy-owned `speed` + beta (anthropic-speed wire) and its usage.speed observation would be // silently skipped. Translation reaches the adapter, which owns both. const messagesBody = anthropicBody; - if (isRec(messagesBody) && resolveProtocolSettings(config).unrepresentable === "reject") { + nativeComboSource = isRec(messagesBody) && typeof messagesBody.model === "string" + && resolveComboId(config, resolveInboundModel(messagesBody.model, cc)) !== null + && Object.keys(config.providers).some(name => resolveProtocolSettings(config, name).rollout.managedMessagesNative); + if (isRec(messagesBody) && (resolveProtocolSettings(config).unrepresentable === "reject" || nativeComboSource)) { + // Keep the replay source independent of later ingress effort rewrites. envelope = createProtocolEnvelope({ inbound: "messages", body: messagesBody, translatorBudget }); + if (nativeComboSource) envelope = createProtocolEnvelope({ inbound: "messages", body: envelope.freshBody(), translatorBudget }); } const sourceEnvelope = envelope; // The bridge entry mark below reads these before an effort override rewrites `thinking`, @@ -987,7 +994,8 @@ async function handleClaudeMessagesWithBudget( }); } // Combo and policy children are judged per candidate (PF-07); an unknown model has no route. - if (envelope && settledRoute && !settledRoute.combo && settledRoute.routeKind !== "policy") { + if (envelope && resolveProtocolSettings(config).unrepresentable === "reject" + && settledRoute && !settledRoute.combo && settledRoute.routeKind !== "policy") { const verdict = checkRepresentable({ inbound: "messages", requestPath: requestPathForLane( @@ -1007,7 +1015,8 @@ async function handleClaudeMessagesWithBudget( } // PF-10: opaque thinking state reaches only first-party Anthropic; under `reject` a native // route to anyone else refuses the request instead of sending it without that state. - if (envelope && nativeMessagesRoute && !credentialDomainFor(nativeMessagesRoute.provider)?.firstPartyAnthropic + if (envelope && resolveProtocolSettings(config).unrepresentable === "reject" + && nativeMessagesRoute && !credentialDomainFor(nativeMessagesRoute.provider)?.firstPartyAnthropic && messagesBodyHasOpaqueState(anthropicBody as Rec)) { markProtocolBlocked(logCtx, { inbound: "messages", reasonCodes: ["feature-unrepresentable", "opaque-state-stripped"], features: messagesFeatures }); logCtx.errorCode = "unsupported_feature"; @@ -1119,6 +1128,21 @@ async function handleClaudeMessagesWithBudget( // over, and TTL-less breakpoints take the destination default. cacheRetention: config.cacheRetention ?? callerCacheRetention(anthropicBody), }, logCtx, { + ...(nativeComboSource && !effortOverride && envelope && settledRoute?.combo ? { + protocolSource: (await import("./messages-native")).createNativeMessagesComboSource({ + req, config, logCtx, envelope, requestedModel, translatorBudget, selector: nativeSelector, + modelScope: resolveAdmissionModelScope(config, logIds?.admission), + callerAnthropicBeta: req.headers.get("anthropic-beta"), + clientIdentity: captureAnthropicClientIdentity(req.headers), + sessionKey: anthropicSessionKeyFromParts({ + sessionIdHeader: req.headers.get("session_id")?.trim() || req.headers.get("x-claude-code-session-id"), + threadIdHeader: req.headers.get("thread_id"), + clientThreadId: logCtx.conversationId, + promptCacheKey: typeof internalBody.prompt_cache_key === "string" ? internalBody.prompt_cache_key : null, + promptCacheKeyIsSharedCohort: cacheKeySource === "system", + }), + }), + } : {}), // Routing keeps Claude-only sidecar overrides; admission policy must follow the live owner. codexAuthPolicy: config, ...(logIds?.admission ? { admission: logIds.admission } : {}), @@ -1162,6 +1186,8 @@ async function handleClaudeMessagesWithBudget( ? { directRequest: messagesDirectRequestOption(directPlan, anthropicBody, cc, translatorBudget) } : {}), }); + // Native combo children report through gated callbacks, not the Responses log inspector. + if (clientWireOf(upstream) === "messages" && upstream.ok && settledRoute?.combo) return upstream; const response = logIds ? responseWithDeferredRequestLog(upstream, logIds.requestId, logIds.start, logCtx) : upstream; // Already in the Messages wire (direct encoder): no conversion. if (clientWireOf(upstream) === "messages") return response; diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts new file mode 100644 index 00000000000..10b55b0f0fa --- /dev/null +++ b/src/server/messages-native-preflight.ts @@ -0,0 +1,98 @@ +import { comboFailureDecision } from "../combos"; +import { httpStatusFromTerminalError } from "../lib/errors"; +import { redactSecretString } from "../lib/redact"; +import type { TranslatorBudget } from "../lib/translator-budget"; +import { createSseInspector } from "./relay"; +import { anthropicUsageToOcx } from "./messages-shared"; +import type { RequestLogContext } from "./request-log"; +import { MAX_CLIENT_SSE_FRAME_BYTES } from "./sse-frame-buffer"; +import { UNDECLARED_TOOL_CALL_ERROR_CODE } from "./responses-undeclared-tool-guard"; + +type Rec = Record; +const isRec = (value: unknown): value is Rec => value !== null && typeof value === "object" && !Array.isArray(value); + +/** Inspect only the bounded protocol prelude. Any block, unknown payload or cap commits. */ +export async function preflightNativeMessagesResponse( + response: Response, logCtx: RequestLogContext, budget: TranslatorBudget, +): Promise<{ kind: "accepted" | "failed"; response: Response }> { + if (!response.ok || !response.body || !response.headers.get("content-type")?.includes("text/event-stream")) { + return { kind: "accepted", response }; + } + let committed = false; + let terminal = false; + let failure: Rec | undefined; + let failureStatus = 502; + let usage: Rec = {}; + const inspector = createSseInspector({ + onOpaquePayload: () => { if (!failure) committed = true; }, + onParsedPayload: payload => { + if (committed || terminal || failure) return; + if (!isRec(payload)) { committed = true; return; } + if (payload.type === "message_start" && isRec(payload.message)) { + if (isRec(payload.message.usage)) usage = { ...usage, ...payload.message.usage }; + if (!Array.isArray(payload.message.content) || payload.message.content.length !== 0) committed = true; + } else if (payload.type === "ping") { + // Protocol liveness is not output. + } else if (payload.type === "error" && isRec(payload.error)) { + const error = payload.error; + const status = httpStatusFromTerminalError({ type: typeof error.type === "string" ? error.type : undefined, + code: typeof error.code === "string" ? error.code : undefined, + message: typeof error.message === "string" ? error.message : undefined }) ?? 502; + if (logCtx.terminalSource !== "synthetic" && logCtx.errorCode !== UNDECLARED_TOOL_CALL_ERROR_CODE + && comboFailureDecision(status, typeof error.message === "string" ? error.message : "", { + code: typeof error.code === "string" ? error.code : undefined, + }) === "hop") { failure = error; failureStatus = status; } + else committed = true; + } else if (payload.type === "message_stop") terminal = true; + else committed = true; + }, + }); + const reader = response.body.getReader(); + const chunks: Uint8Array[] = []; + let retained = 0; + const release = () => { budget.releaseRetained(retained, { kind: "retained_collectors" }); retained = 0; }; + try { + for (;;) { + const next = await reader.read(); + if (next.done) { terminal = true; break; } + // The one upstream chunk already exists; retain it without a copy if the cap commits. + if (retained + next.value.byteLength > MAX_CLIENT_SSE_FRAME_BYTES || chunks.length >= 1024) { + chunks.push(next.value); committed = true; break; + } + budget.chargeRetained(next.value.byteLength, { kind: "retained_collectors" }); + retained += next.value.byteLength; + chunks.push(next.value); + inspector.feed(next.value); + if (committed || terminal || failure) break; + } + if (failure && !committed) { + logCtx.usage = anthropicUsageToOcx(usage); + logCtx.upstreamError = redactSecretString(typeof failure.message === "string" ? failure.message : "Provider stream failed before output"); + void reader.cancel("retrying native Messages pre-output refusal").catch(() => {}); + release(); + const headers = new Headers(response.headers); + headers.set("content-type", "application/json"); headers.delete("content-length"); headers.delete("content-encoding"); + return { kind: "failed", response: Response.json({ error: { + ...(typeof failure.type === "string" ? { type: failure.type } : {}), + ...(typeof failure.code === "string" ? { code: failure.code } : {}), + message: logCtx.upstreamError, + } }, { status: failureStatus, headers }) }; + } + let index = 0; + const body = new ReadableStream({ + async pull(controller) { + try { + if (index < chunks.length) { controller.enqueue(chunks[index++]!); return; } + chunks.length = 0; + release(); + const next = await reader.read(); + if (next.done) controller.close(); else controller.enqueue(next.value); + } catch (error) { release(); controller.error(error); } + }, + cancel(reason) { release(); void reader.cancel(reason).catch(() => {}); }, + }, { highWaterMark: 0 }); + return { kind: "accepted", response: new Response(body, { status: response.status, headers: response.headers }) }; + } catch (error) { + release(); void reader.cancel(error).catch(() => {}); throw error; + } finally { inspector.dispose(); } +} diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 0baffa8e020..27a58610181 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -1,5 +1,5 @@ import { captureAccountAutoPauseAttempt, wasAccountAutoPaused } from "../oauth/account-auto-pause"; -import { createPhysicalSendReporter } from "../lib/request-execution-budget"; +import { createPhysicalSendReporter, reportDispatchSends, type RequestExecutionBudget, type SingleUseDispatchPermit } from "../lib/request-execution-budget"; import { createInferenceSendBudget } from "./inference/context"; /** * Managed native Messages lane (PF-08, behind `protocols.rollout.managedMessagesNative`). @@ -37,7 +37,7 @@ import { buildAnthropicMessagesPassthroughRequest, type AnthropicMessagesPassthroughRequest, } from "../adapters/anthropic/passthrough"; -import { effortFromOutputConfig, resolveInboundModel } from "../claude/inbound"; +import { effortFromOutputConfig, nativeAnthropicProjection, resolveInboundModel } from "../claude/inbound"; import { anthropicErrorBody, anthropicErrorResponse, claudeOverflowSsePayload, claudePromptTooLongMessage, collectAnthropicMessage, isContextOverflowText, isThroughputLimitText } from "../claude/outbound"; import type { AdmissionLease } from "../lib/admission"; import { readBoundedResponseBody } from "../lib/bounded-body"; @@ -60,6 +60,7 @@ import { REPLAY_REFUSED_STATUS, retainReplayRefusal, UpstreamRetryEvidenceError, + TRANSIENT_RETRY_MAX_ATTEMPTS, type UpstreamSendRecovery, UPSTREAM_RESET_REPLAY_REFUSED_CODE, } from "../lib/upstream-retry"; @@ -96,7 +97,10 @@ import { sanitizePassthroughToolCallIds, tapAnthropicSseForLog, } from "./messages-shared"; -import { beginInferenceAttempt } from "./inference/attempt"; +import { beginInferenceAttempt, type InferenceAttempt } from "./inference/attempt"; +import type { ComboProtocolSource } from "./responses/core-combo-native"; +import type { ProtocolEnvelope } from "../protocols/envelope"; +import { preflightNativeMessagesResponse } from "./messages-native-preflight"; import { createFinalRequestLog, type FinalRequestLogMeta } from "./inference/final-log"; import { registerTurn, unregisterTurn } from "./lifecycle"; import { nativeMessagesDeclineReason, type NativeMessagesSelector } from "./messages-native-eligibility"; @@ -185,7 +189,35 @@ export interface HandleNativeMessagesOptions { modelScope?: AdmissionModelScope; } -type FinishLog = (status: number, message?: string, meta?: FinalRequestLogMeta) => void; +export type NativeMessagesFinishLog = (status: number, message?: string, meta?: FinalRequestLogMeta) => void; +type FinishLog = NativeMessagesFinishLog; + +interface NativeMessagesExecution extends HandleNativeMessagesOptions { + finishLog: NativeMessagesFinishLog; + sendBudget?: RequestExecutionBudget; + comboDispatchPermit?: SingleUseDispatchPermit; + turnAdmissionLease?: AdmissionLease; + onFirstOutput?: () => void; +} + +/** The combo retains routing, attempts and the final row; the native lane owns only this send. */ +export function createNativeMessagesComboSource(input: Omit & { + envelope: ProtocolEnvelope; +}): ComboProtocolSource { + return { + inbound: "messages", + envelope: input.envelope, + selector: input.selector, + dispatchNativeChild: child => runNativeMessagesAttempt({ + ...input, logIds: undefined, logCtx: child.childLog, route: child.route, + body: nativeAnthropicProjection(child.body, input.translatorBudget), + selector: { ...input.selector, routeSelector: String(child.body.model ?? "") }, + finishLog: child.finishLog, sendBudget: child.sendBudget, + comboDispatchPermit: child.comboDispatchPermit, turnAdmissionLease: child.turnAdmissionLease, + onFirstOutput: child.onFirstOutput, + }, child.attemptHandle), + }; +} /** Relay an upstream body unchanged, recording first output on its first non-empty chunk. */ function observeFirstChunk(body: ReadableStream, onFirst: () => void): ReadableStream { @@ -320,7 +352,21 @@ async function prepareNativeBody(body: Rec, signal: AbortSignal): Promise * log row, and answers in the Messages wire (SSE or JSON per the caller's own `stream`). */ export async function handleNativeMessages(options: HandleNativeMessagesOptions): Promise { - const { req, config, logCtx, logIds, route, body, requestedModel, translatorBudget } = options; + const { logCtx, logIds, route } = options; + const attemptHandle = beginInferenceAttempt(logCtx, { provider: route.providerName, model: route.modelId, adapter: "anthropic" }); + const finalLog = createFinalRequestLog(logIds, logCtx); + const finishLog: FinishLog = (status, message, meta = { closeReason: "non_stream" }) => { + if (finalLog.finished()) return; + if (message) logCtx.upstreamError = redactSecretString(message).slice(0, 500); + finalLog.finish(status, meta); + }; + return runNativeMessagesAttempt({ ...options, finishLog }, attemptHandle); +} + +/** Execute on the ingress or combo's existing attempt, never opening a second final-row owner. */ +async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemptHandle: InferenceAttempt): Promise { + const { req, config, logCtx, logIds, route, body, requestedModel, translatorBudget, finishLog } = options; + const onFirstOutput = options.onFirstOutput ?? (logIds ? () => recordFirstOutput(logCtx, logIds.start) : undefined); const nativeInstance = configuredAnthropicInstance(config, route.providerName); const requestedStream = body.stream === true; const clientDeclaredEmptyToolCatalog = Array.isArray(body.tools) && body.tools.length === 0; @@ -351,20 +397,9 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) logCtx.spendOutputCeilingTokens = Math.trunc(body.max_tokens); } - const attemptHandle = beginInferenceAttempt(logCtx, { - provider: route.providerName, - model: route.modelId, - adapter: "anthropic", - }); attemptHandle.seal(logCtx.accountLogLabel); recordAttemptRequestedEffort(logCtx); const { attempt } = attemptHandle; - const finalLog = createFinalRequestLog(logIds, logCtx); - const finishLog: FinishLog = (status, message, meta = { closeReason: "non_stream" }) => { - if (finalLog.finished()) return; - if (message) logCtx.upstreamError = redactSecretString(message).slice(0, 500); - finalLog.finish(status, meta); - }; const bindUsage = (usage: OcxUsage | undefined) => { if (!usage) return; if (!recordKeyWireAttemptUsage(logCtx, usage)) { @@ -416,7 +451,7 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) const cleanupAbort = linkAbortSignal(upstream, req.signal); let streamTurnRegistered = false; const transferTurnToStream = () => { - const lease = logIds?.turnAdmissionLease; + const lease = options.turnAdmissionLease ?? logIds?.turnAdmissionLease; if (!lease || typeof (lease as { bindAbortController?: unknown }).bindAbortController !== "function") return; registerTurn(upstream, lease); streamTurnRegistered = true; @@ -461,8 +496,8 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) let activeProvider: OcxProviderConfig = oauthBinding ? oauthProvider(oauthBinding) : route.provider; stampApiKeyAccountLabel(logCtx, route.providerName, activeProvider); if (oauthBinding) logCtx.provider = formatAnthropicProviderForLog(route.providerName, oauthBinding.snapshot.accountId, config); - const spendTracker = attachRequestSpendTracker(req, logCtx); - const physicalBudget = createInferenceSendBudget(req, logCtx); + const spendTracker = options.sendBudget ? undefined : attachRequestSpendTracker(req, logCtx); + const physicalBudget = options.sendBudget ?? createInferenceSendBudget(req, logCtx); // Observe-only spend is already booked by the tracker; count billing sends without booking again. let physicalSendsUsed = 0; let activeRequest: AnthropicMessagesPassthroughRequest; @@ -483,7 +518,9 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) const rejectUnrepresentable = resolveProtocolSettings(config).unrepresentable === "reject"; const buildActiveRequest = () => { recordAttemptCredentialSource(attempt, route.providerName, activeProvider, "anthropic"); - const built = buildAnthropicMessagesPassthroughRequest(activeProvider, route.modelId, body, config, { + // Exact combo replay preserves TTL-less breakpoints; an explicit operator TTL still wins. + const wireConfig = options.sendBudget && config.cacheRetention === undefined ? { ...config, cacheRetention: "short" as const } : config; + const built = buildAnthropicMessagesPassthroughRequest(activeProvider, route.modelId, body, wireConfig, { callerAnthropicBeta: options.callerAnthropicBeta, providerAccountUuid: oauthBinding?.providerAccountUuid, clientIdentity: options.clientIdentity, @@ -526,9 +563,13 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) // One inbound request owns one transient send allowance, captured before any key rotation. const requestTransientPolicy = transientRetryPolicyFor(activeProvider); let transientSendsUsed = 0; - const remainingTransientSends = (): number => requestTransientPolicy - ? Math.max(0, requestTransientPolicy.attempts - transientSendsUsed) - : Number.POSITIVE_INFINITY; + const sharedSendCap = requestTransientPolicy?.attempts ?? TRANSIENT_RETRY_MAX_ATTEMPTS; + let initialSendStarted = false; + const remainingTransientSends = (): number => Math.min( + requestTransientPolicy ? Math.max(0, requestTransientPolicy.attempts - transientSendsUsed) : Number.POSITIVE_INFINITY, + options.sendBudget ? Math.min(sharedSendCap, options.sendBudget.remainingBaseSends(sharedSendCap) + + (!initialSendStarted && options.comboDispatchPermit ? 1 : 0)) : Number.POSITIVE_INFINITY, + ); const transientSendAvailable = (): boolean => remainingTransientSends() > 0; const selector: NativeMessagesSelector = options.selector ?? {}; const nativeOAuthRouteIsCurrent = (binding: NativeOAuthBinding): boolean => { @@ -553,6 +594,7 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) /** Send the native Messages request under the transient retry policy, re-validating the selected OAuth account or API key at the dispatch boundary. */ const send = async (recovery?: "rate-limit-429" | "oauth-401" | "oauth-account-403" | "key-429" | "key-401"): Promise => { const remaining = remainingTransientSends(); + if (options.sendBudget && remaining <= 0) throw new SendBudgetExhaustedError(); if (requestTransientPolicy && remaining <= 0) { throw new Error("native Messages transient send budget exhausted before recovery dispatch"); } @@ -568,6 +610,12 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) providerFetch(activeProvider, undefined, { providerName: route.providerName, modelId: route.modelId, + onPhysicalDispatch: options.sendBudget && options.comboDispatchPermit ? () => { + if (!physicalBudget.spendEnforced) { + reportDispatchSends(options.sendBudget!, 1, options.comboDispatchPermit); + initialSendStarted = true; + } + } : undefined, dispatchOverride: async (_input, init, execute, reacquirePacing) => { if (oauthBinding) { if (!nativeOAuthRouteIsCurrent(oauthBinding)) throw new NativeOAuthSelectionChangedError(); @@ -627,16 +675,19 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) let dispatched: Response; const spendReport = createPhysicalSendReporter(physicalBudget, () => ({ poolId: logCtx.spendPoolId ?? route.providerName, identityId: logCtx.accountLogLabel, - })); + }), options.comboDispatchPermit); let sendOwner: AnthropicPhysicalSendOwnership | null = null; try { if (init.signal?.aborted) throw init.signal.reason; // Capture before either accounting path so this request retains its start policy. if (spendReport.beforeSend && !spendReport.beforeSend()) { - if (spendTracker.refusals) throw new NativeMessagesSpendRefusal(); + if ((logCtx.spendTracker as { refusals?: number } | undefined)?.refusals) throw new NativeMessagesSpendRefusal(); throw new SendBudgetExhaustedError(); } - if (!physicalBudget.spendEnforced && !spendTracker.charge()) throw new NativeMessagesSpendRefusal(); + if (physicalBudget.spendEnforced) initialSendStarted = true; + else if (options.sendBudget) { + if (initialSendStarted && options.sendBudget.remainingBaseSends(sharedSendCap) <= 0) throw new SendBudgetExhaustedError(); + } else if (!spendTracker?.charge()) throw new NativeMessagesSpendRefusal(); noteProviderAttemptSend(logCtx, route.providerName, activeProvider, logCtx.usageLogInputTokens, transportRecovery ?? recovery); // Last synchronous ownership reservation before the physical fetch. Retain this // exact incarnation across its await; a returned response can never reserve one. @@ -889,7 +940,7 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) // `upstream` follows req.signal and is also what shutdown and turn release abort, so both // the client leaving and this lane's own abort read as a cancel, not as a failed stream. const bodyGuard = resolvePassthroughBodyGuard(config, upstream.signal); - const observed = logIds ? observeFirstChunk(response.body, () => recordFirstOutput(logCtx, logIds.start)) : response.body; + const observed = logIds && onFirstOutput ? observeFirstChunk(response.body!, onFirstOutput) : response.body!; const renamed = activeRequest.oauthToolNames ? restoreOAuthToolNamesInSse(observed, activeRequest.oauthToolNames, translatorBudget) : observed; @@ -908,9 +959,11 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) cleanupAbort(); bindUsage(logCtx.usage); finishLog(toolRefusal ? 502 : status, toolRefusal, toolRefusal ? { ...meta, terminalStatus: "failed" } : meta); - if (toolRefusal || meta.closeReason !== "terminal") upstream.abort(); } finally { + // A discarded child cancels its reader, not the shared parent turn. Detach its + // controller before aborting local transport work, while the ladder remains bound. releaseStreamTurn(); + if (toolRefusal || meta.closeReason !== "terminal") upstream.abort(); } }, bodyGuard); // A configured Messages provider words an oversized-input refusal its own way; Claude Code @@ -918,18 +971,42 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) // rewrite buffers whole frames, so it sits after the tap: stall detection keeps timing raw // upstream bytes. Real Anthropic pools refuse pre-stream in that wording and keep one relay. const clientStream = nativeInstance ? relayed : relaySseWithPayloadRewrite(relayed, claudeOverflowSsePayload, translatorBudget); - return retainUpstreamMessagesRequestId(new Response(clientStream, { + let delivered = retainUpstreamMessagesRequestId(new Response(clientStream, { status: 200, headers: { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache", Connection: "keep-alive" }, }), response.headers); + if (options.sendBudget) { + try { + const preflight = await preflightNativeMessagesResponse(delivered, logCtx, translatorBudget); + if (preflight.kind === "failed") { bindUsage(logCtx.usage); return preflight.response; } + delivered = preflight.response; + onFirstOutput?.(); + } catch (error) { + cleanupAbort(); releaseStreamTurn(); upstream.abort(); + if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); + if (isTranslatorBudgetExceededError(error)) return fail(413, error.message, "request_too_large", "translation_buffer_limit"); + return fail(502, error instanceof Error ? error.message : String(error), "api_error"); + } + } + return delivered; } // A non-streaming caller whose upstream streamed anyway: fold the stream into one message. const tapState: { closeReason?: FinalRequestLogMeta["closeReason"]; meta?: FinalRequestLogMeta } = {}; - const tapped = tapAnthropicSseForLog(source, logCtx, (_status, meta) => { + let tapped = tapAnthropicSseForLog(source, logCtx, (_status, meta) => { tapState.closeReason = meta.closeReason; tapState.meta = meta; }, bodyGuard); try { + if (options.sendBudget) { + const preflight = await preflightNativeMessagesResponse(new Response(tapped, { headers: response.headers }), logCtx, translatorBudget); + if (preflight.kind === "failed") { + cleanupAbort(); upstream.abort(); bindUsage(logCtx.usage); + finishLog(preflight.response.status, logCtx.upstreamError); + return preflight.response; + } + tapped = preflight.response.body!; + onFirstOutput?.(); + } const message = await collectAnthropicMessage(tapped, requestedModel, translatorBudget); cleanupAbort(); if (tapState.closeReason === "client_cancel" || req.signal.aborted) { @@ -1002,7 +1079,7 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) return fail(502, undeclaredToolCallMessage(undeclaredTool), "api_error", UNDECLARED_TOOL_CALL_ERROR_CODE); } bindUsage(anthropicUsageToOcx(isRec(message.usage) ? message.usage : undefined)); - if (logIds) recordFirstOutput(logCtx, logIds.start); + onFirstOutput?.(); try { translatorBudget.chargeRetained(Buffer.byteLength(read.text) * 2, { kind: "live_transient" }); } catch (error) { diff --git a/src/server/messages-shared.ts b/src/server/messages-shared.ts index 8444fd73737..5b026cebf6d 100644 --- a/src/server/messages-shared.ts +++ b/src/server/messages-shared.ts @@ -16,6 +16,7 @@ import { sniffImageDimensions } from "../adapters/anthropic-image-guard"; import { anthropicCacheWrite1hTokens } from "../adapters/anthropic/cache-retention"; import { createToolCallIdAllocator } from "../adapters/tool-call-id"; import { idleDeadline } from "../lib/abort"; +import { httpStatusFromTerminalError } from "../lib/errors"; import { CLAUDE_NATIVE_THINKING, projectClaudeRequest, @@ -86,6 +87,9 @@ export function tapAnthropicSseForLog( let usageAcc: Rec = {}; // message_stop or an upstream error event: the turn's own terminal already went through. let terminalSeen = false; + let terminalErrorStatus: number | undefined; + const terminalMeta = (): PassthroughFinalizeMeta => terminalErrorStatus === undefined + ? { closeReason: "terminal" } : { closeReason: "terminal", terminalStatus: "failed" }; const inspectFrame = (frame: string) => { const dataLine = frame .split("\n") @@ -103,6 +107,10 @@ export function tapAnthropicSseForLog( usageAcc = { ...usageAcc, ...data.usage }; } else if (data.type === "message_stop" || data.type === "error") { terminalSeen = true; + if (data.type === "error" && isRec(data.error)) { + terminalErrorStatus = httpStatusFromTerminalError(data.error) ?? 502; + logCtx.upstreamError = redactSecretString(typeof data.error.message === "string" ? data.error.message : "Provider stream failed").slice(0, 500); + } } }; const inspect = (chunk: Uint8Array) => { @@ -158,7 +166,7 @@ export function tapAnthropicSseForLog( // The turn already ended (message_stop or an upstream error event): an upstream that then // idles or keeps sending did not cut it short. Same rule as the read-error branch, // including the restored blank line for a terminal found only in the tail. - finalize(200, { closeReason: "terminal" }); + finalize(terminalErrorStatus ?? 200, terminalMeta()); try { if (terminalInTail) tapController?.enqueue(encoder.encode("\n\n")); tapController?.close(); @@ -189,6 +197,7 @@ export function tapAnthropicSseForLog( settled = true; idle.cancel(); detachAbort(); + recordUsage(); finalize(499, { closeReason: "client_cancel" }); try { tapController?.close(); } catch { /* downstream already torn down */ } reader.cancel(guard?.reqSignal?.reason).catch(() => {}); @@ -220,7 +229,7 @@ export function tapAnthropicSseForLog( idle.cancel(); detachAbort(); recordUsage(); - finalize(200, { closeReason: "terminal" }); + finalize(terminalErrorStatus ?? 200, terminalMeta()); controller.close(); return; } @@ -254,14 +263,14 @@ export function tapAnthropicSseForLog( if (isTranslatorBudgetExceededError(err)) { // A local cap, not an upstream failure: the non-streaming native Messages fold // maps this error to a 413 itself, so it still errors the stream. - finalize(200, { closeReason: "terminal" }); + finalize(terminalErrorStatus ?? 200, terminalMeta()); try { controller.error(err); } catch { /* torn down */ } return; } if (terminalSeen) { // Only the transport trailer was lost; the client already has the turn's terminal. // The Responses relay likewise reports a read error only without a seen terminal. - finalize(200, { closeReason: "terminal" }); + finalize(terminalErrorStatus ?? 200, terminalMeta()); try { // An SSE parser drops an event that EOF cuts off before its blank line, so restore // the delimiter when the terminal was only found in that unterminated tail. @@ -291,6 +300,7 @@ export function tapAnthropicSseForLog( settled = true; idle.cancel(); detachAbort(); + recordUsage(); finalize(499, { closeReason: "client_cancel" }); } reader.cancel(reason).catch(() => {}); diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index cf2c4afd8ee..0f873ec4ec4 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -1,8 +1,9 @@ /** - * Native Chat candidates inside the combo loop (PF-07). + * Native Chat and Messages candidates inside the combo loop. * - * The Chat ingress hands the combo a `ComboProtocolSource` only for a combo route with - * `protocols.rollout.nativeChatCombos` on; without one nothing here runs. For the candidate the + * Each opted-in ingress hands the combo its `ComboProtocolSource`; without one nothing here + * runs. Messages supports canonical strict failover children without effort rewriting. + * For the candidate the * combo is about to dispatch, the concrete route is settled the way the Chat ingress settles a * single route, and the native Chat lane's own eligibility rule decides whether the child goes * native. A native child runs `runNativeChatAttempt` (through the source) on the attempt the combo @@ -40,7 +41,11 @@ import { resolveOpenCodeGoTransport } from "../../providers/opencode-go-transpor import { getOrAllocateRequestSessionLane } from "../request-log-conversation"; import { assertRouteAllowedByScope } from "../admission-model-scope"; import { isNativeChatRouteEligible } from "../chat-native-eligibility"; +import { nativeMessagesDeclineReason, type NativeMessagesSelector } from "../messages-native-eligibility"; +import type { NativeMessagesFinishLog } from "../messages-native"; +import { credentialDomainFor } from "../../protocols/opaque-state"; import { chatCompletionsErrorResponse } from "../../chat/outbound"; +import { anthropicErrorResponse } from "../../claude/outbound"; import { isRequestExecutionBudget } from "../../lib/request-execution-budget"; import { markResponseNonReplayable } from "../../lib/upstream-retry"; import { redactSecretString } from "../../lib/redact"; @@ -57,7 +62,7 @@ type ComboTarget = { provider: string; model: string }; /** What the source needs to run one native child on the combo's open attempt. */ export interface NativeComboChildRun { route: RouteResult; - /** A fresh copy of the source Chat body, never another candidate's rewritten one. */ + /** A fresh copy of the source body, never another candidate's rewritten one. */ body: Rec; childLog: RequestLogContext; attemptHandle: InferenceAttempt; @@ -70,13 +75,22 @@ export interface NativeComboChildRun { } /** Supplied by the Chat ingress for a combo route when `nativeChatCombos` is on. */ -export interface ComboProtocolSource { +interface NativeChatComboSource { readonly inbound: "chat"; readonly envelope: ProtocolEnvelope; /** Runs `runNativeChatAttempt`; the response it returns is in the Chat wire. */ dispatchNativeChild(input: NativeComboChildRun): Promise; } +interface NativeMessagesComboSource { + readonly inbound: "messages"; + readonly envelope: ProtocolEnvelope; + readonly selector?: NativeMessagesSelector; + dispatchNativeChild(input: Omit & { finishLog: NativeMessagesFinishLog }): Promise; +} + +export type ComboProtocolSource = NativeChatComboSource | NativeMessagesComboSource; + /** The native plan for one dispatch: the settled route and its own fresh body. */ export interface NativeComboChildPlan { route: RouteResult; @@ -117,6 +131,8 @@ export function createComboProtocolLanes(input: { const { source, req, config, logCtx, admissionScope, droidDefaultEffort, comboId, targets } = input; if (!source) return undefined; const envelope = source.envelope; + const inbound = source.inbound; + const inboundWire = inbound === "messages" ? "anthropic" : "chat"; const reject = resolveProtocolSettings(config).unrepresentable === "reject"; const selector = (target: ComboTarget): string => `${target.provider}/${target.model}`; @@ -129,9 +145,9 @@ export function createComboProtocolLanes(input: { assertRouteAllowedByScope(admissionScope, selector(target), route); const routedProvider = route.provider; route.staticPolicy = captureRouteStaticPolicy( - route.providerName, route.modelId, routedProvider, route.staticPolicy.effectiveAlias, "chat", + route.providerName, route.modelId, routedProvider, route.staticPolicy.effectiveAlias, inboundWire, ); - const wireProvider = resolveWireProtocolOverride(route.providerName, route.modelId, routedProvider, "chat", route.staticPolicy); + const wireProvider = resolveWireProtocolOverride(route.providerName, route.modelId, routedProvider, inboundWire, route.staticPolicy); route.provider = resolveOpenCodeGoTransport(wireProvider, getOrAllocateRequestSessionLane(req), routedProvider); return route; } catch { @@ -142,12 +158,21 @@ export function createComboProtocolLanes(input: { // Eligibility reads a body, and a body is a charged copy, so the adapter is checked first: // a candidate that can never go native costs no copy at all. const nativeBody = (target: ComboTarget, route: RouteResult | undefined): Rec | undefined => { - if (!route || route.provider.adapter !== "openai-chat") return undefined; + if (!route || route.provider.adapter !== (inbound === "messages" ? "anthropic" : "openai-chat")) return undefined; + if (inbound === "messages") { + const combo = getCombo(config, comboId); + // ponytail: exact replay supports no combo effort rewrite; the existing bridge owns those policies. + if (!combo || combo.strategy !== "failover" || comboDefaultEffort(config, comboId) + || combo.reasoningEffortMode !== "strict" || !credentialDomainFor(route.provider)?.firstPartyAnthropic + || supportedLadderFor({ provider: route.provider, modelId: route.modelId })?.length === 0) return undefined; + } const body = envelope.freshBody(); // The concrete selector, as the bridge child's body carries it; the wire model comes from // the route either way, and the pinned-effort lookup reads this. body.model = selector(target); - return isNativeChatRouteEligible(route, body, config) ? body : undefined; + return (inbound === "messages" + ? nativeMessagesDeclineReason(route, body, config, { ...(source.inbound === "messages" ? source.selector : undefined), routeSelector: selector(target) }) === undefined + : isNativeChatRouteEligible(route, body, config)) ? body : undefined; }; const verdicts = new Map(); @@ -167,10 +192,10 @@ export function createComboProtocolLanes(input: { if (targetRoute) { const settled = settle(target, targetRoute); const native = nativeBody(target, settled) !== undefined; - const upstream = native ? "chat" : upstreamWireForAdapter((settled ?? targetRoute).provider.adapter); + const upstream = native ? inbound : upstreamWireForAdapter((settled ?? targetRoute).provider.adapter); const checked = checkRepresentable({ - inbound: "chat", - requestPath: requestPathForLane("chat", native ? "native" : "bridge", upstream), + inbound, + requestPath: requestPathForLane(inbound, native ? "native" : "bridge", upstream), features: envelope.features(), policy: "reject", }); @@ -200,11 +225,11 @@ export function createComboProtocolLanes(input: { const features = [...new Set(skipped.flatMap(verdict => verdict.skip ?? []))]; // The same refusal the ingress gives a single route: 400 in the Chat shape, feature keys // only, a blocked trace, and no upstream send. - markProtocolBlocked(logCtx, { inbound: "chat", reasonCodes: ["feature-unrepresentable"], features }); + markProtocolBlocked(logCtx, { inbound, reasonCodes: ["feature-unrepresentable"], features }); logCtx.errorCode = "unsupported_feature"; - return markClientWire(chatCompletionsErrorResponse( + return markClientWire((inbound === "messages" ? anthropicErrorResponse : chatCompletionsErrorResponse)( 400, unrepresentableMessage(features), "invalid_request_error", "unsupported_feature", - ), "chat"); + ), inbound); }, nativeChild: (target, targetRoute, targetSendBudget, initialDecision) => { // Without the request's execution budget a native child could not share its sends, so it @@ -213,6 +238,7 @@ export function createComboProtocolLanes(input: { const route = settle(target, targetRoute); const body = nativeBody(target, route); if (!route || !body) return undefined; + if (inbound === "messages") return initialDecision ? undefined : { route, body, sendBudget: targetSendBudget }; applyDroidReasoningDefault(body, droidDefaultEffort, { provider: targetRoute.provider, modelId: targetRoute.modelId, @@ -280,7 +306,7 @@ export interface ComboChildCallbacks { } /** - * Run one native child on the attempt the combo opened, and mark its answer as Chat wire. + * Run one native child on the attempt the combo opened, and mark its client wire. * * The child's outcomes map onto the combo's existing child callbacks: a terminal or cancellation * after the response was handed over publishes through the gate (and so reaches the parent's @@ -301,10 +327,11 @@ export async function dispatchNativeComboChild(input: { callbacks: ComboChildCallbacks; }): Promise { const { plan, logCtx, childLog, attempt, startedAt, callbacks } = input; + const inbound = input.source.inbound; childLog.providerAdapter = plan.route.provider.adapter; markAttemptProtocolPath(attempt, { - mode: deliveryModeForLane("chat", "native", "chat"), - requestPath: requestPathForLane("chat", "native", "chat"), + mode: deliveryModeForLane(inbound, "native", inbound), + requestPath: requestPathForLane(inbound, "native", inbound), }); let outputSeen = false; const finishLog: NativeChatFinishLog = (status, message, closeReason = "non_stream") => { @@ -331,7 +358,7 @@ export async function dispatchNativeComboChild(input: { seal: label => sealRequestAttemptIdentity(attempt, childLog.provider, childLog.providerAdapter ?? attempt.adapter, label), finish: (status, usage) => finishRequestAttempt(attempt, status, Date.now() - startedAt, usage), }; - const response = await input.source.dispatchNativeChild({ + const child = { route: plan.route, body: plan.body, childLog, @@ -343,8 +370,19 @@ export async function dispatchNativeComboChild(input: { outputSeen = true; input.onFirstOutput(); }, - finishLog, - }); + }; + const response = await (input.source.inbound === "messages" + ? input.source.dispatchNativeChild({ ...child, finishLog: (status, message, meta) => { + if (meta?.terminalStatus === "incomplete") { + childLog.terminalHttpStatus = status; + if (childLog.usage !== undefined) logCtx.usage = childLog.usage; + callbacks.onTerminal("incomplete"); + return; + } + finishLog(status, message, meta?.closeReason === "client_cancel" ? "client_cancel" + : meta?.closeReason && meta.closeReason !== "non_stream" ? "terminal" : "non_stream"); + } }) + : input.source.dispatchNativeChild({ ...child, finishLog })); if (!response.ok && outputSeen) markResponseNonReplayable(response); - return markClientWire(response, "chat"); + return markClientWire(response, inbound); } diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts index d64bb0e0e70..879d43bed67 100644 --- a/src/server/responses/core-combo.ts +++ b/src/server/responses/core-combo.ts @@ -399,7 +399,7 @@ export async function executeComboResponses( // internal request carries the caller's captured scope on options.admissionScope even though // its own admission is the host's broad key; recomputing from options.admission would lose it. const admissionScope = options.admissionScope ?? resolveAdmissionModelScope(config, options.admission); - // PF-07: present only for a Chat combo with `nativeChatCombos` on; otherwise every child + // Present only when an opted-in ingress supplied a native source; otherwise every child // takes the bridge below exactly as before. const protocolLanes = createComboProtocolLanes({ source: options.protocolSource, @@ -1184,7 +1184,7 @@ export async function executeComboResponses( // A native Chat child reports a pre-stream failure by status before any byte, so its body // is never peeked; a non-OK one takes the ordinary failure path below. - if (response.ok && !runTurnAdapterSseResponses.has(response) && clientWireOf(response) !== "chat") { + if (response.ok && !runTurnAdapterSseResponses.has(response) && clientWireOf(response) === undefined) { const nativePassthrough = isNativePassthroughSseResponse(response); const eagerRelay = isEagerRelaySseResponse(response); let preflight; diff --git a/src/server/responses/core-options.ts b/src/server/responses/core-options.ts index 034227c2878..9316086ab05 100644 --- a/src/server/responses/core-options.ts +++ b/src/server/responses/core-options.ts @@ -193,7 +193,7 @@ export interface HandleResponsesOptions { anthropicToolLoadingSource?: Readonly>; /** Droid's per-request effort default; each concrete combo or policy target applies it only if its ladder allows it. */ droidDefaultEffort?: string; - /** PF-07: the Chat source a combo child may send natively; set only by the Chat ingress. */ + /** The Chat or Messages source a combo child may send natively; set only by that ingress. */ protocolSource?: import("./core-combo-native").ComboProtocolSource; /** Internal transport identity for route-scoped upstream compatibility policy. */ inboundTransport?: "websocket"; diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 68ae998da50..27390c4a85e 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -378,12 +378,13 @@ replay config; backend credentials are decided at dispatch, so this errs toward ingress, `count_tokens` and the planner all ask it. The planner can judge only the config-and-route parts: skill elision and web search depend on body content no feature describes. +Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. + With the switch on, a declined route re-marks its bridge entry with the decline reason (after any `effort-row` / `fast-row`), so the trace says why the bridge was taken; with it off nothing is added. `claudeCode.stabilizePromptCache` is not a decline rule: it is a Claude-app cache optimization rather than routing policy, applies only on the translated path, and is a recorded gap of the native lane. - `src/server/claude-messages.ts` decides the lane after the route and its wire settle and after the managed-client steps already applied to the body (alias/modelMap resolution, `ocx-route`, effort directives). The caller-forward passthrough is decided earlier, on the caller's own credential, @@ -441,8 +442,7 @@ The TypeScript Agent SDK's `sdk-ts` entrypoint is accepted alongside `cli`, `sdk with the same required headers and bounds, so its first-party native feature betas are retained. Duplicates, oversized/invalid values and headers named by `Connection` cannot gain forwarding authority. The handle stores its headers privately in a WeakMap and serializes without them. -The native builder applies it only for first-party Anthropic, independently of bearer/UUID -selection. Operator `provider.headers` names take precedence case-insensitively over the observed +The native builder applies it only for first-party Anthropic, independently of bearer/UUID selection. Operator `provider.headers` names take precedence case-insensitively over the observed identity bundle without duplicate spelling; unconfigured names retain the observed client values. Caller credentials, proxy/hop headers, arbitrary SDK names and betas are never part of the bundle. These are compatibility observations, not authorization or proof of client provenance. Missing diff --git a/structure/providers-and-adapters.md b/structure/providers-and-adapters.md index 0d0488f0bb0..6bdc15c96c9 100644 --- a/structure/providers-and-adapters.md +++ b/structure/providers-and-adapters.md @@ -205,7 +205,7 @@ rewrite rules and the routed-id settlement. | `src/adapters/openai-responses/transport.ts` | Shared Responses URL, configured auth and caller-header construction for ordinary requests and explicitly enabled custom native v1 compact. Its compact-handoff metadata copier uses the existing client-header allowlist, never credentials, through internal request construction and Codex auth materialization. Protocol capability never grants canonical identity; v2 trigger and native blob replay remain independently declared. On the noncanonical forward path, configured `baseUrl` trailing-slash stripping is a linear scan that preserves interior slash runs and leaves the canonical forward base and non-forward auth modes untouched; `tests/responses/responses-forward-transport-baseurl.test.ts` pins it. | | `src/responses/muse-tool-name-alias.ts` | Host-gated Meta Muse 64-char tool-name alias/restore used by the Responses passthrough. | | `src/adapters/openai-chat.ts`, `src/adapters/openai-chat/` | OpenAI-compatible Chat Completions bridge, split into leaves (`wire.ts`, `messages.ts`, `response-events.ts`, `passthrough.ts`, `parallel-tool-calls.ts`, `reasoning-wire.ts`, `serialized-tool-call-content.ts`, `serialized-tool-call-markup.ts`, `standalone-bare-tool-calls.ts`, `tool-call-validation.ts`, `tool-call-id-remint.ts`, `tool-schema.ts`, `errors.ts`). `parallel-tool-calls.ts` owns the `parallel_tool_calls` wire value for both the translated and native builders, so the three provider states — configured opt-out, configured opt-in, and the unset default that forwards only a caller's explicit `false` — cannot drift between them. `reasoning-wire.ts` applies explicit gateway-object and tool-bearing effort-omission declarations to both builders; absent declarations leave native raw forwarding unchanged. Its client delivery shapes in `src/chat/outbound.ts` and `src/server/chat-native-sse.ts` relay the upstream `service_tier` echo on non-stream, folded-stream, and synthesized-SSE bodies, never inventing the key when the upstream omits it. | -| `src/adapters/anthropic.ts` | Anthropic Messages bridge. A `refusal` or `content_filter` stop reason yields an explicit `incomplete` event with `retryable: false` rather than `done` with that stopReason (#4312); `max_tokens` remains `done`. For a Responses client the bridge delivers that event, and the same one from the Kiro adapter, as a terminal `response.failed` ([Adapter-to-Responses bridge](transports/responses.md#adapter-to-responses-bridge)). It is the wire that defines `tools[*].strict` and `tools[*].allowed_callers`, so a rebuilt declaration carries both: an explicit `strict: true` and any `allowed_callers` the caller declared. An absent `strict` stays absent, because the Messages inbound records it as `false` and a `false` on the wire would read as an opt-out nobody asked for. Anthropic Fast uses the native `anthropic-speed` FastWire: a set decision sends `speed: "fast"` with `fast-mode-2026-02-01` in one case-insensitively merged, deduplicated `anthropic-beta` header that preserves OAuth betas. Stream and buffered `usage.speed` echoes confirm fast or downgrade to standard; no echo leaves the request assumed. `tests/adapters/anthropic/anthropic-fast-speed.test.ts` pins the wire and echoes. Anthropic Fast is opt-in: the registry marks both Anthropic entries `fastOptIn`, and `src/providers/fast-opt-in.ts` (`providerFastSwitchOff`) keeps Fast off until `providers..fastEnabled` is `true`. An off switch is provider capability `false`, applied in the FastPolicy authority (`service-tier.ts`), `resolveModelPolicy`, and router registry enrichment, so no model-level Fast toggle, `--fast` row, or proxy-generated `speed` field is produced. Native Claude Messages passthrough still forwards a `speed` field the caller sends itself, outside the proxy Fast policy. `tests/adapters/anthropic/anthropic-fast-opt-in.test.ts` pins the default, the switch, and the management PATCH/GET. For `api.anthropic.com` only, the Claude client's typed web tools (`parsed._anthropicServerTools`) follow the function tools unchanged, minus the caller's `cache_control`; a function tool of the same name wins, so the web-search sidecar's `web_search` keeps priority and `web_fetch` is still carried. No beta header is added: both tools are GA. Each streamed `web_search_tool_result` or `web_fetch_tool_result` becomes a `web_search_call_begin`/`web_search_call_end` pair (the query of its `server_tool_use` in the same response, hit URLs and titles, or `failed` for an error result) carrying `anthropic`: that `server_tool_use` with its complete streamed input and the result block, verbatim. An orphan result, a result with a different call id, or a result whose call input cannot be parsed is dropped with no synthetic call; the surrounding text is preserved. Streamed server-tool input retains at most 4,096 characters, slicing each incoming fragment to the remaining capacity; an oversized input that cannot be recovered drops its exchange under the same rule. The bridge and the direct encoder keep both on the `web_search_call` item as `anthropic_blocks` (a fetch is an `open_page` action), and the Claude outbound renders them back unchanged, with the original `srvtoolu_` id, `encrypted_content` and `page_age`; a fetch is not counted in `web_search_requests`. A search Anthropic defers beside a client `tool_use` has no result in that response and surfaces nothing. On the next turn, `messagesToAnthropicFormat` re-inserts the client's replayed pairs (`parsed._anthropicServerReplay`) ahead of the `tool_use`, text or thinking block that followed them, for `api.anthropic.com` only. An anchor must occur exactly once across assistant history, counting string and block content alike; repeated anchors drop their exchanges and preserve the other turns. Replay requires one-to-one call/result ids and matching tool types within the same assistant turn, with the call preceding its result in the original sequence. A single forward pass tracks pending calls and rejects reversed pairs, duplicate ids and orphan results. Calls require a plain input object with a nonempty `web_search.query` or `web_fetch.url` string; malformed input, a pair without `encrypted_content`, or a pair with no following anchor stays dropped. Stored `anthropic_blocks` restore only when call id, result id, tool/result types and input object agree; malformed metadata falls back to the rebuilt pair. Citations, a deferred search and `pause_turn` are not resumed. `tests/claude-integration/claude-translated-server-tools.test.ts` pins the wire, the sidecar priority, the bridge's client stream and JSON, and the direct encoder's stream. Prompt-cache TTL: an omitted `cacheRetention` resolves through `src/adapters/anthropic/cache-retention.ts` to 1h (`ttl: "1h"`) for first-party `api.anthropic.com` (API key and OAuth pools) and to 5m for Anthropic-compatible endpoints, which do not document the 1h TTL; an explicit value wins everywhere. The native Messages passthrough upgrades caller breakpoints without a TTL to 1h under the same rule unless the caller pinned any breakpoint to another TTL. Usage carries `cache_creation.ephemeral_1h_input_tokens` as `cacheCreation1hInputTokens`, which `src/usage/cost.ts` prices at 1.6x the 5m write rate (2x base input). `tests/adapters/adapter-usage.test.ts`, `tests/adapters/anthropic/anthropic-messages-passthrough.test.ts` and `tests/claude-integration/claude-translated-cache-ttl.test.ts` pin the defaults. | +| `src/adapters/anthropic.ts` | Anthropic Messages bridge. A `refusal` or `content_filter` stop reason yields an explicit `incomplete` event with `retryable: false` rather than `done` with that stopReason (#4312); `max_tokens` remains `done`. For a Responses client the bridge delivers that event, and the same one from the Kiro adapter, as a terminal `response.failed` ([Adapter-to-Responses bridge](transports/responses.md#adapter-to-responses-bridge)). It is the wire that defines `tools[*].strict` and `tools[*].allowed_callers`, so a rebuilt declaration carries both: an explicit `strict: true` and any `allowed_callers` the caller declared. An absent `strict` stays absent, because the Messages inbound records it as `false` and a `false` on the wire would read as an opt-out nobody asked for. Anthropic Fast uses the native `anthropic-speed` FastWire: a set decision sends `speed: "fast"` with `fast-mode-2026-02-01` in one case-insensitively merged, deduplicated `anthropic-beta` header that preserves OAuth betas. Stream and buffered `usage.speed` echoes confirm fast or downgrade to standard; no echo leaves the request assumed. `tests/adapters/anthropic/anthropic-fast-speed.test.ts` pins the wire and echoes. Anthropic Fast is opt-in: the registry marks both Anthropic entries `fastOptIn`, and `src/providers/fast-opt-in.ts` (`providerFastSwitchOff`) keeps Fast off until `providers..fastEnabled` is `true`. An off switch is provider capability `false`, applied in the FastPolicy authority (`service-tier.ts`), `resolveModelPolicy`, and router registry enrichment, so no model-level Fast toggle, `--fast` row, or proxy-generated `speed` field is produced. Native Claude Messages passthrough still forwards a `speed` field the caller sends itself, outside the proxy Fast policy. `tests/adapters/anthropic/anthropic-fast-opt-in.test.ts` pins the default, the switch, and the management PATCH/GET. For `api.anthropic.com` only, the Claude client's typed web tools (`parsed._anthropicServerTools`) follow the function tools unchanged, minus the caller's `cache_control`; a function tool of the same name wins, so the web-search sidecar's `web_search` keeps priority and `web_fetch` is still carried. No beta header is added: both tools are GA. Each streamed `web_search_tool_result` or `web_fetch_tool_result` becomes a `web_search_call_begin`/`web_search_call_end` pair (the query of its `server_tool_use` in the same response, hit URLs and titles, or `failed` for an error result) carrying `anthropic`: that `server_tool_use` with its complete streamed input and the result block, verbatim. An orphan result, a result with a different call id, or a result whose call input cannot be parsed is dropped with no synthetic call; the surrounding text is preserved. Streamed server-tool input retains at most 4,096 characters, slicing each incoming fragment to the remaining capacity; an oversized input that cannot be recovered drops its exchange under the same rule. The bridge and the direct encoder keep both on the `web_search_call` item as `anthropic_blocks` (a fetch is an `open_page` action), and the Claude outbound renders them back unchanged, with the original `srvtoolu_` id, `encrypted_content` and `page_age`; a fetch is not counted in `web_search_requests`. A search Anthropic defers beside a client `tool_use` has no result in that response and surfaces nothing. On the next turn, `messagesToAnthropicFormat` re-inserts the client's replayed pairs (`parsed._anthropicServerReplay`) ahead of the `tool_use`, text or thinking block that followed them, for `api.anthropic.com` only. An anchor must occur exactly once across assistant history, counting string and block content alike; repeated anchors drop their exchanges and preserve the other turns. Replay requires one-to-one call/result ids and matching tool types within the same assistant turn, with the call preceding its result in the original sequence. A single forward pass tracks pending calls and rejects reversed pairs, duplicate ids and orphan results. Calls require a plain input object with a nonempty `web_search.query` or `web_fetch.url` string; malformed input, a pair without `encrypted_content`, or a pair with no following anchor stays dropped. Stored `anthropic_blocks` restore only when call id, result id, tool/result types and input object agree; malformed metadata falls back to the rebuilt pair. Citations, a deferred search and `pause_turn` are not resumed. `tests/claude-integration/claude-translated-server-tools.test.ts` pins the wire, the sidecar priority, the bridge's client stream and JSON, and the direct encoder's stream. Prompt-cache TTL: an omitted `cacheRetention` resolves through `src/adapters/anthropic/cache-retention.ts` to 1h (`ttl: "1h"`) for first-party `api.anthropic.com` (API key and OAuth pools) and to 5m for Anthropic-compatible endpoints, which do not document the 1h TTL; an explicit value wins everywhere. The direct native Messages passthrough upgrades caller breakpoints without a TTL to 1h under the same rule unless the caller pinned any breakpoint to another TTL; exact native combo replay preserves those source breakpoints unless the operator explicitly configures retention. Usage carries `cache_creation.ephemeral_1h_input_tokens` as `cacheCreation1hInputTokens`, which `src/usage/cost.ts` prices at 1.6x the 5m write rate (2x base input). `tests/adapters/adapter-usage.test.ts`, `tests/adapters/anthropic/anthropic-messages-passthrough.test.ts` and `tests/claude-integration/claude-translated-cache-ttl.test.ts` pin the defaults. | | `src/adapters/anthropic-model-contract.ts` | Per-family Anthropic Messages wire rules, shared by the adapter and the web-search and vision sidecars, measured live on 2026-09-29, with Haiku 5.5 documented on 2026-10-08: adaptive vs budget thinking (Haiku 5.5+ is adaptive); explicit `thinking: disabled` (Sonnet 5.0 up to but excluding 5.5, Haiku 5.5+ without effort); the `between_tools` floor that replaces it on Sonnet 5.5+ (sent without an effort, since xhigh/max reject it); forced `tool_choice` downgraded to `auto` on Opus 5.5, Fable 5.1+ and Sonnet 5.5+; `temperature`/`top_p` dropped on Opus 4.7+, Sonnet 5+, Haiku 5.5+ and every Fable, which 400 on any non-default value; `top_p` dropped when sent with `temperature` on the 4.5/4.6 families, which take either alone but not both; and sidecar thinking-off fields, a low effort with no `thinking` for Opus 5.5 and Fable, which reject both off switches. Dotted Bedrock ids do not parse as a family, and the Messages-native passthrough forwards caller fields unchanged. `tests/adapters/anthropic/anthropic-sonnet-5-5-contract.test.ts` and `tests/adapters/anthropic/anthropic-haiku-5-5-contract.test.ts` pin the family table. | | `src/adapters/google.ts` | Gemini bridge. The final wire compiler owns [endpoint-scoped tool-schema loss policy](providers/google.md#google-tool-schema-loss-reporting): compatible mode changes no request bytes, strict initial loss creates no physical send, and strict non-direct repair creates no changed repair send. A caller-declared strict tool selects `functionCallingConfig.mode: "VALIDATED"` in place of the absent-choice default; `NONE`, `ANY` and a forced-name choice are stronger constraints the caller asked for and are never overwritten. | | `src/adapters/unique-tool-call-ids.ts` | Request-scoped tool-call-id uniqueness for every `openai-chat` provider. An upstream that mints an id from the call's position in its response repeats `call-0-0` on every turn; a Messages client has already paired that id, drops the duplicate, and is left with a call that has no result, so the turn reads as empty and the model re-issues it indefinitely. Only a **repeat** is rewritten — the first occurrence stays byte-identical, leaving prompt-cache keys, reasoning-replay lookups and already-unique upstreams untouched. The ids to avoid come from the caller's history, captured in `buildRequest` (the only point that sees it) and applied at emission, never at ingestion: ingestion matches streamed deltas against the id upstream sent, so rewriting there would strip a pending call of its identity mid-stream. A repeat takes a `-` suffix, never `_`, because `_` reads as a batch sub-call of ``; occupied-set search resumes by suffix width and retained base prefix, including when siblings converge as `-9` becomes `-10`. Covered by `tests/adapters/openai/openai-chat-tool-call-id-remint.test.ts`. | diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 0068233f701..58a8e46e02a 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -116,6 +116,16 @@ redacted before whitespace flattening and bounded to 200 characters; unparsed bo A `runTurn` adapter has the equivalent boundary in `preflightAdapterEvents` (`src/adapters/run-turn-queue.ts`), streaming and non-streaming alike: when its first meaningful event is a tool call the current request did not declare, and no earlier replay-unsafe heartbeat recorded a side effect, `src/server/responses/run-turn-execution.ts` projects the fail-closed undeclared-tool refusal as a pre-commit 502 so the combo can hop with the unchanged catalog. After any output or a replay-unsafe heartbeat the refusal stays with that child. Chat Completions and Anthropic Messages inbound requests do not use this classification. The same heartbeat also decides an ordinary pre-output adapter error or an empty end: after a replay-unsafe heartbeat the child's 502 is marked non-replayable, so the combo stops on it instead of sending the turn to the next target. +A native Messages child uses `src/server/messages-native-preflight.ts` after the native log tap, +so the tap retains byte-stall/size/cancellation ownership. Its bounded prelude accepts only an +empty-content `message_start` and `ping` as replayable. A structured retryable `error` before +any other event returns to the ordinary combo failure classifier; any content block (including +a tool start), unknown/malformed payload, terminal or byte/chunk cap commits conservatively. +Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable. +Preflight retention is charged to the translator budget and released on delivery/cancellation. +The same native executor binds the accepted stream to the parent's held lease, carries observed +usage on cancellation, and reports terminal errors without treating them as successful rows. + HTTP 410 remains terminal by default. It advances and cools only the exact combo target when the structured code or message explicitly identifies a model lifecycle event (end-of-life, retired, deprecated, sunset, decommissioned, or no longer available). An unrelated application-level 410 is diff --git a/structure/transports/responses-spend.md b/structure/transports/responses-spend.md index 83b6912bca8..0f44220b54a 100644 --- a/structure/transports/responses-spend.md +++ b/structure/transports/responses-spend.md @@ -89,7 +89,9 @@ physical send back through `onSendsConsumed`; that is what `countedExternally: t reporter's first send settles only its named pending permit instead of adding a second charge. `transientSendReporter` captures that permit before entering a helper; a later handoff cannot replace it. `reportDispatchSends` verifies shared-ledger ownership and consumes one receipt only. -Native Chat combo children carry the same exact permit to their physical-send boundary. +Native Chat and Messages combo children carry the same exact permit to their physical-send boundary. +Native Messages inherits the per-target budget and opens no second tracker; enforced starts use +the shared reporter, and observe-only starts confirm the exact permit at the final HTTP receipt. Reset-only generic combo helpers report their named prepaid receipt too, without changing the selected retry cap; compaction reconciliation therefore does not count that source again. Without enforced spend, numeric `used` updates and unnamed, foreign, released or already-reported permits charge diff --git a/structure/transports/responses.md b/structure/transports/responses.md index 810c8d782e4..0543aeff94d 100644 --- a/structure/transports/responses.md +++ b/structure/transports/responses.md @@ -568,8 +568,8 @@ in the Chat shape, blocked trace) with no send. `n > 1` is never emulated with s inferences. Policy routes resolve one candidate in `routeModel` and never reach this loop, so they are not migrated. `tests/responses/chat-native-combo.test.ts` pins the source body, failover within the shared budget, no resend after output, the switch-off path and both reject outcomes. -Managed native Messages ([Protocol Paths](../data-planes/protocol-paths.md#managed-native-messages)) -reuses `beginInferenceAttempt` and `createFinalRequestLog` with its own 401/429 key-rotation loop. +Managed native Messages uses the same seam and shared executor for [eligible canonical failover children](../data-planes/protocol-paths.md#managed-native-messages), with the combo's existing attempt, budget/permit and gated final-row callbacks. +Messages successes are marked `messages`, bypass Responses inspection/translation and use their own [bounded preflight](responses-combo-boundaries.md); standalone ingress keeps its own attempt/final row. ## Adapter-to-Responses bridge diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts new file mode 100644 index 00000000000..61fa7dccaef --- /dev/null +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -0,0 +1,277 @@ +import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { handleClaudeMessages } from "../../src/server/claude-messages"; +import { clearComboSelectionState, clearComboTargetCooldowns } from "../../src/combos"; +import { clearComboRecallForTests } from "../../src/server/responses/combo-session-recall"; +import { clearKeyCooldowns } from "../../src/providers/key-failover"; +import { getRequestLogEntries } from "../../src/server/request-log"; +import { activeRegistryMetrics, getActiveTurnCount, tryAdmitTurn } from "../../src/server/lifecycle"; +import { saveCredential } from "../../src/oauth/store"; +import { clearAnthropicAccountPoolState, forgetAnthropicFailoverQuorum } from "../../src/oauth/anthropic-routing"; +import { configureSharedSpendLedger, spendPolicyFromConfig } from "../../src/lib/spend-reservation-ledger"; +import { clearResponseStateForTests, flushResponseState } from "../../src/responses/state"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "../helpers/isolated-codex-home"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import type { OcxConfig } from "../../src/types"; + +type Rec = Record; +const source = { + model: "combo/waterfall", max_tokens: 64, top_k: 7, + system: [{ type: "text", text: "stable system prefix", cache_control: { type: "ephemeral", ttl: "1h" } }], + tools: [{ name: "lookup", description: "stable tool", input_schema: { type: "object", properties: { query: { type: "string" } } }, cache_control: { type: "ephemeral" } }], + messages: [{ role: "user", content: [{ type: "text", text: "stable user prefix", cache_control: { type: "ephemeral" } }] }], +}; +const message = { + id: "msg_fixture", type: "message", role: "assistant", model: "claude-sonnet-4-6", + content: [{ type: "text", text: "native reply" }], stop_reason: "end_turn", stop_sequence: null, + usage: { input_tokens: 11, output_tokens: 3, cache_read_input_tokens: 9 }, +}; +const frame = (type: string, data: Rec = {}) => `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`; +const prelude = frame("message_start", { message: { ...message, content: [], stop_reason: null } }); +const textStart = frame("content_block_start", { index: 0, content_block: { type: "text", text: "" } }); +const textDelta = frame("content_block_delta", { index: 0, delta: { type: "text_delta", text: "native reply" } }); +const terminal = frame("content_block_stop", { index: 0 }) + frame("message_delta", { + delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 3 }, +}) + frame("message_stop"); +const stream = (text: string) => new Response(text, { headers: { "content-type": "text/event-stream" } }); +const refusal = frame("error", { error: { type: "overloaded_error", message: "fixture overloaded" } }); + +let home = ""; +let previousHome: string | undefined; +let codex: IsolatedCodexHome; +let releaseSpend: () => void; +let fetchMock: ReturnType>; +let seen: { url: string; body: Rec; headers: Headers }[]; +let answer: (body: Rec, native: boolean) => Response; + +beforeEach(() => { + previousHome = process.env.OPENCODEX_HOME; + home = mkdtempSync(join(tmpdir(), "ocx-native-messages-combo-")); + process.env.OPENCODEX_HOME = home; + codex = installIsolatedCodexHome("ocx-native-messages-combo-codex-"); + releaseSpend = acquireOwnedSpendHome(); + clearComboSelectionState(); clearComboTargetCooldowns(); clearComboRecallForTests(); clearKeyCooldowns(); clearAnthropicAccountPoolState(); forgetAnthropicFailoverQuorum(); clearResponseStateForTests(); + seen = []; + answer = body => body.stream === true ? stream(prelude + textStart + textDelta + terminal) : Response.json(message); + // All destinations, including canonical Anthropic, are intercepted: no paid generation. + const physicalFetch = globalThis.fetch; + fetchMock = spyOn(globalThis, "fetch").mockImplementation(async (input, init) => { + const request = new Request(input, init); + if (new URL(request.url).hostname === "127.0.0.1") return physicalFetch(request); + const body = await request.json() as Rec; + seen.push({ url: request.url, body, headers: request.headers }); + const native = new URL(request.url).hostname === "api.anthropic.com"; + if (native) return answer(body, true); + return Response.json({ id: "resp_backup", object: "response", status: "completed", model: "fallback", + output: [{ id: "msg_backup", type: "message", role: "assistant", status: "completed", content: [{ type: "output_text", text: "fallback reply", annotations: [] }] }], + usage: { input_tokens: 2, output_tokens: 1, total_tokens: 3 } }); + }); +}); +afterEach(async () => { + fetchMock.mockRestore(); + releaseSpend(); + await flushResponseState(); clearResponseStateForTests(); + clearComboSelectionState(); clearComboTargetCooldowns(); clearComboRecallForTests(); clearKeyCooldowns(); clearAnthropicAccountPoolState(); forgetAnthropicFailoverQuorum(); + codex.restore(); + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = previousHome; + removeTreeWithRetry(home); +}); +function config(on = true): OcxConfig { + return { port: 0, defaultProvider: "anth", providers: { + anth: { adapter: "anthropic", baseUrl: "https://api.anthropic.com", authMode: "key", apiKey: "fixture-managed-key", models: ["claude-sonnet-4-6"] }, + backup: { adapter: "openai-responses", baseUrl: "https://backup.fixture.invalid/v1", authMode: "key", apiKey: "fixture-backup-key", models: ["fallback"] }, + }, combos: { waterfall: { strategy: "failover", targets: [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "backup", model: "fallback" }] } }, + protocols: { rollout: { managedMessagesNative: on } } }; +} +async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown } = {}) { + const requestId = `native-combo-${crypto.randomUUID()}`; + const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", headers: { "content-type": "application/json", authorization: "Bearer fixture-admission", "x-api-key": "fixture-caller" }, + body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}) }), + }), options.config ?? config(), { model: "", provider: "" }, { requestId, start: Date.now() }); + const text = await response.text(); + return { response, text, rows: getRequestLogEntries().filter(row => row.requestId === requestId) }; +} + +describe("native Messages candidates in a combo", () => { + test("canonical child preserves source system/messages/tools/cache prefix and managed auth", async () => { + const { response, text, rows } = await send(); + expect(response.status).toBe(200); + expect(text).toContain("native reply"); + expect(seen).toHaveLength(1); + expect(seen[0]!.body).toEqual({ ...source, model: "claude-sonnet-4-6", stream: true }); + expect(seen[0]!.headers.get("x-api-key")).toBe("fixture-managed-key"); + expect(seen[0]!.headers.get("authorization")).toBeNull(); + expect(rows).toHaveLength(1); + expect(rows[0]!.provider).toBe("combo"); + expect(rows[0]!.attempts).toHaveLength(1); + expect(rows[0]!.attempts![0]!.usage).toMatchObject({ cacheReadInputTokens: 9, outputTokens: 3 }); + expect(rows[0]!.spend).toMatchObject({ sends: 1, unresolved: 0 }); + expect(rows[0]!.protocolTrace?.attempts?.[0]?.requestPath).toEqual(["messages", "messages"]); + }); + test("stable serialized cache prefix survives an appended turn", async () => { + await send(); + const prefix = JSON.stringify({ system: seen[0]!.body.system, tools: seen[0]!.body.tools, messages: seen[0]!.body.messages }); + await send({ messages: [...source.messages, { role: "assistant", content: "previous" }, { role: "user", content: "next" }] }); + expect(JSON.stringify({ system: seen[1]!.body.system, tools: seen[1]!.body.tools, messages: (seen[1]!.body.messages as unknown[]).slice(0, 1) })).toBe(prefix); + expect(seen[1]!.body.system).toEqual(source.system); + expect(seen[1]!.body.tools).toEqual(source.tools); + }); + test("pre-output streamed refusal falls back without leaking rejected child's prelude", async () => { + answer = () => stream(prelude + refusal); + const { response, text, rows } = await send(); + expect(response.status).toBe(200); + expect(text).toContain("fallback reply"); + expect(text).not.toContain("fixture overloaded"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "backup.fixture.invalid"]); + expect(rows).toHaveLength(1); + expect(rows[0]!.attempts).toHaveLength(2); + expect(rows[0]!.attempts![0]!.usage).toMatchObject({ cacheReadInputTokens: 9 }); + expect(rows[0]!.spend).toMatchObject({ sends: 2, unresolved: 0 }); + }); + for (const committed of [textStart + textDelta, frame("content_block_start", { index: 0, content_block: { type: "tool_use", id: "tool_fixture", name: "lookup", input: {} } })]) { + test(`never replays after ${committed.includes("tool_use") ? "tool" : "text"} commitment`, async () => { + answer = () => stream(prelude + committed + refusal); + const { text, rows } = await send(); + expect(text).toContain("fixture overloaded"); + expect(seen).toHaveLength(1); + expect(rows).toHaveLength(1); + expect(rows[0]!.status).toBe(503); + }); + } + test("non-streaming caller retains exact native body and one final row", async () => { + const { response, text, rows } = await send({ stream: false }); + expect(response.status).toBe(200); + expect(JSON.parse(text)).toEqual({ ...message, model: source.model }); + expect(seen[0]!.body).toEqual({ ...source, model: "claude-sonnet-4-6", stream: false }); + expect(rows).toHaveLength(1); + expect(rows[0]!.spend).toMatchObject({ sends: 1, unresolved: 0 }); + }); + for (const output of [false, true]) test(`non-streaming fold ${output ? "never replays output" : "hops before output"}`, async () => { + answer = () => stream(prelude + (output ? textStart + textDelta : "") + refusal); + const { text } = await send({ stream: false }); + expect(text).toContain(output ? "fixture overloaded" : "fallback reply"); + expect(seen).toHaveLength(output ? 1 : 2); + }); + test("ineligible pinned-effort route keeps translated policy", async () => { + const settings = config(); + settings.providers.anth!.pinnedReasoningEffort = "low"; + const { response } = await send({ config: settings }); + expect(response.status).toBe(200); + expect(seen[0]!.body).not.toHaveProperty("top_k"); + expect(seen[0]!.body.thinking).toMatchObject({ type: "enabled" }); + }); + test("stream cancellation retains usage and releases attached lease exactly once", async () => { + let upstreamCancelled = false; + answer = () => new Response(new ReadableStream({ + start(controller) { controller.enqueue(new TextEncoder().encode(prelude + textStart + textDelta)); }, + cancel() { upstreamCancelled = true; }, + }), { headers: { "content-type": "text/event-stream" } }); + const caller = new AbortController(); + const lease = tryAdmitTurn(undefined, caller.signal); + if (!lease) throw new Error("fixture admission unavailable"); + let releases = 0; + lease.attach({ release() { releases++; } }); + const misses = activeRegistryMetrics().activeTurns!.releaseMisses; + const requestId = `native-combo-${crypto.randomUUID()}`; + try { + const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", headers: { "content-type": "application/json" }, + body: JSON.stringify({ ...source, stream: true }), signal: caller.signal, + }), config(), { model: "", provider: "" }, { requestId, start: Date.now(), turnAdmissionLease: lease }); + expect(getActiveTurnCount()).toBe(1); + expect(releases).toBe(0); + const reader = response.body!.getReader(); + await reader.read(); + caller.abort(new Error("fixture cancel")); + await reader.cancel(); + await Bun.sleep(10); + expect(upstreamCancelled).toBe(true); + expect(releases).toBe(1); + expect(getActiveTurnCount()).toBe(0); + expect(activeRegistryMetrics().activeTurns!.releaseMisses).toBe(misses); + const rows = getRequestLogEntries().filter(row => row.requestId === requestId); + expect(rows).toHaveLength(1); + expect(rows[0]!.status).toBe(499); + expect(rows[0]!.attempts![0]!.usage).toMatchObject({ cacheReadInputTokens: 9 }); + expect(seen).toHaveLength(1); + } finally { lease.release(); } + expect(releases).toBe(1); + }); + test("eligible OAuth child reuses native credential binding and tool-name policy", async () => { + await saveCredential("anthropic", { access: "synthetic-combo-access", refresh: "synthetic-combo-refresh", expires: Date.now() + 3_600_000, accountId: "synthetic-combo-uuid", source: "oauth" }); + const settings = config(); + settings.providers.anthropic = { ...settings.providers.anth!, authMode: "oauth", apiKey: undefined }; + delete settings.providers.anth; + settings.defaultProvider = "anthropic"; + settings.combos!.waterfall!.targets[0]!.provider = "anthropic"; + settings.anthropicAccountPool = { enabled: true }; + settings.protocols!.rollout!.managedMessagesNativeOAuth = true; + const { response, rows } = await send({ config: settings, stream: false }); + expect(response.status).toBe(200); + expect(seen).toHaveLength(1); + expect(seen[0]!.headers.get("authorization")).toBe("Bearer synthetic-combo-access"); + expect(seen[0]!.headers.get("x-api-key")).toBeNull(); + expect(seen[0]!.body.messages).toEqual(source.messages); + expect(seen[0]!.body.top_k).toBe(7); + expect((seen[0]!.body.tools as Rec[])[0]!.cache_control).toEqual(source.tools[0]!.cache_control); + expect(rows[0]!.attempts).toHaveLength(1); + expect(rows[0]!.spend).toMatchObject({ sends: 1, unresolved: 0 }); + }); + test("a refused native child cannot settle the lease held by its accepted native sibling", async () => { + const settings = config(); + settings.providers.anth!.models!.push("claude-haiku-4-5"); + settings.combos!.waterfall!.targets[1] = { provider: "anth", model: "claude-haiku-4-5" }; + let close!: () => void; + answer = body => body.model === "claude-sonnet-4-6" ? stream(prelude + refusal) + : new Response(new ReadableStream({ + start(controller) { controller.enqueue(new TextEncoder().encode(prelude + textStart + textDelta)); close = () => { controller.enqueue(new TextEncoder().encode(terminal)); controller.close(); }; }, + }), { headers: { "content-type": "text/event-stream" } }); + const lease = tryAdmitTurn(); + if (!lease) throw new Error("fixture admission unavailable"); + let releases = 0; + lease.attach({ release() { releases++; } }); + try { + const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ ...source, stream: true }), + }), settings, { model: "", provider: "" }, { requestId: `lease-${crypto.randomUUID()}`, start: Date.now(), turnAdmissionLease: lease }); + expect(seen).toHaveLength(2); + expect(releases).toBe(0); + expect(getActiveTurnCount()).toBe(1); + close(); + await response.text(); + expect(releases).toBe(1); + } finally { lease.release(); } + expect(releases).toBe(1); + }); + test("enforced shared send ceiling counts one physical send and never doubles its prepaid permit", async () => { + configureSharedSpendLedger(spendPolicyFromConfig({ pool: { maxTokens: 100_000 } }, undefined, ["anth", "backup"])); + const { response, rows } = await send(); + expect(response.status).toBe(200); + expect(seen).toHaveLength(1); + expect(rows[0]!.spend).toMatchObject({ sends: 1, unresolved: 0 }); + }); + test("real HTTP ingress serves native combo bytes with synthetic canonical egress", async () => { + const settings = config(); + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: request => handleClaudeMessages( + request, settings, { model: "", provider: "" }, { requestId: `http-${crypto.randomUUID()}`, start: Date.now() }, + ) }); + try { + const response = await fetch(new URL("/v1/messages", server.url), { method: "POST", + headers: { "content-type": "application/json" }, body: JSON.stringify({ ...source, stream: true }) }); + expect(response.status).toBe(200); + expect(await response.text()).toContain("native reply"); + expect(seen).toHaveLength(1); + expect(seen[0]!.body).toEqual({ ...source, model: "claude-sonnet-4-6", stream: true }); + } finally { await server.stop(true); } + }); + test("native disabled keeps existing translated path", async () => { + const { response } = await send({ config: config(false) }); + expect(response.status).toBe(200); + expect(seen).toHaveLength(1); + expect(seen[0]!.body).not.toHaveProperty("top_k"); + }); +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 0b5ed542af8..c0578d38365 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -466,6 +466,7 @@ "claude-manual-env.test.ts": "gui", "claude-messages-endpoint.test.ts": "claude-integration", "claude-native-fast-1m-ingress.test.ts": "claude-integration", "claude-messages-thread.test.ts": "claude-integration", "messages-native.test.ts": "claude-integration", + "messages-native-combo.test.ts": "claude-integration", "messages-native-decline-trace.test.ts": "claude-integration", "messages-native-oauth.test.ts": "claude-integration", "messages-native-opaque-state.test.ts": "claude-integration", "messages-surface-matrix.test.ts": "claude-integration", "claude-model-info.test.ts": "claude-integration", "claude-models-discovery.test.ts": "claude-integration", From 723a8c3fc1e99f9e27adba0909545f7e43697256 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 20 Oct 2026 02:06:51 +0800 Subject: [PATCH 02/23] fix(combo): never fall back to a translated child for a native-only tool A source declaring a tool the translated body drops with no out-of-body lane (bash_*, text_editor_*, ...) now makes every translated combo child unpickable, so a refused native child surfaces its own failure instead of continuing without the tool. web_search/web_fetch stay portable; non-combo translated routes are unchanged. Co-Authored-By: Claude Code --- src/claude/inbound-content-options.ts | 6 ++++++ src/server/responses/core-combo-native.ts | 15 +++++++++++++-- structure/data-planes/protocol-paths.md | 2 +- .../messages-native-combo.test.ts | 12 ++++++++++-- 4 files changed, 30 insertions(+), 5 deletions(-) diff --git a/src/claude/inbound-content-options.ts b/src/claude/inbound-content-options.ts index 25139ea20f4..b2c615dbb35 100644 --- a/src/claude/inbound-content-options.ts +++ b/src/claude/inbound-content-options.ts @@ -87,6 +87,12 @@ export function declaredAnthropicServerTools(raw: unknown): Rec[] | undefined { return tools.length > 0 ? tools : undefined; } +/** A declared tool the translated body drops and no out-of-body lane carries (bash_*, text_editor_*, ...). */ +export function hasNativeOnlyTool(raw: Rec): boolean { + return Array.isArray(raw.tools) + && raw.tools.some(tool => !toolsToResponses([tool]) && !declaredAnthropicServerTools({ tools: [tool] })); +} + /** Validate original call/result order without inventing inputs or pairing a later call backwards. */ function replayableServerRun(run: readonly Rec[]): boolean { const pending = new Map(); diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index 0f873ec4ec4..cebb347178e 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -46,6 +46,7 @@ import type { NativeMessagesFinishLog } from "../messages-native"; import { credentialDomainFor } from "../../protocols/opaque-state"; import { chatCompletionsErrorResponse } from "../../chat/outbound"; import { anthropicErrorResponse } from "../../claude/outbound"; +import { hasNativeOnlyTool } from "../../claude/inbound-content-options"; import { isRequestExecutionBudget } from "../../lib/request-execution-budget"; import { markResponseNonReplayable } from "../../lib/upstream-retry"; import { redactSecretString } from "../../lib/redact"; @@ -99,7 +100,7 @@ export interface NativeComboChildPlan { } export interface ComboProtocolLanes { - /** Pick predicate. False only for a candidate the `reject` policy skips. */ + /** Pick predicate. False for a candidate the `reject` policy skips or a translated child of a native-only tool. */ pickable(target: ComboTarget): boolean; /** The ingress refusal when every enabled candidate was skipped; otherwise undefined. */ refusal(): Response | undefined; @@ -114,6 +115,8 @@ export interface ComboProtocolLanes { interface CandidateVerdict { skip?: ProtocolFeature[]; + /** A translated child would silently drop the request's native-only tool. */ + nativeOnly?: true; } /** Build opt-in native eligibility and child plans; only the initial plan applies a JEV choice. */ @@ -177,6 +180,7 @@ export function createComboProtocolLanes(input: { const verdicts = new Map(); let skipRecorded = false; + let nativeOnlyTool: boolean | undefined; const judge = (target: ComboTarget): CandidateVerdict => { const key = selector(target); const cached = verdicts.get(key); @@ -206,13 +210,20 @@ export function createComboProtocolLanes(input: { addProtocolEntryReason(logCtx, "feature-unrepresentable"); } } + // Under any policy: the source chose a native-only tool, so a translated child never runs it. + if (!native && inbound === "messages" && (nativeOnlyTool ??= hasNativeOnlyTool(envelope.freshBody()))) { + verdict.nativeOnly = true; + } } verdicts.set(key, verdict); return verdict; }; return { - pickable: target => !reject || judge(target).skip === undefined, + pickable: target => { + const verdict = judge(target); + return verdict.nativeOnly !== true && (!reject || verdict.skip === undefined); + }, refusal: () => { if (!reject) return undefined; const enabled = targets.filter(target => { diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 27390c4a85e..9a7862b198a 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -378,7 +378,7 @@ replay config; backend credentials are decided at dispatch, so this errs toward ingress, `count_tokens` and the planner all ask it. The planner can judge only the config-and-route parts: skill elision and web search depend on body content no feature describes. -Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. +Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. A source declaring a tool the translated body drops with no out-of-body lane (`hasNativeOnlyTool` in `src/claude/inbound-content-options.ts`: `bash_*`, `text_editor_*`, ...; `web_search_*`/`web_fetch_*` stay portable) makes every translated child unpickable under any `unrepresentable` policy, so a refused native child surfaces its own failure instead of continuing without the tool; non-combo translated routes are unchanged. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. With the switch on, a declined route re-marks its bridge entry with the decline reason (after any `effort-row` / `fast-row`), so the trace says why the bridge was taken; with it off nothing is diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 61fa7dccaef..0e712da8874 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -86,11 +86,11 @@ function config(on = true): OcxConfig { }, combos: { waterfall: { strategy: "failover", targets: [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "backup", model: "fallback" }] } }, protocols: { rollout: { managedMessagesNative: on } } }; } -async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown } = {}) { +async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown; tools?: unknown } = {}) { const requestId = `native-combo-${crypto.randomUUID()}`; const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer fixture-admission", "x-api-key": "fixture-caller" }, - body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}) }), + body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}), ...(options.tools ? { tools: options.tools } : {}) }), }), options.config ?? config(), { model: "", provider: "" }, { requestId, start: Date.now() }); const text = await response.text(); return { response, text, rows: getRequestLogEntries().filter(row => row.requestId === requestId) }; @@ -142,6 +142,14 @@ describe("native Messages candidates in a combo", () => { expect(rows[0]!.status).toBe(503); }); } + for (const [tool, portable] of [[{ type: "bash_20250124", name: "bash" }, false], [{ type: "web_search_20250305", name: "web_search" }, true]] as const) { + test(`${portable ? "portable" : "native-only"} ${tool.name} tool ${portable ? "may" : "never"} fall back to a translated child`, async () => { + answer = () => stream(prelude + refusal); + const { text } = await send({ tools: [...source.tools, tool] }); + expect(text).toContain(portable ? "fallback reply" : "fixture overloaded"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(portable ? ["api.anthropic.com", "backup.fixture.invalid"] : ["api.anthropic.com"]); + }); + } test("non-streaming caller retains exact native body and one final row", async () => { const { response, text, rows } = await send({ stream: false }); expect(response.status).toBe(200); From fe15584194782e105e54d7a19e031f1270e64f91 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Sat, 24 Oct 2026 16:46:20 +0800 Subject: [PATCH 03/23] fix(combo): register the native combo test in a shard and pin the shared tap contract Move the messages-native-combo mapping out of layout.json into the layout-replay shard (fixture registration unchanged), so layout.json stays under its size cap. The shared Anthropic SSE tap now logs an upstream error event as that failure instead of a 200 and records usage on both client-cancel paths. That changes the direct native passthrough and managed lane too, so the claude-desktop structure invariant is updated and two direct-lane tests pin it; both fail on dev without this branch. Co-Authored-By: Claude Code --- scripts/test-layout/layout-replay.json | 3 ++- scripts/test-layout/layout.json | 1 - structure/clients/claude-desktop.md | 2 +- .../claude-native-passthrough.test.ts | 21 +++++++++++++++++++ 4 files changed, 24 insertions(+), 3 deletions(-) diff --git a/scripts/test-layout/layout-replay.json b/scripts/test-layout/layout-replay.json index b9c41e0c9f7..6621ddba789 100644 --- a/scripts/test-layout/layout-replay.json +++ b/scripts/test-layout/layout-replay.json @@ -7,5 +7,6 @@ "combo-replay-refusal-attribution.test.ts": "responses", "responses-replay-metadata-principal.test.ts": "responses", "responses-combo-replay-principal.test.ts": "responses", - "responses-translated-replay-principal.test.ts": "responses" + "responses-translated-replay-principal.test.ts": "responses", + "messages-native-combo.test.ts": "claude-integration" } diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index d496dc16d38..a61f3c0d32b 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -451,7 +451,6 @@ "claude-messages-endpoint.test.ts": "claude-integration", "claude-messages-thread.test.ts": "claude-integration", "messages-native.test.ts": "claude-integration", - "messages-native-combo.test.ts": "claude-integration", "messages-native-decline-trace.test.ts": "claude-integration", "messages-native-oauth.test.ts": "claude-integration", "messages-native-opaque-state.test.ts": "claude-integration", diff --git a/structure/clients/claude-desktop.md b/structure/clients/claude-desktop.md index 4c259199390..5ca1158189c 100644 --- a/structure/clients/claude-desktop.md +++ b/structure/clients/claude-desktop.md @@ -593,7 +593,7 @@ Native Anthropic passthrough in `src/server/claude-messages.ts` forwards the cal Native passthrough retains upstream `anthropic-ratelimit-*` response headers for Claude Code quota/statusLine consumers on SSE, JSON and upstream errors. `src/server/anthropic-rate-limit-headers.ts` selects only that family instead of copying all upstream headers, so cookies and unrelated metadata are not relayed. Missing rate-limit headers are not fabricated; body, status, content type and existing non-stream `Retry-After` behavior stay unchanged. `tests/claude-integration/claude-native-rate-limit-headers.test.ts` exercises the production ingress against a synthetic upstream for all four response shapes (SSE, JSON, upstream error and count_tokens). -`tapAnthropicSseForLog` in `src/server/claude-messages.ts` relays the streamed body of both the native passthrough and the managed native Messages lane (`src/server/messages-native.ts`). The response headers are already sent, so a stall, a byte-cap overflow, or an upstream read failure ends the body with an Anthropic `event: error` frame after a blank-line boundary and a clean close: `timeout_error` for an idle stall, `api_error` for the byte cap, and `api_error` when an upstream read fails mid-stream (a socket reset). The mid-stream reset is logged like the Responses relay's read error: status 502, `terminalStatus: "failed"`, `closeReason: "terminal"`, `transportPhase: "mid_stream"`, a synthetic terminal source, the attempt marked `streamAborted`, the redacted reason in `upstreamError`, and the usage seen before the reset. The non-streaming fold in the managed lane closes its row with the tap's meta for a reset, a stall or the byte cap, so its row matches the streaming lane's. The request is not replayed. Every one of these frames logs status 502. A stall is `terminalStatus: "incomplete"` with `closeReason: "body_stall"`, the status the Responses relay gives a stall-timeout incomplete. The byte cap is `incomplete` with `closeReason: "body_overflow"`, the same 502 the non-stream passthrough answers for it. Both carry the proxy's message in `upstreamError`, and a reset is `terminalStatus: "failed"`. Either way the row keeps its failure diagnostics in usage.jsonl. A stall or overflow after the turn's own terminal (`message_stop` or an upstream `error` event) is a finished turn: it logs 200 and closes without an error frame, as the read-error branch does. The tap finds frames in a copy normalized to LF, because SSE lines may end in CRLF, LF or CR; it holds a trailing CR until the next chunk so a split CRLF stays one line ending, and forwards the original bytes unchanged. Some read failures are not upstream failures. When the cancel signal is already aborted, the rejection is a `499` client cancel, because Bun can reject the read before it dispatches the abort listener. The managed lane passes its upstream controller's signal, so shutdown and turn release count as cancels too. When `message_stop` or an upstream `error` event has been seen, including one still in the buffer without its blank-line delimiter, the turn is complete: it logs 200 and closes with no error frame. A terminal found only in that unterminated tail gets its blank line restored, because an SSE parser drops an event that EOF cuts off. A translator budget overflow is a local cap, so it still errors the stream, and the non-streaming fold answers it with 413. `tests/claude-integration/claude-native-passthrough.test.ts` and `tests/claude-integration/messages-native.test.ts` cover both lanes against an upstream that resets after a partial or a complete body, plus both cancel paths. +`tapAnthropicSseForLog` in `src/server/claude-messages.ts` relays the streamed body of both the native passthrough and the managed native Messages lane (`src/server/messages-native.ts`). The response headers are already sent, so a stall, a byte-cap overflow, or an upstream read failure ends the body with an Anthropic `event: error` frame after a blank-line boundary and a clean close: `timeout_error` for an idle stall, `api_error` for the byte cap, and `api_error` when an upstream read fails mid-stream (a socket reset). The mid-stream reset is logged like the Responses relay's read error: status 502, `terminalStatus: "failed"`, `closeReason: "terminal"`, `transportPhase: "mid_stream"`, a synthetic terminal source, the attempt marked `streamAborted`, the redacted reason in `upstreamError`, and the usage seen before the reset. The non-streaming fold in the managed lane closes its row with the tap's meta for a reset, a stall or the byte cap, so its row matches the streaming lane's. The request is not replayed. Every one of these frames logs status 502. A stall is `terminalStatus: "incomplete"` with `closeReason: "body_stall"`, the status the Responses relay gives a stall-timeout incomplete. The byte cap is `incomplete` with `closeReason: "body_overflow"`, the same 502 the non-stream passthrough answers for it. Both carry the proxy's message in `upstreamError`, and a reset is `terminalStatus: "failed"`. Either way the row keeps its failure diagnostics in usage.jsonl. A stall or overflow after the turn's own terminal (`message_stop` or an upstream `error` event) is a finished turn: it closes without an error frame, as the read-error branch does, and logs that terminal's status. The tap finds frames in a copy normalized to LF, because SSE lines may end in CRLF, LF or CR; it holds a trailing CR until the next chunk so a split CRLF stays one line ending, and forwards the original bytes unchanged. Some read failures are not upstream failures. When the cancel signal is already aborted, the rejection is a `499` client cancel, because Bun can reject the read before it dispatches the abort listener. The managed lane passes its upstream controller's signal, so shutdown and turn release count as cancels too. When `message_stop` or an upstream `error` event has been seen, including one still in the buffer without its blank-line delimiter, the turn is complete and closes with no error frame. `message_stop` logs 200. An upstream `error` event is relayed unchanged and logs the status `httpStatusFromTerminalError` maps it to, with `terminalStatus: "failed"` and the redacted message in `upstreamError`, so a failed turn never reads as a success; the managed lane's combo preflight reads the same failure before the first output byte. A terminal found only in that unterminated tail gets its blank line restored, because an SSE parser drops an event that EOF cuts off. A translator budget overflow is a local cap, so it still errors the stream, and the non-streaming fold answers it with 413. `tests/claude-integration/claude-native-passthrough.test.ts` and `tests/claude-integration/messages-native.test.ts` cover both lanes against an upstream that resets after a partial or a complete body, plus both cancel paths, which record the usage seen before the client left. A native passthrough answered without a stream records its reason in `upstreamError`, so the row and the failure diagnostics in usage.jsonl name the cause. An upstream error response (status 400 or above) is relayed verbatim; the stored diagnostic is `Provider error : ` only for a valid Anthropic error envelope with one of the closed types `invalid_request_error`, `authentication_error`, `permission_error`, `not_found_error`, `rate_limit_error`, `api_error`, `overloaded_error` or `request_too_large`. Upstream messages and arbitrary type strings never enter this diagnostic. Unknown or malformed envelopes, non-JSON bodies and bodies over 64 Ki characters log `Provider error `. Local header timeout, body stall, byte-cap overflow and cancel diagnostics contain fixed text plus validated guard limits. Fetch failures log the fixed reason `anthropic passthrough failed: upstream connection error`; the existing redacted client response is preserved. Classification uses these stored reasons and HTTP status, so a `permission_error` at 403 is `permission_denied` regardless of upstream message wording. `tests/claude-integration/claude-native-passthrough.test.ts` covers these cases and checks that echoed account identifiers and request content are absent from both history sinks. diff --git a/tests/claude-integration/claude-native-passthrough.test.ts b/tests/claude-integration/claude-native-passthrough.test.ts index 1b7bf39fbc5..e37e6b5ea81 100644 --- a/tests/claude-integration/claude-native-passthrough.test.ts +++ b/tests/claude-integration/claude-native-passthrough.test.ts @@ -939,6 +939,27 @@ test("a read rejection that lands before the client abort listener still finaliz expect(calls).toEqual([{ status: 499, closeReason: "client_cancel" }]); expect(logCtx.transportPhase).toBeUndefined(); expect(logCtx.upstreamError).toBeUndefined(); + // A cancelled turn still bills what it consumed before the client left. + expect(logCtx.usage?.inputTokens).toBe(12); +}); + +test("an upstream error event is the turn's terminal, logged as that failure rather than a 200", async () => { + const errorFrame = `event: error\ndata: ${JSON.stringify({ type: "error", error: { type: "overloaded_error", message: "Overloaded" } })}\n\n`; + const source = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(PARTIAL_TURN_SSE + errorFrame)); + controller.close(); + }, + }); + const calls: unknown[] = []; + const logCtx: RequestLogContext = { model: "claude-fable-5", provider: "anthropic-native" }; + const tapped = tapAnthropicSseForLog(source, logCtx, (status, meta) => calls.push({ status, ...meta }), { stallMs: 5_000, maxBytes: 0 }); + // The client receives the upstream's own error event unchanged. + expect(await new Response(tapped).text()).toBe(PARTIAL_TURN_SSE + errorFrame); + // The status every other surface maps this error to (`httpStatusFromTerminalError`). + expect(calls).toEqual([{ status: 503, closeReason: "terminal", terminalStatus: "failed" }]); + expect(logCtx.upstreamError).toBe("Overloaded"); + expect(logCtx.usage?.inputTokens).toBe(12); }); const COMPLETE_TURN_SSE = PARTIAL_TURN_SSE + [ From 8201ecf0cee0bfe99210228f69574afb51b38a44 Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Wed, 28 Oct 2026 16:04:11 +0900 Subject: [PATCH 04/23] test(combo): pin native Messages caller scope and credential isolation --- .../src/content/docs/guides/protocol-paths.md | 3 +- .../content/docs/tr/guides/protocol-paths.md | 10 ++-- .../transports/responses-combo-boundaries.md | 2 +- .../messages-native-combo.test.ts | 52 ++++++++++++++++++- 4 files changed, 58 insertions(+), 9 deletions(-) diff --git a/docs-site/src/content/docs/guides/protocol-paths.md b/docs-site/src/content/docs/guides/protocol-paths.md index 76917c4ecb8..8b28bc10d83 100644 --- a/docs-site/src/content/docs/guides/protocol-paths.md +++ b/docs-site/src/content/docs/guides/protocol-paths.md @@ -141,7 +141,8 @@ These paths still use the internal Responses bridge or are not covered: source Messages envelope, preserving system/messages/tools/cache breakpoints under the existing native allowlist and credential rules. Without explicit operator retention, TTL-less combo breakpoints stay TTL-less. Pre-output retryable errors may fall back; any content/tool block - commits the child and forbids replay. Preserved prefixes do not prove live cache hits. + commits the child and forbids replay. Every child enforces the caller's model/provider scope. + Preserved prefixes do not prove live cache hits. - Direct encoding (`directEncoders`) covers only a concrete, non-Responses route in streaming delivery. Combo and policy children, routed compaction, run-turn adapters (Cursor, Devin, coding-agent CLIs, CodeBuddy), and sidecar turns keep the bridge. diff --git a/docs-site/src/content/docs/tr/guides/protocol-paths.md b/docs-site/src/content/docs/tr/guides/protocol-paths.md index 4c5c7a18149..e05fd070f04 100644 --- a/docs-site/src/content/docs/tr/guides/protocol-paths.md +++ b/docs-site/src/content/docs/tr/guides/protocol-paths.md @@ -99,8 +99,8 @@ varsayılan olarak kapalıdır; kapalı anahtar değişiklik yapmaz. | Anahtar | Açıkken | | --- | --- | | `nativeChatCombos` | Combo içindeki uygun Chat adayı, istemci gövdesinin kendine ait kopyasından yerel gönderilir; diğer adaylar köprüde kalır. | -| `managedMessagesNative` | Doğrudan, anahtarla doğrulanan Anthropic sağlayıcısına giden Messages isteği köprü yerine Messages olarak gönderilir. Sabitlenmiş çaba, engelli beceri çıkarma, web arama yardımcısı, görsel ön işleme veya sentetik satır gibi yalnızca köprüde çalışan davranışlar köprüde kalır. | -| `managedMessagesNativeOAuth` | `api.anthropic.com` üzerindeki havuzsuz `anthropic` OAuth sağlayıcısı için yerel Messages. Yalnızca `managedMessagesNative` ile birlikte etkilidir; Anthropic OAuth hesap havuzu köprüde kalır. | +| `managedMessagesNative` | Uygun doğrudan Anthropic rotaları ve varsayılan çabası olmayan strict failover combo içindeki birinci taraf Anthropic adayları Messages olarak gönderilir. Çaba yeniden yazımı ve yalnızca köprüde çalışan davranışlar köprüde kalır. | +| `managedMessagesNativeOAuth` | `api.anthropic.com` üzerindeki yapılandırılmış Anthropic OAuth hesapları, hesap havuzları ve uygun failover adayları için yerel Messages. Yalnızca `managedMessagesNative` ile birlikte etkilidir. | | `directEncoders` | Responses dışı üst sunucuda Chat/Messages yanıtı, iç Responses akışı yerine doğrudan bağdaştırıcı olaylarından kodlanır. İstek tarafı değişmez. | | `antigravityDirect` | **Varsayılan olarak açık**; kapatmak için `false` yapın. Tek Google Antigravity sağlayıcısına giden Chat/Messages, iç Responses gövdesini iki yönde de atlar: konuşma doğrudan Google bağdaştırıcısının derlediği biçime çözülür, yanıt `directEncoders` gibi kodlanır. Yol `chat -> ir -> other` veya `messages -> ir -> other` olur. Model, çaba, sınırlar ve araç seçimi normal ilkelerden geçtiği için hesap dönüşümü, çaba kuralları ve kullanım değişmez. Combo'lar, ilkeler, barındırılan web araması aracı, yalnızca metin modeline gelen görseller, gölge çağrı yakalamasının değiştirebileceği modeller ve proxy'nin yeniden yazdığı metinler (beceri kataloğu, üretilmiş görsel bağlantısı) köprüde kalır; neden günlükte belirtilir. | | `shadowPlan` | Her Chat/Messages isteğinin sonunda önizlemenin öngöreceği plan gerçek yolla karşılaştırılır. Uyuşmazsa yol kaydına `planMismatch: true` eklenir. İkinci istek gönderilmez. | @@ -136,7 +136,7 @@ Aşağıdakiler iç Responses köprüsünü kullanmaya devam eder veya kapsam d gövdeye çözülür. Tek Google Antigravity rotası istisnadır; ancak `antigravityDirect: false` ise o da köprü kullanır. - Yönlendirme ilkesi adayları, çaba satırından ulaşılan combo'lar, - `nativeChatCombos` kapalıyken combo adayları ve Messages combo'ları. + `nativeChatCombos` kapalıyken Chat combo adayları. Messages combo'larında varsayılan çabası olmayan strict failover içindeki uygun birinci taraf Anthropic adayları yerel yolu kullanır; diğerleri köprüde kalır. Her aday çağıranın model/sağlayıcı kapsamını uygular. - `directEncoders`, yalnızca somut Responses dışı rotalarda akışlı teslimi kapsar. Combo/ilke alt istekleri, yönlendirilmiş sıkıştırma, tur çalıştıran bağdaştırıcılar (Cursor, Devin, kod ajanı CLI'ları, CodeBuddy) ve yardımcı servis turları köprüde kalır. @@ -154,8 +154,8 @@ Aşağıdakiler iç Responses köprüsünü kullanmaya devam eder veya kapsam d Başlıklar tek başına istek yetkisi veya resmi destek sağlamaz. TypeScript Agent SDK'nın `sdk-ts` user-agent giriş noktası da aynı zorunlu kimlik başlıklarıyla tanınır; birinci taraf Anthropic isteklerinde izin listeli yerel betalarını korur. -- OAuth üzerinden yerel Chat planlanmıyor. Anthropic OAuth ile yerel Messages - yalnızca havuzsuz hesabı kapsar; hesap havuzu köprüde kalır. +- OAuth üzerinden yerel Chat planlanmıyor. Anthropic OAuth ile yerel Messages, + aynı uygunluk ve kimlik bilgisi kurallarıyla hesap havuzlarını da kapsar. - Yönetilen yerel Messages, çağıranın `anthropic-beta` değerlerinden yalnızca kısa izin listesindekileri ve yalnızca `api.anthropic.com` hedefine iletir. İzin listesi dışı üst düzey alanları özellik etkisi kaydetmeden düşürür ve diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 58a8e46e02a..31f0eb40337 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -123,7 +123,7 @@ any other event returns to the ordinary combo failure classifier; any content bl a tool start), unknown/malformed payload, terminal or byte/chunk cap commits conservatively. Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable. Preflight retention is charged to the translator budget and released on delivery/cancellation. -The same native executor binds the accepted stream to the parent's held lease, carries observed +Native plans enforce the captured caller model scope, including internal calls with broad host admission. The same native executor binds the accepted stream to the parent's held lease, carries observed usage on cancellation, and reports terminal errors without treating them as successful rows. HTTP 410 remains terminal by default. It advances and cools only the exact combo target when the diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 0e712da8874..2aa8691a151 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -16,6 +16,10 @@ import { installIsolatedCodexHome, type IsolatedCodexHome } from "../helpers/iso import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; import { removeTreeWithRetry } from "../helpers/remove-tree"; import type { OcxConfig } from "../../src/types"; +import type { DataPlaneAdmission } from "../../src/server/auth-cors"; +import { handleResponses } from "../../src/server/responses/core"; +import { createProtocolEnvelope } from "../../src/protocols/envelope"; +import { createTranslatorBudget } from "../../src/lib/translator-budget"; type Rec = Record; const source = { @@ -86,12 +90,12 @@ function config(on = true): OcxConfig { }, combos: { waterfall: { strategy: "failover", targets: [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "backup", model: "fallback" }] } }, protocols: { rollout: { managedMessagesNative: on } } }; } -async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown; tools?: unknown } = {}) { +async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown; tools?: unknown; admission?: DataPlaneAdmission } = {}) { const requestId = `native-combo-${crypto.randomUUID()}`; const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer fixture-admission", "x-api-key": "fixture-caller" }, body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}), ...(options.tools ? { tools: options.tools } : {}) }), - }), options.config ?? config(), { model: "", provider: "" }, { requestId, start: Date.now() }); + }), options.config ?? config(), { model: "", provider: "" }, { requestId, start: Date.now(), admission: options.admission }); const text = await response.text(); return { response, text, rows: getRequestLogEntries().filter(row => row.requestId === requestId) }; } @@ -131,6 +135,50 @@ describe("native Messages candidates in a combo", () => { expect(rows[0]!.attempts).toHaveLength(2); expect(rows[0]!.attempts![0]!.usage).toMatchObject({ cacheReadInputTokens: 9 }); expect(rows[0]!.spend).toMatchObject({ sends: 2, unresolved: 0 }); + expect(seen[1]!.headers.get("authorization")).toBe("Bearer fixture-backup-key"); + expect(seen[1]!.headers.get("x-api-key")).toBeNull(); + }); + for (const allowedModels of [[], ["anth/claude-sonnet-4-6"], ["anth/claude-sonnet-4-6", "anth/claude-haiku-4-5"]]) { + test(`native failover enforces the caller model scope with ${allowedModels.length} allowed destinations`, async () => { + const settings = config(); + settings.providers.anth!.models!.push("claude-haiku-4-5"); + settings.combos!.waterfall!.targets[1] = { provider: "anth", model: "claude-haiku-4-5" }; + settings.apiKeys = [{ id: "caller", name: "fixture", key: "ocx_data_" + "c".repeat(40), + createdAt: "2026-01-01T00:00:00.000Z", allowedProviders: allowedModels.length ? ["anth"] : ["backup"], allowedModels }]; + answer = body => body.model === "claude-sonnet-4-6" ? stream(prelude + refusal) : Response.json(message); + const { response, text } = await send({ config: settings, stream: false, + admission: { kind: "configured", keyId: "caller", source: "dedicated" } }); + expect(response.status).toBe(allowedModels.length === 2 ? 200 : 403); + expect(seen).toHaveLength(allowedModels.length); + if (allowedModels.length < 2) expect(text).toContain("not allowed for this API key"); + for (const child of seen) expect(child.body.top_k).toBe(source.top_k); + if (allowedModels.length === 2) { + expect(seen[1]!.body).toEqual({ ...source, model: "claude-haiku-4-5", stream: false }); + expect(seen[1]!.body.system).toEqual(seen[0]!.body.system); + } + }); + } + test("captured scope overrides broad host admission for a native Messages sibling", async () => { + const { createNativeMessagesComboSource } = await import("../../src/server/messages-native"); + const settings = config(); + settings.providers.anth!.models!.push("claude-haiku-4-5"); + settings.combos!.waterfall!.targets[1] = { provider: "anth", model: "claude-haiku-4-5" }; + settings.apiKeys = [{ id: "host", name: "host", key: "ocx_data_" + "h".repeat(40), createdAt: "2026-01-01T00:00:00.000Z" }]; + const admissionScope = { providers: ["anth"], models: ["anth/claude-sonnet-4-6"] }; + const translatorBudget = createTranslatorBudget(); + const req = new Request("http://localhost/v1/messages", { method: "POST", body: JSON.stringify({ ...source, stream: false }) }); + const logCtx = { model: source.model, provider: "combo" }; + const protocolSource = createNativeMessagesComboSource({ req, config: settings, logCtx, + envelope: createProtocolEnvelope({ inbound: "messages", body: { ...source, stream: false }, translatorBudget }), + requestedModel: source.model, translatorBudget, modelScope: admissionScope }); + answer = () => stream(prelude + refusal); + const response = await handleResponses(new Request("http://localhost/v1/responses", { method: "POST", + headers: { "content-type": "application/json" }, body: JSON.stringify({ model: source.model, input: "fixture question", stream: false }) }), + settings, logCtx, { protocolSource, admissionScope, admission: { kind: "configured", keyId: "host", source: "dedicated" }, inboundWire: "anthropic" }); + expect(response.status).toBe(403); + expect(await response.text()).toContain("not allowed for this API key"); + expect(seen).toHaveLength(1); + expect(seen[0]!.body.top_k).toBe(source.top_k); }); for (const committed of [textStart + textDelta, frame("content_block_start", { index: 0, content_block: { type: "tool_use", id: "tool_fixture", name: "lookup", input: {} } })]) { test(`never replays after ${committed.includes("tool_use") ? "tool" : "text"} commitment`, async () => { From 37a459fa4d629675c321fb63f0016698c9850f0b Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Wed, 28 Oct 2026 16:57:30 +0900 Subject: [PATCH 05/23] fix(combo): sanitize native Messages failure diagnostics --- src/server/messages-native-preflight.ts | 5 +++-- src/server/messages-shared.ts | 3 ++- src/server/responses/core-combo.ts | 6 +++--- structure/clients/claude-desktop.md | 2 +- .../transports/responses-combo-boundaries.md | 2 +- .../claude-native-passthrough.test.ts | 4 ++-- .../messages-native-combo.test.ts | 16 ++++++++++++++++ 7 files changed, 28 insertions(+), 10 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 10b55b0f0fa..626e94848bb 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -67,7 +67,8 @@ export async function preflightNativeMessagesResponse( } if (failure && !committed) { logCtx.usage = anthropicUsageToOcx(usage); - logCtx.upstreamError = redactSecretString(typeof failure.message === "string" ? failure.message : "Provider stream failed before output"); + const message = redactSecretString(typeof failure.message === "string" ? failure.message : "Provider stream failed before output"); + logCtx.upstreamError = `Provider stream failed (${failureStatus})`; void reader.cancel("retrying native Messages pre-output refusal").catch(() => {}); release(); const headers = new Headers(response.headers); @@ -75,7 +76,7 @@ export async function preflightNativeMessagesResponse( return { kind: "failed", response: Response.json({ error: { ...(typeof failure.type === "string" ? { type: failure.type } : {}), ...(typeof failure.code === "string" ? { code: failure.code } : {}), - message: logCtx.upstreamError, + message, } }, { status: failureStatus, headers }) }; } let index = 0; diff --git a/src/server/messages-shared.ts b/src/server/messages-shared.ts index 5b026cebf6d..881611d58f4 100644 --- a/src/server/messages-shared.ts +++ b/src/server/messages-shared.ts @@ -109,7 +109,8 @@ export function tapAnthropicSseForLog( terminalSeen = true; if (data.type === "error" && isRec(data.error)) { terminalErrorStatus = httpStatusFromTerminalError(data.error) ?? 502; - logCtx.upstreamError = redactSecretString(typeof data.error.message === "string" ? data.error.message : "Provider stream failed").slice(0, 500); + // Upstream prose can echo request content or account identifiers; retain only status. + logCtx.upstreamError = `Provider stream failed (${terminalErrorStatus})`; } } }; diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts index 879d43bed67..b8522ee3b34 100644 --- a/src/server/responses/core-combo.ts +++ b/src/server/responses/core-combo.ts @@ -1382,10 +1382,10 @@ export async function executeComboResponses( } return lastFailure; } - // A bare status made a revoked credential read as a provider outage. Log only the parsed - // code and message, never the raw envelope, as one redacted, bounded line. + // Native Messages failures can echo client content; their hop warning uses only status. + // Other wires retain their parsed, redacted and bounded code/message diagnostic. const failureReason = sanitizeLogMetadataString( - redactSecretString([failure.upstreamCode, failure.upstreamMessage].filter(Boolean).join(": ")).replace(/\s+/g, " "), + redactSecretString(options.protocolSource?.inbound === "messages" ? `Provider error ${response.status}` : [failure.upstreamCode, failure.upstreamMessage].filter(Boolean).join(": ")).replace(/\s+/g, " "), 200, ); console.warn( diff --git a/structure/clients/claude-desktop.md b/structure/clients/claude-desktop.md index 5ca1158189c..f64d435c68f 100644 --- a/structure/clients/claude-desktop.md +++ b/structure/clients/claude-desktop.md @@ -593,7 +593,7 @@ Native Anthropic passthrough in `src/server/claude-messages.ts` forwards the cal Native passthrough retains upstream `anthropic-ratelimit-*` response headers for Claude Code quota/statusLine consumers on SSE, JSON and upstream errors. `src/server/anthropic-rate-limit-headers.ts` selects only that family instead of copying all upstream headers, so cookies and unrelated metadata are not relayed. Missing rate-limit headers are not fabricated; body, status, content type and existing non-stream `Retry-After` behavior stay unchanged. `tests/claude-integration/claude-native-rate-limit-headers.test.ts` exercises the production ingress against a synthetic upstream for all four response shapes (SSE, JSON, upstream error and count_tokens). -`tapAnthropicSseForLog` in `src/server/claude-messages.ts` relays the streamed body of both the native passthrough and the managed native Messages lane (`src/server/messages-native.ts`). The response headers are already sent, so a stall, a byte-cap overflow, or an upstream read failure ends the body with an Anthropic `event: error` frame after a blank-line boundary and a clean close: `timeout_error` for an idle stall, `api_error` for the byte cap, and `api_error` when an upstream read fails mid-stream (a socket reset). The mid-stream reset is logged like the Responses relay's read error: status 502, `terminalStatus: "failed"`, `closeReason: "terminal"`, `transportPhase: "mid_stream"`, a synthetic terminal source, the attempt marked `streamAborted`, the redacted reason in `upstreamError`, and the usage seen before the reset. The non-streaming fold in the managed lane closes its row with the tap's meta for a reset, a stall or the byte cap, so its row matches the streaming lane's. The request is not replayed. Every one of these frames logs status 502. A stall is `terminalStatus: "incomplete"` with `closeReason: "body_stall"`, the status the Responses relay gives a stall-timeout incomplete. The byte cap is `incomplete` with `closeReason: "body_overflow"`, the same 502 the non-stream passthrough answers for it. Both carry the proxy's message in `upstreamError`, and a reset is `terminalStatus: "failed"`. Either way the row keeps its failure diagnostics in usage.jsonl. A stall or overflow after the turn's own terminal (`message_stop` or an upstream `error` event) is a finished turn: it closes without an error frame, as the read-error branch does, and logs that terminal's status. The tap finds frames in a copy normalized to LF, because SSE lines may end in CRLF, LF or CR; it holds a trailing CR until the next chunk so a split CRLF stays one line ending, and forwards the original bytes unchanged. Some read failures are not upstream failures. When the cancel signal is already aborted, the rejection is a `499` client cancel, because Bun can reject the read before it dispatches the abort listener. The managed lane passes its upstream controller's signal, so shutdown and turn release count as cancels too. When `message_stop` or an upstream `error` event has been seen, including one still in the buffer without its blank-line delimiter, the turn is complete and closes with no error frame. `message_stop` logs 200. An upstream `error` event is relayed unchanged and logs the status `httpStatusFromTerminalError` maps it to, with `terminalStatus: "failed"` and the redacted message in `upstreamError`, so a failed turn never reads as a success; the managed lane's combo preflight reads the same failure before the first output byte. A terminal found only in that unterminated tail gets its blank line restored, because an SSE parser drops an event that EOF cuts off. A translator budget overflow is a local cap, so it still errors the stream, and the non-streaming fold answers it with 413. `tests/claude-integration/claude-native-passthrough.test.ts` and `tests/claude-integration/messages-native.test.ts` cover both lanes against an upstream that resets after a partial or a complete body, plus both cancel paths, which record the usage seen before the client left. +`tapAnthropicSseForLog` in `src/server/claude-messages.ts` relays the streamed body of both the native passthrough and the managed native Messages lane (`src/server/messages-native.ts`). The response headers are already sent, so a stall, a byte-cap overflow, or an upstream read failure ends the body with an Anthropic `event: error` frame after a blank-line boundary and a clean close: `timeout_error` for an idle stall, `api_error` for the byte cap, and `api_error` when an upstream read fails mid-stream (a socket reset). The mid-stream reset is logged like the Responses relay's read error: status 502, `terminalStatus: "failed"`, `closeReason: "terminal"`, `transportPhase: "mid_stream"`, a synthetic terminal source, the attempt marked `streamAborted`, the redacted reason in `upstreamError`, and the usage seen before the reset. The non-streaming fold in the managed lane closes its row with the tap's meta for a reset, a stall or the byte cap, so its row matches the streaming lane's. The request is not replayed. Every one of these frames logs status 502. A stall is `terminalStatus: "incomplete"` with `closeReason: "body_stall"`, the status the Responses relay gives a stall-timeout incomplete. The byte cap is `incomplete` with `closeReason: "body_overflow"`, the same 502 the non-stream passthrough answers for it. Both carry the proxy's message in `upstreamError`, and a reset is `terminalStatus: "failed"`. Either way the row keeps its failure diagnostics in usage.jsonl. A stall or overflow after the turn's own terminal (`message_stop` or an upstream `error` event) is a finished turn: it closes without an error frame, as the read-error branch does, and logs that terminal's status. The tap finds frames in a copy normalized to LF, because SSE lines may end in CRLF, LF or CR; it holds a trailing CR until the next chunk so a split CRLF stays one line ending, and forwards the original bytes unchanged. Some read failures are not upstream failures. When the cancel signal is already aborted, the rejection is a `499` client cancel, because Bun can reject the read before it dispatches the abort listener. The managed lane passes its upstream controller's signal, so shutdown and turn release count as cancels too. When `message_stop` or an upstream `error` event has been seen, including one still in the buffer without its blank-line delimiter, the turn is complete and closes with no error frame. `message_stop` logs 200. An upstream `error` event is relayed unchanged and logs the status `httpStatusFromTerminalError` maps it to, with `terminalStatus: "failed"` and a fixed status-only diagnostic in `upstreamError` (upstream prose may echo client content), so a failed turn never reads as a success; the managed lane's combo preflight reads the same failure before the first output byte. A terminal found only in that unterminated tail gets its blank line restored, because an SSE parser drops an event that EOF cuts off. A translator budget overflow is a local cap, so it still errors the stream, and the non-streaming fold answers it with 413. `tests/claude-integration/claude-native-passthrough.test.ts` and `tests/claude-integration/messages-native.test.ts` cover both lanes against an upstream that resets after a partial or a complete body, plus both cancel paths, which record the usage seen before the client left. A native passthrough answered without a stream records its reason in `upstreamError`, so the row and the failure diagnostics in usage.jsonl name the cause. An upstream error response (status 400 or above) is relayed verbatim; the stored diagnostic is `Provider error : ` only for a valid Anthropic error envelope with one of the closed types `invalid_request_error`, `authentication_error`, `permission_error`, `not_found_error`, `rate_limit_error`, `api_error`, `overloaded_error` or `request_too_large`. Upstream messages and arbitrary type strings never enter this diagnostic. Unknown or malformed envelopes, non-JSON bodies and bodies over 64 Ki characters log `Provider error `. Local header timeout, body stall, byte-cap overflow and cancel diagnostics contain fixed text plus validated guard limits. Fetch failures log the fixed reason `anthropic passthrough failed: upstream connection error`; the existing redacted client response is preserved. Classification uses these stored reasons and HTTP status, so a `permission_error` at 403 is `permission_denied` regardless of upstream message wording. `tests/claude-integration/claude-native-passthrough.test.ts` covers these cases and checks that echoed account identifiers and request content are absent from both history sinks. diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 31f0eb40337..0cb5adf105b 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -111,7 +111,7 @@ Plain admission leases without controller binding do not register or unregister Reasonless incomplete snapshots containing output items stay committed; an explicit `reason: null` also stays terminal. `tests/routing/combo-stream-preflight.test.ts` pins these conservative boundaries. Hop warnings in `src/server/responses/core-combo.ts` include only parsed upstream code/message, -redacted before whitespace flattening and bounded to 200 characters; unparsed bodies stay out. +redacted before whitespace flattening and bounded to 200 characters; native Messages source warnings use only status, and unparsed bodies stay out. `tests/responses/responses-combo-exhausted-error.test.ts` pins that log boundary. A `runTurn` adapter has the equivalent boundary in `preflightAdapterEvents` (`src/adapters/run-turn-queue.ts`), streaming and non-streaming alike: when its first meaningful event is a tool call the current request did not declare, and no earlier replay-unsafe heartbeat recorded a side effect, `src/server/responses/run-turn-execution.ts` projects the fail-closed undeclared-tool refusal as a pre-commit 502 so the combo can hop with the unchanged catalog. After any output or a replay-unsafe heartbeat the refusal stays with that child. Chat Completions and Anthropic Messages inbound requests do not use this classification. The same heartbeat also decides an ordinary pre-output adapter error or an empty end: after a replay-unsafe heartbeat the child's 502 is marked non-replayable, so the combo stops on it instead of sending the turn to the next target. diff --git a/tests/claude-integration/claude-native-passthrough.test.ts b/tests/claude-integration/claude-native-passthrough.test.ts index e37e6b5ea81..85b655fe19d 100644 --- a/tests/claude-integration/claude-native-passthrough.test.ts +++ b/tests/claude-integration/claude-native-passthrough.test.ts @@ -944,7 +944,7 @@ test("a read rejection that lands before the client abort listener still finaliz }); test("an upstream error event is the turn's terminal, logged as that failure rather than a 200", async () => { - const errorFrame = `event: error\ndata: ${JSON.stringify({ type: "error", error: { type: "overloaded_error", message: "Overloaded" } })}\n\n`; + const errorFrame = `event: error\ndata: ${JSON.stringify({ type: "error", error: { type: "overloaded_error", message: "fixture-private-account: private fixture prompt" } })}\n\n`; const source = new ReadableStream({ start(controller) { controller.enqueue(new TextEncoder().encode(PARTIAL_TURN_SSE + errorFrame)); @@ -958,7 +958,7 @@ test("an upstream error event is the turn's terminal, logged as that failure rat expect(await new Response(tapped).text()).toBe(PARTIAL_TURN_SSE + errorFrame); // The status every other surface maps this error to (`httpStatusFromTerminalError`). expect(calls).toEqual([{ status: 503, closeReason: "terminal", terminalStatus: "failed" }]); - expect(logCtx.upstreamError).toBe("Overloaded"); + expect(logCtx.upstreamError).toBe("Provider stream failed (503)"); expect(logCtx.usage?.inputTokens).toBe(12); }); diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 2aa8691a151..ea03f7dc705 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -206,6 +206,22 @@ describe("native Messages candidates in a combo", () => { expect(rows).toHaveLength(1); expect(rows[0]!.spend).toMatchObject({ sends: 1, unresolved: 0 }); }); + test("pre-output refusal keeps upstream account and prompt echoes out of the final diagnostic", async () => { + const echo = "fixture-private-account: private fixture prompt"; + answer = () => stream(prelude + frame("error", { error: { type: "overloaded_error", message: echo } })); + const warnings: unknown[][] = []; + const warning = spyOn(console, "warn").mockImplementation((...args) => { warnings.push(args); }); + let result: Awaited>; + try { result = await send({ tools: [{ type: "bash_20250124", name: "bash" }] }); } + finally { warning.mockRestore(); } + const { response, text, rows } = result; + expect(response.status).toBe(529); + expect(text).toContain(echo); + expect(rows).toHaveLength(1); + expect(rows[0]!.upstreamError).toBe("Provider stream failed (503)"); + expect(JSON.stringify(warnings)).not.toContain(echo); + expect(seen).toHaveLength(1); + }); for (const output of [false, true]) test(`non-streaming fold ${output ? "never replays output" : "hops before output"}`, async () => { answer = () => stream(prelude + (output ? textStart + textDelta : "") + refusal); const { text } = await send({ stream: false }); From 560e414eed5ff0785ad4fb99804908794e23f2a1 Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Wed, 28 Oct 2026 17:33:18 +0900 Subject: [PATCH 06/23] fix(combo): treat inline system tool declarations as native-only The native-only-tool fence read only top-level tools. A system message can declare a tool inline through tool_addition/tool_definition; the native child receives it, but the translated fallback keeps only system text, so a pre-output refusal fell back to a child without that tool and returned 200. Any active inline declaration now blocks the translated fallback, using the existing permanent-removal semantics. --- src/server/responses/core-combo-native.ts | 7 ++++++- tests/claude-integration/messages-native-combo.test.ts | 9 +++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index cebb347178e..ed807f19478 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -47,6 +47,7 @@ import { credentialDomainFor } from "../../protocols/opaque-state"; import { chatCompletionsErrorResponse } from "../../chat/outbound"; import { anthropicErrorResponse } from "../../claude/outbound"; import { hasNativeOnlyTool } from "../../claude/inbound-content-options"; +import { nativeMessagesInlineToolNames } from "../messages-native-tool-guard"; import { isRequestExecutionBudget } from "../../lib/request-execution-budget"; import { markResponseNonReplayable } from "../../lib/upstream-retry"; import { redactSecretString } from "../../lib/redact"; @@ -181,6 +182,10 @@ export function createComboProtocolLanes(input: { const verdicts = new Map(); let skipRecorded = false; let nativeOnlyTool: boolean | undefined; + // The bridge keeps only text from system messages, so an active inline tool_addition definition + // has no translated equivalent either; fail closed like a top-level native-only tool. + const declaresNativeOnlyTool = (body: Record) => + hasNativeOnlyTool(body) || nativeMessagesInlineToolNames(body).size > 0; const judge = (target: ComboTarget): CandidateVerdict => { const key = selector(target); const cached = verdicts.get(key); @@ -211,7 +216,7 @@ export function createComboProtocolLanes(input: { } } // Under any policy: the source chose a native-only tool, so a translated child never runs it. - if (!native && inbound === "messages" && (nativeOnlyTool ??= hasNativeOnlyTool(envelope.freshBody()))) { + if (!native && inbound === "messages" && (nativeOnlyTool ??= declaresNativeOnlyTool(envelope.freshBody()))) { verdict.nativeOnly = true; } } diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index ea03f7dc705..70e3a09fdfd 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -198,6 +198,15 @@ describe("native Messages candidates in a combo", () => { expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(portable ? ["api.anthropic.com", "backup.fixture.invalid"] : ["api.anthropic.com"]); }); } + test("an inline system-message tool declaration never falls back to a translated child", async () => { + const definition = { type: "advisor_20260301", name: "advisor", model: "claude-haiku-4-5" }; + const addition = { role: "system", content: [{ type: "tool_addition", tool: { type: "tool_definition", definition } }] }; + answer = () => stream(prelude + refusal); + const { text } = await send({ messages: [addition, ...source.messages] }); + expect(seen[0]!.body.messages).toEqual([addition, ...source.messages]); + expect(text).toContain("fixture overloaded"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); test("non-streaming caller retains exact native body and one final row", async () => { const { response, text, rows } = await send({ stream: false }); expect(response.status).toBe(200); From 22a769a43de3933608ed3c1a273d607cf45dcb98 Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Wed, 28 Oct 2026 17:50:13 +0900 Subject: [PATCH 07/23] fix(combo): fence unnamed inline tool definitions as native-only nativeMessagesInlineToolNames tracks named callable tools only, so an inline tool_definition with a missing, empty or non-string name still let a pre-output refusal fall back to a translated child without it. Such a definition cannot be withdrawn by a tool_reference and now always blocks the translated fallback. --- src/server/responses/core-combo-native.ts | 16 +++++++++++++++- .../messages-native-combo.test.ts | 4 ++-- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index ed807f19478..b0d73e6ec8f 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -48,6 +48,20 @@ import { chatCompletionsErrorResponse } from "../../chat/outbound"; import { anthropicErrorResponse } from "../../claude/outbound"; import { hasNativeOnlyTool } from "../../claude/inbound-content-options"; import { nativeMessagesInlineToolNames } from "../messages-native-tool-guard"; + +const isRecord = (value: unknown): value is Record => + value !== null && typeof value === "object" && !Array.isArray(value); + +/** + * An inline definition without a usable name cannot be withdrawn by a tool_reference, so it stays in + * force; nativeMessagesInlineToolNames tracks only named, callable tools and skips it. + */ +function hasUnnamedInlineToolDefinition(body: Record): boolean { + return Array.isArray(body.messages) && body.messages.some(message => isRecord(message) && message.role === "system" + && Array.isArray(message.content) && message.content.some(block => isRecord(block) && block.type === "tool_addition" + && isRecord(block.tool) && block.tool.type === "tool_definition" + && !(isRecord(block.tool.definition) && typeof block.tool.definition.name === "string" && block.tool.definition.name.length > 0))); +} import { isRequestExecutionBudget } from "../../lib/request-execution-budget"; import { markResponseNonReplayable } from "../../lib/upstream-retry"; import { redactSecretString } from "../../lib/redact"; @@ -185,7 +199,7 @@ export function createComboProtocolLanes(input: { // The bridge keeps only text from system messages, so an active inline tool_addition definition // has no translated equivalent either; fail closed like a top-level native-only tool. const declaresNativeOnlyTool = (body: Record) => - hasNativeOnlyTool(body) || nativeMessagesInlineToolNames(body).size > 0; + hasNativeOnlyTool(body) || nativeMessagesInlineToolNames(body).size > 0 || hasUnnamedInlineToolDefinition(body); const judge = (target: ComboTarget): CandidateVerdict => { const key = selector(target); const cached = verdicts.get(key); diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 70e3a09fdfd..d0c89374ac2 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -198,8 +198,8 @@ describe("native Messages candidates in a combo", () => { expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(portable ? ["api.anthropic.com", "backup.fixture.invalid"] : ["api.anthropic.com"]); }); } - test("an inline system-message tool declaration never falls back to a translated child", async () => { - const definition = { type: "advisor_20260301", name: "advisor", model: "claude-haiku-4-5" }; + for (const name of ["advisor", undefined, "", 42]) test(`an inline system-message tool declaration (name ${JSON.stringify(name)}) never falls back to a translated child`, async () => { + const definition = { type: "advisor_20260301", ...(name === undefined ? {} : { name }), model: "claude-haiku-4-5" }; const addition = { role: "system", content: [{ type: "tool_addition", tool: { type: "tool_definition", definition } }] }; answer = () => stream(prelude + refusal); const { text } = await send({ messages: [addition, ...source.messages] }); From 24f734c7d50c2b33e6ba2e2b53a6cf392875d090 Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Wed, 28 Oct 2026 18:07:50 +0900 Subject: [PATCH 08/23] fix(combo): fence inline withdrawal of a top-level tool as native-only A system-message tool_removal of a top-level tool reached the native child, but the translated fallback drops inline changes and re-enabled the tool. Any inline removal that names a declared top-level tool now blocks the translated fallback; a permanently withdrawn inline-only definition still falls back. --- src/server/responses/core-combo-native.ts | 13 ++++++++++++- .../messages-native-combo.test.ts | 16 ++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index b0d73e6ec8f..8ee1eb61832 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -56,6 +56,16 @@ const isRecord = (value: unknown): value is Record => * An inline definition without a usable name cannot be withdrawn by a tool_reference, so it stays in * force; nativeMessagesInlineToolNames tracks only named, callable tools and skips it. */ +/** The bridge drops inline removals too, so withdrawing a top-level tool would re-enable it there. */ +function withdrawsTopLevelTool(body: Record): boolean { + const declared = new Set(Array.isArray(body.tools) + ? body.tools.flatMap(tool => isRecord(tool) && typeof tool.name === "string" ? [tool.name] : []) : []); + return declared.size > 0 && Array.isArray(body.messages) && body.messages.some(message => isRecord(message) + && message.role === "system" && Array.isArray(message.content) && message.content.some(block => isRecord(block) + && block.type === "tool_removal" && isRecord(block.tool) && block.tool.type === "tool_reference" + && typeof block.tool.name === "string" && declared.has(block.tool.name))); +} + function hasUnnamedInlineToolDefinition(body: Record): boolean { return Array.isArray(body.messages) && body.messages.some(message => isRecord(message) && message.role === "system" && Array.isArray(message.content) && message.content.some(block => isRecord(block) && block.type === "tool_addition" @@ -199,7 +209,8 @@ export function createComboProtocolLanes(input: { // The bridge keeps only text from system messages, so an active inline tool_addition definition // has no translated equivalent either; fail closed like a top-level native-only tool. const declaresNativeOnlyTool = (body: Record) => - hasNativeOnlyTool(body) || nativeMessagesInlineToolNames(body).size > 0 || hasUnnamedInlineToolDefinition(body); + hasNativeOnlyTool(body) || nativeMessagesInlineToolNames(body).size > 0 || hasUnnamedInlineToolDefinition(body) + || withdrawsTopLevelTool(body); const judge = (target: ComboTarget): CandidateVerdict => { const key = selector(target); const cached = verdicts.get(key); diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index d0c89374ac2..6cdce86de32 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -207,6 +207,22 @@ describe("native Messages candidates in a combo", () => { expect(text).toContain("fixture overloaded"); expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); }); + for (const clearAt of [undefined, "never", "end_of_turn"]) test(`withdrawing a top-level tool inline (clear_at ${clearAt ?? "unset"}) never falls back to a translated child`, async () => { + const removal = { role: "system", ...(clearAt ? { clear_at: clearAt } : {}), content: [{ type: "tool_removal", tool: { type: "tool_reference", name: "lookup" } }] }; + answer = () => stream(prelude + refusal); + const { text } = await send({ messages: [removal, ...source.messages] }); + expect(text).toContain("fixture overloaded"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); + test("a permanently withdrawn inline-only definition may still fall back to a translated child", async () => { + const definition = { name: "inline_only", description: "inline", input_schema: { type: "object" } }; + const addition = { role: "system", content: [{ type: "tool_addition", tool: { type: "tool_definition", definition } }] }; + const removal = { role: "system", content: [{ type: "tool_removal", tool: { type: "tool_reference", name: "inline_only" } }] }; + answer = () => stream(prelude + refusal); + const { text } = await send({ messages: [addition, removal, ...source.messages] }); + expect(text).toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "backup.fixture.invalid"]); + }); test("non-streaming caller retains exact native body and one final row", async () => { const { response, text, rows } = await send({ stream: false }); expect(response.status).toBe(200); From 16c3f3dbb68b0f04dbbf59bc108fd870746eeeef Mon Sep 17 00:00:00 2001 From: JUN <243035832+lidge-jun@users.noreply.github.com> Date: Thu, 29 Oct 2026 13:47:19 +0900 Subject: [PATCH 09/23] test: pin native combo frame cap to one upstream chunk --- .../messages-native-combo-framing.test.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/claude-integration/messages-native-combo-framing.test.ts b/tests/claude-integration/messages-native-combo-framing.test.ts index 27f2e1398e7..9cbdc092b6d 100644 --- a/tests/claude-integration/messages-native-combo-framing.test.ts +++ b/tests/claude-integration/messages-native-combo-framing.test.ts @@ -68,7 +68,12 @@ for (const raw of [ ':'.repeat(MAX_CLIENT_SSE_FRAME_BYTES + 1), ]) { test(`native combo preflight commits opaque or capped bytes (${raw.length} bytes)`, async () => { - const response = new Response(raw + `event: error\ndata: ${refusal}\n\n`, { + // The frame-count cap is per upstream chunk. A Response string body may split itself; + // enqueue one explicit byte chunk so the cap, rather than Bun's chunking, is the oracle. + const bytes = new TextEncoder().encode(raw + `event: error\ndata: ${refusal}\n\n`); + const response = new Response(new ReadableStream({ + start(controller) { controller.enqueue(bytes); controller.close(); }, + }), { headers: { "content-type": "text/event-stream" }, }); const result = await preflightNativeMessagesResponse(response, From 656b10f1ea5cafad3d1afe51cca2b6d3d943ff97 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:38:31 +0800 Subject: [PATCH 10/23] fix(combo): keep inspecting native prelude frames after a retryable error A retryable error no longer stops inspection of the rest of its chunk, so content or unknown frames read with it commit the child instead of letting the combo replay a turn that carried output. Co-Authored-By: Claude Code --- src/server/messages-native-preflight.ts | 3 ++- .../transports/responses-combo-boundaries.md | 2 +- .../messages-native-combo-framing.test.ts | 18 ++++++++++++++++++ 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 41d0936ebbb..d291d310163 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -26,7 +26,8 @@ export async function preflightNativeMessagesResponse( const inspector = new BoundedSseFrameBuffer(); const decoder = new TextDecoder(); const inspectFrame = (block: Uint8Array) => { - if (committed || terminal || failure) return; + // A retryable failure keeps reading its chunk: later output in it still commits. + if (committed || terminal) return; const lines = decoder.decode(block).split(/\r\n|\r|\n/); const event = lines.map(line => sseFieldValue(line, "event")).findLast(value => value !== null) ?? null; const data = lines.map(line => sseFieldValue(line, "data")).filter(value => value !== null).join("\n"); diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 263c56101b7..9c66992ee0f 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -148,7 +148,7 @@ A `runTurn` adapter has the equivalent boundary in `preflightAdapterEvents` (`sr A native Messages child uses `src/server/messages-native-preflight.ts` after the native log tap, so the tap retains byte-stall/size/cancellation ownership. Its bounded prelude accepts only an empty-content `message_start` and `ping` as replayable. As in the native tap, the last untrimmed SSE event name must match the payload type (or be absent); a mismatch commits conservatively. A structured retryable `error` before -any other event returns to the ordinary combo failure classifier; any content block (including +any other event returns to the ordinary combo failure classifier unless a later frame read with it commits; any content block (including a tool start), unknown/malformed payload, terminal or byte/chunk cap commits conservatively. Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable. Preflight retention is charged to the translator budget and released on delivery/cancellation. diff --git a/tests/claude-integration/messages-native-combo-framing.test.ts b/tests/claude-integration/messages-native-combo-framing.test.ts index 9cbdc092b6d..b4a9c17fca9 100644 --- a/tests/claude-integration/messages-native-combo-framing.test.ts +++ b/tests/claude-integration/messages-native-combo-framing.test.ts @@ -82,3 +82,21 @@ for (const raw of [ expect(await result.response.text()).toBe(raw + `event: error\ndata: ${refusal}\n\n`); }); } + +const oneChunk = (raw: string) => new Response(new ReadableStream({ + start(controller) { controller.enqueue(new TextEncoder().encode(raw)); controller.close(); }, +}), { headers: { "content-type": "text/event-stream" } }); + +for (const [name, after, failed] of [ + ["content after a retryable error commits", 'event: content_block_start\ndata: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}\n\n', false], + ["unknown event after a retryable error commits", 'event: fixture_unknown\ndata: {"type":"fixture_unknown"}\n\n', false], + ["ping and message_stop after a retryable error keep it replayable", 'event: ping\ndata: {"type":"ping"}\n\nevent: message_stop\ndata: {"type":"message_stop"}\n\n', true], +] as const) { + test(`native combo preflight inspects frames after a retryable error in the same chunk: ${name}`, async () => { + const raw = prelude + `event: error\ndata: ${refusal}\n\n` + after; + const result = await preflightNativeMessagesResponse(oneChunk(raw), { model: "fixture", provider: "anth" }, createTestTranslatorBudget()); + expect(result.kind).toBe(failed ? "failed" : "accepted"); + if (!failed) expect(await result.response.text()).toBe(raw); + else await result.response.body?.cancel(); + }); +} From 4e3aac2a3321bc47e5f7d54bce0162d28ba8f563 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:38:51 +0800 Subject: [PATCH 11/23] fix(combo): commit a native prelude event block without data An event-named SSE block with no payload is malformed, so the native Messages preflight now commits on it as the prelude rule requires. Comment-only blocks remain skipped. Co-Authored-By: Claude Code --- src/server/messages-native-preflight.ts | 3 ++- structure/transports/responses-combo-boundaries.md | 2 +- .../messages-native-combo-framing.test.ts | 14 ++++++++++++++ 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index d291d310163..022de35003a 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -31,7 +31,8 @@ export async function preflightNativeMessagesResponse( const lines = decoder.decode(block).split(/\r\n|\r|\n/); const event = lines.map(line => sseFieldValue(line, "event")).findLast(value => value !== null) ?? null; const data = lines.map(line => sseFieldValue(line, "data")).filter(value => value !== null).join("\n"); - if (!data) return; + // A named event without a payload is malformed and commits; a comment-only block is skipped. + if (!data) { if (event !== null) committed = true; return; } let payload: unknown; try { payload = JSON.parse(data); } catch { committed = true; return; } // Match the native log tap: the last, untrimmed event field is authoritative. A payload diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 9c66992ee0f..fb6f19126de 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -149,7 +149,7 @@ A native Messages child uses `src/server/messages-native-preflight.ts` after the so the tap retains byte-stall/size/cancellation ownership. Its bounded prelude accepts only an empty-content `message_start` and `ping` as replayable. As in the native tap, the last untrimmed SSE event name must match the payload type (or be absent); a mismatch commits conservatively. A structured retryable `error` before any other event returns to the ordinary combo failure classifier unless a later frame read with it commits; any content block (including -a tool start), unknown/malformed payload, terminal or byte/chunk cap commits conservatively. +a tool start), unknown/malformed payload (a named event without data included; comment-only blocks are skipped), terminal or byte/chunk cap commits conservatively. Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable. Preflight retention is charged to the translator budget and released on delivery/cancellation. Native plans enforce the captured caller model scope, including internal calls with broad host admission. The same native executor binds the accepted stream to the parent's held lease, carries observed diff --git a/tests/claude-integration/messages-native-combo-framing.test.ts b/tests/claude-integration/messages-native-combo-framing.test.ts index b4a9c17fca9..1435b2e32a3 100644 --- a/tests/claude-integration/messages-native-combo-framing.test.ts +++ b/tests/claude-integration/messages-native-combo-framing.test.ts @@ -100,3 +100,17 @@ for (const [name, after, failed] of [ else await result.response.body?.cancel(); }); } + +for (const [name, block, failed] of [ + ["a named event without data commits", "event: content_block_start\n\n", false], + ["a named event with an empty data line commits", "event: content_block_start\ndata:\n\n", false], + ["a comment-only block stays skipped", ": keepalive\n\n", true], +] as const) { + test(`native combo preflight prelude rule: ${name}`, async () => { + const raw = prelude + block + `event: error\ndata: ${refusal}\n\n`; + const result = await preflightNativeMessagesResponse(oneChunk(raw), { model: "fixture", provider: "anth" }, createTestTranslatorBudget()); + expect(result.kind).toBe(failed ? "failed" : "accepted"); + if (!failed) expect(await result.response.text()).toBe(raw); + else await result.response.body?.cancel(); + }); +} From 5cb67c754e4edb14b62441602b1f2e1f4044e079 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:40:08 +0800 Subject: [PATCH 12/23] fix(combo): record native first output only after a content frame A native Messages combo child recorded first output as soon as preflight accepted, so ping-only or empty streams reported a TTFT. First output now waits for a content frame seen by the log tap, while a committed fold still marks its later failure non-replayable on its own. Co-Authored-By: Claude Code --- src/server/messages-native.ts | 24 ++++++++++++------- .../transports/responses-combo-boundaries.md | 2 +- .../messages-native-combo.test.ts | 17 +++++++++++++ 3 files changed, 34 insertions(+), 9 deletions(-) diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 5f37e9e429a..2eefd308e21 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -56,6 +56,7 @@ import { isNonReplayableResponse, isReplayRefusalResponse, isTransientUpstreamStatus, + markResponseNonReplayable, prepareSameTarget429Wait, REPLAY_REFUSED_STATUS, retainReplayRefusal, @@ -219,8 +220,8 @@ export function createNativeMessagesComboSource(input: Omit, onFirst: () => void): ReadableStream { +/** Relay an upstream body unchanged, recording first output on its first non-empty chunk once `ready`. */ +function observeFirstChunk(body: ReadableStream, onFirst: () => void, ready = () => true): ReadableStream { const reader = body.getReader(); let seen = false; return new ReadableStream({ @@ -230,7 +231,7 @@ function observeFirstChunk(body: ReadableStream, onFirst: () => void controller.close(); return; } - if (!seen && value.byteLength > 0) { + if (!seen && value.byteLength > 0 && ready()) { seen = true; onFirst(); } @@ -407,11 +408,19 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp attempt.usage = usage; } }; + // A combo child whose preflight accepted has committed: a later failure must not replay, + // even when no content frame (and so no first output) was seen. + let committed = false; const fail = (status: number, message: string, type?: string, code?: string): Response => { const safeMessage = redactSecretString(message); finishLog(status, safeMessage); - return anthropicErrorResponse(status, safeMessage, type, code); + const response = anthropicErrorResponse(status, safeMessage, type, code); + if (committed) markResponseNonReplayable(response); + return response; }; + // After preflight, first output waits for a content frame the log tap has seen. + const watchFirstOutput = (body: ReadableStream) => onFirstOutput + ? observeFirstChunk(body, onFirstOutput, () => logCtx.generationStartedAt !== undefined) : body; const localOAuthFailure = (error: unknown): Response | null => { if (error instanceof AnthropicAccountCooldownError) { const response = fail(429, error.message, "rate_limit_error"); @@ -985,8 +994,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp try { const preflight = await preflightNativeMessagesResponse(delivered, logCtx, translatorBudget); if (preflight.kind === "failed") { bindUsage(logCtx.usage); return preflight.response; } - delivered = preflight.response; - onFirstOutput?.(); + delivered = new Response(watchFirstOutput(preflight.response.body!), { status: preflight.response.status, headers: preflight.response.headers }); } catch (error) { cleanupAbort(); releaseStreamTurn(); upstream.abort(); if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); @@ -1010,8 +1018,8 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp finishLog(preflight.response.status, logCtx.upstreamError); return preflight.response; } - tapped = preflight.response.body!; - onFirstOutput?.(); + committed = true; + tapped = watchFirstOutput(preflight.response.body!); } const message = await collectAnthropicMessage(tapped, requestedModel, translatorBudget); cleanupAbort(); diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index fb6f19126de..3c325cde8b1 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -150,7 +150,7 @@ so the tap retains byte-stall/size/cancellation ownership. Its bounded prelude a empty-content `message_start` and `ping` as replayable. As in the native tap, the last untrimmed SSE event name must match the payload type (or be absent); a mismatch commits conservatively. A structured retryable `error` before any other event returns to the ordinary combo failure classifier unless a later frame read with it commits; any content block (including a tool start), unknown/malformed payload (a named event without data included; comment-only blocks are skipped), terminal or byte/chunk cap commits conservatively. -Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable. +Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable, while first output waits for a content frame the tap has seen. Preflight retention is charged to the translator budget and released on delivery/cancellation. Native plans enforce the captured caller model scope, including internal calls with broad host admission. The same native executor binds the accepted stream to the parent's held lease, carries observed usage on cancellation, and reports terminal errors without treating them as successful rows. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index fb795083dbc..36dc1f97ce3 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -366,6 +366,23 @@ describe("native Messages candidates in a combo", () => { expect(seen[0]!.body).toEqual({ ...source, model: "claude-sonnet-4-6", stream: true }); } finally { await server.stop(true); } }); + for (const streaming of [true, false]) test(`first output waits for a content frame (${streaming ? "stream" : "fold"})`, async () => { + const empty = frame("message_delta", { delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 0 } }) + frame("message_stop"); + answer = () => stream(prelude + empty); + const silent = await send({ stream: streaming }); + expect(silent.rows[0]!.firstOutputMs).toBeUndefined(); + expect(silent.rows[0]!.attempts![0]!.firstOutputMs).toBeUndefined(); + answer = () => stream(prelude + textStart + textDelta + terminal); + const spoken = await send({ stream: streaming }); + expect(spoken.rows[0]!.firstOutputMs).toBeNumber(); + expect(spoken.rows[0]!.attempts![0]!.firstOutputMs).toBeNumber(); + }); + test("a fold committed by a non-content frame still never replays", async () => { + answer = () => stream(prelude + frame("fixture_unknown") + refusal); + const { text } = await send({ stream: false }); + expect(text).toContain("fixture overloaded"); + expect(seen).toHaveLength(1); + }); test("native disabled keeps existing translated path", async () => { const { response } = await send({ config: config(false) }); expect(response.status).toBe(200); From b256a7e057c57d4b54483943034c56e3af92b318 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:42:01 +0800 Subject: [PATCH 13/23] fix(combo): release judged native body copies Combo eligibility charged a fresh source-body copy for every judged candidate and never released it, then charged again for the dispatched child, so several Anthropic candidates with a large body could exceed the translator budget before dispatch. The envelope now records each copy's charge and releaseBody hands it back; judging and rejected candidates release theirs, leaving only the dispatched child's copy. Co-Authored-By: Claude Code --- src/protocols/envelope.ts | 17 +++++++++++-- src/server/responses/core-combo-native.ts | 16 +++++++++--- structure/data-planes/protocol-paths.md | 3 ++- .../messages-native-combo.test.ts | 25 +++++++++++++++++++ .../responses-combo-exhausted-error.test.ts | 8 +++--- 5 files changed, 58 insertions(+), 11 deletions(-) diff --git a/src/protocols/envelope.ts b/src/protocols/envelope.ts index cdd2af534f1..76d9f6745ca 100644 --- a/src/protocols/envelope.ts +++ b/src/protocols/envelope.ts @@ -19,6 +19,8 @@ export interface ProtocolEnvelope { features(): ReadonlySet; /** A structured clone of the source body, charged to the translator budget. */ freshBody(): Record; + /** Hand back the charge of a `freshBody()` copy no longer needed; later calls are no-ops. */ + releaseBody(copy: Record): void; } export function createProtocolEnvelope(input: { @@ -28,6 +30,8 @@ export function createProtocolEnvelope(input: { }): ProtocolEnvelope { const { inbound, body, translatorBudget } = input; let features: ReadonlySet | undefined; + // The copy's own size may change once its holder rewrites it, so the charge is remembered. + const charged = new WeakMap(); return { inbound, features() { @@ -36,8 +40,17 @@ export function createProtocolEnvelope(input: { }, freshBody() { // Charged before the copy exists, so an over-budget request fails without allocating it. - translatorBudget.chargeRetained(jsonUtf8Bytes(body), { kind: "request_copies" }); - return structuredClone(body); + const bytes = jsonUtf8Bytes(body); + translatorBudget.chargeRetained(bytes, { kind: "request_copies" }); + const copy = structuredClone(body); + charged.set(copy, bytes); + return copy; + }, + releaseBody(copy) { + const bytes = charged.get(copy); + if (bytes === undefined) return; + charged.delete(copy); + translatorBudget.releaseRetained(bytes, { kind: "request_copies" }); }, }; } diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index 8ee1eb61832..192704b5047 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -198,9 +198,11 @@ export function createComboProtocolLanes(input: { // The concrete selector, as the bridge child's body carries it; the wire model comes from // the route either way, and the pinned-effort lookup reads this. body.model = selector(target); - return (inbound === "messages" + if (inbound === "messages" ? nativeMessagesDeclineReason(route, body, config, { ...(source.inbound === "messages" ? source.selector : undefined), routeSelector: selector(target) }) === undefined - : isNativeChatRouteEligible(route, body, config)) ? body : undefined; + : isNativeChatRouteEligible(route, body, config)) return body; + envelope.releaseBody(body); + return undefined; }; const verdicts = new Map(); @@ -225,7 +227,10 @@ export function createComboProtocolLanes(input: { } if (targetRoute) { const settled = settle(target, targetRoute); - const native = nativeBody(target, settled) !== undefined; + // Judging needs only the verdict; the dispatched child takes its own copy. + const judged = nativeBody(target, settled); + if (judged) envelope.releaseBody(judged); + const native = judged !== undefined; const upstream = native ? inbound : upstreamWireForAdapter((settled ?? targetRoute).provider.adapter); const checked = checkRepresentable({ inbound, @@ -241,7 +246,10 @@ export function createComboProtocolLanes(input: { } } // Under any policy: the source chose a native-only tool, so a translated child never runs it. - if (!native && inbound === "messages" && (nativeOnlyTool ??= declaresNativeOnlyTool(envelope.freshBody()))) { + if (!native && inbound === "messages" && (nativeOnlyTool ??= (() => { + const body = envelope.freshBody(); + try { return declaresNativeOnlyTool(body); } finally { envelope.releaseBody(body); } + })())) { verdict.nativeOnly = true; } } diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 5cb2e0efd59..2d1ddebc158 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -184,7 +184,8 @@ covers the source mapping, the cap, the 404 and the parameter bounds. ingress parsed. It keeps that body by reference for the request only, scans its features on the first `features()` call and caches them, and hands out `freshBody()` copies, each a `structuredClone` charged under `request_copies`, so a consumer that rewrites its body cannot -leak the rewrite into another consumer's input. +leak the rewrite into another consumer's input; `releaseBody(copy)` hands that charge back, so combo judging keeps only +the dispatched native child's copy charged. `src/protocols/codecs/{chat,messages,responses}.ts` are named entry points over the existing translators — `chatToResponsesBody` is `chatCompletionsToResponsesBody`, diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 36dc1f97ce3..3ebf3044b67 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -383,6 +383,31 @@ describe("native Messages candidates in a combo", () => { expect(text).toContain("fixture overloaded"); expect(seen).toHaveLength(1); }); + test("judging candidates leaves only the dispatched native body copy charged", async () => { + const { createComboProtocolLanes } = await import("../../src/server/responses/core-combo-native"); + const { routeConcreteModel } = await import("../../src/router"); + const { createRequestExecutionBudget } = await import("../../src/lib/request-execution-budget"); + const { jsonUtf8Bytes } = await import("../../src/lib/json-byte-size"); + const settings = config(); + settings.providers.anth!.models!.push("claude-haiku-4-5"); + settings.providers.pinned = { ...settings.providers.anth!, models: ["claude-sonnet-4-6"], pinnedReasoningEffort: "low" }; + const targets = [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "anth", model: "claude-haiku-4-5" }, + { provider: "pinned", model: "claude-sonnet-4-6" }, { provider: "backup", model: "fallback" }]; + settings.combos!.waterfall!.targets = targets; + const translatorBudget = createTranslatorBudget(); + const body = { ...source, stream: true }; + const lanes = createComboProtocolLanes({ + source: { inbound: "messages", envelope: createProtocolEnvelope({ inbound: "messages", body, translatorBudget }), dispatchNativeChild: async () => new Response() }, + req: new Request("http://localhost/v1/messages"), config: settings, logCtx: { model: "", provider: "" }, + admissionScope: undefined, comboId: "waterfall", targets, + })!; + for (const target of targets) expect(lanes.pickable(target)).toBe(true); + expect(translatorBudget.snapshot().currentBytes).toBe(0); + expect(lanes.nativeChild(targets[2]!, routeConcreteModel(settings, "pinned/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeUndefined(); + expect(translatorBudget.snapshot().currentBytes).toBe(0); + expect(lanes.nativeChild(targets[0]!, routeConcreteModel(settings, "anth/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeDefined(); + expect(translatorBudget.snapshot().currentBytes).toBe(jsonUtf8Bytes(body)); + }); test("native disabled keeps existing translated path", async () => { const { response } = await send({ config: config(false) }); expect(response.status).toBe(200); diff --git a/tests/responses/responses-combo-exhausted-error.test.ts b/tests/responses/responses-combo-exhausted-error.test.ts index 1faef1b33d9..1ccb16b4ac1 100644 --- a/tests/responses/responses-combo-exhausted-error.test.ts +++ b/tests/responses/responses-combo-exhausted-error.test.ts @@ -93,15 +93,15 @@ function upstream(fallbackStatus: number, fallbackMessage: string, primaryStatus return hosts; } -// Exercise the real reject-policy eligibility path and native dispatch seam with a budget -// for one selected target: its eligibility copy and dispatch copy, never an unused target. +// Exercise the real reject-policy eligibility path and native dispatch seam with a budget for +// one retained copy: eligibility releases its copy, so only the dispatched child's stays charged. async function nativeCombo(cfg = config, status = 200) { const nativeConfig: OcxConfig = { ...cfg, protocols: { unrepresentable: "reject", rollout: { nativeChatCombos: true } }, }; const chatBody = { model: "combo/fan", stream: false, n: 2, messages: [{ role: "user", content: "hello" }] }; - const budget = createTranslatorBudget({ maxTurnBytes: 2 * jsonUtf8Bytes(chatBody) }); + const budget = createTranslatorBudget({ maxTurnBytes: jsonUtf8Bytes(chatBody) }); const envelope = createProtocolEnvelope({ inbound: "chat", body: chatBody, translatorBudget: budget }); const copies: Record[] = []; const dispatched: string[] = []; @@ -237,7 +237,7 @@ describe("combo exhaustion reports the primary's quota refusal", () => { expect(dispatched).toEqual(["fallback"]); expect(response.status).toBe(401); expect(response.headers.get("retry-after")).toBeNull(); - // Only lazy eligibility for the cooled primary attempts a third charged body copy. + // Only lazy eligibility for the cooled primary attempts a second retained body copy. expect(overflows).toBe(1); }); From 67223c80fc39d5f93d2f9b85e0bbebbb7489405c Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:43:34 +0800 Subject: [PATCH 14/23] fix(combo): stop charging an ingress copy for the native Messages source Any combo request charged a full body copy to request_copies whenever some provider enabled managedMessagesNative, even with no native child, so a body that fits the translated path could fail with 413 translation_buffer_limit. The copy only guarded against the effort override rewrite, and an overridden request never uses the native source; the ingress now offers an unrewritten body by reference instead. Co-Authored-By: Claude Code --- src/server/claude-messages.ts | 8 ++++---- structure/transports/responses-combo-boundaries.md | 2 ++ tests/claude-integration/messages-native-combo.test.ts | 9 +++++++++ 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index 580e9455ec4..673fd771ed1 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -711,13 +711,13 @@ async function handleClaudeMessagesWithBudget( // proxy-owned `speed` + beta (anthropic-speed wire) and its usage.speed observation would be // silently skipped. Translation reaches the adapter, which owns both. const messagesBody = anthropicBody; - nativeComboSource = isRec(messagesBody) && typeof messagesBody.model === "string" + // An effort override rewrites the body below, so only an unrewritten body is a replay source; + // the envelope then keeps it by reference and charges no ingress copy. + nativeComboSource = !effortOverride && isRec(messagesBody) && typeof messagesBody.model === "string" && resolveComboId(config, resolveInboundModel(messagesBody.model, cc)) !== null && Object.keys(config.providers).some(name => resolveProtocolSettings(config, name).rollout.managedMessagesNative); if (isRec(messagesBody) && (resolveProtocolSettings(config).unrepresentable === "reject" || nativeComboSource)) { - // Keep the replay source independent of later ingress effort rewrites. envelope = createProtocolEnvelope({ inbound: "messages", body: messagesBody, translatorBudget }); - if (nativeComboSource) envelope = createProtocolEnvelope({ inbound: "messages", body: envelope.freshBody(), translatorBudget }); } const sourceEnvelope = envelope; // The bridge entry mark below reads these before an effort override rewrites `thinking`, @@ -1232,7 +1232,7 @@ async function handleClaudeMessagesWithBudget( // over, and TTL-less breakpoints take the destination default. cacheRetention: config.cacheRetention ?? callerCacheRetention(anthropicBody), }, logCtx, { - ...(nativeComboSource && !effortOverride && envelope && settledRoute?.combo ? { + ...(nativeComboSource && envelope && settledRoute?.combo ? { protocolSource: (await import("./messages-native")).createNativeMessagesComboSource({ req, config, logCtx, envelope, requestedModel, translatorBudget, selector: nativeSelector, modelScope: resolveAdmissionModelScope(config, logIds?.admission), diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index 3c325cde8b1..327a3326b39 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -152,6 +152,8 @@ any other event returns to the ordinary combo failure classifier unless a later a tool start), unknown/malformed payload (a named event without data included; comment-only blocks are skipped), terminal or byte/chunk cap commits conservatively. Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable, while first output waits for a content frame the tap has seen. Preflight retention is charged to the translator budget and released on delivery/cancellation. +The Messages ingress offers its parsed body as that source by reference, with no ingress copy, and +only when no effort override rewrites it. Native plans enforce the captured caller model scope, including internal calls with broad host admission. The same native executor binds the accepted stream to the parent's held lease, carries observed usage on cancellation, and reports terminal errors without treating them as successful rows. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 3ebf3044b67..a695e8738aa 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -408,6 +408,15 @@ describe("native Messages candidates in a combo", () => { expect(lanes.nativeChild(targets[0]!, routeConcreteModel(settings, "anth/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeDefined(); expect(translatorBudget.snapshot().currentBytes).toBe(jsonUtf8Bytes(body)); }); + test("a large body on a combo without a native child is not charged a replay copy", async () => { + // 7 MiB fits the translated path's budget but not one extra retained copy of the body. + for (const on of [false, true]) { + const settings = config(on); + settings.combos!.waterfall!.targets = [{ provider: "backup", model: "fallback" }]; + const { response } = await send({ config: settings, stream: false, messages: [{ role: "user", content: "x".repeat(7 * 1024 * 1024) }] }); + expect(response.status).toBe(200); + } + }); test("native disabled keeps existing translated path", async () => { const { response } = await send({ config: config(false) }); expect(response.status).toBe(200); From 6b26e58af22429a36c2245123315f5a443f6b7e2 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Tue, 3 Nov 2026 13:44:17 +0800 Subject: [PATCH 15/23] fix(combo): keep the envelope release note within the protocol-paths budget Co-Authored-By: Claude Code --- structure/data-planes/protocol-paths.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 2d1ddebc158..77601974537 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -184,8 +184,7 @@ covers the source mapping, the cap, the 404 and the parameter bounds. ingress parsed. It keeps that body by reference for the request only, scans its features on the first `features()` call and caches them, and hands out `freshBody()` copies, each a `structuredClone` charged under `request_copies`, so a consumer that rewrites its body cannot -leak the rewrite into another consumer's input; `releaseBody(copy)` hands that charge back, so combo judging keeps only -the dispatched native child's copy charged. +leak the rewrite into another consumer's input; `releaseBody(copy)` hands that charge back, so combo judging keeps only the dispatched native child's copy charged. `src/protocols/codecs/{chat,messages,responses}.ts` are named entry points over the existing translators — `chatToResponsesBody` is `chatCompletionsToResponsesBody`, From 3df2b15a5e642adf53e36abcaeb00892b7aa30c6 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Fri, 6 Nov 2026 12:52:10 +0800 Subject: [PATCH 16/23] fix(combo): address review on native Messages combo replay - Judge candidates against the source in place (`envelope.readBody()`), so judging charges no copy and can no longer refuse the turn on the translator budget; only a dispatched native child takes a charged copy. - Treat a declared `web_fetch_*` as native-only unless the translated child is first-party Anthropic, the only lane that carries it out of body. - Answer with the caller's model when an `ocx-route` directive chose the combo, as the translated lane does; admission and tool scope keep the routed selector. - Record no first output for a JSON answer with no content blocks. - protocol-paths: a legacy-policy Messages combo request still builds a replay envelope. Co-Authored-By: Claude Code --- src/protocols/envelope.ts | 5 ++ src/server/claude-messages.ts | 1 + src/server/messages-native.ts | 10 ++-- src/server/responses/core-combo-native.ts | 46 +++++++++---------- structure/data-planes/protocol-paths.md | 8 ++-- .../messages-native-combo.test.ts | 31 ++++++++++++- 6 files changed, 68 insertions(+), 33 deletions(-) diff --git a/src/protocols/envelope.ts b/src/protocols/envelope.ts index 76d9f6745ca..7c779dac19b 100644 --- a/src/protocols/envelope.ts +++ b/src/protocols/envelope.ts @@ -17,6 +17,8 @@ export interface ProtocolEnvelope { readonly inbound: Protocol; /** The request features, scanned from the source body on the first call and cached. */ features(): ReadonlySet; + /** The source body itself, uncharged, for checks that only read it; a rewrite takes `freshBody()`. */ + readBody(): Readonly>; /** A structured clone of the source body, charged to the translator budget. */ freshBody(): Record; /** Hand back the charge of a `freshBody()` copy no longer needed; later calls are no-ops. */ @@ -38,6 +40,9 @@ export function createProtocolEnvelope(input: { features ??= featuresFromBody(inbound, body); return features; }, + readBody() { + return body; + }, freshBody() { // Charged before the copy exists, so an over-budget request fails without allocating it. const bytes = jsonUtf8Bytes(body); diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index 9154e0abca9..da26632962e 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -1274,6 +1274,7 @@ async function handleClaudeMessagesWithBudget( ...(nativeComboSource && envelope && settledRoute?.combo ? { protocolSource: (await import("./messages-native")).createNativeMessagesComboSource({ req, config, logCtx, envelope, requestedModel, translatorBudget, selector: nativeSelector, + echoModel: clientFacingModelText(clientModelLabel), modelScope: resolveAdmissionModelScope(config, logIds?.admission), callerAnthropicBeta: req.headers.get("anthropic-beta"), clientIdentity: captureAnthropicClientIdentity(req.headers), diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 937c422d72a..718a03ba328 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -183,6 +183,8 @@ export interface HandleNativeMessagesOptions { body: Rec; /** The selector the client sent, echoed on the request log. */ requestedModel: string; + /** The model the answer reports, when it differs from `requestedModel` (a combo the route directive chose). */ + echoModel?: string; translatorBudget: TranslatorBudget; /** The facts the ingress judged eligibility with; re-applied if key selection changes. */ selector?: NativeMessagesSelector; @@ -992,7 +994,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp ? restoreOAuthToolNamesInSse(observed, activeRequest.oauthToolNames, translatorBudget) : observed; const frames = new MessagesSseFrameContext(); - const echoed = echoRequestedModel(renamed, requestedModel, frames); + const echoed = echoRequestedModel(renamed, options.echoModel ?? requestedModel, frames); let toolRefusal: string | undefined; const source = clientDeclaredEmptyToolCatalog ? guardNativeMessagesEmptyToolCatalog(echoed, translatorBudget, message => { @@ -1056,7 +1058,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp committed = true; tapped = watchFirstOutput(preflight.response.body!); } - const message = await collectAnthropicMessage(tapped, requestedModel, translatorBudget, collectorVerdictSink); + const message = await collectAnthropicMessage(tapped, options.echoModel ?? requestedModel, translatorBudget, collectorVerdictSink); cleanupAbort(); if (tapState.closeReason === "client_cancel" || req.signal.aborted) { return fail(499, "Client cancelled request", "api_error"); @@ -1154,7 +1156,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp return fail(502, undeclaredToolCallMessage(), "api_error", UNDECLARED_TOOL_CALL_ERROR_CODE); } bindUsage(anthropicUsageToOcx(isRec(message.usage) ? message.usage : undefined)); - onFirstOutput?.(); + if (Array.isArray(message.content) && message.content.length > 0) onFirstOutput?.(); try { translatorBudget.chargeRetained(Buffer.byteLength(read.text) * 2, { kind: "live_transient" }); } catch (error) { @@ -1164,7 +1166,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp throw error; } // The client's selector, not the wire id, as the translated lane answers. - message.model = requestedModel; + message.model = options.echoModel ?? requestedModel; const serialized = JSON.stringify(message); finishLog(200); if (requestedStream) { diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index cb8b24c5e72..7a06e999822 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -47,7 +47,7 @@ import type { NativeMessagesFinishLog } from "../messages-native"; import { credentialDomainFor } from "../../protocols/opaque-state"; import { chatCompletionsErrorResponse } from "../../chat/outbound"; import { anthropicErrorResponse } from "../../claude/outbound"; -import { hasNativeOnlyTool } from "../../claude/inbound-content-options"; +import { declaredAnthropicServerTools, hasNativeOnlyTool } from "../../claude/inbound-content-options"; import { nativeMessagesInlineToolNames } from "../messages-native-tool-guard"; const isRecord = (value: unknown): value is Record => @@ -184,31 +184,29 @@ export function createComboProtocolLanes(input: { } }; - // Eligibility reads a body, and a body is a charged copy, so the adapter is checked first: - // a candidate that can never go native costs no copy at all. - const nativeBody = (target: ComboTarget, route: RouteResult | undefined): Rec | undefined => { - if (!route || route.provider.adapter !== (inbound === "messages" ? "anthropic" : "openai-chat")) return undefined; + // Eligibility only reads the body, so it reads the source in place: judging a candidate charges + // no copy and can never refuse the turn on the translator budget. Only a dispatch takes a copy. + const nativeEligible = (target: ComboTarget, route: RouteResult | undefined): route is RouteResult => { + if (!route || route.provider.adapter !== (inbound === "messages" ? "anthropic" : "openai-chat")) return false; if (inbound === "messages") { const combo = getCombo(config, comboId); // ponytail: exact replay supports no combo effort rewrite; the existing bridge owns those policies. if (!combo || combo.strategy !== "failover" || comboDefaultEffort(config, comboId) || combo.reasoningEffortMode !== "strict" || !credentialDomainFor(route.provider)?.firstPartyAnthropic - || supportedLadderFor({ provider: route.provider, modelId: route.modelId })?.length === 0) return undefined; + || supportedLadderFor({ provider: route.provider, modelId: route.modelId })?.length === 0) return false; } - const body = envelope.freshBody(); // The concrete selector, as the bridge child's body carries it; the wire model comes from // the route either way, and the pinned-effort lookup reads this. - body.model = selector(target); - if (inbound === "messages" + const body = { ...envelope.readBody(), model: selector(target) }; + return inbound === "messages" ? nativeMessagesDeclineReason(route, body, config, { ...(source.inbound === "messages" ? source.selector : undefined), routeSelector: selector(target) }) === undefined - : isNativeChatRouteEligible(route, body, config)) return body; - envelope.releaseBody(body); - return undefined; + : isNativeChatRouteEligible(route, body, config); }; const verdicts = new Map(); let skipRecorded = false; let nativeOnlyTool: boolean | undefined; + let declaresWebFetch: boolean | undefined; // The bridge keeps only text from system messages, so an active inline tool_addition definition // has no translated equivalent either; fail closed like a top-level native-only tool. const declaresNativeOnlyTool = (body: Record) => @@ -228,10 +226,7 @@ export function createComboProtocolLanes(input: { } if (targetRoute) { const settled = settle(target, targetRoute); - // Judging needs only the verdict; the dispatched child takes its own copy. - const judged = nativeBody(target, settled); - if (judged) envelope.releaseBody(judged); - const native = judged !== undefined; + const native = nativeEligible(target, settled); const upstream = native ? inbound : upstreamWireForAdapter((settled ?? targetRoute).provider.adapter); const checked = checkRepresentable({ inbound, @@ -247,11 +242,15 @@ export function createComboProtocolLanes(input: { } } // Under any policy: the source chose a native-only tool, so a translated child never runs it. - if (!native && inbound === "messages" && (nativeOnlyTool ??= (() => { - const body = envelope.freshBody(); - try { return declaresNativeOnlyTool(body); } finally { envelope.releaseBody(body); } - })())) { - verdict.nativeOnly = true; + // web_fetch has no translated equivalent; only a first-party Anthropic child carries it out of body. + if (!native && inbound === "messages") { + const provider = (settled ?? targetRoute).provider; + if ((nativeOnlyTool ??= declaresNativeOnlyTool(envelope.readBody())) + || ((declaresWebFetch ??= declaredAnthropicServerTools(envelope.readBody()) + ?.some(tool => String(tool.type).startsWith("web_fetch_")) === true) + && !(provider.adapter === "anthropic" && credentialDomainFor(provider)?.firstPartyAnthropic))) { + verdict.nativeOnly = true; + } } } verdicts.set(key, verdict); @@ -286,8 +285,9 @@ export function createComboProtocolLanes(input: { // keeps the bridge rather than opening a tracker of its own. if (!isRequestExecutionBudget(targetSendBudget)) return undefined; const route = settle(target, targetRoute); - const body = nativeBody(target, route); - if (!route || !body) return undefined; + if (!nativeEligible(target, route)) return undefined; + const body = envelope.freshBody(); + body.model = selector(target); if (inbound === "messages") return initialDecision ? undefined : { route, body, sendBudget: targetSendBudget }; applyDroidReasoningDefault(body, droidDefaultEffort, { provider: targetRoute.provider, diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 997229c520a..3a943b7ce1d 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -182,7 +182,7 @@ covers the source mapping, the cap, the 404 and the parameter bounds. ingress parsed. It keeps that body by reference for the request only, scans its features on the first `features()` call and caches them, and hands out `freshBody()` copies, each a `structuredClone` charged under `request_copies`, so a consumer that rewrites its body cannot -leak the rewrite into another consumer's input; `releaseBody(copy)` hands that charge back, so combo judging keeps only the dispatched native child's copy charged. +leak the rewrite into another consumer's input; `releaseBody(copy)` hands that charge back. `readBody()` returns the source itself, uncharged, for read-only checks, so combo judging charges no copy; only a dispatched native child holds one. `src/protocols/codecs/{chat,messages,responses}.ts` are named entry points over the existing translators — `chatToResponsesBody` is `chatCompletionsToResponsesBody`, @@ -197,7 +197,7 @@ Messages ingresses call the bridge through them. hop (the internal Responses body) still does. Only when `resolveProtocolSettings(config).unrepresentable === "reject"` do the Chat and Messages -ingresses build an envelope and run the guard, after the route and its wire settle and before the +ingresses run the guard, and build an envelope for it, after the route and its wire settle and before the request is sent: Chat on the native path when the native lane was chosen, otherwise on the bridge path to the settled adapter's wire; Messages on the native path when the managed native lane was chosen, otherwise on the bridge path. Combo and policy routes @@ -205,8 +205,8 @@ and an unroutable model are not judged at ingress; with `nativeChatCombos` on, a candidates are judged one by one inside the combo loop (below). A refusal answers 400 in the ingress's own error shape (Chat `invalid_request_error` / `unsupported_feature`; Anthropic `invalid_request_error`) naming feature keys only, marks the trace blocked, and writes the -final log row with no upstream send. Under the default `legacy` policy nothing is built and the -would-be loss appears only as the trace's `featureEffects`. The Messages envelope's features are +final log row with no upstream send. Under the default `legacy` policy the guard does not run and the would-be loss appears only as the trace's +`featureEffects`; a Messages combo-model request still builds an envelope, as a replay source, when some provider enables `managedMessagesNative`. The Messages envelope's features are fixed at the bridge entry mark, before an effort override rewrites `thinking`. `tests/responses/protocol-envelope.test.ts`, `tests/responses/protocol-guard.test.ts` and `tests/responses/protocol-ingress-guard.test.ts` pin them. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 14d4eae0153..acf3893f51c 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -90,11 +90,11 @@ function config(on = true): OcxConfig { }, combos: { waterfall: { strategy: "failover", targets: [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "backup", model: "fallback" }] } }, protocols: { rollout: { managedMessagesNative: on } } }; } -async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown; tools?: unknown; admission?: DataPlaneAdmission } = {}) { +async function send(options: { stream?: boolean; config?: OcxConfig; messages?: unknown; tools?: unknown; admission?: DataPlaneAdmission; extra?: Rec } = {}) { const requestId = `native-combo-${crypto.randomUUID()}`; const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { method: "POST", headers: { "content-type": "application/json", authorization: "Bearer fixture-admission", "x-api-key": "fixture-caller" }, - body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}), ...(options.tools ? { tools: options.tools } : {}) }), + body: JSON.stringify({ ...source, stream: options.stream ?? true, ...(options.messages ? { messages: options.messages } : {}), ...(options.tools ? { tools: options.tools } : {}), ...options.extra }), }), options.config ?? config(), { model: "", provider: "" }, { requestId, start: Date.now(), admission: options.admission }); const text = await response.text(); return { response, text, rows: getRequestLogEntries().filter(row => row.requestId === requestId) }; @@ -406,6 +406,33 @@ describe("native Messages candidates in a combo", () => { expect(translatorBudget.snapshot().currentBytes).toBe(0); expect(lanes.nativeChild(targets[0]!, routeConcreteModel(settings, "anth/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeDefined(); expect(translatorBudget.snapshot().currentBytes).toBe(jsonUtf8Bytes(body)); + // A budget with no room for one more copy still judges every candidate: judging reads in place. + const tight = createTranslatorBudget({ maxTurnBytes: jsonUtf8Bytes(body) - 1 }); + const tightLanes = createComboProtocolLanes({ + source: { inbound: "messages", envelope: createProtocolEnvelope({ inbound: "messages", body, translatorBudget: tight }), dispatchNativeChild: async () => new Response() }, + req: new Request("http://localhost/v1/messages"), config: settings, logCtx: { model: "", provider: "" }, + admissionScope: undefined, comboId: "waterfall", targets, + })!; + for (const target of targets) expect(tightLanes.pickable(target)).toBe(true); + expect(tight.snapshot().currentBytes).toBe(0); + }); + test("a declared web_fetch never falls back to a translated child that drops it", async () => { + answer = () => stream(prelude + refusal); + const { text } = await send({ tools: [...source.tools, { type: "web_fetch_20250910", name: "web_fetch" }] }); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); + test("a combo chosen by an ocx-route directive answers with the caller's model", async () => { + const system = [...source.system, { type: "text", text: "" }]; + const { response, text } = await send({ stream: false, extra: { model: "claude-haiku-4-5", system } }); + expect(response.status).toBe(200); + expect(JSON.parse(text).model).toBe("claude-haiku-4-5"); + }); + test("a JSON answer with no content blocks records no first output", async () => { + answer = () => Response.json({ ...message, content: [] }); + const { rows } = await send({ stream: false }); + expect(rows[0]!.firstOutputMs).toBeUndefined(); + expect(rows[0]!.attempts![0]!.firstOutputMs).toBeUndefined(); }); test("a large body on a combo without a native child is not charged a replay copy", async () => { // 7 MiB fits the translated path's budget but not one extra retained copy of the body. From 863dc3e519cb95cdf976f127649402e9fece0c50 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Fri, 6 Nov 2026 13:17:29 +0800 Subject: [PATCH 17/23] fix(combo): fence restricted search and release a failed native child's copy - A domain- or location-restricted `web_search_*` is carried only by the out-of-body lane to a first-party Anthropic child, like `web_fetch_*`; any other translated child is unpickable for it. Unrestricted search keeps its sidecar fallback. - A native child that fails hands its charged body copy back before failover, so a later child's copy is not refused on a stale charge. - protocol-paths: describe which web tools stay portable to which children. Co-Authored-By: Claude Code --- src/claude/inbound-content-options.ts | 15 ++++++-- src/server/responses/core-combo-native.ts | 17 ++++++--- structure/data-planes/protocol-paths.md | 2 +- .../messages-native-combo.test.ts | 37 ++++++++++++++++++- 4 files changed, 59 insertions(+), 12 deletions(-) diff --git a/src/claude/inbound-content-options.ts b/src/claude/inbound-content-options.ts index 2349d9dc700..bf67365bc33 100644 --- a/src/claude/inbound-content-options.ts +++ b/src/claude/inbound-content-options.ts @@ -53,14 +53,17 @@ export function systemCachePrefix(system: unknown): string | undefined { return last >= 0 && last < system.length - 1 ? systemToInstructions(system.slice(0, last + 1)) : undefined; } +function isRestrictedSearch(raw: unknown): boolean { + return isRec(raw) && typeof raw.type === "string" && raw.type.startsWith("web_search") + && ["allowed_domains", "blocked_domains", "user_location"].some(key => Object.hasOwn(raw, key)); +} + export function toolsToResponses(tools: unknown): Rec[] | undefined { if (!Array.isArray(tools) || tools.length === 0) return undefined; // The shared hosted-search lane cannot enforce these constraints on every destination or // sidecar backend. Omit search rather than broaden it; a duplicate declaration cannot opt out. // Typed originals still travel separately to an Anthropic destination. - const restrictedSearch = tools.some(raw => isRec(raw) && typeof raw.type === "string" - && raw.type.startsWith("web_search") - && ["allowed_domains", "blocked_domains", "user_location"].some(key => Object.hasOwn(raw, key))); + const restrictedSearch = tools.some(isRestrictedSearch); const out: Rec[] = []; for (const raw of tools) { if (!isRec(raw)) continue; @@ -112,6 +115,12 @@ export function hasNativeOnlyTool(raw: Rec): boolean { && raw.tools.some(tool => !toolsToResponses([tool]) && !declaredAnthropicServerTools({ tools: [tool] })); } +/** A declared web_fetch or restricted web_search: only the out-of-body lane to first-party Anthropic carries it. */ +export function hasAnthropicOnlyServerTool(raw: Rec): boolean { + return Array.isArray(raw.tools) && raw.tools.some(tool => isRestrictedSearch(tool) + || (isRec(tool) && typeof tool.type === "string" && tool.type.startsWith("web_fetch_"))); +} + /** Validate original call/result order without inventing inputs or pairing a later call backwards. */ function replayableServerRun(run: readonly Rec[]): boolean { const pending = new Map(); diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index 7a06e999822..d68ecd6b8e3 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -47,7 +47,7 @@ import type { NativeMessagesFinishLog } from "../messages-native"; import { credentialDomainFor } from "../../protocols/opaque-state"; import { chatCompletionsErrorResponse } from "../../chat/outbound"; import { anthropicErrorResponse } from "../../claude/outbound"; -import { declaredAnthropicServerTools, hasNativeOnlyTool } from "../../claude/inbound-content-options"; +import { hasAnthropicOnlyServerTool, hasNativeOnlyTool } from "../../claude/inbound-content-options"; import { nativeMessagesInlineToolNames } from "../messages-native-tool-guard"; const isRecord = (value: unknown): value is Record => @@ -206,7 +206,7 @@ export function createComboProtocolLanes(input: { const verdicts = new Map(); let skipRecorded = false; let nativeOnlyTool: boolean | undefined; - let declaresWebFetch: boolean | undefined; + let anthropicOnlyTool: boolean | undefined; // The bridge keeps only text from system messages, so an active inline tool_addition definition // has no translated equivalent either; fail closed like a top-level native-only tool. const declaresNativeOnlyTool = (body: Record) => @@ -242,12 +242,11 @@ export function createComboProtocolLanes(input: { } } // Under any policy: the source chose a native-only tool, so a translated child never runs it. - // web_fetch has no translated equivalent; only a first-party Anthropic child carries it out of body. + // web_fetch and a restricted web_search only travel out of body, to a first-party Anthropic child. if (!native && inbound === "messages") { const provider = (settled ?? targetRoute).provider; if ((nativeOnlyTool ??= declaresNativeOnlyTool(envelope.readBody())) - || ((declaresWebFetch ??= declaredAnthropicServerTools(envelope.readBody()) - ?.some(tool => String(tool.type).startsWith("web_fetch_")) === true) + || ((anthropicOnlyTool ??= hasAnthropicOnlyServerTool(envelope.readBody())) && !(provider.adapter === "anthropic" && credentialDomainFor(provider)?.firstPartyAnthropic))) { verdict.nativeOnly = true; } @@ -421,7 +420,9 @@ export async function dispatchNativeComboChild(input: { input.onFirstOutput(); }, }; - const response = await (input.source.inbound === "messages" + let response: Response | undefined; + try { + response = await (input.source.inbound === "messages" ? input.source.dispatchNativeChild({ ...child, finishLog: (status, message, meta) => { if (meta?.terminalStatus === "incomplete") { childLog.terminalHttpStatus = status; @@ -433,6 +434,10 @@ export async function dispatchNativeComboChild(input: { : meta?.closeReason && meta.closeReason !== "non_stream" ? "terminal" : "non_stream"); } }) : input.source.dispatchNativeChild({ ...child, finishLog })); + } finally { + // A failed child's copy is spent; the next target takes its own, so this charge must not outlive it. + if (!response?.ok) input.source.envelope.releaseBody(plan.body); + } if (!response.ok && outputSeen) markResponseNonReplayable(response); return markClientWire(response, inbound); } diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 3a943b7ce1d..2d11a72f4cc 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -376,7 +376,7 @@ probe uses the bridge's credential-free affinity headers, existing plan and auth short-lived selection lease; uncertain admission keeps the bridge. Without a runnable plan search alone does not decline native Messages. `count_tokens` and the planner lack runtime facts and remain conservative. -Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. A source declaring a tool the translated body drops with no out-of-body lane (`hasNativeOnlyTool` in `src/claude/inbound-content-options.ts`: `bash_*`, `text_editor_*`, ...; `web_search_*`/`web_fetch_*` stay portable) makes every translated child unpickable under any `unrepresentable` policy, so a refused native child surfaces its own failure instead of continuing without the tool; non-combo translated routes are unchanged. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. +Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. A source declaring a tool the translated body drops with no out-of-body lane (`hasNativeOnlyTool` in `src/claude/inbound-content-options.ts`: `bash_*`, `text_editor_*`, ...; unrestricted `web_search_*` stays portable, while `web_fetch_*` and a domain- or location-restricted `web_search_*` stay portable only to a first-party Anthropic bridge child) makes every translated child unpickable under any `unrepresentable` policy, so a refused native child surfaces its own failure instead of continuing without the tool; non-combo translated routes are unchanged. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. With the switch on, a declined route re-marks its bridge entry with the decline reason (after any `effort-row` / `fast-row`), so the trace says why the bridge was taken; with it off nothing is diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index acf3893f51c..63063f68308 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -395,8 +395,9 @@ describe("native Messages candidates in a combo", () => { settings.combos!.waterfall!.targets = targets; const translatorBudget = createTranslatorBudget(); const body = { ...source, stream: true }; + const envelope = createProtocolEnvelope({ inbound: "messages", body, translatorBudget }); const lanes = createComboProtocolLanes({ - source: { inbound: "messages", envelope: createProtocolEnvelope({ inbound: "messages", body, translatorBudget }), dispatchNativeChild: async () => new Response() }, + source: { inbound: "messages", envelope, dispatchNativeChild: async () => new Response() }, req: new Request("http://localhost/v1/messages"), config: settings, logCtx: { model: "", provider: "" }, admissionScope: undefined, comboId: "waterfall", targets, })!; @@ -404,8 +405,18 @@ describe("native Messages candidates in a combo", () => { expect(translatorBudget.snapshot().currentBytes).toBe(0); expect(lanes.nativeChild(targets[2]!, routeConcreteModel(settings, "pinned/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeUndefined(); expect(translatorBudget.snapshot().currentBytes).toBe(0); - expect(lanes.nativeChild(targets[0]!, routeConcreteModel(settings, "anth/claude-sonnet-4-6"), createRequestExecutionBudget())).toBeDefined(); + const plan = lanes.nativeChild(targets[0]!, routeConcreteModel(settings, "anth/claude-sonnet-4-6"), createRequestExecutionBudget()); + expect(plan).toBeDefined(); expect(translatorBudget.snapshot().currentBytes).toBe(jsonUtf8Bytes(body)); + // A failed child hands its copy back before failover, so the next target's copy fits again. + const { dispatchNativeComboChild } = await import("../../src/server/responses/core-combo-native"); + const failed = await dispatchNativeComboChild({ + source: { inbound: "messages", envelope, dispatchNativeChild: async () => new Response("{}", { status: 503 }) }, + plan: plan!, logCtx: { model: "", provider: "" }, childLog: { model: "", provider: "" }, attempt: {} as never, startedAt: Date.now(), + onFirstOutput: () => {}, callbacks: { onTerminal: () => {}, onCancel: () => {}, onResponseComplete: () => {} }, + }); + expect(failed.status).toBe(503); + expect(translatorBudget.snapshot().currentBytes).toBe(0); // A budget with no room for one more copy still judges every candidate: judging reads in place. const tight = createTranslatorBudget({ maxTurnBytes: jsonUtf8Bytes(body) - 1 }); const tightLanes = createComboProtocolLanes({ @@ -422,6 +433,28 @@ describe("native Messages candidates in a combo", () => { expect(text).not.toContain("fallback reply"); expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); }); + test("a restricted web_search falls back only to a child that carries its restriction", async () => { + answer = () => stream(prelude + refusal); + const restricted = [...source.tools, { type: "web_search_20250305", name: "web_search", allowed_domains: ["example.com"] }]; + const blocked = await send({ tools: restricted }); + expect(blocked.text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + seen = []; + clearComboTargetCooldowns(); + const open = await send({ tools: [...source.tools, { type: "web_search_20250305", name: "web_search" }] }); + expect(open.text).toContain("fallback reply"); + // A first-party Anthropic bridge child carries the typed original, so it stays a fallback. + seen = []; + clearComboTargetCooldowns(); + const settings = config(); + settings.providers.pinned = { ...settings.providers.anth!, pinnedReasoningEffort: "low" }; + settings.combos!.waterfall!.targets = [{ provider: "anth", model: "claude-sonnet-4-6" }, { provider: "pinned", model: "claude-sonnet-4-6" }]; + answer = (_body, _native) => seen.length === 1 ? stream(prelude + refusal) : stream(prelude + textStart + textDelta + terminal); + const bridged = await send({ config: settings, tools: restricted }); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "api.anthropic.com"]); + expect(seen[1]!.body.tools).toContainEqual(expect.objectContaining({ type: "web_search_20250305", allowed_domains: ["example.com"] })); + expect(bridged.response.status).toBe(200); + }); test("a combo chosen by an ocx-route directive answers with the caller's model", async () => { const system = [...source.system, { type: "text", text: "" }]; const { response, text } = await send({ stream: false, extra: { model: "claude-haiku-4-5", system } }); From 68152bdb1ef6a418ee8070f654b7916a209485ce Mon Sep 17 00:00:00 2001 From: Vadym O Date: Sun, 8 Nov 2026 01:32:01 +0800 Subject: [PATCH 18/23] fix(combo): keep committed native Messages failures unreplayable and their verdicts Addresses the 2026-11-07 maintainer reviews on head e73d594: - A non-streaming fold that committed and then collected a malformed or mismatched error event returns its 502 marked non-replayable. - A preflight buffer failure after the tap saw a content frame is marked non-replayable on both lanes. - An observe-only combo send reports its workflow start as already counted, as native Chat does, so a prepaid start is not charged to the root twice. - A reader cancel or signal abort publishes the usage already received before settling, and the fold binds it before its 499. - A 200 stream whose terminal was an upstream error event closes its combo child as failed with that error's status instead of completed. The 895a85989 parity goldens now allow the cancel rows' newly kept usage. Co-Authored-By: Claude Code --- src/server/messages-native.ts | 13 ++++- src/server/messages-shared.ts | 8 +++ src/server/request-log.ts | 2 + src/server/responses/core-combo-native.ts | 2 + .../messages-native-combo.test.ts | 58 +++++++++++++++++++ .../messages-native-stream-parity.test.ts | 6 +- .../messages-shared-sse.test.ts | 11 ++++ tests/helpers/messages-sse-assertions.ts | 6 +- 8 files changed, 100 insertions(+), 6 deletions(-) diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 60f3d27aaf9..61f83490b5b 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -667,7 +667,8 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp modelId: route.modelId, onPhysicalDispatch: options.sendBudget && options.comboDispatchPermit ? () => { if (!physicalBudget.spendEnforced) { - reportDispatchSends(options.sendBudget!, 1, options.comboDispatchPermit); + // As native Chat: an observe-only start was already claimed before the send. + reportDispatchSends(options.sendBudget!, 1, options.comboDispatchPermit, true); initialSendStarted = true; } } : undefined, @@ -1052,6 +1053,8 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp delivered = new Response(watchFirstOutput(preflight.response.body!), { status: preflight.response.status, headers: preflight.response.headers }); } catch (error) { cleanupAbort(); releaseStreamTurn(); upstream.abort(); + // A content frame the tap already saw ran upstream: the failure must not replay. + if (logCtx.generationStartedAt !== undefined) committed = true; if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); if (isTranslatorBudgetExceededError(error)) return fail(502, "upstream translation buffer exceeded the safe limit", "api_error", "translation_buffer_limit"); return fail(502, "upstream Messages stream failed before output", "api_error"); @@ -1079,10 +1082,10 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp } const message = await collectAnthropicMessage(tapped, options.echoModel ?? requestedModel, translatorBudget, collectorVerdictSink); cleanupAbort(); + bindUsage(logCtx.usage); if (tapState.closeReason === "client_cancel" || req.signal.aborted) { return fail(499, "Client cancelled request", "api_error"); } - bindUsage(logCtx.usage); if (message.type === "error") { const error = isRec(message.error) ? message.error : {}; const text = typeof error.message === "string" ? error.message : "upstream stream failed"; @@ -1125,10 +1128,14 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp overflow: messagesRelayedOverflowPresentation(JSON.stringify(collected), collected), evidence: { status: 200, message: typeof originalError.message === "string" ? originalError.message : undefined }, }) : message; - return retainUpstreamMessagesRequestId(Response.json(clientMessage, { status: finalStatus }), response.headers); + const collectedResponse = retainUpstreamMessagesRequestId(Response.json(clientMessage, { status: finalStatus }), response.headers); + if (committed && finalStatus >= 400) markResponseNonReplayable(collectedResponse); + return collectedResponse; } catch (error) { cleanupAbort(); upstream.abort(); + bindUsage(logCtx.usage); + if (logCtx.generationStartedAt !== undefined) committed = true; if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); if (isTranslatorBudgetExceededError(error)) { return fail(413, "upstream translation buffer exceeded the safe limit", "request_too_large", "translation_buffer_limit"); diff --git a/src/server/messages-shared.ts b/src/server/messages-shared.ts index ed0de1d9ef4..6f6c13e5e24 100644 --- a/src/server/messages-shared.ts +++ b/src/server/messages-shared.ts @@ -28,6 +28,7 @@ import { redactSecretString } from "../lib/redact"; import { sseFieldValue } from "../lib/sse-decoder"; import { estimateTokens } from "../lib/token-estimate"; import { isTranslatorBudgetExceededError } from "../lib/translator-budget"; +import { httpStatusFromTerminalError } from "../lib/errors"; import { AnthropicRequestError, isRec, type Rec } from "../claude/inbound-records"; import type { RequestLogContext } from "./request-log"; import { recordGenerationEvent } from "./request-log-generation-window"; @@ -178,6 +179,11 @@ export function tapAnthropicSseForLog( } if (named(event, data, "message_stop") || named(event, data, "error")) { terminalSeen = true; + if (data.type === "error") { + const error = isRec(data.error) ? data.error : {}; + logCtx.streamErrorStatus = httpStatusFromTerminalError({ type: typeof error.type === "string" ? error.type : undefined, + code: typeof error.code === "string" ? error.code : undefined, message: typeof error.message === "string" ? error.message : undefined }); + } } else if (delivered && !named(event, data, "ping")) { const delta = isRec(data.delta) ? data.delta : undefined; lastDeliveredStop = named(event, data, "message_delta") && typeof delta?.stop_reason === "string" @@ -312,6 +318,7 @@ export function tapAnthropicSseForLog( settled = true; idle.cancel(); detachAbort(); + recordUsage(); finalize(499, { closeReason: "client_cancel" }); clientFilter?.dispose(); try { tapController?.close(); } catch { /* downstream already torn down */ } @@ -457,6 +464,7 @@ export function tapAnthropicSseForLog( settled = true; idle.cancel(); detachAbort(); + recordUsage(); finalize(499, { closeReason: "client_cancel" }); } reader.cancel(reason).catch(() => {}); diff --git a/src/server/request-log.ts b/src/server/request-log.ts index 03668088068..fd87db72e3f 100644 --- a/src/server/request-log.ts +++ b/src/server/request-log.ts @@ -252,6 +252,8 @@ export interface RequestLogContext extends UpstreamErrorDiagnostics { upstreamCondition?: UpstreamCondition; // classified at capture where the evidence is not kept on the context /** HTTP status derived from a terminal `response.failed` SSE payload (429/401/503/etc.). */ terminalHttpStatus?: number; + /** Status of an upstream `error` event that ended a 200 Messages stream; the row stays 200, a combo child reads it. */ + streamErrorStatus?: number; /** Recognized structured terminal code whose exact identity must survive status mapping. */ terminalErrorCode?: typeof CYBER_POLICY_ERROR_CODE; /** diff --git a/src/server/responses/core-combo-native.ts b/src/server/responses/core-combo-native.ts index d68ecd6b8e3..6d4caabf6ec 100644 --- a/src/server/responses/core-combo-native.ts +++ b/src/server/responses/core-combo-native.ts @@ -430,6 +430,8 @@ export async function dispatchNativeComboChild(input: { callbacks.onTerminal("incomplete"); return; } + // A 200 stream whose terminal was an upstream error event did not complete. + if (childLog.streamErrorStatus !== undefined && status < 400) return finishLog(childLog.streamErrorStatus, message, "terminal"); finishLog(status, message, meta?.closeReason === "client_cancel" ? "client_cancel" : meta?.closeReason && meta.closeReason !== "non_stream" ? "terminal" : "non_stream"); } }) diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 63063f68308..a3d7f829a6d 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -476,6 +476,64 @@ describe("native Messages candidates in a combo", () => { expect(response.status).toBe(200); } }); + for (const folded of ["event: error\ndata: {oops\n\n", `event: error\ndata: ${JSON.stringify({ type: "ping" })}\n\n`]) { + test(`a fold committed by a malformed error event never replays (${folded.includes("oops") ? "bad JSON" : "type mismatch"})`, async () => { + answer = () => stream(prelude + folded + frame("message_stop")); + const { response } = await send({ stream: false }); + expect(response.status).not.toBe(200); + expect(seen).toHaveLength(1); + }); + } + test("a preflight buffer failure after a content frame never replays", async () => { + const budgets = await import("../../src/lib/translator-budget"); + const create = budgets.createTranslatorBudget; + // The prelude's chunk is retained; the content chunk the tap already read overflows. + let charges = 0; + const spy = spyOn(budgets, "createTranslatorBudget").mockImplementation(options => { + const budget = create(options); + const charge = budget.chargeRetained.bind(budget); + budget.chargeRetained = (delta, scope) => { + if (scope.kind === "retained_collectors" && ++charges === 2) throw new budgets.TranslatorBudgetExceededError(scope.kind, 1); + charge(delta, scope); + }; + return budget; + }); + try { + answer = () => stream(prelude + textStart + textDelta + terminal); + const { response } = await send(); + expect(charges).toBeGreaterThanOrEqual(2); + expect(response.status).not.toBe(200); + expect(seen).toHaveLength(1); + } finally { spy.mockRestore(); } + }); + test("a streamed error after content is a failed child, not a completion", async () => { + answer = () => stream(prelude + textStart + textDelta + refusal); + const { text, rows } = await send(); + expect(text).toContain("overloaded_error"); + expect(rows[0]!.status).toBe(503); + expect(rows[0]!.terminalStatus).toBe("failed"); + }); + test("an observe-only workflow root counts one native send once", async () => { + const { resetWorkflowBudgetsForTest, workflowBudgetSnapshot } = await import("../../src/lib/workflow-budget"); + const { bindWorkflowBudgetConfig } = await import("../../src/lib/workflow-budget-config"); + resetWorkflowBudgetsForTest(); + const settings = config(); + settings.workflowBudget = { maxPhysicalSends: 2 }; + bindWorkflowBudgetConfig(settings); + const headers = { "content-type": "application/json", "x-codex-parent-thread-id": "native-messages-root" }; + // The HTTP ingress opens the root turn; this direct call opens it the same way. + expect((await import("../../src/server/workflow-refusal")).admitHttpWorkflowTurn(new Headers(headers))?.admitted).toBe(true); + try { + answer = () => Response.json({ type: "error", error: { type: "api_error", message: "fixture" } }, { status: 503, headers: { "retry-after": "0" } }); + const requestId = `native-combo-${crypto.randomUUID()}`; + await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", headers, body: JSON.stringify({ ...source, stream: false }), + }), settings, { model: "", provider: "" }, { requestId, start: Date.now() }); + // Counted twice, the native send would leave the two-send root no room for the fallback. + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "backup.fixture.invalid"]); + expect(workflowBudgetSnapshot("native-messages-root")?.sends).toBe(2); + } finally { bindWorkflowBudgetConfig(null); resetWorkflowBudgetsForTest(); } + }); test("native disabled keeps existing translated path", async () => { const { response } = await send({ config: config(false) }); expect(response.status).toBe(200); diff --git a/tests/claude-integration/messages-native-stream-parity.test.ts b/tests/claude-integration/messages-native-stream-parity.test.ts index f1aa3e47c58..3354d2b2dd0 100644 --- a/tests/claude-integration/messages-native-stream-parity.test.ts +++ b/tests/claude-integration/messages-native-stream-parity.test.ts @@ -31,7 +31,11 @@ for (const lane of ["tap", "native-fixture", "native-anthropic"]) for (const sce expect(base).toBeDefined(); const { bytes: baseBytes, ...baseFacts } = base; const { bytes: headBytes, ...headFacts } = head; - expect(headFacts).toEqual(baseFacts); + // A cancel now keeps the usage it already received (#7374 review); the base dropped it. + if (scenario === "cancel" && lane !== "tap") { + expect(head).toHaveProperty("usage", { inputTokens: 5, outputTokens: 0 }); + expect({ ...headFacts, usage: undefined }).toEqual({ ...baseFacts, usage: undefined }); + } else expect(headFacts).toEqual(baseFacts); // Changed client rows only: owned overflow data; fixed throughput/stall/cap/read wording. const changed = lane !== "tap" && ["overflow", "throughput", "cap", "stall", "read-error"].includes(scenario); if (changed) { diff --git a/tests/claude-integration/messages-shared-sse.test.ts b/tests/claude-integration/messages-shared-sse.test.ts index 8f3599e05d8..5177288b3eb 100644 --- a/tests/claude-integration/messages-shared-sse.test.ts +++ b/tests/claude-integration/messages-shared-sse.test.ts @@ -122,3 +122,14 @@ test("client cancellation still settles once and cancels the source", async () = expect(result.calls).toEqual([{ status: 499, meta: { closeReason: "client_cancel" } }]); expect(result.cancelled()).toBe(true); }); + +for (const how of ["reader cancel", "signal abort"] as const) test(`a ${how} keeps the usage already received`, async () => { + const abort = new AbortController(); + const result = tap(encoder.encode(`${start}\n\n`), 4096, "stall", { stallMs: 0, maxBytes: 0, reqSignal: abort.signal }); + const reader = result.stream.getReader(); + await reader.read(); + if (how === "reader cancel") await reader.cancel(); + else { abort.abort(); await new Promise(resolve => setTimeout(resolve, 0)); } + expect(result.calls).toEqual([{ status: 499, meta: { closeReason: "client_cancel" } }]); + expect(result.log.usage).toMatchObject({ inputTokens: 12 }); +}); diff --git a/tests/helpers/messages-sse-assertions.ts b/tests/helpers/messages-sse-assertions.ts index aa8cb98d816..2667b98845b 100644 --- a/tests/helpers/messages-sse-assertions.ts +++ b/tests/helpers/messages-sse-assertions.ts @@ -36,10 +36,12 @@ export function registerSseLane(lane: string) { expect(facts.sends, key).toBe(before.sends); const status = input.kind === "FOLD-502" ? 502 : before.status; expect(after.status, key).toBe(status); + // A cancel now keeps the usage it already received (#7374 review); the base dropped it. + const internal = (row: any) => input.fault === "cancel" ? { ...row, usage: "cancel" } : row; if (lane === "fold") { expect(after.internal.settlements, key).toEqual(before.internal.settlements.map((row: any) => ({ ...row, status: row.status === 200 ? 502 : row.status }))); - expect({ ...after.internal, settlements: [] }, key).toEqual({ ...before.internal, settlements: [] }); - } else expect(after.internal, key).toEqual(before.internal); + expect(internal({ ...after.internal, settlements: [] }), key).toEqual(internal({ ...before.internal, settlements: [] })); + } else expect(internal(after.internal), key).toEqual(internal(before.internal)); const ordinary = input.kind === "non-error" || input.kind === "rename" || lane === "tap"; if (ordinary) { expect(after.bodySha256, key).toBe(before.bodySha256); From ee0e35f80f77645bac117829173845d77ed33e97 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Sun, 8 Nov 2026 01:59:30 +0800 Subject: [PATCH 19/23] fix(combo): classify a native Messages preflight chunk before retaining it A chunk carrying an unknown event commits without setting the log tap's generation marker, so a retention refusal on that chunk still answered a replayable 502 and the combo sent the turn to the fallback too. The preflight now inspects each chunk first and holds a committing chunk without a charge, as the frame cap already does; the budget can then only refuse a prelude that is still safe to replay. The generation-marker check this replaces is removed. Co-Authored-By: Claude Code --- src/server/messages-native-preflight.ts | 10 +++-- src/server/messages-native.ts | 3 -- .../messages-native-combo.test.ts | 43 ++++++++++--------- 3 files changed, 29 insertions(+), 27 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 022de35003a..73e3a75e33e 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -68,15 +68,19 @@ export async function preflightNativeMessagesResponse( if (retained + next.value.byteLength > MAX_CLIENT_SSE_FRAME_BYTES || chunks.length >= 1024) { chunks.push(next.value); committed = true; break; } - budget.chargeRetained(next.value.byteLength, { kind: "retained_collectors" }); - retained += next.value.byteLength; - chunks.push(next.value); + // Classify before retaining: a chunk that commits is held like the cap's, so a budget + // refusal can only meet a prelude that is still safe to replay. try { for (const frame of inspector.feed(next.value)) if (!frame.continuation) inspectFrame(frame.block); } catch (error) { if (!(error instanceof SseFrameTooLargeError || error instanceof SseFrameCountLimitError)) throw error; committed = true; } + if (!committed) { + budget.chargeRetained(next.value.byteLength, { kind: "retained_collectors" }); + retained += next.value.byteLength; + } + chunks.push(next.value); if (committed || terminal || failure) break; } if (failure && !committed) { diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 61f83490b5b..1217a7d9e34 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -1053,8 +1053,6 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp delivered = new Response(watchFirstOutput(preflight.response.body!), { status: preflight.response.status, headers: preflight.response.headers }); } catch (error) { cleanupAbort(); releaseStreamTurn(); upstream.abort(); - // A content frame the tap already saw ran upstream: the failure must not replay. - if (logCtx.generationStartedAt !== undefined) committed = true; if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); if (isTranslatorBudgetExceededError(error)) return fail(502, "upstream translation buffer exceeded the safe limit", "api_error", "translation_buffer_limit"); return fail(502, "upstream Messages stream failed before output", "api_error"); @@ -1135,7 +1133,6 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp cleanupAbort(); upstream.abort(); bindUsage(logCtx.usage); - if (logCtx.generationStartedAt !== undefined) committed = true; if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); if (isTranslatorBudgetExceededError(error)) { return fail(413, "upstream translation buffer exceeded the safe limit", "request_too_large", "translation_buffer_limit"); diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index a3d7f829a6d..00214e3a6ff 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -484,28 +484,29 @@ describe("native Messages candidates in a combo", () => { expect(seen).toHaveLength(1); }); } - test("a preflight buffer failure after a content frame never replays", async () => { - const budgets = await import("../../src/lib/translator-budget"); - const create = budgets.createTranslatorBudget; - // The prelude's chunk is retained; the content chunk the tap already read overflows. - let charges = 0; - const spy = spyOn(budgets, "createTranslatorBudget").mockImplementation(options => { - const budget = create(options); - const charge = budget.chargeRetained.bind(budget); - budget.chargeRetained = (delta, scope) => { - if (scope.kind === "retained_collectors" && ++charges === 2) throw new budgets.TranslatorBudgetExceededError(scope.kind, 1); - charge(delta, scope); - }; - return budget; + for (const committing of [textStart + textDelta, frame("fixture_unknown")]) { + test(`a buffer refusal cannot replay ${committing.includes("unknown") ? "an unknown event" : "a content frame"}`, async () => { + const budgets = await import("../../src/lib/translator-budget"); + const create = budgets.createTranslatorBudget; + // Only the prelude's chunk fits; a committing chunk must be held without a charge. + let charges = 0; + const spy = spyOn(budgets, "createTranslatorBudget").mockImplementation(options => { + const budget = create(options); + const charge = budget.chargeRetained.bind(budget); + budget.chargeRetained = (delta, scope) => { + if (scope.kind === "retained_collectors" && ++charges === 2) throw new budgets.TranslatorBudgetExceededError(scope.kind, 1); + charge(delta, scope); + }; + return budget; + }); + try { + answer = () => stream(prelude + committing + terminal); + const { text } = await send(); + expect(text).not.toContain("fallback reply"); + expect(seen).toHaveLength(1); + } finally { spy.mockRestore(); } }); - try { - answer = () => stream(prelude + textStart + textDelta + terminal); - const { response } = await send(); - expect(charges).toBeGreaterThanOrEqual(2); - expect(response.status).not.toBe(200); - expect(seen).toHaveLength(1); - } finally { spy.mockRestore(); } - }); + } test("a streamed error after content is a failed child, not a completion", async () => { answer = () => stream(prelude + textStart + textDelta + refusal); const { text, rows } = await send(); From 9ee2b41a51590dbd6c4f5d805fd7fbc74b356bb6 Mon Sep 17 00:00:00 2001 From: Vadym O Date: Sun, 8 Nov 2026 12:50:21 +0800 Subject: [PATCH 20/23] fix(combo): keep a native Messages refusal committed when output shared its upstream read The streaming relay reframes an upstream chunk before the preflight reads it, and a partial frame never reaches it. The tap reads the original chunk first, so it now records any non-ping bytes after an upstream error event, and the preflight treats that error as committed instead of replaying it. Co-Authored-By: Claude Code --- src/server/messages-native-preflight.ts | 3 ++- src/server/messages-shared.ts | 5 +++++ src/server/request-log.ts | 2 ++ tests/claude-integration/messages-native-combo.test.ts | 8 ++++++++ 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 73e3a75e33e..cb221d50f72 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -83,7 +83,8 @@ export async function preflightNativeMessagesResponse( chunks.push(next.value); if (committed || terminal || failure) break; } - if (failure && !committed) { + // The tap read the original upstream chunk; output after the error there commits too. + if (failure && !committed && !logCtx.streamErrorFollowed) { logCtx.usage = anthropicUsageToOcx(usage); const message = redactSecretString(typeof failure.message === "string" ? failure.message : "Provider stream failed before output"); logCtx.upstreamError = `Provider stream failed (${failureStatus})`; diff --git a/src/server/messages-shared.ts b/src/server/messages-shared.ts index 6f6c13e5e24..1817abcdc59 100644 --- a/src/server/messages-shared.ts +++ b/src/server/messages-shared.ts @@ -147,6 +147,7 @@ export function tapAnthropicSseForLog( let usageAcc: Rec = {}; // message_stop or an upstream error event: the turn's own terminal already went through. let terminalSeen = false; + let errorSeen = false; // stop_reason of the latest message event the client received as a complete SSE event. Any // later message event (resumed content) clears it; pings and comments do not. let lastDeliveredStop: string | null = null; @@ -166,6 +167,7 @@ export function tapAnthropicSseForLog( if (!dataLine) return; let data: unknown; try { data = JSON.parse(dataLine); } catch { data = null; } + if (errorSeen && !(isRec(data) && named(event, data, "ping"))) logCtx.streamErrorFollowed = true; if (!isRec(data)) { // A malformed event after a stop reason means the turn kept going and was cut. if (delivered) lastDeliveredStop = null; @@ -180,6 +182,7 @@ export function tapAnthropicSseForLog( if (named(event, data, "message_stop") || named(event, data, "error")) { terminalSeen = true; if (data.type === "error") { + errorSeen = true; const error = isRec(data.error) ? data.error : {}; logCtx.streamErrorStatus = httpStatusFromTerminalError({ type: typeof error.type === "string" ? error.type : undefined, code: typeof error.code === "string" ? error.code : undefined, message: typeof error.message === "string" ? error.message : undefined }); @@ -216,6 +219,8 @@ export function tapAnthropicSseForLog( const end = text.length - (heldLf ? 1 : 0); appendPart(text.slice(start, end)); } + // The relay may reframe this read: a combo child judges a retryable error by what followed it here. + if (errorSeen && !isHarmlessSseTail([...frameParts, ...pendingParts].join(""))) logCtx.streamErrorFollowed = true; }; // The last block can sit in the buffer without its blank line: an upstream that stopped after // it, or a held trailing CR. Count it before deciding how the turn ended (the Responses relay diff --git a/src/server/request-log.ts b/src/server/request-log.ts index fd87db72e3f..389d552239f 100644 --- a/src/server/request-log.ts +++ b/src/server/request-log.ts @@ -254,6 +254,8 @@ export interface RequestLogContext extends UpstreamErrorDiagnostics { terminalHttpStatus?: number; /** Status of an upstream `error` event that ended a 200 Messages stream; the row stays 200, a combo child reads it. */ streamErrorStatus?: number; + /** Bytes other than a ping followed that upstream `error` event, so a retry would repeat output. */ + streamErrorFollowed?: true; /** Recognized structured terminal code whose exact identity must survive status mapping. */ terminalErrorCode?: typeof CYBER_POLICY_ERROR_CODE; /** diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 00214e3a6ff..c53a9208e55 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -507,6 +507,14 @@ describe("native Messages candidates in a combo", () => { } finally { spy.mockRestore(); } }); } + for (const [following, streamed] of [[textStart, true], [textStart, false], [textStart.slice(0, 30), true], [textStart.slice(0, 30), false]] as const) { + test(`a ${following === textStart ? "complete" : "partial"} frame read with a refusal never replays (${streamed ? "stream" : "fold"})`, async () => { + answer = () => stream(prelude + refusal + following); + const { text } = await send({ stream: streamed }); + expect(text).not.toContain("fallback reply"); + expect(seen).toHaveLength(1); + }); + } test("a streamed error after content is a failed child, not a completion", async () => { answer = () => stream(prelude + textStart + textDelta + refusal); const { text, rows } = await send(); From 48c301467e319856a9d7a4a664d7ea32739285c0 Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 14:45:46 +0900 Subject: [PATCH 21/23] Preserve native Messages replay boundaries --- src/server/messages-native-preflight.ts | 43 +++++++++++++++++++ src/server/messages-native.ts | 5 ++- src/server/sse-frame-buffer.ts | 5 +++ structure/data-planes/error-detail.md | 1 + structure/data-planes/protocol-paths.md | 2 +- structure/transports/byte-accounting.md | 1 + .../transports/responses-combo-boundaries.md | 1 + .../messages-native-combo.test.ts | 41 ++++++++++++++++++ 8 files changed, 96 insertions(+), 3 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index cb221d50f72..908fb6eb04d 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -11,6 +11,49 @@ import { UNDECLARED_TOOL_CALL_ERROR_CODE } from "./responses-undeclared-tool-gua type Rec = Record; const isRec = (value: unknown): value is Rec => value !== null && typeof value === "object" && !Array.isArray(value); +/** Capture replay evidence before any relay can split a physical upstream read into frames. */ +export function observeNativeMessagesReplayBoundary(body: ReadableStream, logCtx: RequestLogContext): ReadableStream { + const reader = body.getReader(); + const inspector = new BoundedSseFrameBuffer(); + const decoder = new TextDecoder(); + let active = true; + return new ReadableStream({ + async pull(controller) { + try { + const next = await reader.read(); + if (next.done) { inspector.dispose(); controller.close(); return; } + if (active) { + let errorSeen = false; + let finished = false; + try { + for (const frame of inspector.feed(next.value)) { + if (frame.continuation) continue; + const lines = decoder.decode(frame.block).split(/\r\n|\r|\n/); + const event = lines.map(line => sseFieldValue(line, "event")).findLast(value => value !== null) ?? null; + const data = lines.map(line => sseFieldValue(line, "data")).filter(value => value !== null).join("\n"); + let payload: unknown; + try { payload = JSON.parse(data); } catch { payload = undefined; } + const type = isRec(payload) && (event === null || event === payload.type) ? payload.type : undefined; + if (errorSeen && (event !== null || data) && type !== "ping") logCtx.streamErrorFollowed = true; + if (type === "error") errorSeen = true; + else if ((event !== null || data) && type !== "ping" && type !== "message_start") finished = true; + } + // An unfinished event can still acquire a different name or payload on the next read. + if (errorSeen && inspector.pendingBytes > 0) logCtx.streamErrorFollowed = true; + if (errorSeen || finished) { active = false; inspector.dispose(); } + } catch (error) { + if (!(error instanceof SseFrameTooLargeError || error instanceof SseFrameCountLimitError)) throw error; + logCtx.streamErrorFollowed = true; + active = false; inspector.dispose(); + } + } + controller.enqueue(next.value); + } catch (error) { inspector.dispose(); controller.error(error); } + }, + cancel(reason) { inspector.dispose(); return reader.cancel(reason); }, + }, { highWaterMark: 0 }); +} + /** Inspect only the bounded protocol prelude. Any block, unknown payload or cap commits. */ export async function preflightNativeMessagesResponse( response: Response, logCtx: RequestLogContext, budget: TranslatorBudget, diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 1217a7d9e34..943560349af 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -107,7 +107,7 @@ import { import { beginInferenceAttempt, type InferenceAttempt } from "./inference/attempt"; import type { ComboProtocolSource } from "./responses/core-combo-native"; import type { ProtocolEnvelope } from "../protocols/envelope"; -import { preflightNativeMessagesResponse } from "./messages-native-preflight"; +import { observeNativeMessagesReplayBoundary, preflightNativeMessagesResponse } from "./messages-native-preflight"; import { createFinalRequestLog, type FinalRequestLogMeta } from "./inference/final-log"; import { registerTurn, unregisterTurn } from "./lifecycle"; import { nativeMessagesDeclineReason, type NativeMessagesSelector } from "./messages-native-eligibility"; @@ -1009,7 +1009,8 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp ...resolvePassthroughBodyGuard(config, upstream.signal), ...(activeProvider.anthropicEofTolerance === true ? { eofTolerant: true } : {}), }; - const observed = logIds && onFirstOutput ? observeFirstChunk(response.body, onFirstOutput) : response.body; + const raw = options.sendBudget ? observeNativeMessagesReplayBoundary(response.body, logCtx) : response.body; + const observed = logIds && onFirstOutput ? observeFirstChunk(raw, onFirstOutput) : raw; const renamed = activeRequest.oauthToolNames ? restoreOAuthToolNamesInSse(observed, activeRequest.oauthToolNames, translatorBudget) : observed; diff --git a/src/server/sse-frame-buffer.ts b/src/server/sse-frame-buffer.ts index 8309d376c50..4fea8cec97d 100644 --- a/src/server/sse-frame-buffer.ts +++ b/src/server/sse-frame-buffer.ts @@ -288,6 +288,11 @@ export class BoundedSseFrameBuffer { } } + /** Bytes retained for an unfinished frame, including a possible delimiter prefix. */ + get pendingBytes(): number { + return this.candidateBytes + this.delimiterTail.byteLength; + } + /** Return the final unterminated block bytes and release all retained state. */ finish(): Uint8Array { if (this.disposed) return EMPTY_BYTES; diff --git a/structure/data-planes/error-detail.md b/structure/data-planes/error-detail.md index e853325b1cc..eeddf2c4580 100644 --- a/structure/data-planes/error-detail.md +++ b/structure/data-planes/error-detail.md @@ -441,6 +441,7 @@ inspection or serialization. `src/server/messages-native-tool-guard.ts`, `src/server/messages-shared.ts` and `src/server/sse-payload-rewrite.ts` carry complete-frame construction identity. +Native combo replay evidence is collected before framing transforms; the accounting tap remains the owner of stream outcomes and error presentation. Native and passthrough Messages SSE apply error-response normalization after echo, guard, accounting tap and the existing configured-only overflow rewrite. The final enqueue stage in `src/server/messages-sse-error-filter.ts` introduces no stream queue, timer or possible-prefix hold. diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 6862917fe80..c93f3b41046 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -376,7 +376,7 @@ probe uses the bridge's credential-free affinity headers, existing plan and auth short-lived selection lease; uncertain admission keeps the bridge. Without a runnable plan search alone does not decline native Messages. `count_tokens` and the planner lack runtime facts and remain conservative. -Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. A source declaring a tool the translated body drops with no out-of-body lane (`hasNativeOnlyTool` in `src/claude/inbound-content-options.ts`: `bash_*`, `text_editor_*`, ...; unrestricted `web_search_*` stays portable, while `web_fetch_*` and a domain- or location-restricted `web_search_*` stay portable only to a first-party Anthropic bridge child) makes every translated child unpickable under any `unrepresentable` policy, so a refused native child surfaces its own failure instead of continuing without the tool; non-combo translated routes are unchanged. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover; `tests/claude-integration/messages-native-combo.test.ts` proves mocked prefix preservation, not live cache-hit metrics. +Failover combos reuse this lane for eligible concrete first-party Anthropic children through the existing `core-combo-native.ts` source/dispatcher seam. Only strict failover without a default effort qualifies; policy/JEV/other strategies and bridge-only rewrites retain the bridge. The shared executor inherits the combo's attempt, target budget, prepaid permit and gated callbacks, opening no second tracker/final row. Key source system/messages/tools/cache markers retain the native allowlist, image/tool-id safety; explicit operator TTL wins, otherwise TTL-less combo breakpoints stay unchanged. OAuth keeps its preamble/metadata/tool transformations. Counts/previews still decline logical combos without picking accounts. A source declaring a tool the translated body drops with no out-of-body lane (`hasNativeOnlyTool` in `src/claude/inbound-content-options.ts`: `bash_*`, `text_editor_*`, ...; unrestricted `web_search_*` stays portable, while `web_fetch_*` and a domain- or location-restricted `web_search_*` stay portable only to a first-party Anthropic bridge child) makes every translated child unpickable under any `unrepresentable` policy, so a refused native child surfaces its own failure instead of continuing without the tool; non-combo translated routes are unchanged. [Commit boundaries](../transports/responses-combo-boundaries.md) own failover and preserve upstream read evidence before OAuth framing; `tests/claude-integration/messages-native-combo.test.ts` covers the boundary and mocked prefix preservation, not live cache-hit metrics. With the switch on, a declined route re-marks its bridge entry with the decline reason (after any `effort-row` / `fast-row`), so the trace says why the bridge was taken; with it off nothing is diff --git a/structure/transports/byte-accounting.md b/structure/transports/byte-accounting.md index 7037df8712e..139dea783e8 100644 --- a/structure/transports/byte-accounting.md +++ b/structure/transports/byte-accounting.md @@ -252,6 +252,7 @@ original wire bytes. `src/server/relay.ts` re-exports the canonical extractor an Empty byte results across the relay and `src/server/sse-frame-buffer.ts` reuse one immutable zero-length view; non-empty frame ownership, frame limits, cancellation and wire ownership remain bounded. The byte framer and inspection share +an unfinished-byte count for the native combo replay observer, without exposing or copying its buffer. They share one CR/LF/CRLF delimiter scanner; WebSocket projection uses the canonical data extractor. A terminal CR dispatches without awaiting EOF. A later LF extends that delimiter: byte delivery returns a framing-only continuation, excluded from event-count limits and the next event's byte cap; diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index e8c0b6a3d93..ec02c39a45e 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -174,6 +174,7 @@ empty-content `message_start` and `ping` as replayable. As in the native tap, th any other event returns to the ordinary combo failure classifier unless a later frame read with it commits; any content block (including a tool start), unknown/malformed payload (a named event without data included; comment-only blocks are skipped), terminal or byte/chunk cap commits conservatively. Buffered bytes replay unchanged. A collected failure after that commitment is non-replayable, while first output waits for a content frame the tap has seen. +For combo children, a bounded observer before frame transforms preserves the original read boundary for the replay decision. Complete harmless pings may follow a refusal; an unfinished following frame or a named empty frame commits conservatively. The observer stops after that decision or semantic output, and leaves ordinary standalone streaming and log ownership unchanged. Preflight retention is charged to the translator budget and released on delivery/cancellation. The Messages ingress offers its parsed body as that source by reference, with no ingress copy, and only when no effort override rewrites it. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index c53a9208e55..d8cfceb2dfd 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -101,6 +101,47 @@ async function send(options: { stream?: boolean; config?: OcxConfig; messages?: } describe("native Messages candidates in a combo", () => { + for (const streamed of [true, false]) { + test(`OAuth error and content in one upstream read cannot replay (${streamed ? "stream" : "fold"})`, async () => { + await saveCredential("anthropic", { access: "synthetic-combo-access", refresh: "synthetic-combo-refresh", expires: Date.now() + 3_600_000, accountId: "synthetic-combo-uuid", source: "oauth" }); + const settings = config(); + settings.providers.anthropic = { ...settings.providers.anth!, authMode: "oauth", apiKey: undefined }; + delete settings.providers.anth; + settings.defaultProvider = "anthropic"; + settings.combos!.waterfall!.targets[0]!.provider = "anthropic"; + settings.anthropicAccountPool = { enabled: true }; + settings.protocols!.rollout!.managedMessagesNativeOAuth = true; + answer = () => stream(prelude + refusal + textStart + textDelta + terminal); + const { text } = await send({ config: settings, stream: streamed }); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); + test(`error followed by an incomplete ping frame cannot replay (${streamed ? "stream" : "fold"})`, async () => { + answer = () => new Response(new ReadableStream({ + start(controller) { + const enc = new TextEncoder(); + controller.enqueue(enc.encode(prelude + refusal + "event: ping\n")); + controller.enqueue(enc.encode(textStart + textDelta + terminal)); + controller.close(); + }, + }), { headers: { "content-type": "text/event-stream" } }); + const { text } = await send({ stream: streamed }); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); + test(`error followed by an empty named content frame cannot replay (${streamed ? "stream" : "fold"})`, async () => { + answer = () => stream(prelude + refusal + "event: content_block_start\ndata:\n\n"); + const { text } = await send({ stream: streamed }); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); + test(`complete ping after an error remains replayable (${streamed ? "stream" : "fold"})`, async () => { + answer = () => stream(prelude + refusal + frame("ping")); + const { text } = await send({ stream: streamed }); + expect(text).toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "backup.fixture.invalid"]); + }); + } test("canonical child preserves source system/messages/tools/cache prefix and managed auth", async () => { const { response, text, rows } = await send(); expect(response.status).toBe(200); From 31a6ae08a49117561aa0a48c930521659f014a31 Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 15:07:10 +0900 Subject: [PATCH 22/23] Preserve raw Messages commitment across rewrite failures Retain raw replay evidence until preflight and bound post-error tail inspection. --- src/server/messages-native-preflight.ts | 19 ++++-- src/server/messages-native.ts | 9 ++- src/server/messages-shared.ts | 3 +- structure/data-planes/error-detail.md | 2 +- .../messages-native-combo.test.ts | 66 +++++++++++++++++++ .../messages-shared-sse.test.ts | 39 +++++++++++ 6 files changed, 126 insertions(+), 12 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 908fb6eb04d..29873acafa2 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -12,7 +12,7 @@ type Rec = Record; const isRec = (value: unknown): value is Rec => value !== null && typeof value === "object" && !Array.isArray(value); /** Capture replay evidence before any relay can split a physical upstream read into frames. */ -export function observeNativeMessagesReplayBoundary(body: ReadableStream, logCtx: RequestLogContext): ReadableStream { +export function observeNativeMessagesReplayBoundary(body: ReadableStream, logCtx: RequestLogContext, replay: { committed: boolean; pending: boolean }): ReadableStream { const reader = body.getReader(); const inspector = new BoundedSseFrameBuffer(); const decoder = new TextDecoder(); @@ -30,20 +30,25 @@ export function observeNativeMessagesReplayBoundary(body: ReadableStream sseFieldValue(line, "event")).findLast(value => value !== null) ?? null; - const data = lines.map(line => sseFieldValue(line, "data")).filter(value => value !== null).join("\n"); + const dataFields = lines.map(line => sseFieldValue(line, "data")).filter(value => value !== null); + const data = dataFields.join("\n"); let payload: unknown; try { payload = JSON.parse(data); } catch { payload = undefined; } const type = isRec(payload) && (event === null || event === payload.type) ? payload.type : undefined; - if (errorSeen && (event !== null || data) && type !== "ping") logCtx.streamErrorFollowed = true; - if (type === "error") errorSeen = true; - else if ((event !== null || data) && type !== "ping" && type !== "message_start") finished = true; + if (errorSeen && (event !== null || dataFields.length > 0) && type !== "ping") logCtx.streamErrorFollowed = true; + if (type === "error" && isRec(payload) && isRec(payload.error)) errorSeen = true; + else if ((event !== null || dataFields.length > 0) && type !== "ping" && !(type === "message_start" && isRec(payload) && isRec(payload.message) && Array.isArray(payload.message.content) && payload.message.content.length === 0)) { + replay.committed = true; finished = true; + } } + replay.pending = inspector.pendingBytes > 0; // An unfinished event can still acquire a different name or payload on the next read. if (errorSeen && inspector.pendingBytes > 0) logCtx.streamErrorFollowed = true; if (errorSeen || finished) { active = false; inspector.dispose(); } } catch (error) { if (!(error instanceof SseFrameTooLargeError || error instanceof SseFrameCountLimitError)) throw error; logCtx.streamErrorFollowed = true; + replay.committed = true; active = false; inspector.dispose(); } } @@ -56,7 +61,7 @@ export function observeNativeMessagesReplayBoundary(body: ReadableStream { if (!response.ok || !response.body || !response.headers.get("content-type")?.includes("text/event-stream")) { return { kind: "accepted", response }; @@ -127,7 +132,7 @@ export async function preflightNativeMessagesResponse( if (committed || terminal || failure) break; } // The tap read the original upstream chunk; output after the error there commits too. - if (failure && !committed && !logCtx.streamErrorFollowed) { + if (failure && !committed && !replay?.committed && !logCtx.streamErrorFollowed) { logCtx.usage = anthropicUsageToOcx(usage); const message = redactSecretString(typeof failure.message === "string" ? failure.message : "Provider stream failed before output"); logCtx.upstreamError = `Provider stream failed (${failureStatus})`; diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index 943560349af..79ae6c323ca 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -1009,7 +1009,8 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp ...resolvePassthroughBodyGuard(config, upstream.signal), ...(activeProvider.anthropicEofTolerance === true ? { eofTolerant: true } : {}), }; - const raw = options.sendBudget ? observeNativeMessagesReplayBoundary(response.body, logCtx) : response.body; + const replay = { committed: false, pending: false }; + const raw = options.sendBudget ? observeNativeMessagesReplayBoundary(response.body, logCtx, replay) : response.body; const observed = logIds && onFirstOutput ? observeFirstChunk(raw, onFirstOutput) : raw; const renamed = activeRequest.oauthToolNames ? restoreOAuthToolNamesInSse(observed, activeRequest.oauthToolNames, translatorBudget) @@ -1049,10 +1050,11 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp }), response.headers); if (options.sendBudget) { try { - const preflight = await preflightNativeMessagesResponse(delivered, logCtx, translatorBudget); + const preflight = await preflightNativeMessagesResponse(delivered, logCtx, translatorBudget, replay); if (preflight.kind === "failed") { bindUsage(logCtx.usage); return preflight.response; } delivered = new Response(watchFirstOutput(preflight.response.body!), { status: preflight.response.status, headers: preflight.response.headers }); } catch (error) { + committed ||= replay.committed || replay.pending; cleanupAbort(); releaseStreamTurn(); upstream.abort(); if (req.signal.aborted) return fail(499, "Client cancelled request", "api_error"); if (isTranslatorBudgetExceededError(error)) return fail(502, "upstream translation buffer exceeded the safe limit", "api_error", "translation_buffer_limit"); @@ -1070,7 +1072,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp const collectorVerdictSink: MessagesFailureVerdictSink = {}; try { if (options.sendBudget) { - const preflight = await preflightNativeMessagesResponse(new Response(tapped, { headers: response.headers }), logCtx, translatorBudget); + const preflight = await preflightNativeMessagesResponse(new Response(tapped, { headers: response.headers }), logCtx, translatorBudget, replay); if (preflight.kind === "failed") { cleanupAbort(); upstream.abort(); bindUsage(logCtx.usage); finishLog(preflight.response.status, logCtx.upstreamError); @@ -1131,6 +1133,7 @@ async function runNativeMessagesAttempt(options: NativeMessagesExecution, attemp if (committed && finalStatus >= 400) markResponseNonReplayable(collectedResponse); return collectedResponse; } catch (error) { + committed ||= replay.committed || replay.pending; cleanupAbort(); upstream.abort(); bindUsage(logCtx.usage); diff --git a/src/server/messages-shared.ts b/src/server/messages-shared.ts index 1817abcdc59..f8b9d567c2b 100644 --- a/src/server/messages-shared.ts +++ b/src/server/messages-shared.ts @@ -220,7 +220,8 @@ export function tapAnthropicSseForLog( appendPart(text.slice(start, end)); } // The relay may reframe this read: a combo child judges a retryable error by what followed it here. - if (errorSeen && !isHarmlessSseTail([...frameParts, ...pendingParts].join(""))) logCtx.streamErrorFollowed = true; + if (errorSeen && !logCtx.streamErrorFollowed + && (frameParts.length > 0 || !isHarmlessSseTail(pendingParts.join("")))) logCtx.streamErrorFollowed = true; }; // The last block can sit in the buffer without its blank line: an upstream that stopped after // it, or a held trailing CR. Count it before deciding how the turn ended (the Responses relay diff --git a/structure/data-planes/error-detail.md b/structure/data-planes/error-detail.md index eeddf2c4580..dbba4779fa1 100644 --- a/structure/data-planes/error-detail.md +++ b/structure/data-planes/error-detail.md @@ -441,7 +441,7 @@ inspection or serialization. `src/server/messages-native-tool-guard.ts`, `src/server/messages-shared.ts` and `src/server/sse-payload-rewrite.ts` carry complete-frame construction identity. -Native combo replay evidence is collected before framing transforms; the accounting tap remains the owner of stream outcomes and error presentation. +Native combo replay evidence is collected before framing transforms; if a transform fails before preflight, complete output or opaque frames and an unfinished raw frame prevent replay. A complete empty message prelude and ping remain replayable. The accounting tap remains the owner of stream outcomes and error presentation; after an error it conservatively marks a long unfinished frame as followed, bounding repeated tail inspection. Native and passthrough Messages SSE apply error-response normalization after echo, guard, accounting tap and the existing configured-only overflow rewrite. The final enqueue stage in `src/server/messages-sse-error-filter.ts` introduces no stream queue, timer or possible-prefix hold. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index d8cfceb2dfd..34242a69f73 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -101,6 +101,66 @@ async function send(options: { stream?: boolean; config?: OcxConfig; messages?: } describe("native Messages candidates in a combo", () => { + for (const [name, raw, replayable] of [ + ["empty prelude", prelude + frame("ping"), true], + ["unknown frame", prelude + frame("fixture_unknown"), false], + ["empty data", prelude + "data:\n\n", false], + ["malformed error", prelude + frame("error"), false], + ["nonempty message start", frame("message_start", { message }), false], + ["unfinished frame", prelude + "event: content_block_start\n", false], + ] as const) { + test(`rewrite failure preserves raw replay evidence: ${name}`, async () => { + const budgets = await import("../../src/lib/translator-budget"); + const create = budgets.createTranslatorBudget; + let refused = false; + const spy = spyOn(budgets, "createTranslatorBudget").mockImplementation(options => { + const budget = create(options); + const reserve = budget.reserveTransient.bind(budget); + budget.reserveTransient = (bytes, scope) => { + if (!refused && scope.kind === "live_transient") { + refused = true; + throw new budgets.TranslatorBudgetExceededError(scope.kind, 1); + } + return reserve(bytes, scope); + }; + return budget; + }); + try { + answer = () => stream(raw); + const { text } = await send({ tools: [] }); + expect(refused).toBe(true); + expect(text.includes("fallback reply")).toBe(replayable); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(replayable ? ["api.anthropic.com", "backup.fixture.invalid"] : ["api.anthropic.com"]); + } finally { spy.mockRestore(); } + }); + } + for (const streamed of [true, false]) for (const budgetFailure of [true, false]) { + test(`a rewrite ${budgetFailure ? "budget refusal" : "read failure"} after raw tool output cannot replay (${streamed ? "stream" : "fold"})`, async () => { + const budgets = await import("../../src/lib/translator-budget"); + const create = budgets.createTranslatorBudget; + let refused = 0; + const spy = spyOn(budgets, "createTranslatorBudget").mockImplementation(options => { + const budget = create(options); + const charge = budget.chargeRetained.bind(budget); + budget.chargeRetained = (bytes, scope) => { + if (scope.kind === "live_transient" && bytes === 32) { + refused++; + if (budgetFailure) throw new budgets.TranslatorBudgetExceededError(scope.kind, 1); + throw new Error("fixture rewrite failure"); + } + charge(bytes, scope); + }; + return budget; + }); + try { + answer = () => stream(prelude + frame("content_block_start", { index: 0, content_block: { type: "server_tool_use", id: "fixture", name: "web_search", input: {} } }) + terminal); + const { text } = await send({ stream: streamed, tools: [] }); + expect(refused).toBe(1); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + } finally { spy.mockRestore(); } + }); + } for (const streamed of [true, false]) { test(`OAuth error and content in one upstream read cannot replay (${streamed ? "stream" : "fold"})`, async () => { await saveCredential("anthropic", { access: "synthetic-combo-access", refresh: "synthetic-combo-refresh", expires: Date.now() + 3_600_000, accountId: "synthetic-combo-uuid", source: "oauth" }); @@ -135,6 +195,12 @@ describe("native Messages candidates in a combo", () => { expect(text).not.toContain("fallback reply"); expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); }); + test(`error followed by empty data cannot replay (${streamed ? "stream" : "fold"})`, async () => { + answer = () => stream(prelude + refusal + "data:\n\n"); + const { text } = await send({ stream: streamed }); + expect(text).not.toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com"]); + }); test(`complete ping after an error remains replayable (${streamed ? "stream" : "fold"})`, async () => { answer = () => stream(prelude + refusal + frame("ping")); const { text } = await send({ stream: streamed }); diff --git a/tests/claude-integration/messages-shared-sse.test.ts b/tests/claude-integration/messages-shared-sse.test.ts index 5177288b3eb..95647db2463 100644 --- a/tests/claude-integration/messages-shared-sse.test.ts +++ b/tests/claude-integration/messages-shared-sse.test.ts @@ -7,6 +7,45 @@ const start = 'data: {"type":"message_start","message":{"usage":{"input_tokens": const delta = 'data: {"type":"message_delta","usage":{"output_tokens":5}}'; const stop = 'data: {"type":"message_stop"}'; +test("post-error fragments do not repeatedly join an already committed tail", async () => { + const bytes = encoder.encode(`data: {"type":"error","error":{"type":"overloaded_error"}}\n\ndata: ${"x".repeat(256 * 1024)}`); + const original = Array.prototype.join; + let joinedChars = 0; + const join = spyOn(Array.prototype, "join"); + join.mockImplementation(function (this: unknown[], separator?: string) { + if (separator === "") for (const part of this) if (typeof part === "string") joinedChars += part.length; + return Reflect.apply(original, this, [separator]); + }); + try { + const result = tap(bytes, 256); + const output = new Uint8Array(await new Response(result.stream).arrayBuffer()); + join.mockRestore(); + expect(output).toEqual(bytes); + expect(result.log.streamErrorFollowed).toBe(true); + expect(result.calls).toEqual([{ status: 200, meta: { closeReason: "terminal" } }]); + expect(joinedChars).toBeLessThanOrEqual(bytes.length * 5); + } finally { join.mockRestore(); } +}); + +test("an oversized unfinished comment after an error is bounded and conservatively commits", async () => { + const bytes = encoder.encode(`data: {"type":"error","error":{"type":"overloaded_error"}}\n\n:${"x".repeat(256 * 1024)}`); + const original = Array.prototype.join; + let joinedChars = 0; + const join = spyOn(Array.prototype, "join"); + join.mockImplementation(function (this: unknown[], separator?: string) { + if (separator === "") for (const part of this) if (typeof part === "string") joinedChars += part.length; + return Reflect.apply(original, this, [separator]); + }); + try { + const result = tap(bytes, 256); + const output = new Uint8Array(await new Response(result.stream).arrayBuffer()); + join.mockRestore(); + expect(output).toEqual(bytes); + expect(result.log.streamErrorFollowed).toBe(true); + expect(joinedChars).toBeLessThanOrEqual(bytes.length * 5); + } finally { join.mockRestore(); } +}); + function tap(bytes: Uint8Array, chunkSize: number, ending: "close" | "error" | "stall" = "close", guard?: PassthroughBodyGuard) { let offset = 0; let cancelled = false; From 943341523615a803bc6aee46c3ecbcdeef4d1f4d Mon Sep 17 00:00:00 2001 From: JUN Date: Sun, 8 Nov 2026 15:10:41 +0900 Subject: [PATCH 23/23] Handle mixed-case Messages SSE media types --- src/server/messages-native-preflight.ts | 2 +- structure/transports/responses-combo-boundaries.md | 2 +- tests/claude-integration/messages-native-combo.test.ts | 8 ++++++++ 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/src/server/messages-native-preflight.ts b/src/server/messages-native-preflight.ts index 29873acafa2..94714bcf1fb 100644 --- a/src/server/messages-native-preflight.ts +++ b/src/server/messages-native-preflight.ts @@ -63,7 +63,7 @@ export function observeNativeMessagesReplayBoundary(body: ReadableStream { - if (!response.ok || !response.body || !response.headers.get("content-type")?.includes("text/event-stream")) { + if (!response.ok || !response.body || !response.headers.get("content-type")?.toLowerCase().includes("text/event-stream")) { return { kind: "accepted", response }; } let committed = false; diff --git a/structure/transports/responses-combo-boundaries.md b/structure/transports/responses-combo-boundaries.md index ec02c39a45e..d76940d21d9 100644 --- a/structure/transports/responses-combo-boundaries.md +++ b/structure/transports/responses-combo-boundaries.md @@ -169,7 +169,7 @@ messages and bodies stay out. `tests/responses/responses-combo-exhausted-error.t A `runTurn` adapter has the equivalent boundary in `preflightAdapterEvents` (`src/adapters/run-turn-queue.ts`), streaming and non-streaming alike: when its first meaningful event is a tool call the current request did not declare, and no earlier replay-unsafe heartbeat recorded a side effect, `src/server/responses/run-turn-execution.ts` projects the fail-closed undeclared-tool refusal as a pre-commit 502 so the combo can hop with the unchanged catalog. After any output or a replay-unsafe heartbeat the refusal stays with that child. Chat Completions and Anthropic Messages inbound requests do not use this classification. The same heartbeat also decides an ordinary pre-output adapter error or an empty end: after a replay-unsafe heartbeat the child's 502 is marked non-replayable, so the combo stops on it instead of sending the turn to the next target. A native Messages child uses `src/server/messages-native-preflight.ts` after the native log tap, -so the tap retains byte-stall/size/cancellation ownership. Its bounded prelude accepts only an +so the tap retains byte-stall/size/cancellation ownership. SSE media types are matched case-insensitively. Its bounded prelude accepts only an empty-content `message_start` and `ping` as replayable. As in the native tap, the last untrimmed SSE event name must match the payload type (or be absent); a mismatch commits conservatively. A structured retryable `error` before any other event returns to the ordinary combo failure classifier unless a later frame read with it commits; any content block (including a tool start), unknown/malformed payload (a named event without data included; comment-only blocks are skipped), terminal or byte/chunk cap commits conservatively. diff --git a/tests/claude-integration/messages-native-combo.test.ts b/tests/claude-integration/messages-native-combo.test.ts index 34242a69f73..b5dddcc870d 100644 --- a/tests/claude-integration/messages-native-combo.test.ts +++ b/tests/claude-integration/messages-native-combo.test.ts @@ -231,6 +231,14 @@ describe("native Messages candidates in a combo", () => { expect(seen[1]!.body.system).toEqual(source.system); expect(seen[1]!.body.tools).toEqual(source.tools); }); + for (const streamed of [true, false]) { + test(`mixed-case SSE content type permits pre-output failover (${streamed ? "stream" : "fold"})`, async () => { + answer = () => new Response(prelude + refusal, { headers: { "content-type": "Text/Event-Stream" } }); + const { text } = await send({ stream: streamed }); + expect(text).toContain("fallback reply"); + expect(seen.map(entry => new URL(entry.url).hostname)).toEqual(["api.anthropic.com", "backup.fixture.invalid"]); + }); + } test("pre-output streamed refusal falls back without leaking rejected child's prelude", async () => { answer = () => stream(prelude + refusal); const { response, text, rows } = await send();