diff --git a/docs-site/src/content/docs/reference/adapters.md b/docs-site/src/content/docs/reference/adapters.md index 14f420d27a2..b7d038c1854 100644 --- a/docs-site/src/content/docs/reference/adapters.md +++ b/docs-site/src/content/docs/reference/adapters.md @@ -91,7 +91,9 @@ transport; it does not infer subscription attribution from the inbound protocol. keep their existing behavior. This does not add Mistral signature replay. - Canonical key-auth Mistral Chat (`https://api.mistral.ai/v1`) maps `max_completion_tokens` to `max_tokens` while keeping an explicit non-null `max_tokens`, and drops `user`, `store`, `thinking`, - `include_reasoning`, `reasoning` and `top_k`. Its preset folds developer messages to system by + `include_reasoning`, `reasoning` and `top_k`. It also drops `reasoning_content` from every message in the + history, which reasoning-capable Chat clients replay and Mistral rejects with HTTP 422, and no + reasoning replay restores it on that destination. Its preset folds developer messages to system by default; an explicit `foldDeveloperRoleToSystem: false` overrides this. Native Chat also follows the configured model effort ladder: an unsupported `medium` becomes `low` for the bundled DeepSeek/GLM ladders. Mistral Large 4 keeps its supported `medium` rung. diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 4a261447cc7..c490b6b8da6 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -423,7 +423,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity | `transientRetryOn5xx?` | `{ enabled?: boolean; attempts?: number }` | Key-auth `openai-chat` and `openai-responses` providers only. `authMode: "forward"` providers (the ChatGPT account pool) never read this option and keep the default ladder. Opt-in retry for pre-stream transient upstream statuses (500, 502, 503, 504, 520, 521, 522): absent means off, object presence enables it unless `enabled: false`. Covers the initial Responses request, the Responses passthrough lane and each of its recovery legs (OAuth-401 replay, same-target 429 replay, validated rebuild), the terminal-guard continuation, and native `/v1/chat/completions`. `attempts` is the TOTAL number of upstream sends allowed for one request including the first (1..10, default 3) — it is one budget shared with connection-reset recovery, so `3` means at most three real requests reach the provider. On the Responses passthrough lane the configured value is additionally intersected with the request-wide send allowance, so a value below that allowance narrows the ladder exactly while a value above it does not raise the bound. Each Combo target includes its prepaid first send in that target's configured total; `attempts: 1` sends once, without refunding the reservation or widening the shared Combo ceiling. A local refusal or cancellation before dispatch releases an unsent booking. Waits use a fixed 400 ms exponential backoff capped at 5 s and honor `Retry-After`. Separate from `retryOn429`, which handles rate limiting; mid-stream failures are never replayed. | | `retryOnReset?` | `{ enabled?: boolean; replacements?: number }` | Native `openai-responses` providers, including `authMode: "forward"`, plus generic translated Responses dispatch before upstream headers. Opt-in replacement of a send that failed while the caller had observed nothing: absent means off, object presence enables it unless `enabled: false`. Covers both ambiguous stages — a connection that died before any response header, and an SSE body that died after the header while carrying only control events. A canonical ChatGPT upstream WebSocket that closed or errored after its create frame left, before any Responses event, is covered the same way, and its replacement is sent over HTTP. Only a self-contained request is ever replaced: `store: false`, complete `input`, no `previous_response_id`, `conversation` or `stream_id`, and only client-executed tools. A chained turn whose `previous_response_id` this proxy already expanded from its own replay state qualifies, because that id is removed before the send. When a reset is not replaced, the service log records why: `ambiguous-no-policy`, `ambiguous-request-not-replayable` or `ambiguous-allowance-spent`. `replacements` is the number of replacement sends ONE logical request may make across every leg and every combo child (1..2, default 1) — not a per-leg retry count and not a send budget, so a replacement still has to fit inside the send allowance the leg already had. A request that already emitted output or a tool call is never replaced, whatever this is set to. The replacement inference may still be billed if the origin had already started the first one, which is why this is off by default. Generic translated dispatch supports pre-header reset replacement for initial and rebuilt sends under the same grant and send budget. Adapter-owned transports and translated stream failures after upstream headers are excluded. | | `autoToolChoiceOnlyModels?` | `string[]` | Models whose `tool_choice` accepts only `auto` or `none`; forced choices are downgraded. | -| `preserveReasoningContentModels?` | `string[]` | Models requiring prior assistant `reasoning_content` in chat history. Translated Chat requests to `api.mistral.ai` instead reconstruct native thinking chunks before assistant text; supplied thinking replays by default for all hosted families, and an explicit list (including `[]`) overrides enrollment. No placeholder thinking or signatures are invented. Native Chat passthrough is unchanged. A provider save that keeps the destination keeps the stored list, including `[]`; see [What a provider save keeps](#what-a-provider-save-keeps). `PATCH /api/providers?name=` accepts an array or `null` to clear it. | +| `preserveReasoningContentModels?` | `string[]` | Models requiring prior assistant `reasoning_content` in chat history. Translated Chat requests to `api.mistral.ai` instead reconstruct native thinking chunks before assistant text; supplied thinking replays by default for all hosted families, and an explicit list (including `[]`) overrides enrollment. No placeholder thinking or signatures are invented. Native Chat passthrough to canonical Mistral never replays `reasoning_content` and drops any copy the client sent in history, because Mistral rejects the field. A provider save that keeps the destination keeps the stored list, including `[]`; see [What a provider save keeps](#what-a-provider-save-keeps). `PATCH /api/providers?name=` accepts an array or `null` to clear it. | | `inlineThinkTagModels?` | `string[]` | Opt-in recovery for `openai-chat` gateways without a server-side reasoning parser. A leading `` / `` / `` block (optionally after whitespace) activates splitting in streamed and buffered replies. All answer whitespace is preserved. Subsequent tags are delimiters anywhere, including same-line interleaving and code fences; this mode does not interpret Markdown. Ordinary text or a code fence before the first tag keeps the whole reply untouched. Off by default; prefer structured upstream reasoning or `reasoningSplitModels` where supported. | | `reasoningDetailsModels?` | `string[]` | Models whose endpoint returns thinking as a structured `reasoning_details` array (MiniMax M-series with `reasoning_split`); stream deltas are cumulative snapshots that are prefix-diffed, and preserved reasoning replays as a `reasoning_details` array instead of a `reasoning_content` string. | | `requiresReasoningPlaceholderModels?` | `string[]` | Models whose upstream rejects a tool_call continuation missing `reasoning_content` (DeepSeek thinking mode); a minimal placeholder is injected when the replay cache misses. Defaults to `preserveReasoningContentModels`; set `[]` to opt out. A provider save that keeps the destination keeps the stored list, including `[]`; see [What a provider save keeps](#what-a-provider-save-keeps). `PATCH /api/providers?name=` accepts an array or `null` to clear it. | diff --git a/src/adapters/openai-chat/destination-policy.ts b/src/adapters/openai-chat/destination-policy.ts index b97463c7b98..754c8f8668c 100644 --- a/src/adapters/openai-chat/destination-policy.ts +++ b/src/adapters/openai-chat/destination-policy.ts @@ -5,6 +5,8 @@ import type { OcxProviderConfig } from "../../types"; const MISTRAL_CHAT_POLICY = { rename: { max_completion_tokens: "max_tokens" }, omit: ["user", "store", "thinking", "include_reasoning", "reasoning", "top_k"], + /** Replayed inside message history by reasoning-capable Chat clients; Mistral's schema rejects them. */ + omitMessageFields: ["reasoning_content"], } as const; /** Final wire projection shared by native and translated Chat; caller objects are not mutated. */ @@ -15,6 +17,20 @@ export function applyChatDestinationPolicy(body: Record, provid delete body[source]; } for (const field of MISTRAL_CHAT_POLICY.omit) delete body[field]; + const messages = body.messages; + if (Array.isArray(messages)) { + let sanitized: unknown[] | undefined; + for (let index = 0; index < messages.length; index++) { + const message = messages[index]; + if (message === null || typeof message !== "object" || Array.isArray(message)) continue; + const record = message as Record; + if (!MISTRAL_CHAT_POLICY.omitMessageFields.some((field) => field in record)) continue; + const copy = { ...record }; + for (const field of MISTRAL_CHAT_POLICY.omitMessageFields) delete copy[field]; + (sanitized ??= [...messages])[index] = copy; + } + if (sanitized) body.messages = sanitized; + } } /** Native Chat normally preserves effort spelling; Mistral requires its declared model ladder. */ diff --git a/src/adapters/openai-chat/reasoning-replay.ts b/src/adapters/openai-chat/reasoning-replay.ts index 96b717c9a89..881b8b82c68 100644 --- a/src/adapters/openai-chat/reasoning-replay.ts +++ b/src/adapters/openai-chat/reasoning-replay.ts @@ -2,6 +2,7 @@ import { modelInList, type OcxProviderConfig, type OcxReasoningReplayScopeRef } import { peekReasoningForCall } from "../../responses/reasoning-replay-cache"; import { jsonStringPartsUtf8Bytes, jsonUtf8Bytes } from "../../lib/json-byte-size"; import { isTranslatorBudgetExceededError, TranslatorBudgetExceededError } from "../../lib/translator-budget"; +import { isCanonicalMistralChat } from "../../providers/mistral-transport"; type Rec = Record; export const isChatRecord = (value: unknown): value is Rec => @@ -58,7 +59,9 @@ export function planNativeChatReasoningReplay( maxSerializedBytes = Number.MAX_SAFE_INTEGER, ): NativeChatReasoningReplayPlan { const policy = chatReasoningReplayPolicy(provider, modelId); - if (!policy.preserve || !Array.isArray(messages) + // Canonical Mistral Chat rejects `reasoning_content` in history (HTTP 422), and the destination + // policy strips it before this plan is applied; replaying here would restore it (#8432). + if (!policy.preserve || !Array.isArray(messages) || isCanonicalMistralChat(provider) || modelInList(provider.reasoningDetailsModels, modelId)) { return { serializedBytes: 0, hasChanges: false, apply: () => messages as unknown[] }; } diff --git a/structure/providers/mistral-chat.md b/structure/providers/mistral-chat.md index 49cecb72bf8..2cfcea5943f 100644 --- a/structure/providers/mistral-chat.md +++ b/structure/providers/mistral-chat.md @@ -3,7 +3,11 @@ `src/adapters/openai-chat/destination-policy.ts` uses `isCanonicalMistralChat` for key-auth `https://api.mistral.ai/v1` destinations, including custom provider names. Both Chat builders rename `max_completion_tokens` to `max_tokens` without overwriting an existing non-null limit, and omit `user`, `store`, -`thinking`, `include_reasoning`, `reasoning` and `top_k`. Native effort follows the declared model ladder +`thinking`, `include_reasoning`, `reasoning` and `top_k`. The same projection drops `reasoning_content` from +every message in the history before egress: reasoning-capable Chat clients replay the field their responses +carried (#8420), and Mistral's strict Chat schema rejects it with HTTP 422. Caller history is never +mutated: when a message carries a stripped field the projection swaps in a copied array holding +stripped copies of just those messages. Native passthrough applies its reasoning-replay plan after this projection, so `planNativeChatReasoningReplay` (`src/adapters/openai-chat/reasoning-replay.ts`) returns the no-op plan for these destinations; an operator `preserveReasoningContentModels` entry cannot restore the field. Native effort follows the declared model ladder using the shared at-or-below resolver: DeepSeek/GLM `medium` becomes `low`, supported rungs and `none` remain, and an operator ladder wins. Other destinations retain native caller controls. The Mistral registry seeds `foldDeveloperRoleToSystem: true`; explicit false still wins. diff --git a/tests/adapters/openai/openai-chat-mistral-policy.test.ts b/tests/adapters/openai/openai-chat-mistral-policy.test.ts index 75d23d46135..37ab1aa4d34 100644 --- a/tests/adapters/openai/openai-chat-mistral-policy.test.ts +++ b/tests/adapters/openai/openai-chat-mistral-policy.test.ts @@ -60,6 +60,18 @@ describe("canonical Mistral Chat wire policy", () => { .toMatchObject({ max_completion_tokens: 123, user: "synthetic-user", store: false, reasoning_effort: "medium" }); } }); + test("strips replayed reasoning_content from message history without mutating the caller", () => { + const assistant = { role: "assistant", content: "Hi", reasoning_content: "thinking", tool_calls: [] }; + const result = native({ messages: [{ role: "user", content: "hi" }, assistant] }); + expect(result.messages[1]).toEqual({ role: "assistant", content: "Hi", tool_calls: [] }); + expect(assistant).toEqual({ role: "assistant", content: "Hi", reasoning_content: "thinking", tool_calls: [] }); + }); + test("keeps message reasoning_content untouched outside canonical Mistral destinations", () => { + for (const overrides of [{ baseUrl: "https://gateway.example/v1" }, { baseUrl: "https://api.mistral.ai/custom" }, { authMode: "oauth" as const }]) { + const message = { role: "assistant", content: "Hi", reasoning_content: "kept" }; + expect(native({ messages: [{ role: "user", content: "hi" }, message] }, provider(overrides)).messages[1]).toEqual(message); + } + }); }); test("the final policy removes rejected controls even when another serializer generated them", () => { @@ -68,6 +80,16 @@ test("the final policy removes rejected controls even when another serializer ge expect(body).toEqual({ max_tokens: 12 }); }); +test("the final policy strips message reasoning_content without mutating the caller's history", () => { + const assistant = { role: "assistant", content: "Hi", reasoning_content: "gone" }; + const messages = [{ role: "user", content: "hi" }, assistant]; + const body = { messages }; + applyChatDestinationPolicy(body, provider()); + expect(body.messages).toEqual([{ role: "user", content: "hi" }, { role: "assistant", content: "Hi" }]); + expect(body.messages).not.toBe(messages); + expect(messages[1]).toEqual({ role: "assistant", content: "Hi", reasoning_content: "gone" }); +}); + test("typed response normalization preserves choices, tools, usage, and existing reasoning", () => { const value = { id: "fixture", usage: { prompt_tokens: 1 }, choices: [{ index: 0, finish_reason: "tool_calls", delta: { content: [{ type: "thinking", thinking: [{ type: "text", text: "We" }], closed: true }, { type: "text", text: "A" }], @@ -80,3 +102,17 @@ test("typed response normalization preserves choices, tools, usage, and existing expect(Array.isArray(value.choices[0]!.delta.content)).toBe(true); expect(normalizeNativeChatContent(value, provider({ authMode: "oauth" }))).toBe(value); }); + +test("operator reasoning replay cannot restore reasoning_content on canonical Mistral Chat", () => { + const target = provider({ preserveReasoningContentModels: [model] }); + const history = [ + { role: "user", content: "hi" }, + { role: "assistant", content: "", reasoning_content: "replayed by the client", tool_calls: [{ id: "call_a", type: "function", function: { name: "f", arguments: "{}" } }] }, + { role: "tool", tool_call_id: "call_a", content: "ok" }, + { role: "assistant", content: "", tool_calls: [{ id: "call_b", type: "function", function: { name: "f", arguments: "{}" } }] }, + { role: "tool", tool_call_id: "call_b", content: "ok" }, + ]; + const result = native({ messages: history, tools: [{ type: "function", function: { name: "f", parameters: { type: "object" } } }] }, target); + for (const message of result.messages) expect(message).not.toHaveProperty("reasoning_content"); + expect(history[1]).toHaveProperty("reasoning_content", "replayed by the client"); +});