Skip to content

fix(chat): convert Mistral typed content on native Chat passthrough - #8252

Closed
Masih-0x3 wants to merge 1 commit into
lidge-jun:devfrom
Masih-0x3:fix/native-chat-mistral-typed-content
Closed

Masih-0x3 wants to merge 1 commit into
lidge-jun:devfrom
Masih-0x3:fix/native-chat-mistral-typed-content

Conversation

@Masih-0x3

@Masih-0x3 Masih-0x3 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #8251.

On native /v1/chat/completions passthrough, Mistral reasoning models return content as typed thinking/text blocks. The native Chat lane relayed those arrays unchanged, so OpenAI-compatible Chat clients that expect a string delta.content rejected the stream. OpenCode, for example, fails with Invalid ... stream event. The translated path already converts these blocks with chatContentEvents. This PR reuses that converter on the native lane, for canonical key-auth Mistral only (isCanonicalMistralChat).

  • Streaming (src/server/chat-native-sse.ts): a typed delta becomes one chunk per run, in source order. Thinking goes to reasoning_content and text to content. role rides the first chunk. The last chunk keeps the other delta fields, finish_reason and usage, and every emitted choice carries its index. Stall, first-output and EOF-recovery tracking now see the converted text, so a typed-only stream cut before its terminal frame recovers with [DONE] and no longer ends as upstream_sse_truncated.
  • Non-streaming (src/server/chat-native.ts): a typed message.content joins its text into content (null when there is none, matching the streamed collector) and its thinking into reasoning_content. This also feeds the synthesized SSE for streaming clients when the upstream answers with JSON.
  • An unsupported block fails closed with a 502, matching the translated path.
  • Native reasoning replay capture still reads the raw upstream chunk. Native replay writes reasoning_content into history, which is not the thinking-chunk format the translated path sends back to Mistral, so request history is deliberately unchanged here.
  • Other destinations and plain-string Mistral content pass through byte-for-byte as before.

Docs: docs-site/.../reference/adapters.md and structure/providers/chat-compat.md.

Verification

All runs were local on Linux, at commit 99cd96e2 rebased on dev 7ea41c36, with Bun 1.4.4 from the repo's pinned dependency after bun install --frozen-lockfile.

  • bun test tests/responses/chat-native-mistral-content.test.ts: 9 pass, 0 fail.
    • The same file on unmodified dev (7ea41c36): 8 fail, 1 pass. The pass is the "other destinations unchanged" control, as intended.
  • bun run typecheck: no type errors (2,322 files).
  • bun run structure:check: passed.
  • bun run privacy:scan: passed.
  • bun test tests/ci-workflows/file-size-ratchet.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts: all pass.
    • The new test is registered in layout-chat.json, because layout.json sits at its 2,000-line cap.
  • bun run test (full suite, --parallel=4, 895 s, every file ran): 61,445 pass, 712 skip, 6 fail. All 6 failures are pre-existing and unrelated:
    • tests/codex-integration/codex-cli-update-apply.test.ts: 3 fail
    • tests/codex-integration/codex-composed-acceptance.test.ts: 1 fail
    • tests/codex-integration/codex-config-write-lock.test.ts: 2 fail
    • The same 6 cases fail when these files run alone on unmodified dev 7ea41c36 in this environment.

Not covered:

  • No live call to Mistral through a patched build; the fixtures replay the documented chunk shapes.
  • Clients now receive reasoning_content, and some echo it back on assistant messages. Native passthrough forwards those messages unchanged; this PR does not alter that, and I have not verified how Mistral treats that field.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (No auth, logging or credential paths touched; test fixtures use synthetic keys.)

🤖 Generated with Claude Code

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Mistral reasoning models return `content` as typed `thinking`/`text`
blocks. The native /v1/chat/completions lane relayed those arrays as-is,
so OpenAI-compatible Chat clients that expect a string `delta.content`
rejected the stream.

For canonical key-auth Mistral, reuse the translated path's
`chatContentEvents` converter:
- streamed deltas split into one chunk per run in source order
  (thinking -> `reasoning_content`, text -> `content`); `role` leads,
  the last chunk keeps the other delta fields, `finish_reason`, `usage`
- buffered `message.content` joins text into `content` (null when empty)
  and thinking into `reasoning_content`, which also feeds synthesized SSE
- unsupported blocks fail closed with a 502

Reasoning replay capture keeps reading the raw upstream chunk, so request
history sent back to Mistral is unchanged. Other destinations and plain
string content pass through unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

@Masih-0x3

Copy link
Copy Markdown
Contributor Author

Closing as superseded: #8278 (merged as 05408a3) fixed #8251 with the same approach, reusing chatContentEvents on the native Chat lane, and also covers Mistral request-field sanitization. This draft never left draft state and will not be updated. Thanks for the quick fix.

@Masih-0x3 Masih-0x3 closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native Chat passthrough relays Mistral typed content arrays, breaking OpenAI-compatible clients

1 participant