Skip to content

fix(responses): repair the Provider API wire for GLM-5.3 and Mistral Large 4 - #215

Merged
pierreraby merged 1 commit into
mainfrom
fix/responses-wire-repairs
Nov 13, 2026
Merged

pierreraby merged 1 commit into
mainfrom
fix/responses-wire-repairs

Conversation

@pierreraby

Copy link
Copy Markdown
Collaborator

Summary

zai-org/GLM-5.3 cannot be used on the /responses route at all, in any thinking mode, and mistral/mistral-large-4 is broken whenever thinking is off. Both are wire-format mismatches between what the host encoder emits and what the Provider API accepts, and both are fixed here in one per-model repair table.

Reported in #214 with a one-message reproduction on 0.8.2 + pi 1.5.2.

What the live API showed

Four request bodies against zai-org/GLM-5.3 on POST /provider/v1/responses, minimal text-only "hey" with max_output_tokens: 16:

body result
input items as the host sends them, no reasoning 400 invalid_request: input item is missing type
items with type: "message", no reasoning 200 + full SSE stream
items with type: "message" + reasoning: {effort: "none"} 400 unsupported output_config.effort: none
items as today + reasoning: {effort: "none"} 400 input item is missing type

So there are two stacked causes:

  1. Untagged message items. pi-ai tags only non-message items (type: "message" for assistant text, function_call, function_call_output, reasoning). The leading system/developer item and every user item go out as {role, content}. OpenAI accepts it; the adapter in front of GLM-5.3 does not. Verified by driving the installed encoder and dumping the body it builds for this model — the first turn emits one untagged {role: "user", ...}.
  2. reasoning.effort: "none". When thinking is off, pi-ai reads thinkingLevelMap and falls back to "none". zai-org/GLM-5.3 answers unsupported output_config.effort: none; mistral/mistral-large-4 answers reasoning_effort none is not supported for this model, supported values: [high, low, ...]. Both accept effort: "low" or no reasoning field at all. Declaring off: null is not a fix — pi then clamps to the lowest allowed level and sends effort: "low", silently forcing reasoning on instead of disabling it.

Scope

Sweep of all 74 models advertising /responses, one minimal text request each:

  • untagged input refused by exactly one model: zai-org/GLM-5.3. Every other /responses model accepts the shorthand, so the default api: "openai-responses" routing is not the problem and keeps its cache win.
  • effort: "none" refused by two models: zai-org/GLM-5.3 and mistral/mistral-large-4. The Mistral case is unrelated to Bug: invalid request "input item is missing type" on zai-org/GLM-5.3 #214 and simply never got reported.

Changes

  • src/responses-compat.ts — RESPONSES_WIRE_REPAIRS, a per-model table (zai-org/GLM-5.3: both repairs; mistral/mistral-large-4: drop the inert effort), plus rewriteResponsesBody() and withResponsesWireRepairs(). Returns the original string when nothing changes, and is a no-op for unlisted models, invalid JSON, non-object bodies, and already-tagged items.
  • index.ts — one line, inside the openai-responses branch only.
  • tests/test-responses-compat.ts — 13 hermetic tests: tagged vs untouched items, idempotence, reasoning dropped only when it is exactly {effort: "none"}, real efforts preserved, Mistral input left untagged, unlisted models byte-identical, malformed bodies, injectable table, header handling.
  • package.json, CHANGELOG.md.

No runtime, routing-default, workflow, version or dependency changes.

Validation

  • tsc --noEmit PASS, prettier --check PASS on the whole repo, git diff --check clean.
  • tests/test-responses-compat.ts 13/13 PASS.
  • Full npm test with PI_LOCAL_REQUIRED=1 OMP_COMPAT_REQUIRED=1 on real hosts (pi 1.5.2, @oh-my-pi 18.18.3, bun 1.4.4): PASS, 11 host suite lines green, 0 failures.
  • Live before/after: the exact bodies the table rewrites are the ones that answered 400 red-handed; the downstream effect was confirmed by a 200 with a complete SSE stream and a real answer.

Not covered here

Two ids are still advertised by /provider/v1/models but answer 404 at request time (Qwen/Qwen3.6-Max-Preview, stepfun/Step-3.5-Flash). Unrelated to this fix — I will file that separately rather than widen this PR.

…Large 4

Two build-level quirks of the Responses wire, both proven with live requests
against https://api.commandcode.ai/provider/v1/responses on 2026-11-13:

- zai-org/GLM-5.3 rejects the shorthand `{role, content}` input items the host
  encoder emits for system and user turns: 400 `input item is missing type`.
  Adding the item-level `type: "message"` answers 200 with a full SSE stream.
  A sweep of all 74 models advertising /responses showed GLM-5.3 is the only
  model refusing the shorthand, so the rewrite is allowlisted.
- zai-org/GLM-5.3 and mistral/mistral-large-4 reject `reasoning.effort: "none"`,
  which pi-ai sends whenever thinking is off: `unsupported output_config.effort:
  none` and `reasoning_effort none is not supported for this model`. Omitting
  the field succeeds. Declaring `off: null` in thinkingLevelMap is not a fix: pi
  clamps to the lowest allowed level and sends `effort: "low"`, silently forcing
  reasoning on.

Both repairs live in a per-model table applied to the /responses fetch wrapper.
Every other model keeps the exact bytes the host sends, and malformed or
unexpected bodies pass through untouched.

Issue: #214
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@github-actions

Copy link
Copy Markdown
Contributor

Memory benchmark

Compared base 9bd0340 with PR head 42ef3c0 on the same GitHub-hosted darwin runner. Lower values are better.

Metric Base PR PR − Base Change
Stable RSS 106.4 MiB ± 1.2 MiB 107.7 MiB ± 0.9 MiB +1.3 MiB ± 2.1 MiB +1.2%
Physical footprint 60.4 MiB ± 0.4 MiB 61.7 MiB ± 0.2 MiB +0.8 MiB ± 0.8 MiB +1.3%
Physical peak 71.1 MiB ± 0.8 MiB 73.1 MiB ± 0.9 MiB +2.0 MiB ± 2.1 MiB +2.9%

Interpretation: Differences larger than their measured MAD: Physical footprint +0.8 MiB ± 0.8 MiB.

Extension overhead above pi baseline

Metric Base overhead PR overhead Difference
Stable RSS +5.5 MiB ± 1.4 MiB +6.5 MiB ± 0.9 MiB +1.3 MiB ± 2.1 MiB
Physical footprint +3.4 MiB ± 0.8 MiB +4.1 MiB ± 0.5 MiB +0.8 MiB ± 0.8 MiB
Physical peak +3.6 MiB ± 2.2 MiB +5.6 MiB ± 0.8 MiB +2.0 MiB ± 2.1 MiB

Values are medians of 6 alternating, paired runs. The value after ± is the median absolute deviation (MAD). Each process was sampled 12 times after a 2500 ms warm-up.

Environment: pi 0.84.4, Bun 1.4.4, darwin arm64. RSS comes from ps; physical footprint and peak come from macOS footprint.

This is a comparative signal, not a pass/fail threshold. GitHub-hosted runner noise can affect absolute values.

@pierreraby
pierreraby merged commit 6094904 into main Nov 13, 2026
15 checks passed
@pierreraby
pierreraby deleted the fix/responses-wire-repairs branch November 13, 2026 11:51
pierreraby added a commit that referenced this pull request Nov 13, 2026
Patch release carrying two user-facing fixes and one pricing refresh:

- Responses wire repair for zai-org/GLM-5.3 and mistral/mistral-large-4 (#215,
  reported in #214).
- Reviewed zero pricing for thinkingmachines/inkling-2-small:free (#213).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant