Skip to content

Commit 2bb2d5f

Browse files
committed
Fix: never send thinking:{type:"disabled"} to a GLM-5.3 model
Z.AI documents thinking as impossible to turn off for this family — thinking.type accepts only "enabled" — and prescribes an explicit migration: switch to enabled and set the effort to low, "otherwise the request will fail". https://docs.z.ai/guides/llm/glm-5.3 (Migration Notice) https://docs.z.ai/guides/vlm/glm-5.3-flash Live probing did not reproduce that failure: a disabled request came back 200, with 54 chars of thinking for glm-5.3 and none for glm-5.3-flash. So this is not a fault we have observed. It is about not sending a shape the vendor documents as unsupported — the warning is written against the OpenAI-protocol upstream, which is exactly what start-plan uses, and a gateway is free to begin enforcing it. "low" is also the closest honest approximation of off: it measured zero thinking on glm-5.3-flash and the family minimum on glm-5.3. A client body can reach an upstream through four routes, and the rewrite has to hold on all of them. The two translated ones were the easy half: - OpenAI client on coding-plan (openai-to-anthropic) previously forwarded "disabled" untouched; it now resolves to the low effort level, budget included. - Anthropic client on start-plan (anthropic-to-openai) now emits enabled + reasoning_effort, with an explicit output_config.effort still winning over the substitute. The other two forward the client body verbatim, with no translator in the path at all, so body-transformer covers them: an Anthropic client on coding-plan gets the enabled form with the effort's paired budget fitted to max_tokens, and an OpenAI client on start-plan gets enabled plus reasoning_effort. Both leave an effort the client already chose alone. Scoped strictly to the GLM-5.3 family. glm-4.7 and friends genuinely support disabling — measured zero thinking when it is sent — and are untouched, which the tests assert. Verified on the wire, not just in unit tests. Upstream bodies captured through the running proxy: Anthropic client, glm-5.3, disabled -> {"thinking":{"type":"enabled","budget_tokens":8000},"output_config":{"effort":"low"}} OpenAI client, glm-5.3-flash, disabled -> {"thinking":{"type":"enabled","budget_tokens":8000},"output_config":{"effort":"low"}} Anthropic client, glm-4.7, disabled -> {"thinking":{"type":"disabled"}} all three 200.
1 parent 46e197e commit 2bb2d5f

7 files changed

Lines changed: 290 additions & 11 deletions

File tree

‎src/provider/reasoning.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,26 @@ export const GLM53_MIN_THINKING_BUDGET = 1_024;
5252
*/
5353
export const GLM53_ANSWER_RESERVE = 1_024;
5454

55+
/**
56+
* Effort level that replaces `thinking:{type:"disabled"}` for this family.
57+
*
58+
* Z.AI documents thinking as impossible to turn off here — `thinking.type`
59+
* accepts only `"enabled"` — and prescribes an explicit migration: switch to
60+
* `enabled` and set the effort to `low`, "otherwise the request will fail".
61+
* @see https://docs.z.ai/guides/llm/glm-5.3 (Migration Notice)
62+
* @see https://docs.z.ai/guides/vlm/glm-5.3-flash
63+
*
64+
* Live probing against the Anthropic gateway did *not* reproduce that
65+
* failure (a disabled request returned 200, with 54 chars of thinking for
66+
* glm-5.3 and none for glm-5.3-flash), so this rewrite is not about a fault
67+
* we have observed. It is about not sending a shape the vendor documents as
68+
* unsupported: the OpenAI-protocol upstream used on start-plan is exactly
69+
* where that warning applies, and a gateway is free to start enforcing it.
70+
* `low` is also the closest honest approximation of "off" — it measured zero
71+
* thinking on glm-5.3-flash and the family minimum on glm-5.3.
72+
*/
73+
export const GLM53_DISABLED_REPLACEMENT_EFFORT: Glm53Effort = "low";
74+
5575
/**
5676
* Match the GLM-5.3 model family, including `glm-5.3-flash`, case-insensitively
5777
* (the upstream also accepts `GLM-5.3`). Deliberately excludes `glm-5`,

‎src/proxy/body-transformer.test.ts‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -345,3 +345,105 @@ describe("transformRequestBody — metadata.user_id (Anthropic)", () => {
345345
expect(parsed.metadata).toBeUndefined();
346346
});
347347
});
348+
349+
350+
// A client body reaches the upstream verbatim on two routes — an Anthropic
351+
// client on coding-plan, and an OpenAI client on start-plan — so the
352+
// translators' GLM-5.3 rewrite would not cover them. Z.AI documents
353+
// thinking:{type:"disabled"} as unsupported for this family and prescribes
354+
// enabled + low effort. https://docs.z.ai/guides/llm/glm-5.3
355+
describe("transformRequestBody — GLM-5.3 disabled-thinking rewrite", () => {
356+
it("Anthropic passthrough: rewrites disabled to enabled with the low-effort budget", () => {
357+
const body = JSON.stringify({
358+
model: "glm-5.3",
359+
max_tokens: 40_000,
360+
thinking: { type: "disabled" },
361+
messages: [{ role: "user", content: "hi" }],
362+
});
363+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
364+
expect(parsed.thinking).toEqual({ type: "enabled", budget_tokens: 8_000 });
365+
expect(parsed.output_config).toEqual({ effort: "low" });
366+
});
367+
368+
it("Anthropic passthrough: fits the budget to a small max_tokens", () => {
369+
const body = JSON.stringify({
370+
model: "glm-5.3",
371+
max_tokens: 4_096,
372+
thinking: { type: "disabled" },
373+
messages: [{ role: "user", content: "hi" }],
374+
});
375+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
376+
expect(parsed.thinking).toEqual({ type: "enabled", budget_tokens: 3_072 });
377+
});
378+
379+
it("Anthropic passthrough: drops the budget rather than starving the answer", () => {
380+
const body = JSON.stringify({
381+
model: "glm-5.3",
382+
max_tokens: 1_500,
383+
thinking: { type: "disabled" },
384+
messages: [{ role: "user", content: "hi" }],
385+
});
386+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
387+
expect(parsed.thinking).toEqual({ type: "enabled" });
388+
});
389+
390+
it("Anthropic passthrough: leaves an output_config the client already set", () => {
391+
const body = JSON.stringify({
392+
model: "glm-5.3",
393+
max_tokens: 40_000,
394+
thinking: { type: "disabled" },
395+
output_config: { effort: "max" },
396+
messages: [{ role: "user", content: "hi" }],
397+
});
398+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
399+
expect(parsed.output_config).toEqual({ effort: "max" });
400+
});
401+
402+
it("OpenAI passthrough: rewrites disabled to enabled + reasoning_effort", () => {
403+
const body = JSON.stringify({
404+
model: "glm-5.3-flash",
405+
thinking: { type: "disabled" },
406+
messages: [{ role: "user", content: "hi" }],
407+
});
408+
const parsed = JSON.parse(transformRequestBody(body, { format: "openai", startPlan: true }) as string);
409+
expect(parsed.thinking).toEqual({ type: "enabled" });
410+
expect(parsed.reasoning_effort).toBe("low");
411+
});
412+
413+
it("OpenAI passthrough: keeps a reasoning_effort the client already set", () => {
414+
const body = JSON.stringify({
415+
model: "glm-5.3",
416+
thinking: { type: "disabled" },
417+
reasoning_effort: "max",
418+
messages: [{ role: "user", content: "hi" }],
419+
});
420+
const parsed = JSON.parse(transformRequestBody(body, { format: "openai", startPlan: true }) as string);
421+
expect(parsed.thinking).toEqual({ type: "enabled" });
422+
expect(parsed.reasoning_effort).toBe("max");
423+
});
424+
425+
it("leaves non-GLM-5.3 models alone — glm-4.7 genuinely supports disabling", () => {
426+
const body = JSON.stringify({
427+
model: "glm-4.7",
428+
max_tokens: 40_000,
429+
thinking: { type: "disabled" },
430+
messages: [{ role: "user", content: "hi" }],
431+
});
432+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
433+
expect(parsed.thinking).toEqual({ type: "disabled" });
434+
expect(parsed.output_config).toBeUndefined();
435+
});
436+
437+
it("leaves an already-enabled GLM-5.3 body untouched", () => {
438+
const body = JSON.stringify({
439+
model: "glm-5.3",
440+
max_tokens: 40_000,
441+
thinking: { type: "enabled", budget_tokens: 32_000 },
442+
output_config: { effort: "max" },
443+
messages: [{ role: "user", content: "hi" }],
444+
});
445+
const parsed = JSON.parse(transformRequestBody(body, { format: "anthropic" }) as string);
446+
expect(parsed.thinking).toEqual({ type: "enabled", budget_tokens: 32_000 });
447+
expect(parsed.output_config).toEqual({ effort: "max" });
448+
});
449+
});

‎src/proxy/body-transformer.ts‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,21 @@
1616
* is safe and matches ZCode's `applyCacheControl: true` default.
1717
* 4. Anthropic format + `ctx.userId` set → inject `metadata: { user_id }`.
1818
* Mirrors `user_id: B.metadata.userId` at bundle offset ~4760586.
19+
* 5. GLM-5.3 family + `thinking: { type: "disabled" }` → rewrite to the
20+
* enabled/low-effort form Z.AI documents, in whichever shape the
21+
* upstream expects. Only reachable on the two routes that forward a
22+
* client body verbatim; the translators handle the rest.
1923
*
2024
* @see _reverse/NOTEPAD.md "How Credential is Used for LLM Calls"
2125
*/
2226
import type { Format } from "../translator/types.js";
2327
import { buildStartPlanSystem } from "./system-prompt.js";
28+
import {
29+
isGlm53Model,
30+
buildGlm53Reasoning,
31+
fitGlm53Budget,
32+
GLM53_DISABLED_REPLACEMENT_EFFORT,
33+
} from "../provider/reasoning.js";
2434

2535
interface TransformContext {
2636
format: Format;
@@ -52,12 +62,14 @@ export function transformRequestBody(body: string | undefined, ctx: TransformCon
5262
modified = applyStartPlanOpenAISystem(parsed as Record<string, unknown>) || modified;
5363
}
5464
modified = applyStreamOptionsIncludeUsage(parsed as Record<string, unknown>) || modified;
65+
modified = applyGlm53OpenAIThinkingCompat(parsed as Record<string, unknown>) || modified;
5566
}
5667
if (ctx.format === "anthropic") {
5768
const obj = parsed as Record<string, unknown>;
5869
if (ctx.startPlan) {
5970
modified = applyStartPlanSystem(obj) || modified;
6071
}
72+
modified = applyGlm53AnthropicThinkingCompat(obj) || modified;
6173
modified = applyAnthropicCacheControl(obj) || modified;
6274
if (ctx.userId) {
6375
modified = applyAnthropicUserId(obj, ctx.userId) || modified;
@@ -84,6 +96,56 @@ function isPlainObject(v: unknown): v is Record<string, unknown> {
8496
return typeof v === "object" && v !== null;
8597
}
8698

99+
/**
100+
* Both of the transforms below exist because a client body can reach an
101+
* upstream without passing through a translator at all: an Anthropic client
102+
* on coding-plan and an OpenAI client on start-plan are each forwarded
103+
* verbatim. The translators already rewrite `thinking:{type:"disabled"}` for
104+
* GLM-5.3 models; these close the two passthrough gaps so the rewrite holds
105+
* on every route rather than only the translated ones.
106+
*
107+
* Z.AI documents `"disabled"` as unsupported for this family — `thinking.type`
108+
* accepts only `"enabled"` — and prescribes enabled + `low` effort instead.
109+
* @see https://docs.z.ai/guides/llm/glm-5.3 (Migration Notice)
110+
*/
111+
function glm53Model(body: Record<string, unknown>): boolean {
112+
return isGlm53Model(typeof body.model === "string" ? body.model : undefined);
113+
}
114+
115+
function thinkingIsDisabled(body: Record<string, unknown>): boolean {
116+
return isPlainObject(body.thinking) && body.thinking.type === "disabled";
117+
}
118+
119+
/** OpenAI-format passthrough (start-plan): `disabled` -> enabled + reasoning_effort. */
120+
function applyGlm53OpenAIThinkingCompat(body: Record<string, unknown>): boolean {
121+
if (!glm53Model(body) || !thinkingIsDisabled(body)) return false;
122+
body.thinking = { type: "enabled" };
123+
if (body.reasoning_effort === undefined) {
124+
body.reasoning_effort = GLM53_DISABLED_REPLACEMENT_EFFORT;
125+
}
126+
return true;
127+
}
128+
129+
/**
130+
* Anthropic-format passthrough (coding-plan): `disabled` -> enabled with the
131+
* effort's paired budget. The budget is fitted to `max_tokens` so it cannot
132+
* consume the whole output allowance; when no usable budget survives, the
133+
* bare `{type:"enabled"}` still beats sending an unsupported `"disabled"`.
134+
* An `output_config` the client already set is left alone.
135+
*/
136+
function applyGlm53AnthropicThinkingCompat(body: Record<string, unknown>): boolean {
137+
if (!glm53Model(body) || !thinkingIsDisabled(body)) return false;
138+
const legal = buildGlm53Reasoning(GLM53_DISABLED_REPLACEMENT_EFFORT);
139+
const budget = fitGlm53Budget(legal.thinking.budget_tokens, body.max_tokens);
140+
body.thinking = budget === undefined
141+
? { type: "enabled" }
142+
: { type: "enabled", budget_tokens: budget };
143+
if (!isPlainObject(body.output_config)) {
144+
body.output_config = legal.output_config;
145+
}
146+
return true;
147+
}
148+
87149
/**
88150
* Anthropic: add `cache_control: { type: "ephemeral" }` to the last content
89151
* block of the last non-system message. Mirrors ZCode's `HLr` algorithm.

‎src/translator/anthropic-to-openai.test.ts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,51 @@ describe("translateRequestAnthropicToOpenAI: output_config.effort", () => {
4646
expect(result.reasoning_effort).toBeUndefined();
4747
});
4848
});
49+
50+
51+
// This translator feeds the OpenAI-protocol upstream on start-plan — the exact
52+
// path where Z.AI says a disabled GLM-5.3 request "will fail".
53+
describe("translateRequestAnthropicToOpenAI — GLM-5.3 disabled thinking", () => {
54+
it("rewrites disabled to enabled + reasoning_effort low", () => {
55+
const req: AnthropicMessagesRequest = {
56+
model: "glm-5.3",
57+
messages: [{ role: "user", content: "Hi" }],
58+
max_tokens: 100,
59+
thinking: { type: "disabled" },
60+
};
61+
62+
const result = translateRequestAnthropicToOpenAI(req);
63+
64+
expect(result.thinking).toEqual({ type: "enabled" });
65+
expect(result.reasoning_effort).toBe("low");
66+
});
67+
68+
it("lets an explicit output_config.effort win over the substitute", () => {
69+
const req: AnthropicMessagesRequest = {
70+
model: "glm-5.3",
71+
messages: [{ role: "user", content: "Hi" }],
72+
max_tokens: 100,
73+
thinking: { type: "disabled" },
74+
output_config: { effort: "high" },
75+
};
76+
77+
const result = translateRequestAnthropicToOpenAI(req);
78+
79+
expect(result.thinking).toEqual({ type: "enabled" });
80+
expect(result.reasoning_effort).toBe("high");
81+
});
82+
83+
it("leaves glm-4.7 disabled thinking untouched", () => {
84+
const req: AnthropicMessagesRequest = {
85+
model: "glm-4.7",
86+
messages: [{ role: "user", content: "Hi" }],
87+
max_tokens: 100,
88+
thinking: { type: "disabled" },
89+
};
90+
91+
const result = translateRequestAnthropicToOpenAI(req);
92+
93+
expect(result.thinking).toEqual({ type: "disabled" });
94+
expect(result.reasoning_effort).toBeUndefined();
95+
});
96+
});

‎src/translator/anthropic-to-openai.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import type {
1515
AnthropicContentBlock,
1616
AnthropicUsage,
1717
} from "./types.js";
18+
import { isGlm53Model, GLM53_DISABLED_REPLACEMENT_EFFORT } from "../provider/reasoning.js";
1819

1920
/** Translate an Anthropic messages request into an OpenAI chat request. */
2021
export function translateRequestAnthropicToOpenAI(req: AnthropicMessagesRequest): OpenAIChatRequest {
@@ -55,6 +56,16 @@ export function translateRequestAnthropicToOpenAI(req: AnthropicMessagesRequest)
5556
result.reasoning_effort = req.output_config.effort as OpenAIChatRequest["reasoning_effort"];
5657
}
5758

59+
// This translator feeds the OpenAI-protocol upstream, which is precisely
60+
// where Z.AI documents `thinking:{type:"disabled"}` on a GLM-5.3 model as
61+
// fatal ("otherwise the request will fail") and prescribes enabled + low
62+
// effort instead. Rewrite rather than forward. An effort the caller set
63+
// explicitly via output_config wins over the substitute.
64+
if (isGlm53Model(req.model) && req.thinking?.type === "disabled") {
65+
result.thinking = { type: "enabled" };
66+
result.reasoning_effort ??= GLM53_DISABLED_REPLACEMENT_EFFORT;
67+
}
68+
5869
if (req.tools?.length) {
5970
result.tools = req.tools.map((t) => ({
6071
type: "function" as const,

‎src/translator/openai-to-anthropic.test.ts‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -631,9 +631,43 @@ describe("translateRequestOpenAIToAnthropic", () => {
631631
expect(result.max_tokens).toBe(4096);
632632
});
633633

634-
it("glm-5.3: explicit thinking:{type:'disabled'} is forwarded as-is, not overridden to an effort level", () => {
634+
// Z.AI documents thinking as impossible to disable for this family, and
635+
// prescribes enabled + low effort as the migration ("otherwise the
636+
// request will fail"). Never forward "disabled" for a GLM-5.3 model.
637+
// https://docs.z.ai/guides/llm/glm-5.3
638+
it("glm-5.3: thinking:{type:'disabled'} is rewritten to enabled + low effort, never forwarded", () => {
635639
const req = {
636640
model: "glm-5.3",
641+
max_tokens: 40_000,
642+
messages: [{ role: "user", content: "Hi" }],
643+
thinking: { type: "disabled" },
644+
} as OpenAIChatRequest;
645+
646+
const result = translateRequestOpenAIToAnthropic(req);
647+
648+
expect(result.thinking).toEqual({ type: "enabled", budget_tokens: 8_000 });
649+
expect(result.output_config).toEqual({ effort: "low" });
650+
});
651+
652+
it("glm-5.3-flash: thinking:{type:'disabled'} is rewritten the same way", () => {
653+
const req = {
654+
model: "glm-5.3-flash",
655+
max_tokens: 40_000,
656+
messages: [{ role: "user", content: "Hi" }],
657+
thinking: { type: "disabled" },
658+
} as OpenAIChatRequest;
659+
660+
const result = translateRequestOpenAIToAnthropic(req);
661+
662+
expect(result.thinking).toEqual({ type: "enabled", budget_tokens: 8_000 });
663+
expect(result.output_config).toEqual({ effort: "low" });
664+
});
665+
666+
// The rewrite is scoped to GLM-5.3: glm-4.7 genuinely supports disabling,
667+
// and live probing measured zero thinking when it is sent.
668+
it("glm-4.7: thinking:{type:'disabled'} is still forwarded untouched", () => {
669+
const req = {
670+
model: "glm-4.7",
637671
messages: [{ role: "user", content: "Hi" }],
638672
thinking: { type: "disabled" },
639673
} as OpenAIChatRequest;

‎src/translator/openai-to-anthropic.ts‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import {
2121
normalizeGlm53Effort,
2222
buildGlm53Reasoning,
2323
fitGlm53Budget,
24+
GLM53_DISABLED_REPLACEMENT_EFFORT,
2425
} from "../provider/reasoning.js";
2526

2627
/** Default max_tokens if the OpenAI request doesn't specify one. */
@@ -125,23 +126,24 @@ function resolveDefaultMaxTokens(model: string): number {
125126
* live testing), so routing it through the effort channel is the closer
126127
* approximation across the whole family.
127128
*
128-
* An explicit `req.thinking:{type:"disabled"}` is still forwarded as-is
129-
* (rather than overridden to an effort level) since it does work for
130-
* glm-5.3-flash, and is the closest available signal for plain glm-5.3.
131-
* An explicit `req.thinking` budget is respected over the effort-level
132-
* default budget, but `output_config.effort` is still attached — without it
133-
* the upstream runs at its own near-zero default regardless of budget.
129+
* An explicit `req.thinking:{type:"disabled"}` is rewritten to the effort
130+
* level named by `GLM53_DISABLED_REPLACEMENT_EFFORT` rather than forwarded:
131+
* Z.AI documents `"disabled"` as unsupported for this family and prescribes
132+
* exactly that migration. An explicit `req.thinking` budget is respected over
133+
* the effort-level default budget, but `output_config.effort` is still
134+
* attached — without it the upstream runs at its own near-zero default
135+
* regardless of budget.
134136
*/
135137
function translateGlm53Reasoning(
136138
req: OpenAIChatRequest,
137139
maxTokens: number,
138140
): { thinking: AnthropicThinkingConfig; output_config?: AnthropicOutputConfig } {
139141
const explicit = req.thinking;
140-
if (explicit && typeof explicit === "object" && explicit.type === "disabled") {
141-
return { thinking: { type: "disabled" } };
142-
}
142+
const disabled = explicit !== undefined && typeof explicit === "object" && explicit.type === "disabled";
143143

144-
const effort = normalizeGlm53Effort(req.reasoning_effort);
144+
const effort = disabled
145+
? GLM53_DISABLED_REPLACEMENT_EFFORT
146+
: normalizeGlm53Effort(req.reasoning_effort);
145147
const base = buildGlm53Reasoning(effort);
146148

147149
let budget: number = base.thinking.budget_tokens;

0 commit comments

Comments
 (0)