Skip to content

raccoon: take thinking levels from the catalog - #1372

Open
OLDnana1 wants to merge 2 commits into
magpie-community:mainfrom
OLDnana1:raccoon-thinking-levels
Open

OLDnana1 wants to merge 2 commits into
magpie-community:mainfrom
OLDnana1:raccoon-thinking-levels

Conversation

@OLDnana1

Copy link
Copy Markdown

PR: raccoon — take a model's thinking levels from the catalog

What this fixes

packages/raccoon gives every model reasoning: true and variants: {}. The
vendor does not take OpenAI's reasoning_effort at all: it takes a private
field, raccoon_thinking_level, whose values are level_1 … level_5, and the
valid set differs per model. Because the plugin neither reads that capability
from the catalog nor maps an effort onto it, no client can make a Raccoon model
think — the effort name is silently ignored upstream.

Measured against https://xiaohuanxiong.com/api/web/llm/v2/chat/completions
directly (same prompt, temperature: 0):

model request reasoning_tokens
sn-qwen4-max — 0
sn-qwen4-max raccoon_thinking_level: level_5 411
sn-deepseek-v4-1-flash-1017 — 799
sn-deepseek-v4-1-flash-1017 raccoon_thinking_level: level_5 775
sn-deepseek-v4-1-flash-1017 reasoning_effort: max 137

That last row matters: sending reasoning_effort does not raise effort there, it
lowers it. So the plugin should translate the effort and not forward it.

What changed

packages/raccoon/index.mjs:

  1. parseCatalog reads params.raccoon_thinking (supported,
    can_disable, default_level, display_levels) and keeps it as thinking
    on the model. reasoning now comes from supported instead of being
    hardcoded — a catalog that names no raccoon_thinking at all keeps the old
    true, so nothing regresses before the vendor serves the field.
  2. runtimeModel builds variants from those levels: off when the model
    can be disabled, plus minimal / low / medium / high / max, each mapped to the
    level the vendor accepts.
  3. the loader's fetch rewrites the request body: it maps reasoning_effort
    onto the model's level, sets raccoon_thinking_level, and removes
    reasoning_effort. A request with no effort is left byte-for-byte alone, so
    the vendor's own default still applies.

The mapping is the vendor's own: the catalog labels level_1 … level_5 as
Minimal / Low / Medium / High / Max, so an effort name maps to the level of the
same number — one to one on a model that publishes all five. A level the model
does not publish rounds up to the next one it does; only at the top does it
fall back to the highest available.

effort five levels level_1/3/5 (qwen4) level_1/2/4/5 (flash-lite)
minimal level_1 level_1 level_1
low level_2 level_3 level_2
medium level_3 level_3 level_4
high level_4 level_5 level_4
max level_5 level_5 level_5
off off off off

The catalog is warmed once inside the loader if it is not cached yet, so a
request that arrives before any model listing still gets levelled.

Tests

packages/raccoon/thinking.test.mjs, 17 tests, bun test:

75 pass, 0 fail   (58 existing + 17 new)

Covered: the capability shape and its refusals; the ranking on three- and
four-level models; exact level names; off with and without can_disable;
unknown effort names; bodies that are not JSON; variants; parseCatalog
honouring supported: false while staying compatible with a catalog that has no
such field; and three end-to-end runs through the plugin's own loader against a
fake vendor — effort translated, effort-less request untouched, and a
provider-prefixed model name (raccoon/sn-qwen4-max) still levelled.

Verified against the vendor, without a gateway

The plugin's own loader was run directly against
https://xiaohuanxiong.com/api/web/llm/v2, with a real account, same prompt:

request body actually sent reasoning_tokens
reasoning_effort: "max" raccoon_thinking_level: "level_5", no reasoning_effort 2526
no effort unchanged, byte for byte 0

One thing that had to be handled: the catalog now spells some ids with a -p
suffix (sn-qwen4-max-p) while a request asks for sn-qwen4-max, and the gateway
prefixes its own provider on top. thinkingOf normalises all three before
matching, so the levels are found however the name is written.

Not in this PR

  • magpie's provider editor does not show Extra body for account/plugin
    providers, so a user cannot inject raccoon_thinking_level from the UI today.
    With this change that is no longer needed: the agent's own effort setting is
    what selects the level.
  • sn-kimi-k3-1-pro rejects temperature: 0 outright (400 "temperature only 1 is allowed for this model"). Separate issue, untouched here.

Verification recipe

cd packages/raccoon
bun test

Then, against a real account: point a client at the provider, send
reasoning_effort: max for sn-qwen4-max, and expect reasoning_tokens > 0
where it used to be 0. A model named by magpie with the provider prefix
(raccoon/sn-qwen4-max) behaves the same as the bare id.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant