Skip to content

catpaw 0.2.3: kimi-k3.1-pro's native tool_use frames become tool_calls; newline-bearing JSON repaired - #1367

Open
Beyfish wants to merge 1 commit into
magpie-community:mainfrom
Beyfish:catpaw-native-tool-use
Open

Beyfish wants to merge 1 commit into
magpie-community:mainfrom
Beyfish:catpaw-native-tool-use

Conversation

@Beyfish

@Beyfish Beyfish commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

What

The tool path was JSON-only: the whole reply had to be the strict {"tool_calls":[…]} the injected system instruction asks for. Two failure modes observed on kimi-k3.1-pro (2026-11-19, through magpie):

  1. Native tool_use frames dropped. The model intermittently ignores the instructed JSON and emits its own native tool_use content blocks in the turn's SSE frames (cumulative toolParams growing across frames). The frame loop read text parts only, so the call was dropped silently: an empty assistant message, finish_reason: stop, no tool_calls — while the reasoning delta still streamed, so the model "thought and said nothing". Reproduced 3/5 runs on a complex prompt (long PowerShell command).

  2. Literal newlines inside the JSON. When the model did follow the instruction, a literal newline inside a string value (long shell commands are where it shows) broke strict JSON.parse and the raw text leaked to the user as content — the visible {"tool_calls":[{"name":"powershell",…}]} blob.

Fix

  • runConversation now collects tool_use parts keyed by toolCallId (latest cumulative params win) and yields them alongside text/reasoning. Every tools-aware path — plain non-stream, streaming end, and the forced tool_choice=required/named retry on both — prefers the native calls and falls back to the instructed-JSON parse.
  • On strict parse failure, the reply text is retried once with control chars (\n \r \t) escaped inside string literals only — no reformatting of anything that already parsed.
  • Name drift beyond case is matched back to the declaration by prefix (read_file → read, seen in a failed run's reasoning; powershell_command → powershell): the longest declared name wins, two equally good matches stay unknown — ambiguous beats wrong. The native and the JSON path resolve names through the same helper.

Tests

8 new tests in packages/catpaw/index.test.mjs against the fake upstream (no network):

  • native frames → tool_calls (non-stream + stream deltas + tool_choice=required without a second round)
  • native + JSON-path prefix drift, unknown native name dropped without leaking params
  • newline-bearing JSON repaired into a call
  • resolveToolName helper table (exact, prefix, ambiguity, null)

bun test packages/catpaw: 82 green; the 7 fix-carrying new tests are red on the previous index.mjs. bun scripts/check.mjs catpaw passes, version bumped to 0.2.3 (bun scripts/bumped.mjs HEAD clean).

Live verification through magpie's gateway with the real upstream (catpaw/kimi-k3.1-pro, alternating stream/non-stream, complex tool prompts): 12/12 + 8/8 + 8/8 runs with correct tool_calls, zero empty replies and zero leaked JSON; before the fix the same harness hit all three failure modes.

…s; a JSON reply with literal newlines is repaired

The tool path was JSON-only: the whole reply had to be the strict
{"tool_calls":[…]} the system instruction asks for. Two failure modes
observed on kimi-k3.1-pro (2026-11-19, via magpie):

1. The model intermittently ignores the instructed JSON and emits its
   own native tool_use content blocks in the turn's SSE frames
   (cumulative toolParams growing across frames). The frame loop read
   text parts only, so the call was dropped silently: an empty
   assistant message, finish stop, no tool_calls — while the reasoning
   delta still streamed, so the model "thought and said nothing".
   Reproduced 3/5 runs on a complex prompt. runConversation now
   collects tool_use parts keyed by toolCallId and yields them; every
   tools-aware path (plain, forced-retry, stream and non-stream)
   prefers the native calls and falls back to the instructed-JSON
   parse, so a native-only retry turn counts as the call too.

2. The reply was the instructed JSON, but with a literal newline
   inside a string value (long shell commands are where it shows).
   Strict JSON.parse fails and the raw text leaks to the user as
   content. On parse failure the candidate is retried once with
   control chars escaped inside string literals only.

Also, name drift beyond case — the model writes read_file for a
declared read (seen in a failed run's reasoning) — now matches back to
the declaration by prefix, the longest declared name wins, and two
equally good matches stay unknown (ambiguous beats wrong). Both the
native and the JSON path resolve names the same way.

Run: bun test packages/catpaw — 82 tests green; the 7 new ones that
the fix carries are red on the previous index.mjs.
Copilot AI balanced review requested due to automatic review settings November 19, 2026 14:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants