Repository navigation
Conversation
…s; a JSON reply with literal newlines is repaired
The tool path was JSON-only: the whole reply had to be the strict
{"tool_calls":[…]} the system instruction asks for. Two failure modes
observed on kimi-k3.1-pro (2026-11-19, via magpie):
1. The model intermittently ignores the instructed JSON and emits its
own native tool_use content blocks in the turn's SSE frames
(cumulative toolParams growing across frames). The frame loop read
text parts only, so the call was dropped silently: an empty
assistant message, finish stop, no tool_calls — while the reasoning
delta still streamed, so the model "thought and said nothing".
Reproduced 3/5 runs on a complex prompt. runConversation now
collects tool_use parts keyed by toolCallId and yields them; every
tools-aware path (plain, forced-retry, stream and non-stream)
prefers the native calls and falls back to the instructed-JSON
parse, so a native-only retry turn counts as the call too.
2. The reply was the instructed JSON, but with a literal newline
inside a string value (long shell commands are where it shows).
Strict JSON.parse fails and the raw text leaks to the user as
content. On parse failure the candidate is retried once with
control chars escaped inside string literals only.
Also, name drift beyond case — the model writes read_file for a
declared read (seen in a failed run's reasoning) — now matches back to
the declaration by prefix, the longest declared name wins, and two
equally good matches stay unknown (ambiguous beats wrong). Both the
native and the JSON path resolve names the same way.
Run: bun test packages/catpaw — 82 tests green; the 7 new ones that
the fix carries are red on the previous index.mjs.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The tool path was JSON-only: the whole reply had to be the strict
{"tool_calls":[…]}the injected system instruction asks for. Two failure modes observed on kimi-k3.1-pro (2026-11-19, through magpie):Native tool_use frames dropped. The model intermittently ignores the instructed JSON and emits its own native
tool_usecontent blocks in the turn's SSE frames (cumulativetoolParamsgrowing across frames). The frame loop read text parts only, so the call was dropped silently: an empty assistant message,finish_reason: stop, notool_calls— while the reasoning delta still streamed, so the model "thought and said nothing". Reproduced 3/5 runs on a complex prompt (long PowerShell command).Literal newlines inside the JSON. When the model did follow the instruction, a literal newline inside a string value (long shell commands are where it shows) broke strict
JSON.parseand the raw text leaked to the user as content — the visible{"tool_calls":[{"name":"powershell",…}]}blob.Fix
runConversationnow collectstool_useparts keyed bytoolCallId(latest cumulative params win) and yields them alongside text/reasoning. Every tools-aware path — plain non-stream, streaming end, and the forcedtool_choice=required/named retry on both — prefers the native calls and falls back to the instructed-JSON parse.\n\r\t) escaped inside string literals only — no reformatting of anything that already parsed.read_file→read, seen in a failed run's reasoning;powershell_command→powershell): the longest declared name wins, two equally good matches stay unknown — ambiguous beats wrong. The native and the JSON path resolve names through the same helper.Tests
8 new tests in
packages/catpaw/index.test.mjsagainst the fake upstream (no network):tool_calls(non-stream + stream deltas +tool_choice=requiredwithout a second round)resolveToolNamehelper table (exact, prefix, ambiguity, null)bun test packages/catpaw: 82 green; the 7 fix-carrying new tests are red on the previousindex.mjs.bun scripts/check.mjs catpawpasses, version bumped to 0.2.3 (bun scripts/bumped.mjs HEADclean).Live verification through magpie's gateway with the real upstream (
catpaw/kimi-k3.1-pro, alternating stream/non-stream, complex tool prompts): 12/12 + 8/8 + 8/8 runs with correcttool_calls, zero empty replies and zero leaked JSON; before the fix the same harness hit all three failure modes.