multimodal image forwarding seems to be broken (kimi k3 + claude code) #888

Activity

1. LPFchan

   added

   bugSomething isn't workingSomething isn't working

   on Aug 2, 2026
2. github-actions commented on Aug 2, 2026

   

   github-actionsBot

   on Aug 2, 2026 – with GitHub Actions

   Last edited by github-actions

   Contributor

   More actions

   _{Automated translation bookkeeping — detected language: unknown.}
3. github-actions

   added

   proxyHTTP proxy, routing, reverse-proxy / management authHTTP proxy, routing, reverse-proxy / management auth

   on Aug 2, 2026
4. devuterian commented on Aug 2, 2026

   

   devuterian

   on Aug 2, 2026

   More actions

   I am also experiencing this with OpenCodex on macOS.

   In my case, I tried it through commandcode.io using deepseek-v4-pro. The image contained only the blue word “green” on a white background. The first response hallucinated an unrelated conversation and even included a local file path that was not present in the image. After I asked it to try again with GPT-5.6 Luna, it correctly read “green”.

   This appears to be the same multimodal image-forwarding issue described here.
5. devmello

   added 3 commits that reference this issue on Aug 2, 2026

   fix(openai-chat): deliver tool-result images to vision models

   ...

   This issue will close once commit a98dc19 is merged into the 'main' branch.

   a98dc19

   fix(openai-chat): deliver tool-result images to vision models

   ...

   This issue will close once commit a719509 is merged into the 'main' branch.

   a719509

   fix(openai-chat): deliver tool-result images to vision models

   ...

   This issue will close once commit 9c8a0c5 is merged into the 'main' branch.

   9c8a0c5
6. lidge-jun commented on Aug 2, 2026

   

   lidge-jun

   on Aug 2, 2026

   Owner

   More actions

   Triage: confirmed still present on dev — openai-chat.ts:227 flattens tool-result parts through contentPartsToText, which replaces images with the literal [image]. PR #912 is the correct fix shape (re-send image parts in a user vision message after the tool round, matching the Google/Kiro adapters); it is in the merge queue right behind #892.
7. lidge-jun

   added a commit that references this issue on Aug 3, 2026

   Merge pull request #912 from DevMello/fix/tool-result-images-openai-chat

   ...

   This issue will close once commit 1b62026 is merged into the 'dev' branch.

   1b62026
8. lidge-jun commented on Aug 3, 2026

   

   lidge-jun

   on Aug 3, 2026

   Owner

   More actions

   Fixed on dev by #912 (merge commit 1b620266b), authored by @devmello.

   What was happening. Chat Completions tool messages accept only strings or text parts, so an image inside a tool result had nowhere to go on the wire. openai-chat flattened the whole tool result through contentPartsToText, which is where the literal [image] you saw came from — and why the model then described an image it had never received.

   The route, traced end to end for your kimicode/k3 + Claude Code case:

   1. Claude Code sends an Anthropic tool_result carrying the image.
   2. src/claude/inbound.ts:96-113 preserves it as input_image.
   3. src/responses/parser.ts:547-555 converts it to an internal toolResult with structured image parts.
   4. Both the Kimi OAuth preset (src/providers/registry.ts:712-741) and the API-key preset (:1428-1444) resolve to openai-chat.
   5. That adapter flattened it.

   The fix collects image parts while the tool round is consumed and flushes them as a following user message once the round closes — the same carrier shape google.ts and kiro.ts already use for the same reason.

   On the second report in this thread (deepseek-v4-pro via commandcode.io): that is a different path and this fix does not cover it. Built-in DeepSeek V4 models are listed in noVisionModels (src/providers/registry.ts:1013-1016), so they route through the vision sidecar rather than direct image forwarding. A custom registry entry without equivalent capability metadata can reach the direct path instead. If you are still seeing it there after updating, please open a separate issue with your provider config — it is worth tracking on its own rather than under this one.

   Verified before merge: typecheck clean on the merged tree, 42 image/EOF tests and 37 parser/vision tests passing, and a red-green ablation confirming four image-forwarding tests fail without the fix while the image-free control still passes.

   Thanks for the report, and thanks @devmello for the fix.
9. lidge-jun

   closed this as completedon Aug 3, 2026
10. agentHits

    added 2 commits that reference this issue on Sep 17, 2026

    fix(openai-chat): deliver tool-result images to vision models

    ...

    This issue will close once commit bb47c2e is merged into the 'main' branch.

    bb47c2e

    Merge pull request lidge-jun#912 from DevMello/fix/tool-result-images…

    ...

    This issue will close once commit e7ac5ba is merged into the 'main' branch.

    e7ac5ba