[Catapult291]

fix(translator): accept content-part arrays in function_call_output so tool images reach the model

Problem

The Responses spec allows function_call_output.output to be a content-part array — OpenAI's own API docs show {type:"input_image", image_url} inside a tool result precisely so the model can look at a screenshot/image the tool returned. Coding agents (e.g. returning read_file results for image files) send exactly this shape.

src/translator/responses-to-chat.ts only accepted string output:

const fco = item as { call_id: string; output?: string };
out.push({ role: "tool", tool_call_id: fco.call_id, content: fco.output ?? "" });

With an array, the raw part array rode into the Chat tool-message content verbatim, and the downstream Chat→Anthropic toolResultContent() mapper only knows image_url / video_url / file parts — the Responses-dialect input_image parts fell through to the unknown-part fallback and became empty text blocks. Every image in a tool result was silently dropped; the model received no trace of it.

Fix

Normalize the output with a new toolOutputToChat():

• string → passes through unchanged (existing wire shape, byte-for-byte).
• array → new outputPartsToChat() maps Responses parts (input_text, input_image with image_url: string | {url}, input_video, input_file) and their already-Chat-shaped aliases onto Chat content parts; unknown parts degrade to a text part carrying their JSON instead of being dropped. All-text arrays collapse to a plain string, so pure-text tool results keep their exact previous shape.
• structured object → stringified (matches OpenAI Chat, where tool-message content is stringly typed).

No downstream changes needed: the Chat→Anthropic hop already maps tool-message image_url parts onto Anthropic tool_result image blocks (base64 or URL source), so images now reach Anthropic-dialect upstreams end to end.

Tests

• responses-to-chat.test.ts: array output (input_text + input_image) → Chat tool message with image_url part; structured (non-array, non-string) output → stringified.
• openai-to-anthropic.test.ts: end-to-end Responses→Chat→Anthropic — input_image in a tool output arrives as a base64 image block inside the tool_result.
• Full suite: 123 pass / 0 fail across the two touched files; bun x tsc --noEmit clean.

Evidence (live)

Sent a 64×64 PNG data-URL as input_image inside a function_call_output to a build running this fix, via /v1/responses → Anthropic-dialect upstream: the model's reasoning cites "The image shows a solid red square." and the final answer is "Red". With the unfixed translator the same request produced an answer with no knowledge of the image (empty text block reached the upstream).