Skip to content

Render a vf Response as each dialect's native response - #2754

Closed
parkerpettit wants to merge 3 commits into
mainfrom
feat/dialect-response-wire
Closed

parkerpettit wants to merge 3 commits into
mainfrom
feat/dialect-response-wire

Conversation

@parkerpettit

@parkerpettit parkerpettit commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Why

Each harness speaks one API format, its dialect: chat completions (bash, rlm, most harnesses), Anthropic Messages (claude_code), or Responses (codex). Normally the interception server passes the provider's response through unchanged. So the dialects only had to parse native responses into a typed Response.

Some responses have no provider behind them. The train client generates its own, and trace replay serves recorded ones again. Then the interception server must build the native response itself. Until now only chat could, inside the train client. Anthropic and Responses could parse responses but not build them.

Change

Dialect.response_to_wire(response, model) builds the native response. Parsing it back gives the same message and usage.

  • Chat: the train client's converter moves into the dialect as completion_to_wire, and both callers share it. The only addition: provider_state is emitted as reasoning_details. The train client never sets it.
  • Anthropic: a message with thinking, text, and tool_use blocks. stream_events now streams every block, not only a single text block.
  • Responses: reuses the recorded output items, or builds them when there are none. stream_events now streams every item.

The rewrite path, which replaces a response with inert text, emits the same events as before.

Testing

  • Round trip for all three dialects with text, tool calls, and reasoning, both plain and streamed: the same message and usage come back.
  • Live through trace replay: claude_code and codex receive replayed replies and finish their rollouts.
  • pytest tests -m "not e2e": green.
  • No change on existing paths. Main streams an Anthropic or Responses reply from stream_events only after a response interceptor rewrote it, and rewrite_response reduces that reply to one text block. On that shape the new stream_events is byte-identical to main's: 44,421 shapes fuzzed from 247 recorded claude_code and codex responses. The train client's chat output is byte-identical too, because it never sets provider_state.

Note

Add response_to_wire so each dialect renders a native vf Response

  • Adds an abstract Dialect.response_to_wire in base.py that converts the internal Response model into each dialect's native payload format
  • Implements native serialization for Anthropic (content blocks, tool use, finish reasons, usage) in anthropic.py and OpenAI Responses (output items, tool calls, completion status, usage) in responses.py
  • Moves the chat completion serializer into the shared chat.py as completion_to_wire, now including provider state under reasoning_details; removes the local serializer from train.py
  • Rewrites stream_events in both the Anthropic and Responses dialects to emit start/delta/stop events for every content block and output item instead of streaming only the first text block
  • Risk: all concrete Dialect subclasses must implement response_to_wire or they can no longer be instantiated; check out-of-tree dialect implementations

Macroscope summarized 8f7db49. (Automatic summaries will resume when PR exits draft mode or review begins).

Dialect.response_to_wire turns a typed Response into the native response a
harness's SDK expects, for responses no provider sent. Chat shares one
converter with the train client (completion_to_wire, which now also carries
provider_state as reasoning_details). Anthropic builds a message with
thinking, text, and tool_use blocks; Responses reuses the recorded output
items. Both now stream responses with several blocks or items; a single text
block streams as before.
…responses

A reply with reasoning but no provider items (a model without signed
thinking, such as the train client's) lost its reasoning on the way to an
Anthropic or Responses harness. Anthropic now emits a thinking block and
Responses a reasoning item for it. Every rendered response is validated
through the dialect's provider SDK model.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant