Skip to content
Open
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ atomically; the mod itself still writes no file.
off overrides. Keep routing activation limited to the V2 and subagent-only flags:
orchestration alone must not enable routing. Add regression tests for the new parameter's
controls. Session overrides must apply after version resolution.
Update the native Claude mod's literal setters and registry coverage test in the same change.
Update the native Claude mod's literal setters and validate its capabilities in the same change.

`_validate_versions` runs at module import and rejects missing keys, unknown keys, and
invalid values. Do not weaken the complete-key check or infer required keys from `_VERSIONS`.
Expand Down
41 changes: 27 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -619,14 +619,15 @@ Use `SMART_ROUTER_CONFIG_VERSION` at launch to select a smart-routing configurat
Each version defines separate Claude Code and Codex flag mappings; the existing versions
use identical mappings for both agents. The Claude-only version disables routing for Codex:

| Version | Routed agents | Subagent routing | First-prompt routing | Orchestrator |
| --- | --- | --- | --- | --- |
| `first_prompt_and_subagent_no_orch_v0` | Claude, Codex | On | On | Off |
| `subagent_only_v0` | Claude, Codex | On | Off | Off |
| `subagent_only_v1` | Claude, Codex | On | Off | Off |
| `subagent_orch_v0` | Claude, Codex | On | Off | On |
| `subagent_orch_v1` | Claude, Codex | On | Off | On |
| `subagent_orch_v0_claude_only` | Claude | On | Off | On |
| Version | Routed agents | Subagent routing | First-prompt routing | Orchestrator | Claude savings |
| --- | --- | --- | --- | --- | --- |
| `first_prompt_and_subagent_no_orch_v0` | Claude, Codex | On | On | Off | Off |
| `subagent_only_v0` | Claude, Codex | On | Off | Off | Off |
| `subagent_only_v1` | Claude, Codex | On | Off | Off | Off |
| `subagent_orch_v0` | Claude, Codex | On | Off | On | Off |
| `subagent_orch_v1` | Claude, Codex | On | Off | On | Off |
| `subagent_orch_v0_claude_only` | Claude | On | Off | On | Off |
| `subagent_orch_savings_v0` | Claude, Codex | On | Off | On | On |

`first_prompt_and_subagent_no_orch_v0` routes the first prompt and subagents without
orchestration: `ENABLE_SMART_ROUTING_V2=1`, `ENABLE_SMART_ROUTING_SUBAGENT_ONLY=0`, and
Expand All @@ -639,21 +640,33 @@ managed default model, without routing or orchestration. `--enable-smart-routing
workspace smart routing keep this version, so Codex stays unrouted: ug warns and launches it
like a plain `ug codex`, without the Smart Router skills or routing hooks.

`subagent_orch_savings_v0` uses the routing and orchestration flags from `subagent_orch_v1`
and sets `ENABLE_SMART_ROUTING_SAVINGS=1`. Savings tracking, endpoint-price refresh,
pricing, and the above-prompt savings row are opt-in and currently
implemented for Claude. Every existing preset sets this flag to `"0"`, overriding an
inherited `"1"`. Without a selector, explicitly set `ENABLE_SMART_ROUTING_SAVINGS=1`
with routing enabled to use the feature.

```bash
SMART_ROUTER_CONFIG_VERSION=subagent_orch_savings_v0 ug claude
```

`SMART_ROUTER_NAME` still selects the router independently of the preset.

Smart-routed Claude and Codex sessions install `smart-router`. The `subagent_orch_v0`
and `subagent_orch_v1` versions also install and activate the bundled `smart-router-orchestrator` skill.
`--enable-smart-routing` and workspace smart routing select `subagent_orch_v0_claude_only`, which
installs both skills for Claude only; Codex gets neither. For example:
Smart-routed Claude and Codex sessions install `smart-router`. The `subagent_orch_v0`,
`subagent_orch_v1`, and `subagent_orch_savings_v0` versions also install and activate the
bundled `smart-router-orchestrator` skill. Without a known version, `--enable-smart-routing`
and workspace smart routing select `subagent_orch_v0_claude_only`, which installs both
skills for Claude only; Codex gets neither. For example:

```bash
SMART_ROUTER_CONFIG_VERSION=subagent_orch_v0 ug claude
```

The version takes precedence over conflicting legacy flags. Once the agent is selected,
UG expands its mapping into
`ENABLE_SMART_ROUTING_V2`, `ENABLE_SMART_ROUTING_SUBAGENT_ONLY`, and
`ENABLE_SMART_ROUTER_ORCHESTRATOR` for the launched session. When the version is
`ENABLE_SMART_ROUTING_V2`, `ENABLE_SMART_ROUTING_SUBAGENT_ONLY`,
`ENABLE_SMART_ROUTER_ORCHESTRATOR`, and `ENABLE_SMART_ROUTING_SAVINGS` for the launched session. When the version is
unset or empty, these legacy flags retain their existing behavior, including
first-prompt routing through `ENABLE_SMART_ROUTING_V2=1`. Unknown versions are ignored:
no preset is applied, the inherited environment is unchanged, and commands continue normally.
Expand Down
53 changes: 53 additions & 0 deletions docs/claude-mod.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,3 +256,56 @@ but no inherited model or counterfactual native choice is guessed. The existing
`requested_model` remains UG's routed target, not Claude's original choice.
Actual execution still requires the child transcript or the separate subagent-start
audit; that event can omit its model. Existing records are not backfilled.

## Savings estimate

`hooks/register.ts` draws a row above the prompt, styled like the status band: the
estimated savings from smart routing, with the amount in a pill (green for a saving,
yellow when routing cost more), and, when the launch activates Smart Router Orchestrator,
the skill's `metadata.version` from `SKILL.claude.md`, labeled
`Unity Gateway Smart Router v<version>`. The row appears only once there is an estimate,
and then takes the status band's place, so the two never stack; the prompt hint row still
shows the routing state. Its handler decides before running the other `AbovePrompt`
handlers, so the band steps aside whichever order Claude runs them in. For each turn
request, main agent and subagents, the mod sums token usage by the model that served it
and its baseline. Main is its own baseline, and a subagent's baseline is the main model
in effect when it ran.
Under first-prompt routing, requests before the user changes the main model are priced
against the model selected when the first prompt was submitted, before the router's
`/model` command. A manual selection before that prompt becomes the baseline. Later
model changes affect only subsequent usage, including changes made during a turn with
savings disabled. Child requests observe those changes even before the next parent request.
Request-specific model overrides leave the selected main model unchanged.
Claude's native `$.session.model()` returns
the resolved model ID, including for configured family aliases such as `sonnet`.

After each turn the mod passes a snapshot of those sums on stdin to `savings_pricer`
(`python -P -m ucode.smart_routing.savings --write-usage`). The pricer atomically replaces
`savings_usage` in the launch's temporary directory, then prices the sums from the
workspace's cached endpoint rates and prints the estimate's parts (direction, amount,
percent) as one JSON line. Each savings-enabled routed launch
refreshes that cache on a background thread. Concurrent refreshes merge their model sets
under a file lock. The cache key includes the workspace ID
on shared hosts, since dollar rates can differ between organizations. An estimate is hidden, not undercounted,
when any served or baseline model has no rate. The orchestrator version is passed
as `orchestrator_version`.

Savings hooks live in `hooks/register.ts` alongside the routing controls: Claude
allows one hook module per plugin and its capability scanner follows engine API
calls only within that module. The shared `session.start` handler seeds the start model,
and `prompt.submit` captures the baseline before first-prompt routing. That capture is
persisted with the totals so a mod reload does not replace it with the routed model.
Savings failures, including denied reads of its environment flag, leave model requests
running. Savings handlers are registered only when the launch options explicitly enable
the feature. A failed usage-file write hides the estimate and retains the in-memory
totals for the next turn's write. If a completed tracked request has no usage, the estimate
stays hidden for that session, including after a mod reload, because its totals are incomplete.

The entire savings mod requires `ENABLE_SMART_ROUTING_SAVINGS=1`. Otherwise no usage
is tracked or written, no endpoint rates are fetched, no pricer runs, and no savings row is
shown; the three options stay at their declared defaults.
All existing versioned configurations set the flag to `"0"`. Select
`SMART_ROUTER_CONFIG_VERSION=subagent_orch_savings_v0` to enable it with subagent
routing and orchestration. Session routing off/on also disables/restores savings
from the launch baseline. Both the mod and pricer read the saved session controls;
CLI toggles update a mounted savings row within one second.
21 changes: 21 additions & 0 deletions docs/demos/pr-850/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[Watch the native offline savings recording](offline-savings.mp4) (21.55 seconds, 241 KB).

Claude Code 2.1.322 runs the savings-enabled `subagent_orch_savings_v0` preset.
The native router selects Sonnet for a child that reads `invoice-note.txt` and
verifies `50 - 8 = 42`. The prompt shows estimated savings of **$0.23 (57%)**;
`/smart-router off` hides the estimate and `/smart-router on` restores it.

**OFFLINE FIXTURE:** Messages responses, token usage, routing selection and endpoint
rates are prepared local inputs in a network namespace with loopback only. This
does not measure actual billing or live-gateway savings. The native Agent and Read
tools, routing CLI, accounting, display and session controls execute normally.

The MP4 replays captured native terminal text in monochrome at its original timing,
with onboarding trimmed and brief viewing holds. T3 preview capture was unavailable
because no automation host was connected. Raw transcripts and profile data remain
private; [provenance](provenance.json) records the runtime, source and media hashes.

The recording uses source commit `1721714aa99d86659735bd729830d5e5b97279c0`,
which passed 468 selected component, integration-contract and native-client cases,
plus Ruff, formatting and type checks. This documentation commit leaves those
validated source bytes unchanged.
Binary file added docs/demos/pr-850/offline-savings.mp4
Binary file not shown.
120 changes: 120 additions & 0 deletions docs/demos/pr-850/provenance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
{
"schema": 1,
"pr": "https://github.com/databricks/unity-gateway/pull/850",
"source_commit": "1721714aa99d86659735bd729830d5e5b97279c0",
"source_tree": "e0311bd5bdddd1bf8958a09c08f92088338cf0c0",
"runtime": {
"name": "Claude Code",
"version": "2.1.322",
"sha256": "bfb3703ad5380242328459bc62ada4bde8ae7ba57c6e1eeadb9c00e08874c478"
},
"offline_fixture": true,
"preset": "subagent_orch_savings_v0",
"method": "Real native interactive PTY output rendered to MP4 at its original timing. Onboarding is trimmed; terminal text is unchanged and rendered in a monochrome palette. An introductory 1.5-second hold and final 2-second hold aid viewing.",
"network": "Network namespace with loopback only. Prepared Messages, routing and endpoint-rate responses. External download attempts were rejected locally.",
"live_model_calls": 0,
"native_exit_code": 0,
"source_unchanged": true,
"route_count": 1,
"rates_count": 1,
"request_models": [
"system.ai.claude-opus-4-8",
"system.ai.claude-opus-4-8",
"system.ai.claude-sonnet-4-6",
"system.ai.claude-sonnet-4-6",
"system.ai.claude-opus-4-8"
],
"actual_native_result": {
"invoice_net_usd": 42,
"estimate_usd": "0.23",
"estimate_percent": 57,
"off_hides_estimate": true,
"on_restores_estimate": true
},
"rate_fixture_usd_per_million": {
"opus_4_8": {
"input": 15,
"output": 75
},
"sonnet_4_6": {
"input": 3,
"output": 15
},
"haiku_4_5": {
"input": 1,
"output": 5
}
},
"usage_fixture": {
"main": [
{
"input_tokens": 2000,
"output_tokens": 100
},
{
"input_tokens": 2000,
"output_tokens": 40
},
{
"input_tokens": 2500,
"output_tokens": 150
}
],
"child": [
{
"input_tokens": 8000,
"output_tokens": 80
},
{
"input_tokens": 10000,
"output_tokens": 200
}
]
},
"billing": "Prepared fixture values; not actual billing or live-gateway savings.",
"raw_capture_sha256": "c54127e492bba68b7a06f84a123da33a3d2659ad20690c1806a009dd5a0615e3",
"raw_capture_private": true,
"rendered_frames_sha256": "f6902ceb3e945f9c209714150e0bdeda31a3ef3ba8bd4322896f087c55ef1eee",
"trim_seconds": {
"start": 7.988,
"end": 26.015
},
"markers_seconds": {
"routing-on-before-work": 0.0,
"task-typed": 0.346,
"completed-task-and-estimate": 7.119,
"estimate-visible": 10.12,
"routing-off-estimate-hidden": 10.573,
"off-visible": 13.574,
"routing-on-estimate-restored": 14.026,
"on-visible": 18.027
},
"audited_distinct_terminal_screens": 78,
"frame_count": 86,
"preview_capture": "T3 preview navigation reported no connected automation host; preserved failure receipt. Video rendered from the verified native capture using installed ffmpeg/libass.",
"media": {
"sha256": "d566167d8921e5993ad26b2f436b5d24c4d3d4a0a30f02d961ef1a1e9c2f1a42",
"bytes": 241039,
"duration_seconds": 21.55,
"codec": "h264",
"width": 1280,
"height": 800,
"full_decode_passed": true
},
"media_markers_seconds": {
"routing-on-before-work": 1.5,
"task-typed": 1.846,
"completed-task-and-estimate": 8.619,
"estimate-visible": 11.62,
"routing-off-estimate-hidden": 12.073,
"off-visible": 15.074,
"routing-on-estimate-restored": 15.526,
"on-visible": 19.527
},
"video_render": {
"ffmpeg_sha256": "ed16af623947494a72e284b6eb8ff225f2da22b38b5d5069c2fd4b4ba3384e41",
"font_sha256": "c805f9436dbc268644c1d9584f01a601a653e028e08fd74b9b949f6cf8304d88",
"rendered_screen_count": 86
},
"verified_at": "2026-11-13T20:42:53.863557+00:00"
}
5 changes: 4 additions & 1 deletion skills/smart-router/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ Continue in the root unless the user explicitly requests a subagent. Honor that
native tools and normal harness model selection, without Smart Router Orchestrator; keep routing off.
Existing children can finish. When turned on, apply the `smart-router-orchestrator`
skill to further work only if the session was launched with `ENABLE_SMART_ROUTER_ORCHESTRATOR=1`.
Do not change the orchestration feature flag. Return the command's result briefly, then stop.
For launches with `ENABLE_SMART_ROUTING_SAVINGS=1`, off/on also disables/restores
savings tracking and its display row. All earlier versioned presets disable savings;
`subagent_orch_savings_v0` enables it with subagent routing and orchestration.
Do not change the orchestration or savings launch baseline. Return the command's result briefly, then stop.
In Codex, the current main turn keeps its original routing attribution; the next user turn
gets the new state. Do not interrupt, restart, or reload any conversation.
Loading
Loading