Repository navigation
Conversation
Carries PR #850's statusline, pricing, and endpoint-rates client onto the Claude mods stack. Conflicts with the mods branch are resolved by keeping both sides; ENABLE_SMART_ROUTING_SAVINGS stays in v2 since the routing flags moved to ucode.constants. Co-authored-by: Isaac <no-reply@databricks.com>
e547da8 to
6f8dc9c
Compare
63aa3f9 to
da5c0b9
Compare
lilly-luo
left a comment
There was a problem hiding this comment.
can you add ENABLE_SMART_ROUTING_SAVINGS to config.py
1cd168b to
98d1a3a
Compare
|
/ug-review |
| RECIPE_FIELD, | ||
| } from './constants.ts'; | ||
| import { mergeExtraBody } from './extra-body.ts'; | ||
| import { startStatusBandTurn } from './smart-routing-status.ts'; |
| let trackSavings = false; | ||
| let baseline: string | null = null; | ||
| try { | ||
| trackSavings = await ready($); |
| lastMainModel = str(request.model) ?? lastMainModel; | ||
| firstMainModel ??= lastMainModel; | ||
| firstPromptSeen = true; | ||
| userSwitched ||= lastMainModel !== firstMainModel; |
There was a problem hiding this comment.
| return {text: `Smart Router is ${action}`}; | ||
| }); | ||
| registerStatusBand(on); | ||
| registerStatusBand(on, () => savingsShown); |
| await $.fs.write( | ||
| usagePath, |
There was a problem hiding this comment.
AGENTS.md says "the mod itself still writes no file", but this rewrites the usage file on every session start, first prompt and turn. Can the pricer own persistence instead (usage passed on stdin, or written by Python)? That would also stop a write that dies halfway from being reloaded as an empty file in loadUsage.
| def fetch_endpoint_rates( | ||
| workspace: str, token: str, model_services: list[str] | ||
| ) -> tuple[list[dict], str | None]: |
| prices = { | ||
| key: price | ||
| for key, raw in models.items() | ||
| if isinstance(key, str) and (price := _load_price(raw)) is not None |
There was a problem hiding this comment.
_load_price only returns None for a non-mapping, so {} loads as an all-None price and the cache still counts as usable. can we drop entries with no input/output rate, the same way prices_from_endpoint_rates does?
Suggestion:
| if isinstance(key, str) and (price := _load_price(raw)) is not None | |
| if isinstance(key, str) | |
| and (price := _load_price(raw)) is not None | |
| and (price.input is not None or price.output is not None) |
| ) | ||
| from ucode.smart_routing.session_env import effective_environment | ||
|
|
||
| MODULE = "ucode.smart_routing.savings" |
| percent = (abs(saved) / baseline * 100).quantize(Decimal(1), rounding=ROUND_HALF_UP) | ||
| magnitude = abs(saved) | ||
| amount = ( | ||
| "<$0.01" if magnitude < _CENT else f"${magnitude.quantize(_CENT, rounding=ROUND_HALF_UP):,}" |
| if savings_enabled(): | ||
| _start_savings_price_refresh( |
Polly AI ReviewBlocking issues
Security vulnerabilitiesNo confirmed security vulnerabilities found in the frozen diff. The new endpoint client reuses authenticated HTTP handling and workspace headers; subprocess arguments remain structured, and cache identity includes the workspace. Non-blocking notes
ApproachThe pricing/accounting split and launch-local configuration fit the existing module boundaries. Background refresh keeps pricing off the request path. The approach is sound; a pricing/cache foundation followed by accounting/display would also make review and revert easier. SummaryDiff size (computed): 28 files; +2514 / -66 lines (2580 changed text lines); 0 binary files.
Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements. ChangesAdds an opt-in preset and flag, endpoint-rate fetching and workspace-scoped caching, native usage accounting, a Python pricing command, and the Claude prompt row. Documentation, component tests and native fixtures accompany those changes; no dependencies or lockfiles change. The cross-language accounting boundary, token-class pricing, session controls and native compatibility checks explain most of the size. No clearly removable implementation scaffolding was established. The version-label surface remains a scope question. TestsTest-by-test assessmentStatic review only; no tests were executed for this review. The reported component, integration-contract and native results are author-provided evidence, not independently reproduced results. Required CI is reported as pending. The table accounts for changed tests, helpers, fixtures and test documentation. Parameterized inputs share rows where their purpose and recommendation match.
Existing unit coverage is sufficient for pricing arithmetic, token classes, malformed records and missing rates. Existing native coverage establishes usage collection, replay and ordinary off/on behavior, but misses their combined model-transition failure. Additional local coverage should establish savings-enabled launch wiring and background-refresh failures. Native title/compaction exclusion, live endpoint compatibility and interactive terminal rendering remain unverified; the provided fixture results do not establish those behaviors. For human verification of the blocker, follow the five-step legacy-mode sequence above. After re-enabling, the newly selected Sonnet usage must not add an Opus-to-Sonnet saving, and earlier recorded usage must remain unchanged. ScopeThe configuration, pricing, accounting, fixtures and documentation support the intended savings estimate. No clearly unrelated change was established. The orchestrator-version trailer and its supporting option/lookup are an unresolved scope question, not a blocking scope conclusion. Automated review by Polly · workflow run |
| lastMainModel = selected; | ||
| firstMainModel ??= lastMainModel; | ||
| firstPromptSeen = true; | ||
| userSwitched ||= lastMainModel !== firstMainModel; |
There was a problem hiding this comment.
can we detect model switches on child steps too? with first-prompt routing, changing /model while a background child runs leaves its next request compared with the old launch model until the parent sends another request.
| const usage = (result as any)?.usage; | ||
| const served = str(request.model) ?? str(usage?.model); | ||
| const base = agent === null ? served : baseline; | ||
| if (trackSavings && await ready($) && usage && served && base) { |
There was a problem hiding this comment.
can we hide the estimate when a completed request has no usage? this skips the request entirely, so a previous positive total stays visible and can be regenerated on the next flush.
| _PRICE_CACHE_VERSION = 1 | ||
| _MILLION = Decimal(1_000_000) | ||
| _CONTEXT_SUFFIX_RE = re.compile(r"\[[^\]]*\]$") | ||
| _ANTHROPIC_AIGW_PREFIX_RE = re.compile(r"^anthropic-aigw-[0-9a-f]{8}-") |
There was a problem hiding this comment.
can we accept 7-digit prefixes here too, like the existing routing normalizer? those aliases already appear in the picker, but this leaves the prefix on and the price lookup fails, hiding the savings row.
|
what does this look like in the UX? can you add screenshots? |
|
there was a recently codex update wihch also allows for displaying info like this in the UI. as a followup, can we expand this functionality there? I can either author that or help review. whatever is easier PR is still WIP though, hopefully we can wrap it up today |
Add an opt-in savings estimate above the Claude Code prompt so users can see how routing affects estimated model cost. The row compares main-agent and subagent token usage with the model selected before routing, using endpoint list prices. It shows either estimated savings or additional cost, and stays hidden when usage or prices are insufficient.
Enable it with
SMART_ROUTER_CONFIG_VERSION=subagent_orch_savings_v0, orENABLE_SMART_ROUTING_SAVINGS=1alongside routing. Existing presets keep it off. The row replaces the routing status band while visible; session off/on controls stop and restore accounting. Manual model changes affect subsequent usage without repricing earlier requests.Prices refresh in the background and are cached per workspace, including its ID on shared hosts. Optional pricing or display failures leave model requests running. The estimate excludes native side queries such as titles and compaction; it is not a billing total.
Validation
6d0110186f56c9d40466ed8a3186d17d5e51ee6b. Native checks used Claude 2.1.322 and prepared local Messages responses in a loopback-only namespace. They cover usage persistence, disabled-turn model history, request overrides, concurrent cache refreshes, pricing and existing routing controls.Required CI is pending on
6d0110186f56c9d40466ed8a3186d17d5e51ee6b.