Skip to content

feat(*): share the provider-error hint across the surfaces that show one - #1342

Open
joei4cm wants to merge 3 commits into
EverMind-AI:mainfrom
joei4cm:fix/provider_error_hint_surfaces
Open

joei4cm wants to merge 3 commits into
EverMind-AI:mainfrom
joei4cm:fix/provider_error_hint_surfaces

Conversation

@joei4cm

@joei4cm joei4cm commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The fix hint for a failed model call lived in the CLI alone, and the screen that needed it most got nothing.

raven/cli/agent_commands.py held a table of category-apt hints next to the renderer that used it. A chat reader got nothing: raven/gateway/spine.py told the channel the failure's category and endpoint and stopped there, so a rate limit reached the room as "The provider is rate limiting this account (openrouter). The runtime log has the provider's own account." -- a diagnosis with nothing to do about it. The TUI got even less: the error event carried the canonical detail and no advice, so a rejected key read as Turn failed - Error calling LLM (auth@openrouter): ... and stopped there.

The knowledge now lives in one module, raven/providers/error_hints.py, and each surface asks for the wording it can honour:

  • the CLI keeps the wording it had, byte for byte -- the Fix: line for a rejected key, the Hint: line for the categories that have an apt one-liner, and no line at all for the ones that do not;
  • the channel gets what its reader can act on: "Try again in a moment." for a rate limit, "The operator can check the provider key." for a rejected key. No command -- nobody in a chat can run one on the operator's behalf -- and nothing about the credentials themselves;
  • the TUI reads as the chat does, with the one difference the screen makes: the credential case names the screen the TUI owns -- "Open Settings to check the provider key." -- rather than a command nobody there can run.

The error event now carries the advice beside the detail. raven/rpc/spine.py's emit_error takes it and sends it when it has one, and the failed-turn sink derives it from the failure's category; a crash carries no category, so it gets no advice rather than a guessed one. rpc-schema/openrpc.json, raven/rpc/models.py and both generated clients carry the new optional hint field, and ui-tui's turnErrorLine closes the line with the advice when the runtime sent one. Every other reading of the frame -- a stop, an unrecognised code, a frame with no advice -- is unchanged.

The channel reply still names the failure by its category and endpoint alone; the provider's own account stays in the operator's log, as before.

This is the first item of #336. The issue's second item -- the credential remedy the runtime already forwards but neither client renders -- is not in this change.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • Operator-facing docs or screenshots are updated when needed

  • PYTHONUTF8=1 uv run --frozen --python 3.12 pytest tests/test_rpc_spine.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_rpc_session.py tests/test_rpc_transport.py tests/test_acp_updates.py tests/test_gateway_spine.py tests/test_cli_agent_commands.py tests/test_provider_error_hints.py tests/test_error_classification.py -q -> 1339 passed

  • PYTHONUTF8=1 uv run --frozen --python 3.12 pytest tests/test_rpc_spine.py tests/test_provider_error_hints.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_rpc_session.py -q --cov=raven.rpc.spine --cov=raven.rpc.models --cov=raven.providers.error_hints --cov-report=term-missing -o addopts='' -> 890 passed; raven/providers/error_hints.py 15 statements, 0 missed, 100% covered, and no line the change touched is uncovered

  • npx vitest run src/__tests__/messages.test.ts src/__tests__/chatStream.test.ts src/__tests__/createGatewayEventHandler.test.ts src/__tests__/directEpisodes.test.ts (ui-tui) -> 181 passed

  • PYTHONUTF8=1 uv run --frozen --python 3.12 ty check raven/providers/error_hints.py raven/cli/agent_commands.py raven/gateway/spine.py raven/rpc/spine.py -> All checks passed!

  • make lint-python, make lint-types, make lint-imports, make lint-deps, make lint-tui, make lint-ui -> all pass

  • node scripts/gen-rpc-types.mjs --check (ui-tui) -> OK: generated.ts in sync; npm run gen:check --prefix ui-web passes

  • make check-source-language, make check-large-files, make check-commits -> all pass

Risk

  • Security impact considered

  • Backward compatibility considered

  • Rollback path is clear for risky changes

  • Security impact considered: nothing new reaches a reader. Each surface's advice is one sentence naming a screen or a retry; none carries vendor text, credential material or a command. The channel reply gains one sentence and keeps everything it had, and the credential case still points at the operator rather than naming a key. The CLI prints the same words it printed before, and the failure's own detail is untouched on every surface.

  • Backward compatibility considered: the CLI's rendered output is unchanged. The channel's reply grows by one sentence. On the wire, hint is optional on both sides, so an older client ignores it and a newer runtime talking to an older client omits it; the rendered line grows by one sentence only when the runtime sent one, and a stop, an unrecognised code and an advisory-less failure read exactly as before.

  • Rollback path is clear for risky changes: revert the commits.

Related Issues

#336

@LivXue

LivXue commented Nov 17, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — 8da9800c59

Reviewed at head 8da9800c59b1e25a86a56b04b930d94f7be2e339 (base a639935478). Scope: the full base-to-head diff (raven/providers/error_hints.py new, raven/gateway/spine.py, raven/cli/agent_commands.py, tests/test_provider_error_hints.py, tests/test_gateway_spine.py), the two call sites, and parse_llm_error's category vocabulary they key on.

No Windows compatibility issue found.

The change is a category→sentence lookup with two wordings (terminal vs chat) and moves the CLI's existing hints verbatim into raven/providers/error_hints.py. It touches no path, no subprocess, no encoding, and no platform branch; the strings contain no separators, and the CLI site still runs the whole hint through rich.markup.escape, so a provider name carrying brackets cannot inject markup on any platform. surface is a closed two-value switch that raises on anything else, which is the right shape.

Verification (native Windows host, Python 3.12.13, shared venv, detached worktree pinned to the head):

  • python -S runner.py tests/test_provider_error_hints.py tests/test_gateway_spine.py -q → 69 passed in 22.52s.

Coverage gaps: no live provider failure was produced, so both surfaces' wording is verified through the unit tests rather than a real 429/network error; the channel path was read, not exercised against a live channel.

@joei4cm

joei4cm commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

The head of this PR moved from 8da9800c5 to 2b43242b3.

The TUI half of #336's first item is folded in here rather than split into a second PR, because it builds on the module this one introduces. The diff now also covers the error event's new optional hint field (rpc-schema/openrpc.json, raven/rpc/models.py, both generated clients) and ui-tui's turnErrorLine. CI on the new head: 22 pass, 1 skipped (docs deploy), 0 failed.

The Windows compatibility review above was pinned to 8da9800c5, so it covers the CLI and channel surfaces but not the TUI half added since. Whenever you get a chance, a look at the new head would be appreciated.

@LivXue

LivXue commented Nov 17, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — 2b43242b39

Reviewed at head 2b43242b39564b59bf84d7d0b985f777f6f1117c (base a639935478; this head follows 8da9800c59, reviewed last round, and adds the TUI surface). Scope: the diff since that head — raven/providers/error_hints.py (+11), raven/rpc/models.py, raven/rpc/spine.py (+26), rpc-schema/openrpc.json, both rpc/generated.ts copies, ui-tui/src/domain/messages.ts, and the tests — plus the two call sites.

No Windows compatibility issue found.

The new tui surface is one more entry in the same string map (_TUI_HINTS spreads _CHANNEL_HINTS and overrides auth), and _turn_failure_hint returns "" when parse_llm_error has no category. The payload change adds an optional hint field on ErrorEventPayload with a default of None, so no consumer is forced to send or read it. Nothing here touches paths, subprocesses, encoding, or platform branches; the wire strings are plain ASCII/UTF-8 text carried through the existing JSON-RPC encoding.

Verification (native Windows host, Python 3.12.13, shared venv, detached worktree pinned to the head):

  • python -S runner.py tests/test_provider_error_hints.py tests/test_rpc_spine.py -q → 106 passed in 21.92s
  • python -S runner.py tests/test_gateway_spine.py -q → 65 passed in 21.33s
  • node node_modules/vitest/vitest.mjs run src/__tests__/messages.test.ts --no-file-parallelism (in ui-tui, modules linked from the sibling checkout) → 64 tests passed

Coverage gaps: no live provider failure and no live TUI session, so the hint's rendering is verified through the unit tests; the openrpc.json schema change was read, not validated against a generated client beyond the checked-in generated.ts diffs.

@joei4cm
joei4cm force-pushed the fix/provider_error_hint_surfaces branch from 2b43242 to dae904c Compare November 18, 2026 09:10
@LivXue

LivXue commented Nov 18, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — dae904cd64

Reviewed at head dae904cd64e7a18270dced1ebfcae6941a57c365 (base 20aaefd1161f). Scope: the full base-to-head diff (13 files) — raven/providers/error_hints.py (new), raven/cli/agent_commands.py, raven/gateway/spine.py, raven/rpc/spine.py, raven/rpc/models.py, rpc-schema/openrpc.json, both generated clients, ui-tui/src/domain/messages.ts, and the tests. Earlier heads 8da9800c59 and 2b43242b39 were reviewed in previous rounds; this head is the same two PR commits rebased onto newer main.

No Windows compatibility issue found.

  • The new logic is a closed category→sentence lookup: provider_error_hint raises ValueError on an unknown surface instead of guessing, _turn_failure_hint returns "" when parse_llm_error finds no category (a crash gets no advice), and the wire field is optional (hint: str | None = None), so older peers simply omit or ignore it. No path, subprocess, encoding, or platform branch is touched; the hints are plain ASCII text carried through the existing JSON-RPC encoding.
  • The CLI's six non-auth hints and the auth Fix: line are reproduced byte-for-byte through the new module (checked against the removed table), and the whole hint is now run through rich.markup.escape, so a provider name or <name> placeholder cannot be swallowed as markup on any console.

Verification, native Windows host (GitHub tarball of dae904cd64, extracted under D:\tmp\rv1342):

  • python -S runner.py tests/test_provider_error_hints.py tests/test_rpc_spine.py tests/test_gateway_spine.py tests/test_cli_agent_commands.py tests/test_rpc_schema_match.py -q → 770 passed
  • node node_modules/vitest/vitest.mjs run src/__tests__/messages.test.ts --no-file-parallelism (ui-tui) → 64 passed
  • node scripts/gen-rpc-types.mjs --check (ui-tui) → OK: generated.ts in sync
  • node scripts/gen-rpc-client.mjs --check (ui-web) → generated.ts matches the contract (263 methods)

Gaps: no live provider failure and no live TUI session, so both surfaces' wording is verified through unit tests; the tree is the commit's export rather than a git worktree.

@joei4cm

joei4cm commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Hi @LivXue, your Windows compatibility review for this PR (posted at head dae904cd64) found no Windows compatibility issue, and CI on this head is green (22 pass / 1 skip). The merge gate still shows "review required" only because that conclusion arrived as an issue comment rather than a formal review. If you agree, could you submit the same conclusion with the Approve button? That is all this PR is waiting on.

@joei4cm
joei4cm force-pushed the fix/provider_error_hint_surfaces branch from dae904c to e245f93 Compare November 18, 2026 15:37
@LivXue

LivXue commented Nov 18, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — e245f93e22

Reviewed at head e245f93e22438ddcb4ef437a5d4e94eaa097ce42 (rebased onto newer main). The PR's own diff is byte-identical to the revision reviewed at dae904cd64: still 550 lines over 13 files, with every blob id matching (13bd17f4f3, 5d33f8b543, 9e6a62c1f0, 88276125a9, 480e2c0bf5, 3c7611b908, 84b3551589, d324d11ef1, 80a962c294, fc9cd7c0f3, cbbc9c14a8, 8ca08275a6, 4a6e4e9288).

No Windows compatibility issue found. Same reasoning as before: a closed category→sentence lookup (ValueError on an unknown surface, "" for a category-less crash), an optional hint wire field, and CLI hints that reproduce the pre-PR strings byte-for-byte and are rich.markup.escaped.

Re-verified at this head on the native Windows host (GitHub tarball of e245f93e22, extracted under D:\tmp\rv1342b):

  • python -S runner.py tests/test_provider_error_hints.py tests/test_rpc_spine.py tests/test_gateway_spine.py tests/test_cli_agent_commands.py tests/test_rpc_schema_match.py -q → 770 passed
  • node node_modules/vitest/vitest.mjs run src/__tests__/messages.test.ts --no-file-parallelism (ui-tui) → 64 passed
  • node scripts/gen-rpc-types.mjs --check (ui-tui) → in sync; node scripts/gen-rpc-client.mjs --check (ui-web) → generated.ts matches the contract (263 methods)

Gaps: no live provider failure and no live TUI session; the tree is the commit's export rather than a git worktree.

joei4cm and others added 3 commits November 19, 2026 02:13
The fix hint for a failed model call lived in the CLI alone, as a table
next to the renderer that used it. A chat reader was told the category
and the endpoint and nothing else, so a rate limit reached the room as a
sentence with no advice attached.

The knowledge now lives in one module, and each surface asks for the
wording it can honour: a terminal is handed the command to run, a chat is
handed what its reader can do -- no command, and the credential case
pointed at the operator. The CLI's own output is unchanged, byte for byte.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
The lint job runs ruff format --check over the tree, and the new test
file was written by hand in a shape ruff would reformat.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
A failed turn reached the TUI as the canonical line and stopped there, so a
rejected key named the failure without naming a way out. The error event now
carries the advice beside the detail, in the words the screen can act on, and
the TUI closes the line with it when the runtime sent one.
@joei4cm
joei4cm force-pushed the fix/provider_error_hint_surfaces branch from e245f93 to c5cc986 Compare November 18, 2026 18:13
@LivXue

LivXue commented Nov 18, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — c5cc986eb1

Reviewed at head c5cc986eb1d2c92777330c74dfc19a66dcea2d98 (rebased onto newer main). The PR's own diff is byte-identical to the revision reviewed at e245f93e22: still 550 lines over 13 files, with every blob id matching (including 13bd17f4f3, 5d33f8b543, 9e6a62c1f0, 88276125a9, 480e2c0bf5, 3c7611b908, 84b3551589, d324d11ef1, 80a962c294, fc9cd7c0f3, cbbc9c14a8, 8ca08275a6, 4a6e4e9288).

No Windows compatibility issue found. Same reasoning as the earlier rounds: a closed category→sentence lookup (ValueError on an unknown surface, "" for a category-less crash), an optional hint wire field, and CLI hints that reproduce the pre-PR strings byte-for-byte and are rich.markup.escaped.

Re-verified at this head on the native Windows host (GitHub tarball of c5cc986eb1, extracted under D:\tmp\rv1342c):

  • python -S runner.py tests/test_provider_error_hints.py tests/test_rpc_spine.py tests/test_gateway_spine.py tests/test_cli_agent_commands.py tests/test_rpc_schema_match.py -q → 770 passed
  • node node_modules/vitest/vitest.mjs run src/__tests__/messages.test.ts --no-file-parallelism (ui-tui) → 64 passed
  • node scripts/gen-rpc-types.mjs --check (ui-tui) → in sync; node scripts/gen-rpc-client.mjs --check (ui-web) → generated.ts matches the contract (263 methods)

Gaps: no live provider failure and no live TUI session; the tree is the commit's export rather than a git worktree.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants