Repository navigation
Conversation
8b73d32 to
8e7c196
Compare
8e7c196 to
dca786e
Compare
|
Thanks for the fix. The token-limit resolution and provider metadata support are useful, but I would hold this PR until the cache-update regression below is fixed. In I reproduced this with an isolated profile and a synthetic Ollama model:
The model-list handler also requests a gateway reload, so this can cause runtime context accounting to substantially exceed the model's configured window. Please merge validated fields into the existing per-model limits, preserving known fields when a response omits them. Add regressions for the show → list → runtime-config round trip and for partial metadata updates that contain only one limit. Web and Desktop CI are both green, but the current tests do not cover this sequence. This needs a fix before merging. |
An ID-only model list replaced the cached limit record, so runtime parsing fell back to the protocol window.
|
Thanks @xhd0728 . Cache writes now merge validated limit fields into the existing per-model record, and a response that omits a field keeps the last known value. Regressions cover the show → list → runtime-config round trip, and a partial update that contains only one validated limit. Both are in |
Main deletes the standalone lookup import. The model-limits route still needs it, and the merged handler was throwing before it could return limits.
Opening model settings could display protocol defaults instead of the model's limits, and saving unrelated options persisted those defaults as explicit capability overrides. For example,
deepseek-chatcould fall from its catalog limits of 1,048,576 context / 393,216 output tokens to 128,000 / 32,768 after saving the dialog.Closes #685.
Rebased onto
main@9d8f1d0b(2026-11-03). Upstream landed adjacent fixes while this PR was open (3a6c4b39preserve catalog limits when enabling models,5410f5ebshare engine token limits across setup flows,c390dd53retire hidden agent context overrides,d57f2aac/5aecccf7model-test dialog and save scoping). They are kept as-is; the conflicts were resolved by combining both sides:ui/src/shared/catalogProviders.tskeeps its upstream role (picker presentation +modelTokenLimits.ts) while model lists and protocol defaults are derived from the engine catalog.ModelSettingsModal.tsxkeeps upstream'ssaveBeforeTest/blockingTasksupport and adds per-field token-source display plus edit tracking.ProviderCard.tsxkeeps upstream'sstartModelTestand catalog seeding when enabling a model, and resolves displayed limits through the newresolveTokenLimitschain.Changes
/api/showonly for the opened model. Ollamanum_ctxwins over architecture context; input and output limits are never added together.$PILOT_HOME/cache/provider-model-limits.json, scoped by provider, protocol, and endpoint. Runtime config parsing reads the cache offline, and successful updates request a gateway reload.tokenLimitOverridesmarkers allow users to deliberately pin those same numbers again.Implementation and migration details: docs/model-token-limits.md.
Note on protocol defaults
The engine now uses a single
262_144 / 32_768protocol default. Legacy-pair recovery is defined against the current protocol defaults, so configurations persisted by older builds with128_000 / 32_768are treated as explicit user values unlesstokenLimitOverridesmarks them. Keep that behavior for now, or extend the legacy list with the old pairs — worth deciding during review.Validation
Run locally with Node 22.23.2 on Windows 11:
tests/model/token-limits.spec.tsui/src/components/settings/view/modelPool,catalogProviders,modelListApi,modelTokenLimits)server/routes/config.test.js, 162 tests)npm run build