fix(models): preserve token limits without persisting fallbacks

docs/model-token-limits.md

@@ -0,0 +1,116 @@
+# Model token limit resolution (#685)
+
+Context and output limits resolve independently in this order:
+
+1. Explicit user configuration.
+2. Positive metadata from the configured provider endpoint.
+3. The engine model catalog, including aliases.
+4. Protocol defaults.
+
+`src/model/metadata/tokenLimits.ts` is a browser-safe shared normalizer and
+resolver. Input-only limits from Anthropic and Gemini are used directly as
+`maxContextTokens`; output is not added to input. Missing, null, fractional,
+non-finite and non-positive metadata fields do not override the next layer.
+
+## Discovery and offline behavior
+
+`/api/config/models` retains the token metadata instead of dropping everything
+except ID and name. Anthropic and Gemini lists follow cursors with a bounded
+page count and reject repeated cursors. Requests retain the existing credential
+scope checks and use the provider URL for all pagination requests.
+
+DashScope's official hosts additionally query `/api/v1/models` for `model_info`.
+An unavailable supplemental endpoint does not break ID discovery. Custom proxies
+use the same field map without making requests to an unrelated official host.
+
+Opening one model's settings calls `/api/config/model-limits`. For Ollama this
+performs one `/api/show` request, with `num_ctx` taking precedence over the model
+architecture's context length. Listing Ollama models never calls show per model.
+No output limit is invented from an Ollama context limit.
+
+Successful discoveries are cached in
+`$PILOT_HOME/cache/provider-model-limits.json`, separately from `pilotdeck.yaml`.
+The cache contains only model IDs and validated numeric limits, scoped by
+provider ID, protocol and endpoint. Inference and discovery URLs such as `/v1`
+and `/models` share a scope. Configuration parsing reads this cache synchronously
+without network access, so server restarts and headless use retain the metadata.
+A successful cache update requests a gateway configuration reload. A later
+discovery merges into the existing per-model record: only validated fields in
+the new response replace stored values, and an omitted field keeps its last
+known value. Offline or damaged-cache reads fall back to the model catalog and
+protocol defaults.
+
+## Saving and legacy configuration
+
+The dialog labels each limit's source and updates untouched fields when provider
+metadata arrives. It preserves edits made during the lookup. Saving compares the
+submitted values to the resolved non-user values. Equal values are omitted from
+`capabilities`; differing values are stored with a per-field marker:
+
+```yaml
+models:
+  deepseek-chat:
+    capabilities:
+      maxContextTokens: 128000
+      maxOutputTokens: 32768
+    tokenLimitOverrides:
+      maxContextTokens: true
+      maxOutputTokens: true
+```
+
+An old configuration containing exactly both protocol defaults, without an
+explicit marker, stops overriding a differing provider/catalog value. This is
+an in-memory interpretation, not an automatic rewrite of the user's YAML.
+Other user values, including a single overridden field, retain priority. Users
+can deliberately save the old default pair again after upgrading, or use
+“Use automatic token limits” to remove their overrides.
+
+## Relationship to #682
+
+This change does not import or replace #682's remote catalog store, generated
+catalog, refresh hooks, model overrides, or selection rules. The provider
+metadata cache is a separate higher-priority layer and must remain separate
+from #682's `model-catalog.json` cache.
+
+Before #682 lands, UI model capabilities and aliases are derived from the engine
+catalog instead of copied into a hand-maintained UI subset. Provider presentation
+fields remain local. OpenRouter's small fallback suggestion list references
+engine entries rather than duplicating their token limits.
+
+Both changes touch config parsing, ProviderCard, modelListApi and the UI catalog
+adapter. When merging them, retain #682's catalog snapshot injection and refresh
+hooks, call the shared provider normalizer, and retain #685's metadata resolver,
+cache input, explicit override markers and non-user-value save comparison.
+`/model-limits` must use #682's active catalog snapshot when that store is present.
+No #682 commit is required for this branch to build or run.
+
+## Validation
+
+Targeted tests cover provider field maps, per-field fallback, user priority,
+legacy-pair recovery and deliberate pins, metadata persistence and endpoint
+isolation, damaged/offline caches, pagination, DashScope enrichment, on-demand
+Ollama metadata, the show-then-list runtime round trip, partial cache updates,
+source labels, asynchronous dialog updates, and automatic-value
+omission on save. Tests inject provider responses; they do not make paid model
+requests or claim validation against every live provider.
+
+### Compatibility rehearsal
+
+A temporary worktree combined this fix with #682 at
+`5f5f9186c4774b8770acdedda2e9566d60e64085`. The four overlapping files above
+required manual merge resolution. Both sets of imports and parsing inputs were
+retained; #682's UI catalog adapter and refresh hooks were kept, and the new
+`/model-limits` lookup was changed to pass `readCatalogSnapshot().catalog` to
+`lookupCatalogModel`. The combined engine type check, 145 model tests, 149 config
+route tests, 74 UI tests and Web production build passed. This is an integration rehearsal,
+not a merge of #682 into the repair branch; later changes to that PR need a new
+check.
+
+### Main-branch checks
+
+The repair branch passed 135 model-layer tests, 149 config-route tests and 89
+settings/onboarding/shared-API UI tests, engine TypeScript compilation, and engine
+and Web production builds. The full UI type check still reports 546 existing
+errors, mainly React 18/19 type incompatibilities. Running the same check on
+unmodified `main` at `5862db79` produced the identical diagnostic set after
+normalizing source paths and line numbers; this fix adds no UI type errors.

src/model/config/parseModelConfig.ts

@@ -1,3 +1,5 @@
+import { readProviderModelLimits } from "../metadata/store.js";
+import { resolveTokenLimits, type TokenLimits, type TokenLimitOverrides } from "../metadata/tokenLimits.js";
 import { sanitizeProviderBody } from "../request/sanitizeProviderBody.js";
 import { parseThinkingSettings } from "../thinking/settings.js";
 import { ANTHROPIC_DEFAULT_CAPABILITIES } from "../providers/anthropic/defaults.js";
@@ -99,9 +101,10 @@ function parseProvider(providerId: string, rawProvider: unknown, env?: Credentia
     });
   }
 
+  const providerLimits = readProviderModelLimits({ providerId, protocol, baseUrl: rawUrl }, env);
   const models: Record<string, ModelDefinition> = {};
   for (const [modelId, rawModel] of Object.entries(provider.models)) {
-    models[modelId] = parseModelDefinition(modelId, protocol, rawModel, providerId);
+    models[modelId] = parseModelDefinition(modelId, protocol, rawModel, providerId, providerLimits[modelId]);
   }
 
   return {
@@ -195,6 +198,7 @@ function parseModelDefinition(
   protocol: ModelProtocol,
   rawModel: unknown,
   providerId: string,
+  providerLimits?: TokenLimits,
 ): ModelDefinition {
   const effectiveRaw = rawModel ?? {};
   if (!isRecord(effectiveRaw)) {
@@ -210,6 +214,8 @@ function parseModelDefinition(
     model.capabilities,
     catalogModel?.capabilities,
     providerId,
+    providerLimits,
+    parseTokenLimitOverrides(model.tokenLimitOverrides),
   );
   // Cross-provider model-name matches are useful for token/capability hints,
   // but they must not silently opt a custom model into image delivery. Aliases
@@ -240,6 +246,8 @@ function parseCapabilities(
   rawCapabilities: unknown,
   catalogCapabilities?: ModelCapabilities,
   providerId?: string,
+  providerLimits?: TokenLimits,
+  explicit?: TokenLimitOverrides,
 ): ModelCapabilities {
   const protocolDefaults =
     protocol === "anthropic"
@@ -249,15 +257,11 @@ function parseCapabilities(
         : OPENAI_DEFAULT_CAPABILITIES;
   const defaults = catalogCapabilities ?? protocolDefaults;
 
-  if (rawCapabilities === undefined) {
-    return applyOfficialSpeedDefault(defaults, providerId);
-  }
-
-  if (!isRecord(rawCapabilities)) {
+  if (rawCapabilities !== undefined && !isRecord(rawCapabilities)) {
     throw new ModelConfigError("invalid_capabilities", "Model capabilities must be an object.");
   }
 
-  const capabilities = rawCapabilities as RawCapabilities;
+  const capabilities = (rawCapabilities ?? {}) as RawCapabilities;
   const overrides: Partial<ModelCapabilities> = {};
 
   for (const key of [
@@ -280,7 +284,7 @@ function parseCapabilities(
 
   // Accept `contextWindow` as an alias for `maxContextTokens` so that
   // YAML configs using the friendlier name are not silently ignored.
-  const raw = rawCapabilities as Record<string, unknown>;
+  const raw = capabilities as Record<string, unknown>;
   if (raw.contextWindow !== undefined && capabilities.maxContextTokens === undefined) {
     overrides.maxContextTokens = readPositiveNumber(raw.contextWindow, "contextWindow");
   }
@@ -291,14 +295,25 @@ function parseCapabilities(
     }
   }
 
+  const limits = resolveTokenLimits({ protocol, user: overrides, explicit, provider: providerLimits, catalog: catalogCapabilities });
   return applyOfficialSpeedDefault({
     ...mergeCapabilities(defaults, overrides),
+    maxContextTokens: limits.maxContextTokens.value,
+    maxOutputTokens: limits.maxOutputTokens.value,
     ...(capabilities.supportsThinking !== undefined
       ? { supportsThinkingExplicit: capabilities.supportsThinking }
       : {}),
   } as ModelCapabilities, providerId, overrides.supportsSpeed);
 }
 
+function parseTokenLimitOverrides(value: unknown): TokenLimitOverrides | undefined {
+  if (value === undefined) return undefined;
+  if (!isRecord(value) || Object.values(value).some(v => typeof v !== "boolean")) {
+    throw new ModelConfigError("invalid_capabilities", "tokenLimitOverrides must contain boolean values.");
+  }
+  return value as TokenLimitOverrides;
+}
+
 function applyOfficialSpeedDefault(
   capabilities: ModelCapabilities,
   providerId: string | undefined,

src/model/config/schema.ts

@@ -20,6 +20,7 @@ export type RawProviderConfig = {
 };
 
 export type RawModelDefinition = {
+  tokenLimitOverrides?: unknown;
   thinking?: unknown;
   displayName?: unknown;
   capabilities?: unknown;

src/model/metadata/discovery.ts

@@ -0,0 +1,90 @@
+import { networkFetch } from "../../network/fetch.js";
+import { buildProviderModelsEndpointCandidates, isExpectedProviderModelsResponseShape } from "../providerEndpoint.js";
+import type { ModelProtocol } from "../protocol/canonical.js";
+import { normalizeModelListItem, parseModelListResponse, positiveTokenLimit, type ProviderModelMetadata } from "./tokenLimits.js";
+
+type Options = { protocol: ModelProtocol; baseUrl: string; apiKey?: string; providerId?: string; signal?: AbortSignal; fetchImpl?: typeof fetch };
+
+export class ModelDiscoveryError extends Error {
+  constructor(message: string, readonly status: number) { super(message); }
+}
+
+function headers(options: Options): Record<string, string> {
+  const key = options.apiKey;
+  if (options.protocol === "anthropic") return { "anthropic-version": "2023-06-01", ...(key ? { "x-api-key": key } : {}) };
+  return key ? options.protocol === "google" ? { "x-goog-api-key": key } : { Authorization: `Bearer ${key}` } : {};
+}
+
+async function request(url: string, options: Options, body?: unknown): Promise<Record<string, unknown>> {
+  const response = await networkFetch(url, {
+    method: body === undefined ? "GET" : "POST", headers: { ...headers(options), ...(body === undefined ? {} : { "Content-Type": "application/json" }) },
+    ...(body === undefined ? {} : { body: JSON.stringify(body) }), signal: options.signal,
+  }, { signal: options.signal, fetchImpl: options.fetchImpl, retry: { maxRetries: 2, baseDelayMs: 500, maxDelayMs: 5000, retryOnPost: body !== undefined } });
+  const text = await response.text();
+  let parsed;
+  try { parsed = text ? JSON.parse(text) : {}; } catch { /* Non-JSON error responses still keep the upstream HTTP status. */ }
+  if (!response.ok) throw new ModelDiscoveryError(parsed?.error?.message || parsed?.message || text || `HTTP ${response.status}`, response.status);
+  if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) throw new ModelDiscoveryError(`Expected JSON from ${url}, but received non-JSON content.`, 502);
+  return parsed;
+}
+
+/** Follow only provider-issued cursors on the same URL; never forward keys to a next-link host. */
+async function listPages(options: Options): Promise<ProviderModelMetadata[]> {
+  const urls = buildProviderModelsEndpointCandidates(options);
+  let body: Record<string, unknown> | undefined, endpoint = urls[0];
+  for (const url of urls) {
+    try {
+      const candidate = await request(url, options);
+      if (isExpectedProviderModelsResponseShape(options.protocol, candidate)) { body = candidate; endpoint = url; break; }
+    } catch (error) {
+      if (!(error instanceof ModelDiscoveryError) || ![400, 404, 405].includes(error.status) || url === urls.at(-1)) throw error;
+    }
+  }
+  if (!body) throw new ModelDiscoveryError("Expected a provider model list.", 502);
+  const models = new Map<string, ProviderModelMetadata>(), cursors = new Set<string>();
+  for (let page = 0; ; page++) {
+    for (const model of parseModelListResponse(body)) models.set(model.id, { ...models.get(model.id), ...model });
+    const cursor = options.protocol === "google" ? body.nextPageToken : options.protocol === "anthropic" && body.has_more === true ? body.last_id : undefined;
+    if (!cursor) {
+      if (options.protocol === "anthropic" && body.has_more === true) throw new ModelDiscoveryError("Provider omitted its pagination cursor.", 502);
+      return [...models.values()];
+    }
+    if (typeof cursor !== "string" || cursors.has(cursor) || page >= 99) throw new ModelDiscoveryError("Provider model pagination did not finish.", 502);
+    cursors.add(cursor);
+    const url = new URL(endpoint);
+    url.searchParams.set(options.protocol === "google" ? "pageToken" : "after_id", cursor);
+    body = await request(url.toString(), options);
+    if (!isExpectedProviderModelsResponseShape(options.protocol, body)) throw new ModelDiscoveryError("Expected a provider model list.", 502);
+  }
+}
+
+export async function discoverProviderModels(options: Options): Promise<ProviderModelMetadata[]> {
+  const models = await listPages(options);
+  const url = new URL(options.baseUrl);
+  // This API is separate from the compatible-mode discovery endpoint. A proxy
+  // must never cause an additional request to the official host with its key.
+  if (["dashscope.aliyuncs.com", "dashscope-intl.aliyuncs.com", "dashscope-us.aliyuncs.com"].includes(url.hostname)) {
+    try {
+      const details = parseModelListResponse(await request(`${url.origin}/api/v1/models`, options));
+      const byId = new Map(details.map(model => [model.id, model]));
+      return models.map(model => ({ ...model, ...byId.get(model.id) }));
+    } catch { /* Discovery still works when the optional metadata endpoint is unavailable. */ }
+  }
+  return models;
+}
+
+/** Ollama's loaded window is available only in /api/show, requested for one opened model. */
+export async function discoverOllamaModel(options: Options, modelId: string): Promise<ProviderModelMetadata> {
+  const url = new URL(options.baseUrl);
+  url.pathname = url.pathname.replace(/\/+$/, "").replace(/\/models$/, "").replace(/\/v1$/, "").replace(/\/api$/, "") + "/api/show";
+  url.search = ""; url.hash = "";
+  const body = await request(url.toString(), options, { model: modelId });
+  const info = body.model_info && typeof body.model_info === "object" ? body.model_info as Record<string, unknown> : {};
+  const parameters = typeof body.parameters === "string" ? body.parameters : "";
+  const configured = /^\s*num_ctx\s+(\d+)\s*$/m.exec(parameters);
+  const configuredValue = configured ? positiveTokenLimit(Number(configured[1])) : undefined;
+  const architecture = typeof info["general.architecture"] === "string" ? info["general.architecture"] : undefined;
+  const context = configuredValue ?? positiveTokenLimit(architecture ? info[`${architecture}.context_length`] : undefined)
+    ?? Object.entries(info).filter(([key]) => key.endsWith(".context_length")).map(([, value]) => positiveTokenLimit(value)).find(v => v !== undefined);
+  return { ...normalizeModelListItem({ id: modelId })!, ...(context !== undefined ? { maxContextTokens: context } : {}) };
+}

src/model/metadata/store.ts

@@ -0,0 +1,84 @@
+import { randomUUID } from "node:crypto";
+import { mkdirSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { resolvePilotHome } from "../../pilot/paths.js";
+import type { ModelProtocol } from "../protocol/canonical.js";
+import {
+  normalizeProviderTokenLimits,
+  TOKEN_LIMIT_KEYS,
+  type ProviderModelMetadata,
+  type TokenLimits,
+} from "./tokenLimits.js";
+
+type Scope = { providerId: string; protocol: ModelProtocol; baseUrl: string };
+type Cache = Record<string, Record<string, TokenLimits>>;
+type Env = Record<string, string | undefined>;
+
+/** Discovery URLs can be siblings of the inference URL, e.g. /models and /v1. */
+export function metadataEndpointScope(value: string): string {
+  const url = new URL(value);
+  url.username = ""; url.password = ""; url.search = ""; url.hash = "";
+  url.pathname = url.pathname.replace(/\/+$/, "").replace(/\/models$/, "").replace(/\/v1(?:beta)?$/, "");
+  return url.toString().replace(/\/+$/, "");
+}
+
+function scopeKey(scope: Scope): string {
+  return JSON.stringify([scope.providerId, scope.protocol, metadataEndpointScope(scope.baseUrl)]);
+}
+
+function cachePath(env?: Env): string {
+  return join(resolvePilotHome(env), "cache", "provider-model-limits.json");
+}
+
+function readCache(env?: Env): Cache {
+  try {
+    const file = cachePath(env);
+    if (statSync(file).size > 10 * 1024 * 1024) return {};
+    const body = JSON.parse(readFileSync(file, "utf8"));
+    return body?.version === 1 && body.scopes && typeof body.scopes === "object" && !Array.isArray(body.scopes) ? body.scopes : {};
+  } catch { return {}; }
+}
+
+/** Read-only, synchronous fallback for config parsing, including headless/offline starts. */
+export function readProviderModelLimits(scope: Scope, env?: Env): Record<string, TokenLimits> {
+  try {
+    const models = readCache(env)[scopeKey(scope)];
+    if (!models || typeof models !== "object" || Array.isArray(models)) return {};
+    return Object.fromEntries(Object.entries(models).map(([id, value]) => [id, normalizeProviderTokenLimits(value)]));
+  } catch { return {}; }
+}
+
+/** Keep a known limit when a later response omits it. A present, validated value replaces that field. */
+function mergeValidatedTokenLimits(current: TokenLimits | undefined, incoming: TokenLimits): TokenLimits {
+  const merged = normalizeProviderTokenLimits(current);
+  for (const key of TOKEN_LIMIT_KEYS) {
+    const value = incoming[key];
+    if (value !== undefined) merged[key] = value;
+  }
+  return merged;
+}
+
+function mergeCachedModelLimits(
+  existing: Record<string, TokenLimits> | undefined,
+  models: ProviderModelMetadata[],
+): Record<string, TokenLimits> {
+  const merged: Record<string, TokenLimits> = { ...existing };
+  for (const model of models) {
+    merged[model.id] = mergeValidatedTokenLimits(merged[model.id], normalizeProviderTokenLimits(model));
+  }
+  return merged;
+}
+
+/** Persist metadata separately from user configuration; never store credentials or raw responses. */
+export function writeProviderModelLimits(scope: Scope, models: ProviderModelMetadata[], env?: Env): void {
+  const file = cachePath(env), temporary = `${file}.${randomUUID()}.tmp`;
+  try {
+    const cache = readCache(env), key = scopeKey(scope);
+    cache[key] = mergeCachedModelLimits(cache[key], models);
+    mkdirSync(dirname(file), { recursive: true });
+    writeFileSync(temporary, JSON.stringify({ version: 1, scopes: cache }), { mode: 0o600 });
+    renameSync(temporary, file);
+  } finally {
+    try { unlinkSync(temporary); } catch { /* Already renamed, or the directory was read-only. */ }
+  }
+}