diff --git a/docs-site/src/content/docs/fr/reference/configuration/server.md b/docs-site/src/content/docs/fr/reference/configuration/server.md index 20e8336a1f8..6808cfe4b49 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/server.md +++ b/docs-site/src/content/docs/fr/reference/configuration/server.md @@ -15,7 +15,7 @@ exécute des fonctionnalités d'assistance autour des demandes du fournisseur. | `proxy?` | `string` | — | URL du proxy HTTP(S) ou SOCKS5 sortant (`socks5://host:port`) ou `${ENV_VAR}`. Les URL HTTP s’appliquent à `HTTP_PROXY` / `HTTPS_PROXY` si elles sont vides. Les URL SOCKS5 utilisent le tunnel SOCKS5 intégré et sont aussi exposées via `ALL_PROXY` (`ocx start --socks5`); `HTTP(S)_PROXY` héritées sont effacées dans ce processus. Le bouclage reste dans `NO_PROXY`. | | `emptyCompletionRetry?` | `boolean` | `false` | Active une nouvelle tentative Responses identique lorsqu’une réponse ne contient ni texte ni appel d’outil. Cette tentative peut être facturée. `OCX_EMPTY_COMPLETION_RETRY=0` la désactive sans modifier la configuration ; les combinaisons et les tours de compactage routés restent exclus. | | `stallTimeoutSec?` | `number` | `300` (public) / désactivé (local) | Secondes sans progression utile en amont (Responses et Chat natif) avant la coupure du flux. Sans réglage, un amont **local** (loopback, privé ou nom `.local`/`.lan`) est désactivé par défaut et un amont public vaut 300 s ; une valeur positive s'applique aux deux (minimum 1 s) ; `0` désactive partout le watchdog de silence. Pour Responses qui replie le SSE canonique ChatGPT en JSON non-streaming, un plafond total indépendant de 15 minutes subsiste même lorsque ce watchdog est désactivé. Les lectures de corps en attente de `/v1/responses/compact` partagent ce budget mais valent 300 s par défaut même pour un amont local ; une valeur explicite, y compris `0`, prime. | -| `connectTimeoutMs?` | `number` | `200000` | Délai maximal par tentative pour DNS/TCP/TLS et les en-têtes finaux ; il prend fin avant la génération du corps. | +| `connectTimeoutMs?` | `number` | `200000` (public) / désactivé (local) | Délai par tentative entre l'envoi de la requête et le premier en-tête de réponse de l'amont (DNS/TCP/TLS et attente des en-têtes) ; il prend fin avant la génération du corps et est indépendant de `stallTimeoutSec`, qui régit le flux du corps. Sans réglage, un amont **local** (loopback, privé ou nom `.local`/`.lan`) est désactivé par défaut et un amont public vaut 200 s ; une valeur positive s'applique aux deux ; `0` (ou toute valeur non positive) désactive partout ce délai d'en-tête. La désactivation permet à un modèle local de passer plusieurs minutes en prompt prefill sur une requête volumineuse sans que l'horloge du premier en-tête ne coupe une requête saine. | | `shutdownTimeoutMs?` | `number` | `5000` | Délai de vidange gracieux avant l’annulation des tours actifs. | | `websockets?` | `boolean` | `false` | Annonce et autorise la route WebSocket Responses destinée aux clients. La valeur false maintient les clients sur HTTP/SSE ; elle ne désactive pas une optimisation WebSocket canonique admissible vers ChatGPT en amont. | | `corsAllowOrigins?` | `string[]` | `[]` | Origines exactes supplémentaires autorisées par CORS. Les origines de bouclage sont toujours autorisées. Les origines d'extensions de navigateur basées sur l'autorité telles que `chrome-extension://` sont prises en charge ; `*` n'est pas un caractère générique. Firefox et Safari régénèrent l'extension UUID (par installation / par lancement de navigateur), mettez donc à jour l'entrée lorsque l'origine change. | @@ -322,3 +322,11 @@ Le retour à 2.80.0 ne garantit ni l’agrégation canonique ni le même solde d Les journaux de versions expérimentales non publiées contenant `pool-current` ou `poolContinuity` sont exclus du contrat C, même après compactage. Leurs soldes d’origine restent un historique inconnu jusqu’à l’expiration normale, sans conversion ni suppression immédiate. Un enregistrement complet mais invalide, y compris un `null` final, bloque l’admission sous plafond. Le compactage peut conserver les données valides dans un point de contrôle propre, mais le refus reste actif jusqu’au redémarrage avec ce journal propre. Une dernière ligne JSON incomplète suit les règles de récupération existantes. Voir les limites et codes d’erreur dans la [référence anglaise](/reference/configuration/server/#historical-pool-continuity). + +## Classification des délais des upstreams locaux + +Cette classification règle les délais, pas les autorisations réseau. Sans requête DNS, les destinations HTTP(S) locales sont : `localhost` exact (casse indifférente, point final admis), `0.0.0.0`, loopback IPv4 (127/8), plages privées RFC 1918 et link-local (169.254/16), loopback IPv6, link-local (fe80::/10), unique-local (fc00::/7), équivalents IPv4-mapped et noms `.local`/`.lan`. Les autres noms DNS gardent les délais publics même s'ils résolvent vers loopback. Les destinations invalides, les schémas non pris en charge et `localhost.example` restent publics. + +Désactiver le délai d'en-tête préserve l'annulation client et toute échéance globale englobante. Un `stallTimeoutSec` explicitement positif borne le silence du corps local ; sans délais locaux header/stall configurés, aucune borne automatique ne précède les en-têtes. Le JSON local de Chat/Messages natif utilise le budget stall ; le JSON public conserve `max(connectTimeoutMs ?? 200000, 5000)`. Le corps compact conserve 300 secondes par défaut, même en local. Les budgets image/recherche restent inchangés. + +Le bloc facultatif `opencodeClientExport` accepte `headerTimeoutMs`, `chunkTimeoutMs` et `timeoutMs` : millisecondes positives ou `false` pour désactiver. Les valeurs absentes préservent les défauts OpenCode ; les champs invalides sont omis. Ces réglages couvrent tout le provider, dans options V1 et settings V2. Voir [modèles locaux lents](/guides/opencode/#slow-local-models-prefill-timeouts). diff --git a/docs-site/src/content/docs/guides/opencode.md b/docs-site/src/content/docs/guides/opencode.md index cc61a911c53..9b25081ac24 100644 --- a/docs-site/src/content/docs/guides/opencode.md +++ b/docs-site/src/content/docs/guides/opencode.md @@ -201,6 +201,43 @@ CLI export, dashboard export, managed refresh and the launcher share the effecti The `opencodex` provider block is regenerated on every launch, so per-model tweaks made inside it will not survive. Keep custom entries under a provider key of your own instead. +## Slow local models (prefill timeouts) + +opencode waits a bounded time for the first response header from a provider (five minutes by +default in current V2), and a separate, longer bound between streamed chunks. Behind the +`opencodex` provider, a local model (LM Studio, vLLM, +Ollama, or any self-hosted endpoint) can spend longer than that in prompt *prefill* on a large +prompt — processing the prompt before it emits a single token. On a non-streaming turn the proxy +sends opencode nothing until the upstream completes, so opencode's own header wait fires first and +the turn fails with `Timed out waiting for response headers`, even though the model is still +working. + +Because the generated `opencodex` provider multiplexes every routed upstream under one block and +opencode's timeouts are provider-wide, raise or disable the wait for the whole provider with +`opencodeClientExport` in your opencodex config. It is omitted by default, so opencode keeps its +native waits (header and the longer chunk bound) unless you opt in: + +```json +{ + "opencodeClientExport": { + "headerTimeoutMs": false, + "chunkTimeoutMs": 1800000 + } +} +``` + +| Field | opencode setting | Meaning | +| --- | --- | --- | +| `headerTimeoutMs` | `settings.headerTimeout` | Wait for the proxy's first response header. A number of milliseconds, or `false` to disable. | +| `chunkTimeoutMs` | `settings.chunkTimeout` | Wait between streamed response chunks. A number of milliseconds, or `false` to disable. | +| `timeoutMs` | `settings.timeout` | Whole-request bound. A number of milliseconds, or `false`. opencode applies no default. | + +Each value is emitted verbatim into both the `provider.opencodex` (`options`) and +`providers.opencodex` (`settings`) blocks. Disabling `headerTimeout` removes opencode's safety net +against a genuinely hung proxy for every model routed through it, so prefer a large explicit value +(for example `1800000` for thirty minutes) unless you specifically want no bound. A malformed value +is dropped and opencode keeps its native default rather than failing to load. + ## Requirements opencode must be installed and on `PATH`: diff --git a/docs-site/src/content/docs/ja/reference/configuration/server.md b/docs-site/src/content/docs/ja/reference/configuration/server.md index cb3c56adb68..e4143c5323f 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/server.md +++ b/docs-site/src/content/docs/ja/reference/configuration/server.md @@ -15,7 +15,7 @@ description: リスナー、リモート アクセス、アドミッション | `emptyCompletionRetry?` | `boolean` | `false` | テキストもツール呼び出しもない Responses ターンを、ターミナルイベント前にストリームが終了した場合も含め、同一リクエストで 1 回再試行するよう明示的に有効化します。再試行は課金対象になる場合があります。`OCX_EMPTY_COMPLETION_RETRY=0` で設定を変更せず無効化できます。combo と routed-compaction turn は対象外です。 | | `dropCodexSafetyBuffering?` | `boolean` | `false` | Codex Responses パススルーから Codex の safety-buffering ヒントを除去します。対象は `x-codex-safety-buffering-enabled` / `x-codex-safety-buffering-faster-model` 応答ヘッダー、`safety_buffering` 型の `response.metadata` SSE イベント、およびその他の SSE イベントにある `safety_buffering` フィールドです。Codex TUI はこれらを、既定の操作でセッションをより弱いモデルに切り替える「より高速なモデルで再試行」プロンプトとして表示します。その他の `x-codex-*` ヘッダーと SSE イベントの内容は、そのフィールドの除去を除いて変更せずに転送されます。既定ではオフです。 | | `stallTimeoutSec?` | `number` | `300`（public）/ 無効（local） | ストリームが遮断されるまでの、有効な上流進捗がない秒数（Responses とネイティブ Chat）。未設定では**ローカル**上流（loopback・プライベート・`.local`/`.lan` 名）は無効が既定、public 上流は 300 秒。正の値は両方に適用（最小 1 秒）、`0` で無通信 watchdog を全面無効化。canonical ChatGPT SSE を非ストリーミング JSON にまとめる Responses リクエストには、watchdog が無効でも独立した 15 分の全体上限が残る。`/v1/responses/compact` の保留ボディ読み取りもこの予算を共有するが、ローカル上流でも既定は 300 秒。明示値（`0` を含む）が優先される。 | -| `connectTimeoutMs?` | `number` | `200000` |試行ごとの DNS/TCP/TLS/最終ヘッダーの期限。本体が生成される前に終了します。 | +| `connectTimeoutMs?` | `number` | `200000`（public）/ 無効（local） | リクエスト送信から上流が最初の応答ヘッダーを返すまでの試行ごとの期限（DNS/TCP/TLS とヘッダー待ち）。本体生成の前に終了し、本体ストリームを制御する `stallTimeoutSec` とは無関係です。未設定では**ローカル**上流（loopback・プライベート・`.local`/`.lan` 名）は無効が既定、public 上流は 200 秒。正の値は両方に適用、`0`（または任意の非正値）でヘッダー期限を全面無効化します。無効にすることで、ローカルモデルが大きなプロンプトで数分の prompt prefill を行っても、最初のヘッダーのタイマーが健全なリクエストを切断しません。 | | `shutdownTimeoutMs?` | `number` | `5000` |アクティブなターンが中止される前の正常な排出期限。 | | `websockets?` | `boolean` | `false` | クライアント向け Responses WebSocket パスを広告して許可します。false の場合クライアントは HTTP/SSE を使いますが、対象となる canonical ChatGPT upstream WS 最適化は無効にしません。 | | `corsAllowOrigins?` | `string[]` | `[]` | 追加の正確な CORS origin。ループバック origin は常に許可します。`chrome-extension://` など authority ベースのブラウザー拡張 origin に対応し、`*` はワイルドカードではありません。Firefox と Safari は拡張 UUID を（インストール/ブラウザー起動ごとに）再生成するため、origin が変わったらエントリを更新してください。 | @@ -246,3 +246,11 @@ CLI 内部の再試行・ツールターンは要求全体の送信回数上限 完全だが不正な記録（最終行の `null` も含む）は上限付きの受け入れを拒否します。圧縮は有効な残高を保持して続行し、 実行中の拒否はクリーンなジャーナルを再起動で読み直すまで残ります。途中で切れた最終 JSON 行は従来の復旧規則に従います。 詳しい境界とエラーは[英語の正本](/reference/configuration/server/#historical-pool-continuity)を参照してください。 + +## ローカル upstream のタイムアウト分類 + +この分類はタイムアウト方針であり、ネットワークアクセスの許可ではありません。DNS を参照せず HTTP(S) の宛先を分類します。正確な `localhost`（大文字小文字と末尾のドットを許容）、`0.0.0.0`、IPv4 loopback (127/8)、RFC 1918 private、link-local (169.254/16)、IPv6 loopback、link-local (fe80::/10)、unique-local (fc00::/7)、IPv4-mapped アドレス、`.local`/`.lan` 名がローカルです。他の DNS 名は loopback に解決されても公開向けの期限を維持します。不正な宛先、非対応スキーム、`localhost.example` も同様です。 + +ヘッダー期限を無効にしてもクライアントのキャンセルと上位の全体期限は維持されます。明示的な正の `stallTimeoutSec` はローカル本文の無応答を制限します。ローカルの header/stall が未設定なら、ヘッダー到着前の自動期限はありません。ネイティブ Chat/Messages のローカル JSON 読み取りは stall 予算を使用し、公開 JSON 読み取りは従来の `max(connectTimeoutMs ?? 200000, 5000)` を維持します。Compact 本文の既定 300 秒と画像・検索 sidecar の予算は変わりません。 + +任意の `opencodeClientExport` は `headerTimeoutMs`、`chunkTimeoutMs`、`timeoutMs` に正のミリ秒または無効化する `false` を指定できます。省略時は OpenCode の既定値を維持し、不正なフィールドは出力しません。V1 options と V2 settings の provider 全体に適用します。[遅いローカルモデル](/guides/opencode/#slow-local-models-prefill-timeouts)を参照してください。 diff --git a/docs-site/src/content/docs/ko/reference/configuration/server.md b/docs-site/src/content/docs/ko/reference/configuration/server.md index 4a180b9a63f..bedee7af881 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/server.md +++ b/docs-site/src/content/docs/ko/reference/configuration/server.md @@ -15,7 +15,7 @@ description: 리스너, 원격 접근, admission 키, 타임아웃, 저장소, | `emptyCompletionRetry?` | `boolean` | `false` | 텍스트나 도구 호출이 없는 Responses 턴을, 터미널 이벤트 전에 스트림이 종료된 경우를 포함해 동일한 요청으로 한 번 재시도하도록 선택합니다. 재시도에는 비용이 발생할 수 있습니다. `OCX_EMPTY_COMPLETION_RETRY=0`은 설정을 바꾸지 않고 비활성화하며, combo 및 routed-compaction turn은 제외됩니다. | | `dropCodexSafetyBuffering?` | `boolean` | `false` | Canonical Codex Responses 응답의 선택적 safety-buffering 헤더 두 개와 SSE 힌트를 제거합니다. 공급자의 안전 정책이나 거절 응답은 바뀌지 않습니다. Native WS 메타데이터와 compact는 제외됩니다. | | `stallTimeoutSec?` | `number` | `300`(public) / 비활성(local) | 스트림이 끊기기까지 유효한 업스트림 진행이 없는 시간(초, Responses 및 네이티브 Chat). 미설정 시 **로컬** 업스트림(loopback, private, `.local`/`.lan` 이름)은 비활성이 기본이고 공개 업스트림은 300초. 양수 값은 둘 다에 적용(최소 1초), `0`은 무응답 watchdog을 전면 비활성화한다. canonical ChatGPT SSE를 비스트리밍 JSON으로 접는 Responses 요청에는 이 watchdog이 꺼져도 별도의 15분 전체 상한이 남는다. `/v1/responses/compact`의 보류 바디 읽기도 이 예산을 공유하지만 로컬 업스트림에서도 기본은 300초. 명시 값(`0` 포함)이 우선한다. | -| `connectTimeoutMs?` | `number` | `200000` | 시도별 DNS/TCP/TLS/최종 헤더 기한입니다. 본문 생성 전에 끝납니다. | +| `connectTimeoutMs?` | `number` | `200000`(public) / 비활성(local) | 요청 전송부터 업스트림이 첫 응답 헤더를 반환할 때까지의 시도별 기한(DNS/TCP/TLS 및 헤더 대기). 본문 생성 전에 끝나며, 본문 스트림을 제어하는 `stallTimeoutSec`와는 무관합니다. 미설정 시 **로컬** 업스트림(loopback, private, `.local`/`.lan` 이름)은 비활성이 기본이고 공개 업스트림은 200초. 양수 값은 둘 다에 적용, `0`(또는 임의의 비양수 값)은 헤더 기한을 전면 비활성화합니다. 비활성 덕분에 로컬 모델이 큰 프롬프트에서 수 분간 prompt prefill을 해도 첫 헤더 시계가 정상 요청을 끊지 않습니다. | | `shutdownTimeoutMs?` | `number` | `5000` | 진행 중인 turn을 중단하기 전에 허용하는 정상 종료 드레인 기한입니다. | | `websockets?` | `boolean` | `false` | 클라이언트용 Responses WebSocket 경로를 광고하고 허용합니다. `false`이면 클라이언트는 HTTP/SSE를 사용하며, 적격 canonical ChatGPT 업스트림 WS 최적화는 비활성화하지 않습니다. | | `corsAllowOrigins?` | `string[]` | `[]` | CORS에서 추가로 허용할 정확한 origin입니다. 루프백 origin은 항상 허용됩니다. `chrome-extension://` 같은 authority 기반 브라우저 확장 origin을 지원하며, `*`는 와일드카드가 아닙니다. Firefox와 Safari는 확장 UUID를 (설치/브라우저 실행 때마다) 새로 만드므로 origin이 바뀌면 항목을 갱신하세요. | @@ -309,3 +309,11 @@ CLI 내부 재시도·도구 턴은 요청 전체 전송 횟수 한도에 따로 압축은 유효한 잔액을 보존하며 계속할 수 있습니다. 실행 중 거부 상태는 재시작 뒤 깨끗한 저널을 읽을 때까지 남습니다. 쓰다 끊긴 마지막 JSON 줄은 기존 복구 규칙을 따릅니다. 세부 조건과 오류 코드는 [영문 원문](/reference/configuration/server/#historical-pool-continuity)을 참조하세요. + +## 로컬 업스트림 타임아웃 분류 + +로컬 분류는 타임아웃 정책이며 네트워크 접근 권한을 부여하지 않습니다. DNS 조회 없이 HTTP(S) 목적지를 분류합니다. 정확한 `localhost`(대소문자 무관, 끝의 점 허용), `0.0.0.0`, IPv4 loopback(127/8), RFC 1918 private 및 link-local(169.254/16), IPv6 loopback·link-local(fe80::/10)·unique-local(fc00::/7), IPv4-mapped 주소, `.local`/`.lan` 이름이 로컬입니다. 다른 DNS 이름은 loopback으로 해석되더라도 공개 기본값을 유지합니다. 잘못된 목적지, 지원하지 않는 스킴, `localhost.example`도 공개 기본값을 유지합니다. + +헤더 제한을 꺼도 클라이언트 취소와 상위 전체 기한은 유지됩니다. 명시적인 양수 `stallTimeoutSec`는 로컬 바디 무응답을 제한합니다. 로컬 헤더·stall 설정이 모두 없으면 헤더 이전의 자동 제한은 없습니다. 네이티브 Chat/Messages 로컬 JSON 읽기는 stall 예산을 쓰며 공개 JSON 읽기는 기존 `max(connectTimeoutMs ?? 200000, 5000)`을 유지합니다. Compact 바디의 기본 300초와 이미지/검색 사이드카 예산도 유지됩니다. + +선택 설정 `opencodeClientExport`의 `headerTimeoutMs`, `chunkTimeoutMs`, `timeoutMs`는 양수 밀리초 또는 비활성화 값 `false`를 받습니다. 생략 시 OpenCode 기본값을 유지하고 잘못된 필드는 내보내지 않습니다. V1 options와 V2 settings의 provider 전체에 적용됩니다. [느린 로컬 모델](/guides/opencode/#slow-local-models-prefill-timeouts)을 참고하세요. diff --git a/docs-site/src/content/docs/reference/configuration/server.md b/docs-site/src/content/docs/reference/configuration/server.md index 2a394d9d697..67362b177e7 100644 --- a/docs-site/src/content/docs/reference/configuration/server.md +++ b/docs-site/src/content/docs/reference/configuration/server.md @@ -18,7 +18,7 @@ runs helper features around provider requests. | `dropCodexSafetyBuffering?` | `boolean` | `false` | Remove optional client-facing hints from canonical Codex Responses passthrough: the two `x-codex-safety-buffering-enabled` / `x-codex-safety-buffering-faster-model` response headers, `response.metadata` events whose metadata type is `safety_buffering`, and top-level `safety_buffering` fields. Other headers, response data, policy refusals and failures are preserved. This does not disable provider safety enforcement or upstream buffering. Native `codex.response.metadata.headers` WebSocket metadata and `/responses/compact` are outside this filter. | | `stallTimeoutSec?` | `number` | `300` (public) / disabled (local) | Seconds without meaningful upstream progress (Responses and native Chat) before the stream is cut. Unset, a **local** upstream (loopback, private, or a `.local`/`.lan` name) defaults to disabled and a public upstream to 300 s; a positive value applies to both (minimum 1 s); `0` disables the silence watchdog everywhere. Disabled leaves a silent-but-healthy local model connected (keep-alives still flow). Canonical ChatGPT Responses folded from SSE into non-streaming JSON retain a separate 15-minute whole-turn ceiling even when the silence watchdog is disabled. Pending `/v1/responses/compact` body reads share this budget but default to 300 s even for a local upstream — the route buffers the complete body while holding an active-turn lease — and an explicit value, including `0`, still wins. | | `oauthOpenBrowser?` | `boolean` | `true` | Whether a login may open a browser on the machine running the proxy. Absent and `true` both open, so an existing install is unchanged; only an explicit `false` declines. Decline when you need the authorization link in a different browser profile, or when the dashboard is not on the proxy's machine — the login still starts and the URL is still returned and displayed. `POST /api/oauth/login` and `POST /api/codex-auth/login` accept a per-request `openBrowser` boolean that overrides this, and the dashboard exposes the same choice beside the login button. Both routes report the outcome as `browserLaunch`: `started`, `failed` (nothing could be opened; the dashboard says so), or `skipped`. Device-code flows never open a browser either way. | -| `connectTimeoutMs?` | `number` | `200000` | Per-attempt DNS/TCP/TLS/final-header deadline; it ends before body generation. | +| `connectTimeoutMs?` | `number` | `200000` (public) / disabled (local) | Per-attempt deadline from dispatch until the upstream returns its first response header (DNS/TCP/TLS and the header wait); it ends before body generation and is unrelated to `stallTimeoutSec`, which governs the body stream. Unset, a **local** upstream (loopback, private, or a `.local`/`.lan` name) defaults to disabled and a public upstream to 200 s; a positive value applies to both; `0` (or any non-positive value) disables the header deadline everywhere. Disabled is what lets a local model spend minutes in prompt prefill on a large prompt without the first-header clock cutting a healthy turn. | | `shutdownTimeoutMs?` | `number` | `5000` | Graceful drain deadline before active turns are aborted. | | `websockets?` | `boolean` | `false` | Advertise and admit the client-facing Responses WebSocket path. False keeps clients on HTTP/SSE; it does not disable an eligible canonical ChatGPT upstream WS optimization. Complete-input requests may reuse an upstream connection within the same selected credential, account, thread and turn; changed handshake policy or missing identity keeps requests on separate connections. This does not trim HTTP input or create previous-response IDs. | | `codexNativeSteering?` | `boolean` | `false` | Experimental, native-only mid-turn steering on the Responses WebSocket endpoint. Requires `websockets: true`, a compatible upstream/client, and a pinned account/model/tool surface. Validated generation settings can change in explicit saved-result continuations. Does not enable translated models or HTTP fallback. See [native steering](/guides/codex-integration/#experimental-native-mid-turn-steering). | @@ -85,8 +85,9 @@ supports protocol pings, the proxy pings it every 15 seconds. Any inbound frame response metadata, or a pong — resets a 90-second silence clock, so a socket without ping support still stays alive on its own frames, and only 90 seconds with nothing at all settles the request as an HTTP 504 with an `upstream_no_response` error. A slow but alive origin therefore waits for the -client's own deadline or for `connectTimeoutMs` (default 200s), whichever comes first; a -connect timeout that fires after the create frame was sent settles as the same 504. A socket +client's own deadline or for `connectTimeoutMs` (default 200s for a public upstream, disabled for +a local one unless set), whichever comes first; a connect timeout that fires after the create frame +was sent settles as the same 504. A socket that closes or errors before the first Responses event settles as an HTTP 502 with `upstream_closed_before_response`. The proxy does not retry either status on its own: the frame may already be executing upstream, so the client applies its own retry policy exactly as it would @@ -1177,3 +1178,11 @@ Compaction can still preserve valid accounting in a clean checkpoint; the live r a clean restart replays it. A torn, unparseable final line keeps the existing recovery behavior. Storage/corruption denials use `workflow_spend_undurable`; unsafe files and ownership failures remain storage errors. Alias validation failures use `workflow_pool_history_unresolved`. + +## Local upstream timeout classification + +Locality is a timeout policy, not network-access authorization. Classification uses the configured HTTP(S) destination without DNS lookups: exact `localhost` (case-insensitive, optional trailing dot), `0.0.0.0`, IPv4 loopback (127/8), RFC 1918 private and link-local (169.254/16) addresses, IPv6 loopback, link-local (fe80::/10), unique-local (fc00::/7), IPv4-mapped equivalents, and names ending in `.local` or `.lan`. Other DNS names keep public deadlines even if they resolve to loopback; malformed destinations and unsupported schemes also keep public deadlines. Names such as `localhost.example` are public. + +Disabling the header clock preserves client cancellation and any enclosing overall deadline. An explicitly positive `stallTimeoutSec` still bounds local body silence; unset local stall/header clocks provide no automatic bound before headers. Native Chat/Messages local JSON reads use the local stall budget; public JSON reads retain their previous `max(connectTimeoutMs ?? 200000, 5000)` budget. Compact body reads retain their 300-second default even locally. Image/search sidecar budgets are unchanged. + +Optional `opencodeClientExport` controls the generated OpenCode provider's `headerTimeoutMs`, `chunkTimeoutMs`, and `timeoutMs` (positive milliseconds or `false` to disable). Unset values preserve OpenCode defaults; malformed fields are omitted. These provider-wide settings are emitted into V1 options and V2 settings. See [slow local models](/guides/opencode/#slow-local-models-prefill-timeouts). diff --git a/docs-site/src/content/docs/ru/reference/configuration/server.md b/docs-site/src/content/docs/ru/reference/configuration/server.md index 27f00766361..a492fceeac0 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/server.md +++ b/docs-site/src/content/docs/ru/reference/configuration/server.md @@ -16,7 +16,7 @@ description: Listener, удалённый доступ, admission key, тайм | `emptyCompletionRetry?` | `boolean` | `false` | Явно включает один идентичный повтор Responses, если в turn нет ни текста, ни tool call, включая случай, когда stream завершается до terminal event. Повтор может тарифицироваться. `OCX_EMPTY_COMPLETION_RETRY=0` отключает его без изменения config; combo и routed-compaction turn исключены. | | `dropCodexSafetyBuffering?` | `boolean` | `false` | Удаляет подсказки Codex safety-buffering из passthrough-ответов Codex Responses: заголовки `x-codex-safety-buffering-enabled` / `x-codex-safety-buffering-faster-model`, SSE-события `response.metadata` типа `safety_buffering` и поле `safety_buffering` в других SSE-событиях. Codex TUI отображает их как предложение повторить запрос с более быстрой моделью, действие по умолчанию в котором переключает сессию на более слабую модель. Остальные заголовки `x-codex-*` и содержимое других SSE-событий передаются без изменений, кроме удаления этого поля. По умолчанию выключено. | | `stallTimeoutSec?` | `number` | `300` (public) / выкл. (local) | Секунды без полезного прогресса upstream (Responses и нативный Chat) до обрыва потока. Без настройки **локальный** upstream (loopback, private, имя `.local`/`.lan`) по умолчанию выключен, публичный — 300 с; положительное значение действует на оба (минимум 1 с); `0` отключает watchdog тишины везде. Для Responses, которые сворачивают canonical ChatGPT SSE в непотоковый JSON, даже при выключенном watchdog остаётся отдельный общий предел 15 минут. Ожидающие чтения тела `/v1/responses/compact` используют этот же бюджет, но по умолчанию 300 с даже для локального upstream; явное значение, включая `0`, имеет приоритет. | -| `connectTimeoutMs?` | `number` | `200000` | Дедлайн одной попытки DNS/TCP/TLS/final-header; он завершается до генерации тела ответа. | +| `connectTimeoutMs?` | `number` | `200000` (public) / выкл. (local) | Дедлайн одной попытки от отправки запроса до возврата upstream первого заголовка ответа (DNS/TCP/TLS и ожидание заголовка); он завершается до генерации тела и не связан с `stallTimeoutSec`, который управляет потоком тела. Без настройки **локальный** upstream (loopback, private, имя `.local`/`.lan`) по умолчанию выключен, публичный — 200 с; положительное значение действует на оба; `0` (или любое неположительное значение) отключает дедлайн заголовка везде. Отключение позволяет локальной модели тратить минуты на prompt prefill при большом промпте, не обрывая исправный запрос по таймеру первого заголовка. | | `shutdownTimeoutMs?` | `number` | `5000` | Дедлайн graceful-drain до принудительного прерывания активных turn'ов. | | `websockets?` | `boolean` | `false` | Объявляет и разрешает клиентский WebSocket-путь Responses. При false клиенты используют HTTP/SSE; это не отключает подходящую upstream WS-оптимизацию canonical ChatGPT. | | `corsAllowOrigins?` | `string[]` | `[]` | Дополнительные точные origin, разрешённые CORS. Loopback-origin разрешены всегда. Поддерживаются authority-based origin браузерных расширений, например `chrome-extension://`; `*` не является маской. Firefox и Safari пересоздают UUID расширения (при каждой установке/запуске браузера), поэтому обновляйте запись при смене origin. | @@ -305,3 +305,11 @@ The dashboard warns about old or unknown CLI versions, unavailable targets, and Уплотнение продолжает сохранять корректные балансы; отказ живого процесса сохраняется до перезапуска с чистым журналом. Оборванная последняя строка JSON обрабатывается по прежним правилам восстановления. Подробности и коды ошибок: [английский оригинал](/reference/configuration/server/#historical-pool-continuity). + +## Классификация локальных upstream для таймаутов + +Классификация определяет таймауты, а не разрешения сетевого доступа. HTTP(S)-адрес проверяется без DNS: локальными считаются точный `localhost` (без учёта регистра, с допустимой конечной точкой), `0.0.0.0`, IPv4 loopback (127/8), private RFC 1918 и link-local (169.254/16), IPv6 loopback, link-local (fe80::/10), unique-local (fc00::/7), IPv4-mapped аналоги и имена `.local`/`.lan`. Другие DNS-имена сохраняют публичные таймауты, даже если разрешаются в loopback. Неверные адреса, неподдерживаемые схемы и `localhost.example` также считаются публичными. + +Отключение срока заголовков сохраняет отмену клиентом и внешний общий срок. Явный положительный `stallTimeoutSec` ограничивает тишину локального тела. Без локальных header/stall настроек автоматического срока до заголовков нет. Локальный JSON native Chat/Messages использует stall-бюджет; публичный JSON сохраняет `max(connectTimeoutMs ?? 200000, 5000)`. Compact сохраняет 300 секунд для тела даже локально. Бюджеты image/search sidecar не меняются. + +Необязательный `opencodeClientExport` принимает `headerTimeoutMs`, `chunkTimeoutMs`, `timeoutMs`: положительные миллисекунды или `false` для отключения. Отсутствующие значения сохраняют настройки OpenCode, неверные поля пропускаются. Настройки действуют на весь provider в V1 options и V2 settings. См. [медленные локальные модели](/guides/opencode/#slow-local-models-prefill-timeouts). diff --git a/docs-site/src/content/docs/tr/reference/configuration/server.md b/docs-site/src/content/docs/tr/reference/configuration/server.md index 70f08045e9c..4a20ef617ad 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/server.md +++ b/docs-site/src/content/docs/tr/reference/configuration/server.md @@ -16,7 +16,7 @@ yardımcı özellikleri nasıl çalıştıracağını kontrol eder. | `proxy?` | `string` | — | Giden HTTP(S) veya SOCKS5 proxy URL'si (`socks5://host:port`) ya da `${ENV_VAR}`. HTTP URL'leri değişkenler boşsa `HTTP_PROXY` / `HTTPS_PROXY`'ye yazılır. SOCKS5 URL'leri yerleşik gerçek SOCKS5 tünelini kullanır ve `ALL_PROXY`'ye de yazılır (`ocx start --socks5`); bu süreçte miras `HTTP(S)_PROXY` temizlenir. Geri döngü `NO_PROXY` içinde kalır. | | `emptyCompletionRetry?` | `boolean` | `false` | Metin veya araç çağrısı içermeyen bir Responses tamamlamasını aynı istekle bir kez yeniden denemeyi açıkça etkinleştirir. Yeniden deneme ücretlendirilebilir. `OCX_EMPTY_COMPLETION_RETRY=0`, yapılandırmayı değiştirmeden devre dışı bırakır; combo ve routed-compaction turları hariçtir. | | `stallTimeoutSec?` | `number` | `300` (public) / kapalı (local) | Akış kesilmeden önce anlamlı üst sunucu ilerlemesi olmadan geçen saniye (Responses ve yerel Chat). Ayarlanmamışsa **yerel** üst sunucu (loopback, private, `.local`/`.lan` adı) için varsayılan kapalı, genel üst sunucu için 300 sn; pozitif değer ikisine de uygulanır (en az 1 sn); `0` sessizlik watchdog'unu her yerde kapatır. Canonical ChatGPT SSE'yi akışsız JSON'a katlayan Responses isteklerinde watchdog kapalı olsa bile bağımsız 15 dakikalık toplam tur sınırı kalır. `/v1/responses/compact` için bekleyen gövde okumaları bu bütçeyi paylaşır ama yerel üst sunucuda bile varsayılan 300 sn'dir; açık değer (`0` dahil) önceliklidir. | -| `connectTimeoutMs?` | `number` | `200000` | Deneme başına DNS/TCP/TLS/nihai başlık son tarihi; gövde üretiminden önce biter. | +| `connectTimeoutMs?` | `number` | `200000` (public) / kapalı (local) | İsteğin gönderilmesinden üst sunucunun ilk yanıt başlığını döndürmesine kadar deneme başına son tarih (DNS/TCP/TLS ve başlık bekleme); gövde üretiminden önce biter ve gövde akışını yöneten `stallTimeoutSec` ile ilgisizdir. Ayarlanmamışsa **yerel** üst sunucu (loopback, private, `.local`/`.lan` adı) için varsayılan kapalı, genel üst sunucu için 200 sn; pozitif değer ikisine de uygulanır; `0` (veya herhangi bir pozitif olmayan değer) başlık son tarihini her yerde kapatır. Kapalı olması, yerel bir modelin büyük istemde dakikalarca prompt prefill yapmasına rağmen ilk başlık saatinin sağlıklı bir isteği kesmemesini sağlar. | | `shutdownTimeoutMs?` | `number` | `5000` | Aktif turlar iptal edilmeden önce zarif boşaltma süresi sınırı. | | `websockets?` | `boolean` | `false` | Responses WebSocket yolu için `supports_websockets` bildirin. False, HTTP/SSE'yi tutar. | | `corsAllowOrigins?` | `string[]` | `[]` | CORS tarafından izin verilen ek tam kaynaklar. Geri döngü kaynaklarına her zaman izin verilir. `chrome-extension://` gibi yetki tabanlı tarayıcı uzantısı kaynakları desteklenir; `*` bir joker karakter değildir. Firefox ve Safari uzantı UUID'sini yeniden oluşturur (yükleme başına / tarayıcı başlatma başına), bu nedenle kaynak değiştiğinde girdiyi güncelleyin. | @@ -353,3 +353,11 @@ Sürüm düşürme, kanonik toplamların korunacağını veya aynı trafik başt Yayımlanmamış deneysel sürümlerin `pool-current` veya `poolContinuity` içeren günlükleri, daha sonra sıkıştırılsalar bile C sözleşmesinin dışındadır. Özgün bakiyeleri normal saklama süresi dolana kadar belirsiz geçmiş olarak tutulur; hemen dönüştürülmez veya silinmez. Tam ama geçersiz bir kayıt, son satırdaki `null` dahil, sınır uygulanan isteklerin kabulünü engeller. Sıkıştırma geçerli muhasebeyi temiz bir kontrol noktasında koruyabilir; çalışan süreç temiz günlükle yeniden başlatılana kadar ret durumu sürer. Yarım kalmış son JSON satırı mevcut kurtarma kurallarına tabidir. Ayrıntılar ve hata kodları için [İngilizce kaynağa](/reference/configuration/server/#historical-pool-continuity) bakın. + +## Yerel upstream zaman aşımı sınıflandırması + +Bu sınıflandırma zaman aşımı politikasıdır; ağ erişim izni vermez. HTTP(S) hedefi DNS sorgusu olmadan değerlendirilir: tam `localhost` (büyük/küçük harf duyarsız, son nokta kabul edilir), `0.0.0.0`, IPv4 loopback (127/8), RFC 1918 özel ve link-local (169.254/16), IPv6 loopback, link-local (fe80::/10), unique-local (fc00::/7), IPv4-mapped eşdeğerleri ve `.local`/`.lan` adları yereldir. Diğer DNS adları loopback adresine çözülse bile genel hedef sürelerini korur. Hatalı hedefler, desteklenmeyen şemalar ve `localhost.example` de geneldir. + +Başlık süresini kapatmak istemci iptalini ve dış toplam süreyi korur. Açıkça pozitif `stallTimeoutSec`, yerel gövde sessizliğini sınırlar. Yerel header/stall ayarları yoksa başlıklardan önce otomatik süre sınırı yoktur. Yerel native Chat/Messages JSON okumaları stall bütçesini kullanır; genel JSON okumaları eski `max(connectTimeoutMs ?? 200000, 5000)` bütçesini korur. Compact gövdesinin yerelde de 300 saniyelik varsayılanı ve image/search sidecar bütçeleri değişmez. + +İsteğe bağlı `opencodeClientExport` içindeki `headerTimeoutMs`, `chunkTimeoutMs`, `timeoutMs` pozitif milisaniye veya kapatmak için `false` alır. Eksik değerler OpenCode varsayılanlarını korur; geçersiz alanlar atlanır. V1 options ve V2 settings içinde provider genelinde uygulanır. [Yavaş yerel modeller](/guides/opencode/#slow-local-models-prefill-timeouts) bölümüne bakın. diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/server.md b/docs-site/src/content/docs/zh-cn/reference/configuration/server.md index dabdc65c6fd..edc4a3a55ac 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/server.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/server.md @@ -16,7 +16,7 @@ description: 监听、远程访问、准入密钥、超时、存储、侧车、 | `emptyCompletionRetry?` | `boolean` | `false` | 显式启用：当 Responses turn 既无文本也无工具调用时，使用相同请求重试一次，包括流在终止事件之前结束的情况。重试可能产生费用。`OCX_EMPTY_COMPLETION_RETRY=0` 可在不修改配置的情况下禁用；combo 与 routed-compaction turn 不参与。 | | `dropCodexSafetyBuffering?` | `boolean` | `false` | 从 Codex Responses 透传响应中移除 Codex safety-buffering 提示：`x-codex-safety-buffering-enabled` / `x-codex-safety-buffering-faster-model` 响应头、类型为 `safety_buffering` 的 `response.metadata` SSE 事件，以及其他 SSE 事件中的 `safety_buffering` 字段。Codex TUI 会将这些提示显示为“使用更快模型重试”的提示框，其默认操作会把会话切换到较弱的模型。其他 `x-codex-*` 响应头和其他所有 SSE 事件内容均保持不变，但会移除该字段。默认关闭。 | | `stallTimeoutSec?` | `number` | `300`（public）/ 禁用（local） | 上游无有效进展（Responses 和原生 Chat）多少秒后切断流。未设置时**本地**上游（loopback、private、`.local`/`.lan` 名称）默认禁用，公网上游默认 300 秒；正值对两者生效（最小 1 秒）；`0` 全面禁用静默 watchdog。对于把 canonical ChatGPT SSE 折叠为非流式 JSON 的 Responses 请求，即使 watchdog 已禁用，仍保留独立的 15 分钟整轮上限。`/v1/responses/compact` 的挂起响应体读取共享此预算，但即使本地上游也默认 300 秒；显式值（含 `0`）优先。 | -| `connectTimeoutMs?` | `number` | `200000` | 每次尝试的 DNS/TCP/TLS/最终响应头截止时间；它在正文生成之前结束。 | +| `connectTimeoutMs?` | `number` | `200000`（public）/ 禁用（local） | 从发出请求到上游返回首个响应头的每次尝试截止时间（DNS/TCP/TLS 及等待响应头）；它在正文生成之前结束，与管理正文流的 `stallTimeoutSec` 无关。未设置时**本地**上游（loopback、private、`.local`/`.lan` 名称）默认禁用，公网上游默认 200 秒；正值对两者生效；`0`（或任意非正值）全面禁用该响应头截止时间。禁用后，本地模型在大提示上可花费数分钟进行 prompt prefill，而首个响应头时钟不会切断健康的请求。 | | `shutdownTimeoutMs?` | `number` | `5000` | 优雅停机截止时间，超过后会中止仍在进行中的请求。 | | `websockets?` | `boolean` | `false` | 声明并允许面向客户端的 Responses WebSocket 路径。设为 false 时客户端使用 HTTP/SSE；它不会禁用符合条件的 canonical ChatGPT 上游 WS 优化。 | | `corsAllowOrigins?` | `string[]` | `[]` | CORS 额外允许的精确 origin。loopback origin 始终允许；支持 `chrome-extension://<扩展 ID>` 等基于 authority 的浏览器扩展 origin，`*` 不是通配符。Firefox 和 Safari 会（每次安装/启动浏览器时）重新生成扩展 UUID，origin 变化后请更新该条目。 | @@ -258,3 +258,11 @@ CLI 内部重试和工具轮次不单独计入请求级发送次数上限，但 完整但无效的记录（包括末行 `null`）会拒绝有额度限制的请求。压缩仍可保留有效余额， 当前进程的拒绝状态持续到重启并读取干净日志。写入中断的最后一行 JSON 沿用原有恢复规则。 详细边界及错误码见[英文原文](/reference/configuration/server/#historical-pool-continuity)。 + +## 本地上游超时分类 + +该分类只决定超时策略，不授予网络访问权限。HTTP(S) 目标按名称或字面地址分类，不查询 DNS：精确的 `localhost`（不区分大小写，允许末尾句点）、`0.0.0.0`、IPv4 loopback（127/8）、RFC 1918 私有地址及 link-local（169.254/16）、IPv6 loopback、link-local（fe80::/10）、unique-local（fc00::/7）、IPv4-mapped 等价地址，以及 `.local`/`.lan` 名称属于本地。其他 DNS 名称即使解析到 loopback，也保留公开上游超时。无效目标、不支持的协议和 `localhost.example` 同样保留公开默认值。 + +关闭响应头时限仍保留客户端取消和外层整体时限。显式正值 `stallTimeoutSec` 仍限制本地响应体静默；本地 header/stall 都未设置时，响应头到达前没有自动时限。原生 Chat/Messages 本地 JSON 读取使用 stall 预算，公开 JSON 读取保留原有 `max(connectTimeoutMs ?? 200000, 5000)`。Compact 响应体即使本地也默认 300 秒。图像/搜索 sidecar 预算不变。 + +可选的 `opencodeClientExport` 包含 `headerTimeoutMs`、`chunkTimeoutMs`、`timeoutMs`，取正毫秒数或表示关闭的 `false`。省略时保留 OpenCode 默认值，无效字段不导出。设置在 V1 options 和 V2 settings 中对整个 provider 生效。参见[慢速本地模型](/guides/opencode/#slow-local-models-prefill-timeouts)。 diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/server.md b/docs-site/src/content/docs/zh-tw/reference/configuration/server.md index df3f820c695..65dabac2f1e 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/server.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/server.md @@ -14,7 +14,7 @@ description: 監聽器、遠端存取、許可金鑰、逾時、儲存、sidecar | `proxy?` | `string` | — | 對外 HTTP(S) 或 SOCKS5 代理 URL（`socks5://host:port`）或 `${ENV_VAR}`。HTTP URL 僅在那些變數未設定時套用至 `HTTP_PROXY` / `HTTPS_PROXY`。SOCKS5 URL 使用內建的真實 SOCKS5 通道，也會套用至 `ALL_PROXY`（`ocx start --socks5`），並清除此行程繼承的 `HTTP(S)_PROXY`。回送保留在 `NO_PROXY` 中。 | | `emptyCompletionRetry?` | `boolean` | `false` | 明確啟用：當 Responses 完成時沒有文字或工具呼叫，以相同請求重試一次。重試可能產生費用。`OCX_EMPTY_COMPLETION_RETRY=0` 可在不變更設定的情況下停用；combo 與 routed-compaction turn 不適用。 | | `stallTimeoutSec?` | `number` | `300`（public）/ 停用（local） | 上游無有效進展（Responses 與原生 Chat）多少秒後切斷串流。未設定時**本地**上游（loopback、private、`.local`/`.lan` 名稱）預設停用，公網上游預設 300 秒；正值對兩者生效（最小 1 秒）；`0` 全面停用靜默 watchdog。對於把 canonical ChatGPT SSE 折疊為非串流 JSON 的 Responses 請求，即使 watchdog 已停用，仍保留獨立的 15 分鐘整體上限。`/v1/responses/compact` 的擱置回應本文讀取共用此預算，但即使本地上游也預設 300 秒；明確值（含 `0`）優先。 | -| `connectTimeoutMs?` | `number` | `200000` | 每次嘗試的 DNS/TCP/TLS/final-header 截止時間；它在 body 生成前結束。 | +| `connectTimeoutMs?` | `number` | `200000`（public）/ 停用（local） | 從送出請求到上游傳回首個回應 header 的每次嘗試截止時間（DNS/TCP/TLS 及等待 header）；它在 body 生成前結束，與管理 body 串流的 `stallTimeoutSec` 無關。未設定時**本地**上游（loopback、private、`.local`/`.lan` 名稱）預設停用，公網上游預設 200 秒；正值對兩者生效；`0`（或任何非正值）全面停用該 header 截止時間。停用後，本地模型在大提示上可花費數分鐘進行 prompt prefill，而首個 header 時鐘不會切斷健康的請求。 | | `shutdownTimeoutMs?` | `number` | `5000` | 在中止活躍回合前的優雅排空截止時間。 | | `websockets?` | `boolean` | `false` | 廣告並允許面向 client 的 Responses WebSocket 路徑。False 時 client 使用 HTTP/SSE；不會停用符合條件的 canonical ChatGPT upstream WS 最佳化。 | | `corsAllowOrigins?` | `string[]` | `[]` | 額外的精確 CORS 來源。回送來源恆被允許。 | @@ -263,3 +263,11 @@ Claude CLI、CodeBuddy 及 Qoder 將每次 CLI 呼叫計為一次傳送，啟動 未發布實驗版本的日誌若含 `pool-current` 或 `poolContinuity`，即使後來經過壓縮，仍不屬於契約 C。其原始餘額會作為未知歷史保留至正常到期，不會立即轉換或刪除。 完整但無效的紀錄，包括最後一行的 `null`，會使有額度限制的請求遭拒。壓縮仍可將有效帳務保留在乾淨的檢查點；目前程序的拒絕狀態會持續到重新啟動並讀取乾淨日誌。寫入中斷的最後一行 JSON 沿用原有復原規則。詳細範圍及錯誤碼請見[英文原文](/reference/configuration/server/#historical-pool-continuity)。 + +## 本機上游逾時分類 + +此分類只決定逾時政策，不授予網路存取權限。HTTP(S) 目的地依名稱或位址字面值分類，不查詢 DNS：精確的 `localhost`（不分大小寫，允許結尾句點）、`0.0.0.0`、IPv4 loopback（127/8）、RFC 1918 私有位址及 link-local（169.254/16）、IPv6 loopback、link-local（fe80::/10）、unique-local（fc00::/7）、IPv4-mapped 等價位址，以及 `.local`/`.lan` 名稱屬於本機。其他 DNS 名稱即使解析到 loopback，也保留公開上游逾時。無效目的地、不支援的協定及 `localhost.example` 亦保留公開預設值。 + +關閉回應標頭期限仍保留用戶端取消及外層整體期限。明確的正值 `stallTimeoutSec` 仍限制本機回應本文靜默；本機 header/stall 都未設定時，標頭抵達前沒有自動期限。原生 Chat/Messages 本機 JSON 讀取使用 stall 預算，公開 JSON 讀取保留原有 `max(connectTimeoutMs ?? 200000, 5000)`。Compact 本文即使本機也預設 300 秒。影像/搜尋 sidecar 預算不變。 + +選用的 `opencodeClientExport` 包含 `headerTimeoutMs`、`chunkTimeoutMs`、`timeoutMs`，接受正毫秒數或表示關閉的 `false`。省略時保留 OpenCode 預設值，無效欄位不匯出。設定在 V1 options 與 V2 settings 中對整個 provider 生效。請參閱[慢速本機模型](/guides/opencode/#slow-local-models-prefill-timeouts)。 diff --git a/scripts/test-layout/layout-chat.json b/scripts/test-layout/layout-chat.json index 88e19056c9f..336d6c74f75 100644 --- a/scripts/test-layout/layout-chat.json +++ b/scripts/test-layout/layout-chat.json @@ -1,4 +1,6 @@ { + "local-upstream-deadlines.test.ts": "responses", + "refreshed-upstream-deadlines.test.ts": "responses", "chat-raw-reasoning-delivery.test.ts": "responses", "chat-incomplete-length-reason.test.ts": "responses", "openai-chat-required-tool.test.ts": "adapters/openai", diff --git a/src/adapters/base.ts b/src/adapters/base.ts index 31f0ed057d8..23c4d31529f 100644 --- a/src/adapters/base.ts +++ b/src/adapters/base.ts @@ -191,7 +191,7 @@ export interface AdapterFetchContext { kiroPreferAccountFailover?: boolean; /** Remains attached to the returned response body after the response headers arrive. */ abortSignal?: AbortSignal; - /** Deadline for receiving response headers on each attempt, not for consuming the response body. */ + /** Response-header deadline per attempt; non-positive disables it. Body cancellation stays linked. */ timeoutMs?: number; /** Return final non-2xx responses untouched so the caller can own the error-body read. */ returnRawErrors?: boolean; diff --git a/src/adapters/command-code.ts b/src/adapters/command-code.ts index 97b9f17a58f..66293ec5ef1 100644 --- a/src/adapters/command-code.ts +++ b/src/adapters/command-code.ts @@ -7,6 +7,7 @@ import { isAllowedToolChoice, namespacedToolName, resolveToolChoiceWireName, too import type { AdapterFetchContext, AdapterRequest, ProviderAdapter } from "./base"; import type { TranslatorBudget } from "../lib/translator-budget"; import { readBoundedResponseBody } from "../lib/bounded-body"; +import { clearableDeadline } from "../lib/abort"; import { debugDroppedFrame } from "../lib/debug"; import { configuredReasoningEfforts, modelRecordValue } from "../reasoning-effort"; import { commandCodeReasoningEfforts, refreshCommandCodeReasoningEfforts } from "../providers/command-code-efforts"; @@ -609,19 +610,17 @@ function requestModelId(request: AdapterRequest): string | undefined { } async function fetchCommandCode(request: AdapterRequest, ctx: AdapterFetchContext | undefined, executor: typeof globalThis.fetch): Promise { - const timeout = new AbortController(); - const timer = setTimeout(() => timeout.abort(new DOMException("Timeout elapsed", "TimeoutError")), ctx?.timeoutMs ?? 200_000); - const callerSignal = ctx?.abortSignal ?? new AbortController().signal; + const deadline = clearableDeadline(ctx?.timeoutMs ?? 200_000, ctx?.abortSignal); try { return await executor(request.url, { method: request.method, headers: request.headers, body: request.body, redirect: "manual", - signal: AbortSignal.any([callerSignal, timeout.signal]), + signal: deadline.signal, }); } finally { - clearTimeout(timer); + deadline.clear(); } } diff --git a/src/adapters/kiro/adapter.ts b/src/adapters/kiro/adapter.ts index 4390f46ba69..11285f97573 100644 --- a/src/adapters/kiro/adapter.ts +++ b/src/adapters/kiro/adapter.ts @@ -64,6 +64,7 @@ export function createKiroAdapter(provider: OcxProviderConfig): ProviderAdapter let firstRequestBodyBytes = 0; let requestAbortSignal: AbortSignal | undefined; let requestExecutor: typeof globalThis.fetch | undefined; + let requestTimeoutMs: number | undefined; // Captured the same way as the abort signal, because the text-fallback rebuild below runs // outside the fetchResponse frame and used to construct a context without either (#4546). let requestSendBudget: RequestExecutionBudget | undefined; @@ -237,6 +238,7 @@ export function createKiroAdapter(provider: OcxProviderConfig): ProviderAdapter const fallbackOrdinalBase = physicalSendsObserved; const response = await fetchKiroWithRetry(retry.request, { abortSignal: requestAbortSignal, + timeoutMs: requestTimeoutMs, ...(requestExecutor ? { executor: requestExecutor } : {}), allowGatewayRotation: !_sawReasoning && !assistantText.trim(), returnRawErrors: true, @@ -322,6 +324,7 @@ export function createKiroAdapter(provider: OcxProviderConfig): ProviderAdapter // both the first Kiro request and its one allowed completion retry. if (ctx?.abortSignal) requestAbortSignal = ctx.abortSignal; requestExecutor = ctx?.executor; + requestTimeoutMs = ctx?.timeoutMs; if (ctx?.sendBudget) requestSendBudget = ctx.sendBudget; if (ctx?.onPhysicalSend) requestOnPhysicalSend = ctx.onPhysicalSend; // Reset per fetch call, because `ordinal` is defined within one call and the caller records diff --git a/src/bridge/sse.ts b/src/bridge/sse.ts index c1fdeb561e8..fbbeefcd002 100644 --- a/src/bridge/sse.ts +++ b/src/bridge/sse.ts @@ -369,10 +369,7 @@ export function bridgeToResponsesSSE( ? encoder.encode(': opencodex heartbeat\n\n') : encoder.encode('event: response.heartbeat\ndata: {"type":"response.heartbeat"}\n\n'); let stallTicks = 0; - const stallSec = resolveStallTimeoutSec(options?.stallTimeoutSec, { - localUpstream: options?.localUpstream, - }); - const maxStallTicks = Math.ceil((stallSec * 1000) / heartbeatMs); + let localUpstream = options?.localUpstream; let currentMsg: { itemId: string; @@ -1494,6 +1491,9 @@ export function bridgeToResponsesSSE( gated = true; beat = setBeatInterval(() => { if (closed || gated) return; + if (localUpstream !== options?.localUpstream) { localUpstream = options?.localUpstream; stallTicks = 0; } + const stallSec = resolveStallTimeoutSec(options?.stallTimeoutSec, { localUpstream }); + const maxStallTicks = Math.ceil((stallSec * 1000) / heartbeatMs); if (upstreamActivity) { upstreamActivity = false; stallTicks = 0; diff --git a/src/clients/config-export.ts b/src/clients/config-export.ts index f477968ccc2..daf3298027d 100644 --- a/src/clients/config-export.ts +++ b/src/clients/config-export.ts @@ -135,6 +135,18 @@ export interface OpencodeProviderConnection { baseURL: string; apiKey?: string; headers?: Record; + /** + * opencode's own client-side wait for the proxy's first response header, in ms, or `false` + * to disable it. Emitted only when `opencodeClientExport.headerTimeoutMs` is set. opencode's + * default is 300000 (5 min); a slow local model's prompt prefill behind a non-streaming turn + * can exceed it, so the proxy sends no bytes and opencode's header wait fires ("Timed out + * waiting for response headers"). + */ + headerTimeout?: number | false; + /** opencode's wait between streamed response chunks (ms) or `false`; emitted from `chunkTimeoutMs`. */ + chunkTimeout?: number | false; + /** opencode's whole-request bound (ms) or `false`; emitted from `timeoutMs`. opencode has no default. */ + timeout?: number | false; } /** opencode V1 provider block: `npm` + `options`. */ @@ -712,6 +724,17 @@ export function raycastConfigPath(env: OpencodeLaunchEnv = process.env, home: st /** Endpoint plus admission, identical for the V1 `options` and V2 `settings` field. */ function opencodeProviderConnection(baseURL: string, config: OcxConfig): OpencodeProviderConnection { const options: OpencodeProviderConnection = { baseURL }; + // Opt-in opencode-side timeout overrides. Omitted by default so opencode keeps its native + // header/chunk waits for every upstream routed through the proxy (current V2 defaults the + // header wait to 5 min and the longer chunk wait to 15 min, but this repo pins no opencode + // release); a user fronting a slow local model sets these to extend or disable the wait that + // otherwise cuts a long prefill. + const timeouts = config.opencodeClientExport; + if (timeouts) { + if (timeouts.headerTimeoutMs !== undefined) options.headerTimeout = timeouts.headerTimeoutMs; + if (timeouts.chunkTimeoutMs !== undefined) options.chunkTimeout = timeouts.chunkTimeoutMs; + if (timeouts.timeoutMs !== undefined) options.timeout = timeouts.timeoutMs; + } // Non-loopback binds accept proxy admission only via x-opencodex-api-key so Authorization // stays free for Codex Direct upstream credentials when applicable. if (shouldInjectApiAuthHeader(config)) { diff --git a/src/config/schema/config-schema.ts b/src/config/schema/config-schema.ts index 72ec9c558f4..0565f9efc35 100644 --- a/src/config/schema/config-schema.ts +++ b/src/config/schema/config-schema.ts @@ -32,6 +32,7 @@ import { apiKeyEntrySchema, asideProfileSyncSchema, clientIntegrationsSchema, + opencodeClientExportSchema, CODEX_ACCOUNT_NAMESPACE_ACCOUNT_ID_COLLISION_ERROR, codexAccountNamespacesSchema, modelPinnedEffortsSchema, @@ -274,6 +275,9 @@ export const configSchema = z.object({ // value degrades to undefined, which simply raises the notice again. multiAgentSurfaceAdvisoryVersion: z.number().int().nonnegative().optional().catch(undefined), clientIntegrations: clientIntegrationsSchema.optional().catch(undefined), + // Opt-in opencode-side export timeouts; a malformed block degrades to undefined so opencode + // keeps its native waits rather than failing the parse and hiding valid providers/accounts. + opencodeClientExport: opencodeClientExportSchema.optional().catch(undefined), // A malformed profile policy must not fall back to legacy all-profile activation. asideProfileSync: asideProfileSyncSchema.optional().catch({ allProfiles: false }), providerContextCaps: z.record(z.string(), z.number().int().positive()).optional(), diff --git a/src/config/schema/leaf-validators.ts b/src/config/schema/leaf-validators.ts index ddd20cf8688..0268b5822ac 100644 --- a/src/config/schema/leaf-validators.ts +++ b/src/config/schema/leaf-validators.ts @@ -849,6 +849,21 @@ export const clientIntegrationsSchema = z.object({ "claude-desktop": z.boolean().optional().catch(undefined), }).passthrough(); +/** + * Opt-in opencode-side request timeouts emitted into the generated opencode provider block. + * Each field is a positive millisecond count or the literal `false` (disable that wait). A + * malformed per-field value degrades to undefined (the field is dropped and opencode keeps its + * native default) rather than rejecting the whole config or emitting an opencode config that + * opencode would refuse to load. `.passthrough()` keeps a forward-compatible key a later version + * might add. + */ +const opencodeClientExportTimeout = z.union([z.number().positive().finite(), z.literal(false)]); +export const opencodeClientExportSchema = z.object({ + headerTimeoutMs: opencodeClientExportTimeout.optional().catch(undefined), + chunkTimeoutMs: opencodeClientExportTimeout.optional().catch(undefined), + timeoutMs: opencodeClientExportTimeout.optional().catch(undefined), +}).passthrough(); + export const asideProfileSyncSchema = z.object({ allProfiles: z.boolean().optional(), profiles: z.record( diff --git a/src/lib/abort.ts b/src/lib/abort.ts index 79f638e8115..99482b5609b 100644 --- a/src/lib/abort.ts +++ b/src/lib/abort.ts @@ -81,10 +81,14 @@ export function idleDeadline(idleMs: number, onIdle: () => void): IdleDeadline { export function clearableDeadline(timeoutMs: number, parent?: AbortSignal): ClearableDeadline { const deadline = new AbortController(); const timeoutReason = new DOMException("Timeout elapsed", "TimeoutError"); - let timer: ReturnType | undefined = setTimeout(() => { - timer = undefined; - if (!deadline.signal.aborted) deadline.abort(timeoutReason); - }, timeoutMs); + // A non-positive budget disables the deadline: the caller asked for no header clock (e.g. a + // local upstream whose prefill can stay silent for minutes). Only `parent` can then abort. + let timer: ReturnType | undefined = timeoutMs > 0 + ? setTimeout(() => { + timer = undefined; + if (!deadline.signal.aborted) deadline.abort(timeoutReason); + }, timeoutMs) + : undefined; const signal = parent ? AbortSignal.any([parent, deadline.signal]) : deadline.signal; return { diff --git a/src/lib/bounded-body.ts b/src/lib/bounded-body.ts index 3286a09afdc..9501784dbac 100644 --- a/src/lib/bounded-body.ts +++ b/src/lib/bounded-body.ts @@ -89,7 +89,13 @@ export function boundedBodyBufferGrowthsForTests(): number { } function bodyDeadline(ms: number, onTimeout: () => void): { clear: () => void } { - const timer = setTimeout(onTimeout, Math.max(0, ms)); + // A non-positive budget is the "disabled" sentinel (same rule as fetchWithHeaderTimeout and + // clearableDeadline): arm no timer so a slow-but-alive body — e.g. a local model streaming a + // buffered JSON response over many seconds — is bounded only by the caller's abort signal. + if (!(ms > 0)) { + return { clear: () => {} }; + } + const timer = setTimeout(onTimeout, ms); return { clear: () => clearTimeout(timer), }; diff --git a/src/lib/local-upstream.ts b/src/lib/local-upstream.ts index 5999a6449cd..f5e50f8d1c1 100644 --- a/src/lib/local-upstream.ts +++ b/src/lib/local-upstream.ts @@ -16,6 +16,7 @@ * `.local`/`.lan` name) earns the disabled budget. Anything else — a public DNS name, an * unreachable host, a bare domain — stays public and keeps its stall budget, because being local * is what switches OFF a safety clock. + * This is lexical timeout policy: it never resolves DNS or grants network access. */ import { isIP } from "node:net"; @@ -52,29 +53,20 @@ export function isLocalUpstream(destination: string | null | undefined): boolean function hostnameOf(destination: string): string | null { const trimmed = destination.trim(); if (!trimmed) return null; - // Only trust `new URL` when it produces a non-empty host. `new URL("localhost:1234")` succeeds - // but reads `localhost` as a scheme and leaves the host empty, so fall through to the schemeless - // parse in that case rather than returning null. - if (/^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed)) { - try { - const parsed = new URL(trimmed).hostname; - if (parsed) return parsed; - } catch { - // Not an absolute URL after all: fall through to the schemeless forms. - } + const absolute = /^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed); + // A schemeless host may carry a numeric port, but never credentials, a path, an + // unmatched bracket or a discarded suffix. Do not rescue malformed absolute URLs. + if (!absolute && !/^(?:\[[0-9a-f:.]+\]|[^:/?#@\s\\]+)(?::\d{1,5})?$/i.test(trimmed)) return null; + try { + const parsed = new URL(absolute ? trimmed : `http://${trimmed}`); + if (parsed.protocol !== "http:" && parsed.protocol !== "https:") return null; + return parsed.hostname || null; + } catch { + return null; } - let host = trimmed; - if (host.startsWith("[")) { - const end = host.indexOf("]"); - host = end > 0 ? host.slice(1, end) : host.slice(1); - } else { - const colon = host.indexOf(":"); - if (colon >= 0) host = host.slice(0, colon); - } - return host || null; } -/** 0.0.0.0/8, 10/8, 127/8 (loopback), 169.254/16 (link-local), 172.16/12, 192.168/16. */ +/** Unspecified 0.0.0.0, 10/8, 127/8, 169.254/16, 172.16/12, 192.168/16. */ function isLocalIpv4(host: string): boolean { const parts = host.split("."); if (parts.length !== 4) return false; @@ -86,7 +78,7 @@ function isLocalIpv4(host: string): boolean { octets.push(value); } const [a, b] = octets; - if (a === 0) return true; + if (a === 0) return octets.every(value => value === 0); if (a === 10) return true; if (a === 127) return true; if (a === 169 && b === 254) return true; @@ -102,8 +94,9 @@ function isLocalIpv6(host: string): boolean { if (!groups) return false; // Loopback in any spelling: every group zero except a trailing one. if (groups.every((group, index) => (index < 7 ? group === 0 : group === 1))) return true; - const mapped = lower.match(/^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/); - if (mapped) return isLocalIpv4(mapped[1]); + if (groups.slice(0, 5).every(group => group === 0) && groups[5] === 0xffff) { + return isLocalIpv4(`${groups[6] >>> 8}.${groups[6] & 255}.${groups[7] >>> 8}.${groups[7] & 255}`); + } const first = groups[0]; if ((first & 0xffc0) === 0xfe80) return true; if ((first & 0xfe00) === 0xfc00) return true; @@ -116,15 +109,18 @@ function ipv6Groups(host: string): number[] | null { const compressed = marker >= 0; const head = compressed ? host.slice(0, marker) : host; const tail = compressed ? host.slice(marker + 2) : ""; - const headGroups = head === "" ? [] : head.split(":"); + let headGroups = head === "" ? [] : head.split(":"); let tailGroups = tail === "" ? [] : tail.split(":"); // A trailing embedded IPv4 (e.g. `64:ff9b::1.2.3.4`) counts as two groups. - const last = tailGroups[tailGroups.length - 1]; + const ending = compressed ? tailGroups : headGroups; + const last = ending[ending.length - 1]; if (last !== undefined && last.includes(".")) { const v4 = last.split("."); if (v4.length === 4 && v4.every((part) => /^\d{1,3}$/.test(part) && Number(part) <= 255)) { const [o0, o1, o2, o3] = v4.map(Number); - tailGroups = [...tailGroups.slice(0, -1), ((o0 << 8) | o1).toString(16), ((o2 << 8) | o3).toString(16)]; + const expanded = [...ending.slice(0, -1), ((o0 << 8) | o1).toString(16), ((o2 << 8) | o3).toString(16)]; + if (compressed) tailGroups = expanded; + else headGroups = expanded; } else return null; } let groups: string[]; diff --git a/src/protocols/encoders/adapter-events.ts b/src/protocols/encoders/adapter-events.ts index 76f09b10b39..716e45879da 100644 --- a/src/protocols/encoders/adapter-events.ts +++ b/src/protocols/encoders/adapter-events.ts @@ -181,9 +181,7 @@ export function encodeAdapterEventStream( const heartbeatMs = options.heartbeatMs ?? 2_000; const setTimer = options.timers?.setInterval ?? ((handler: () => void, ms: number) => setInterval(handler, ms)); const clearTimer = options.timers?.clearInterval ?? ((id: unknown) => clearInterval(id as ReturnType)); - const maxStallTicks = Math.ceil( - (resolveStallTimeoutSec(options.stallTimeoutSec, { localUpstream: options.localUpstream }) * 1000) / heartbeatMs, - ); + let localUpstream = options.localUpstream; const textEncoder = new TextEncoder(); let controller!: ReadableStreamDefaultController; @@ -870,6 +868,8 @@ export function encodeAdapterEventStream( gated = true; beat = setTimer(() => { if (closed || gated) return; + if (localUpstream !== options.localUpstream) { localUpstream = options.localUpstream; stallTicks = 0; } + const maxStallTicks = Math.ceil((resolveStallTimeoutSec(options.stallTimeoutSec, { localUpstream }) * 1000) / heartbeatMs); if (upstreamActivity) { upstreamActivity = false; stallTicks = 0; diff --git a/src/protocols/encoders/messages.ts b/src/protocols/encoders/messages.ts index 9eee75fdb15..d41d8004306 100644 --- a/src/protocols/encoders/messages.ts +++ b/src/protocols/encoders/messages.ts @@ -410,7 +410,7 @@ export function encodeAnthropicMessageSse( return encodeAdapterEventStream( events, sink => createAnthropicMessageWriter(sink, options.model, options.inputTokenFloor), - { ...options, keepaliveMs: MESSAGES_KEEPALIVE_MS }, + { ...options, get localUpstream() { return options.localUpstream; }, keepaliveMs: MESSAGES_KEEPALIVE_MS }, ); } diff --git a/src/server/chat-native.ts b/src/server/chat-native.ts index 6001433b998..ac3d861eb5e 100644 --- a/src/server/chat-native.ts +++ b/src/server/chat-native.ts @@ -68,6 +68,7 @@ import type { RouteResult } from "../router"; import type { OcxConfig, OcxProviderConfig } from "../types"; import { fetchWithHeaderTimeout, physicalSendObserverInit, providerFetch, safeHostLabel, sendWithConnectionPolicy } from "./responses/fetch-helpers"; import { isLocalUpstream } from "../lib/local-upstream"; +import { resolveConnectTimeoutMs, resolveJsonBodyReadTimeoutMs } from "../stall-timeout"; import { linkAbortSignal } from "./responses"; import { noteProviderAttemptSend, @@ -326,7 +327,8 @@ export async function runNativeChatAttempt( streamTurnRegistered = false; unregisterTurn(upstream); }; - const connectMs = config.connectTimeoutMs ?? 200_000; + // Set only at the final send boundary; queued refresh can replace the destination. + let localUpstream = false; // Native chat is its own entry path -- chat-completions.ts routes here directly and never // through the Responses core -- so the pre-dispatch key preference is applied again here // rather than inherited. Assigned before the adapter binds below, for the same reason it is @@ -483,7 +485,7 @@ export async function runNativeChatAttempt( body: request.body, }, effectiveRecovery), upstream.signal, - connectMs, + 0, // Final physical send owns the header clock after queued credential/destination refresh. requestedStream, providerFetch(activeProvider, undefined, { providerName: route.providerName, @@ -545,19 +547,21 @@ export async function runNativeChatAttempt( noteProviderAttemptSend(logCtx, route.providerName, activeProvider, logCtx.usageLogInputTokens, transportRecovery ?? recovery); // A reselected provider transport is still a physical send: the connection policy // and manual-redirect ownership wrap the selected implementation (#4992). - dispatched = await sendWithConnectionPolicy( + const physicalFetch = ((input, physicalInit) => sendWithConnectionPolicy( distributedKeyPhysicalFetch((activeProvider as OcxProviderTransport).fetch ?? execute, route.providerName, activeProvider, request.url), + String(input), physicalInit, { providerName: route.providerName, provider: activeProvider }, + )) as typeof fetch; + localUpstream = isLocalUpstream(request.url); + dispatched = await fetchWithHeaderTimeout( request.url, applyUpstreamRecoveryInit({ ...init, method: request.method, headers, body: request.body, ...physicalSendObserverInit(observeAutoPause, request.url), }, transportRecovery), - // Reselection can replace the provider transport and the wire shape, so the - // egress route is bound to the provider this send actually uses. Omitting it - // here would let a provider transport bypass its configured route entirely, - // because that transport wins over the executor that carries the binding. - { providerName: route.providerName, provider: activeProvider }, + init.signal ?? upstream.signal, + resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream }), + requestedStream, physicalFetch, ); } finally { try { if (physicalBudget.spendEnforced) spendReport(1); } @@ -816,7 +820,7 @@ export async function runNativeChatAttempt( translatorBudget, signal: upstream.signal, stallTimeoutSec: config.stallTimeoutSec, - localUpstream: isLocalUpstream(activeProvider.baseUrl), + localUpstream, clientDeclaredEmptyToolCatalog, declaredLegacyFunctionNames, onFirstOutput, @@ -888,11 +892,12 @@ export async function runNativeChatAttempt( let body; try { + const jsonBodyReadMs = resolveJsonBodyReadTimeoutMs(config.stallTimeoutSec, { localUpstream }, config.connectTimeoutMs); body = await readBoundedResponseBody(response, { signal: upstream.signal, maxBytes: MAX_NATIVE_CHAT_JSON_BYTES, - totalTimeoutMs: Math.max(connectMs, 5_000), - inactivityTimeoutMs: Math.max(connectMs, 5_000), + totalTimeoutMs: localUpstream ? 0 : jsonBodyReadMs, + inactivityTimeoutMs: jsonBodyReadMs, }); } catch (error) { cleanupAbort(); diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index 174e94229dd..c38cbbe2064 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -58,6 +58,8 @@ import { responsesSseToAnthropicSse, } from "../claude/outbound"; import { clearableDeadline, idleDeadline } from "../lib/abort"; +import { isLocalUpstream } from "../lib/local-upstream"; +import { resolveConnectTimeoutMs } from "../stall-timeout"; import { CLAUDE_NATIVE_THINKING, type ClaudeThinkingProjection, @@ -358,7 +360,9 @@ async function anthropicNativePassthrough( const result = await fetchWithHeaderDeadline( `${base}${pathname}${search}`, { method: "POST", headers, body: JSON.stringify(body) }, - config.connectTimeoutMs ?? 200_000, + // Local Anthropic-compatible endpoints (a self-hosted base URL) disable the header deadline + // when the operator sets none, so a slow local prefill is not cut before the first header. + resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(base) }), req.signal, ); if (result.kind === "timeout") { diff --git a/src/server/inference/client-encoder-delivery.ts b/src/server/inference/client-encoder-delivery.ts index ceddcd54961..830531951e6 100644 --- a/src/server/inference/client-encoder-delivery.ts +++ b/src/server/inference/client-encoder-delivery.ts @@ -220,13 +220,14 @@ export async function deliverClientEncodedResponse(input: ClientEncodedDelivery) translatorBudget, ...input.fold, ...(input.stallTimeoutSec !== undefined ? { stallTimeoutSec: input.stallTimeoutSec } : {}), - ...(input.localUpstream !== undefined ? { localUpstream: input.localUpstream } : {}), + get localUpstream() { return input.localUpstream; }, hooks, }; const encoded = encoder.protocol === "chat" ? encodeChatCompletionSse(events, encodeOptions) : encodeAnthropicMessageSse(events, { ...encodeOptions, + get localUpstream() { return input.localUpstream; }, ...(encoder.inputTokenFloor !== undefined ? { inputTokenFloor: encoder.inputTokenFloor } : {}), }); const tracked = trackStreamLifetime(encoded, new AbortController(), input.onStreamDone, input.turnAdmissionLease); diff --git a/src/server/messages-native.ts b/src/server/messages-native.ts index e9e0ccda5f1..0baffa8e020 100644 --- a/src/server/messages-native.ts +++ b/src/server/messages-native.ts @@ -122,6 +122,8 @@ import { type RequestLogContext, } from "./request-log"; import { fetchWithHeaderTimeout, physicalSendObserverInit, providerFetch, safeHostLabel, sendWithConnectionPolicy } from "./responses/fetch-helpers"; +import { isLocalUpstream } from "../lib/local-upstream"; +import { resolveConnectTimeoutMs, resolveJsonBodyReadTimeoutMs } from "../stall-timeout"; import { anthropicBillingPhysicalFetch } from "../providers/anthropic-billing-ownership"; import { rotateAnthropicBillingKeyOnResponse } from "../providers/anthropic-billing-recovery"; import { linkAbortSignal } from "./responses/core-lifetime"; @@ -424,7 +426,8 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) streamTurnRegistered = false; unregisterTurn(upstream); }; - const connectMs = config.connectTimeoutMs ?? 200_000; + // Set only at the final send boundary; queued refresh can replace the destination. + let localUpstream = false; // An OAuth route is served by the account the existing OAuth selection commits now, at // dispatch; the token lives on this lane's provider copy only, never on the shared route. let oauthBinding: NativeOAuthBinding | undefined; @@ -560,7 +563,7 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) request.url, applyUpstreamRecoveryInit({ method: "POST", headers: request.headers, body: request.body }, transportRecovery), upstream.signal, - connectMs, + 0, // Final physical send owns the header clock after queued credential/destination refresh. requestedStream, providerFetch(activeProvider, undefined, { providerName: route.providerName, @@ -640,13 +643,19 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) sendOwner = snapshot ? captureAnthropicPhysicalSendOwnership(snapshot) : null; if (snapshot && !sendOwner) throw new NativeOAuthSelectionChangedError(); physicalSendsUsed++; - dispatched = await sendWithConnectionPolicy( + const physicalFetch = ((input, physicalInit) => sendWithConnectionPolicy( distributedKeyPhysicalFetch( anthropicBillingPhysicalFetch((activeProvider as OcxProviderTransport).fetch ?? execute, route.providerName, activeProvider), route.providerName, activeProvider, wire.url, activeProvider.apiKeyTransport === "bearer" ? "authorization" : "x-api-key"), + String(input), physicalInit, { providerName: route.providerName, provider: activeProvider }, + )) as typeof fetch; + localUpstream = isLocalUpstream(wire.url); + dispatched = await fetchWithHeaderTimeout( wire.url, applyUpstreamRecoveryInit({ ...init, method: "POST", headers, body: wire.body, ...physicalSendObserverInit(observeAutoPause, wire.url) }, transportRecovery), - { providerName: route.providerName, provider: activeProvider }, + init.signal ?? upstream.signal, + resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream }), + requestedStream, physicalFetch, ); } finally { try { if (physicalBudget.spendEnforced) spendReport(1); } @@ -957,11 +966,12 @@ export async function handleNativeMessages(options: HandleNativeMessagesOptions) let read; try { + const jsonBodyReadMs = resolveJsonBodyReadTimeoutMs(config.stallTimeoutSec, { localUpstream }, config.connectTimeoutMs); read = await readBoundedResponseBody(response, { signal: upstream.signal, maxBytes: MAX_NATIVE_MESSAGES_JSON_BYTES, - totalTimeoutMs: Math.max(connectMs, 5_000), - inactivityTimeoutMs: Math.max(connectMs, 5_000), + totalTimeoutMs: localUpstream ? 0 : jsonBodyReadMs, + inactivityTimeoutMs: jsonBodyReadMs, }); } catch (error) { cleanupAbort(); diff --git a/src/server/responses/adapter-continuation.ts b/src/server/responses/adapter-continuation.ts index 1239ac374c7..63db5570166 100644 --- a/src/server/responses/adapter-continuation.ts +++ b/src/server/responses/adapter-continuation.ts @@ -18,6 +18,7 @@ import { import { noteAttemptRecoveryWithheld } from "../request-log"; import { withProviderRequestSlot } from "../../providers/request-pacing"; import { providerFetch, fetchWithHeaderTimeout, safeHostLabel } from "./fetch-helpers"; +import { isLocalUpstream } from "../../lib/local-upstream"; import { transientRetryPolicyFor, rateLimitRetryDelayMs, @@ -139,8 +140,7 @@ export function createAdapterContinuations( } = transportState; const { route, translatorBudget, inboundWire, parsed } = requestState; const { routedCompaction } = sidecarState; - const { upstream, connectMs, rateLimitPolicy, stallTimeoutMs, localUpstream } = adapterExchange; - const bodyInactivityMs = resolveStallTimeoutMs(config.stallTimeoutSec, { localUpstream }); + const { upstream, connectMs, rateLimitPolicy, stallTimeoutMs } = adapterExchange; const { adapterDispatchBudget, noteAdapterPhysicalSend, @@ -249,7 +249,7 @@ export function createAdapterContinuations( transportState.activeAdapter.fetchResponse!(builtContinuationRequest, { kiroPreferAccountFailover: route.providerName === "kiro" && isGenericOAuthFailoverEnabled(config, "kiro"), abortSignal: upstream.signal, - timeoutMs: connectMs, + timeoutMs: route.provider.authMode === "forward" ? connectMs : 0, sendBudget: adapterDispatchBudget, onPhysicalSend: send => noteAdapterPhysicalSend(continuationEstimate, send), onRecoveryWithheld: noteAdapterRecoveryWithheld, @@ -284,7 +284,7 @@ export function createAdapterContinuations( body: builtContinuationRequest.body, }, recovery), upstream.signal, - connectMs, + route.provider.authMode === "forward" ? connectMs : 0, nextParsed.stream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { dispatchOverride: oauthDispatch(builtContinuationRequest, nextParsed), @@ -720,6 +720,7 @@ export function createAdapterContinuations( // The shared signal is not aborted on timeout: this generator must still emit // the error that the already-live Responses bridge turns into a terminal. const parse = async function* (current: Response): AsyncGenerator { + const bodyInactivityMs = resolveStallTimeoutMs(config.stallTimeoutSec, { localUpstream: isLocalUpstream(transportState.sameTargetRequest?.url ?? route.provider.baseUrl) }); if (!current.ok) { yield await adapterExchange.thinkingSignatureRecovery.projectHttpFailure(current, upstream.signal); } else if (nextParsed.stream) { diff --git a/src/server/responses/adapter-delivery.ts b/src/server/responses/adapter-delivery.ts index 40edc8f767d..986bd4c0a45 100644 --- a/src/server/responses/adapter-delivery.ts +++ b/src/server/responses/adapter-delivery.ts @@ -102,6 +102,7 @@ export async function deliverAdapterResponse( return; } try { + const bodyInactivityMs = resolveStallTimeoutMs(config.stallTimeoutSec, { localUpstream: adapterExchange.localUpstream }); const retryEvents = upstreamRequestsStream ? readResponseStreamWithInactivity(response, upstream.signal, bodyInactivityMs, guarded => transportState.activeAdapter.parseStream(guarded, translatorBudget, logCtx.activeTierMetadata)) @@ -239,7 +240,7 @@ export async function deliverAdapterResponse( toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, bareCustomToolNames, toolSearchToolNames, }, stallTimeoutSec: config.stallTimeoutSec, - localUpstream, + get localUpstream() { return adapterExchange.localUpstream; }, turnAdmissionLease: options.turnAdmissionLease, ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), stopUpstream: () => { cancelResponseCompletion(); upstream.abort(); }, @@ -257,7 +258,7 @@ export async function deliverAdapterResponse( ...(outputItemIds ? { outputItemIds } : {}), ...(options.forceEmptyResponseId ? { responseId: "" } : {}), stallTimeoutSec: config.stallTimeoutSec, - localUpstream, + get localUpstream() { return adapterExchange.localUpstream; }, hideThinkingSummary: parsed.options.hideThinkingSummary, chatRawReasoningVisible: options.chatRawReasoningVisible, hideRawReasoning: parsed.options.hideRawReasoning, diff --git a/src/server/responses/adapter-dispatch.ts b/src/server/responses/adapter-dispatch.ts index bed97287048..56d8fbbc32a 100644 --- a/src/server/responses/adapter-dispatch.ts +++ b/src/server/responses/adapter-dispatch.ts @@ -12,6 +12,7 @@ import { transientSendCapFor } from "./request-send-budget"; import { isNonReplayableResponse, isSameTargetRetryHold } from "../../lib/upstream-retry"; import { sanitizeNonReplayableUpstreamError } from "./non-replayable-error"; import { isLocalUpstream } from "../../lib/local-upstream"; +import { resolveConnectTimeoutMs, resolveStallTimeoutMs } from "../../stall-timeout"; import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; import type { PreparedResponsesRequest } from "./request-prepare"; import type { ResponsesTransport } from "./request-transport"; @@ -258,16 +259,12 @@ export async function prepareAdapterExchange( ?? formatErrorResponse(502, "upstream_error", "GitHub Copilot Auto negotiation failed."); } }; - const connectMs = config.connectTimeoutMs ?? 200_000; - // Bridge stall budget (seconds of silence before upstream_stall_timeout); the retry backoff - // heartbeat interval is derived from it so the watchdog is always fed during deliberate waits. - const stallTimeoutMs = typeof config.stallTimeoutSec === "number" && Number.isFinite(config.stallTimeoutSec) && config.stallTimeoutSec > 0 - ? Math.floor(config.stallTimeoutSec * 1000) - : 300_000; // Where this exchange dials. Local upstreams (loopback / private / `.local` / `.lan`) are // operator-trusted and often CPU-bound, so an unset stall budget resolves to disabled for them. - // Key/OAuth rotation re-resolves credentials, never the origin, so the routed baseUrl is stable. const localUpstream = isLocalUpstream(route.provider.baseUrl); + // Header arrival and body silence are separate clocks (structure/transports/timeouts.md). + const connectMs = resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream }); + const sendConnectMs = () => route.provider.authMode === "forward" ? resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(route.provider.baseUrl) }) : 0; transportState.activeAdapter = transportState.adapter; const antigravityPoolActivated = route.providerName === "google-antigravity" && isGenericOAuthFailoverEnabled(config, route.providerName); @@ -416,7 +413,7 @@ export async function prepareAdapterExchange( transportState.activeAdapter.fetchResponse!(builtInitialRequest, { kiroPreferAccountFailover: route.providerName === "kiro" && isGenericOAuthFailoverEnabled(config, "kiro"), abortSignal: upstream.signal, - timeoutMs: connectMs, + timeoutMs: sendConnectMs(), sendBudget: adapterDispatchBudget, onPhysicalSend: send => { antigravityDenial.beforeSend(); noteAdapterPhysicalSend(inputTokenEstimate, send); }, onRecoveryWithheld: noteAdapterRecoveryWithheld, @@ -498,7 +495,7 @@ export async function prepareAdapterExchange( method: builtInitialRequest.method, headers: builtInitialRequest.headers, body: builtInitialRequest.body, - }, recovery), upstream.signal, connectMs, parsed.stream, + }, recovery), upstream.signal, sendConnectMs(), parsed.stream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { dispatchOverride: oauthDispatch(builtInitialRequest), providerName: route.providerName, @@ -715,7 +712,7 @@ export async function prepareAdapterExchange( return transportState.activeAdapter.fetchResponse!(retryRequest, { kiroPreferAccountFailover: route.providerName === "kiro" && isGenericOAuthFailoverEnabled(config, "kiro"), abortSignal: upstream.signal, - timeoutMs: connectMs, + timeoutMs: sendConnectMs(), sendBudget: adapterDispatchBudget, onPhysicalSend: send => { if (preserveFailureResponse) replacementAdmitted = true; @@ -793,7 +790,7 @@ export async function prepareAdapterExchange( return fetchWithHeaderTimeout(retryRequest.url, applyUpstreamRecoveryInit({ method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, - }, recoveryKind), upstream.signal, connectMs, parsed.stream, + }, recoveryKind), upstream.signal, sendConnectMs(), parsed.stream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { dispatchOverride: oauthDispatch(retryRequest), onPhysicalDispatch: deferredRefetch ? () => { @@ -1977,9 +1974,9 @@ export async function prepareAdapterExchange( return { upstream, cleanupUpstreamAbort, - connectMs, - stallTimeoutMs, - localUpstream, + get connectMs() { return resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: this.localUpstream }); }, + get stallTimeoutMs() { return resolveStallTimeoutMs(config.stallTimeoutSec, { localUpstream: this.localUpstream }); }, + get localUpstream() { return isLocalUpstream(transportState.sameTargetRequest?.url ?? route.provider.baseUrl); }, upstreamResponse, thinkingSignatureRecovery, refetchThinkingSignature, diff --git a/src/server/responses/compact.ts b/src/server/responses/compact.ts index ca1697f8135..41e60a18aad 100644 --- a/src/server/responses/compact.ts +++ b/src/server/responses/compact.ts @@ -181,7 +181,7 @@ import { codexAccountSelectionForTurn, registerTurn, trackStreamLifetime, unregi import type { AdmissionLease } from "../../lib/admission"; import { redactSecretString } from "../../lib/redact"; import { readBoundedResponseBytes } from "../../lib/bounded-body"; -import { resolveStallTimeoutMs } from "../../stall-timeout"; +import { resolveConnectTimeoutMs, resolveStallTimeoutMs } from "../../stall-timeout"; import { isLocalUpstream } from "../../lib/local-upstream"; import { isRateLimitOrQuotaFailureMessage } from "../../lib/errors"; import { supportedLadderFor } from "../effort-policy"; @@ -1054,7 +1054,7 @@ export async function handleResponsesCompact( } compactHostAdmissionLease = null; }; - const connectMs = config.connectTimeoutMs ?? 200_000; + const connectMs = () => resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(compactUrl) }); // Takes its context explicitly: the alternate-account flow below records a rejection // against A while promoting B, then records B's own outcome. A closure over a single // `authCtx` cannot express either. @@ -1175,7 +1175,9 @@ export async function handleResponsesCompact( body: JSON.stringify({ ...buildCompactSendBody() as typeof compactBody, model: route.modelId }), }, upstreamRecovery), req.signal, - connectMs, + // Keyed sends can refresh their destination after pacing. Their physical send below + // owns the header clock; the outer wrapper owns pacing and client cancellation only. + sendProvider.authMode !== "forward" && sendProvider.apiKey ? 0 : connectMs(), false, providerFetch(sendProvider, undefined, { providerName: route.providerName, @@ -1219,10 +1221,12 @@ export async function handleResponsesCompact( if (!rebindPhysicalSend(sendBudget, { poolId: logCtx.spendPoolId ?? route.providerName, identityId: logCtx.accountLogLabel })) throw new SendBudgetExhaustedError(safeHostLabel(compactUrl)); noteProviderAttemptSend(logCtx, route.providerName, sendProvider, logCtx.usageLogInputTokens, upstreamRecovery); - return sendWithConnectionPolicy( - distributedKeyPhysicalFetch((sendProvider as { fetch?: typeof fetch }).fetch ?? execute, route.providerName, sendProvider, compactUrl), compactUrl, + const physicalFetch = ((input, physicalInit) => sendWithConnectionPolicy( + distributedKeyPhysicalFetch((sendProvider as { fetch?: typeof fetch }).fetch ?? execute, route.providerName, sendProvider, compactUrl), String(input), + physicalInit, { providerName: route.providerName, provider: sendProvider })) as typeof fetch; + return fetchWithHeaderTimeout(compactUrl, { ...init, headers: sendHeaders, body: JSON.stringify({ ...buildCompactSendBody() as typeof compactBody, model: route.modelId }) }, - { providerName: route.providerName, provider: sendProvider }); + init.signal ?? req.signal, connectMs(), false, physicalFetch); } : undefined, ...(onPhysicalDispatch ? { onPhysicalDispatch } : {}), }), diff --git a/src/server/responses/fetch-helpers.ts b/src/server/responses/fetch-helpers.ts index 7776242db4b..469cbdb8341 100644 --- a/src/server/responses/fetch-helpers.ts +++ b/src/server/responses/fetch-helpers.ts @@ -558,7 +558,7 @@ export async function fetchWithHeaderTimeout( _manualRedirect = false, ): Promise { const pacing = executor as ProviderFetch; - const slot = await pacing.waitForPacing?.(abortSignal); + const slot = pacing.waitForPacing ? await pacing.waitForPacing(abortSignal) : undefined; const fetchExecutor = pacing.unpacedFetch ?? executor; let timer: ReturnType | undefined; try { @@ -568,6 +568,10 @@ export async function fetchWithHeaderTimeout( pause() { if (timer) clearTimeout(timer); timer = undefined; }, resume() { this.pause(); + // A non-positive budget disables the header deadline: a local upstream (LM Studio, Ollama) + // can stay silent for minutes during prompt prefill, so the request is bounded only by the + // caller's own abort signal. See resolveConnectTimeoutMs in src/stall-timeout.ts. + if (timeoutMs <= 0) return; timer = setTimeout(() => { if (!timeout.signal.aborted) timeout.abort(new DOMException("Timeout elapsed", "TimeoutError")); }, timeoutMs); diff --git a/src/server/responses/passthrough-dispatch.ts b/src/server/responses/passthrough-dispatch.ts index 857e31839d1..e253d5c9460 100644 --- a/src/server/responses/passthrough-dispatch.ts +++ b/src/server/responses/passthrough-dispatch.ts @@ -20,6 +20,7 @@ import { transientSendCapFor } from "./request-send-budget"; import { renewMainQuotaDispatchForAttempt } from "../../codex/main-account-cache"; import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; import { isLocalUpstream } from "../../lib/local-upstream"; +import { resolveConnectTimeoutMs } from "../../stall-timeout"; import { codexSafetyBufferingFilterOptions, terminalStatusFromParsed } from "../relay"; import { imageGenToolCallAliases } from "../responses-image-gen-repair"; import { rememberResponseState, isBodyNonPersistable } from "../../responses/state"; @@ -746,7 +747,8 @@ export async function preparePassthroughExchange( // whose cancel() aborts the upstream — preventing leaked connections (RC2, passthrough path). const upstream = new AbortController(); const cleanupUpstreamAbort = linkAbortSignal(upstream, options.abortSignal ?? req.signal); - const connectMs = config.connectTimeoutMs ?? 200_000; + const sendConnectMs = () => route.provider.authMode === "forward" ? resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(request.url) }) : 0; + const connectMs = resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(route.provider.baseUrl) }); /** * This leg's transient-5xx ladder cap, from the provider's own `transientRetryOn5xx`. * @@ -828,7 +830,7 @@ export async function preparePassthroughExchange( body: request.body, }, "connection-reset"), signal, - connectMs, + sendConnectMs(), true, providerFetch(route.provider, options.codexWsRuntimeIdentity, { httpOnly: true, @@ -1025,7 +1027,7 @@ export async function preparePassthroughExchange( method: request.method, headers: request.headers, body: request.body, - }, recovery), upstream.signal, connectMs, upstreamRequestsStream, + }, recovery), upstream.signal, sendConnectMs(), upstreamRequestsStream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { httpOnly: canonicalBufferedJson, nativeControl: nativeResponseControlEligible(route.provider, options.nativeControl) && options.inboundTransport === "websocket" && !options.comboAttempt @@ -1165,7 +1167,7 @@ export async function preparePassthroughExchange( method: request.method, headers: request.headers, body: request.body, - }, innerRecovery), upstream.signal, connectMs, upstreamRequestsStream, + }, innerRecovery), upstream.signal, sendConnectMs(), upstreamRequestsStream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { httpOnly: canonicalBufferedJson, nativeControl: nativeResponseControlEligible(route.provider, options.nativeControl) && options.inboundTransport === "websocket" && !options.comboAttempt @@ -1318,7 +1320,7 @@ export async function preparePassthroughExchange( body: request.body, }, recovery), upstream.signal, - connectMs, + sendConnectMs(), upstreamRequestsStream, oauthReplayExecutor, route.provider.authMode === "forward", @@ -1431,7 +1433,7 @@ export async function preparePassthroughExchange( method: request.method, headers: request.headers, body: request.body, - }, recovery), upstream.signal, connectMs, upstreamRequestsStream, + }, recovery), upstream.signal, sendConnectMs(), upstreamRequestsStream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { httpOnly: canonicalBufferedJson, nativeControl: nativeResponseControlEligible(route.provider, options.nativeControl) && options.inboundTransport === "websocket" && !options.comboAttempt @@ -1578,7 +1580,7 @@ export async function preparePassthroughExchange( method: request.method, headers: request.headers, body: request.body, - }, recovery), upstream.signal, connectMs, upstreamRequestsStream, + }, recovery), upstream.signal, sendConnectMs(), upstreamRequestsStream, providerFetch(route.provider, options.codexWsRuntimeIdentity, { httpOnly: canonicalBufferedJson, nativeControl: nativeResponseControlEligible(route.provider, options.nativeControl) && options.inboundTransport === "websocket" && !options.comboAttempt @@ -1943,10 +1945,8 @@ export async function preparePassthroughExchange( upstreamResponse, request, config, upstream.signal, logCtx, rebuildAndRefetch, ); // Where this relay dials upstream. Local infrastructure (loopback / private / `.local` / `.lan`) - // is operator-trusted and its silent phases are normal, so an unset stall budget resolves to - // disabled for it. Provider rotation above keeps the same endpoint origin, so the routed - // provider's baseUrl is the stable classification source. - const localUpstream = isLocalUpstream(route.provider.baseUrl); + // is operator-trusted and its silent phases are normal. Use the refreshed serving URL. + const localUpstream = isLocalUpstream(request.url); return { kind: undefined, @@ -1986,7 +1986,7 @@ export async function preparePassthroughExchange( noteInspectedPayload, rememberPassthroughResponseChecked, upstream, - connectMs, + connectMs: resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream }), localUpstream, upstreamResponse, terminalOutcomeRecorded: preOutputRefusals.outcomeRecorded, diff --git a/src/server/responses/request-transport.ts b/src/server/responses/request-transport.ts index 44e6303890d..6089549e94c 100644 --- a/src/server/responses/request-transport.ts +++ b/src/server/responses/request-transport.ts @@ -77,7 +77,9 @@ import { resolveCurrentProviderApiKeyTransport, } from "../../providers/api-key-selection"; import { resolveAdapter, resolveWireProtocolOverride } from "../adapter-resolve"; -import { codexWsExecutorBinding, physicalSendObserverInit, providerFetch, sendWithConnectionPolicy } from "./fetch-helpers"; +import { codexWsExecutorBinding, fetchWithHeaderTimeout, physicalSendObserverInit, providerFetch, sendWithConnectionPolicy } from "./fetch-helpers"; +import { resolveConnectTimeoutMs } from "../../stall-timeout"; +import { isLocalUpstream } from "../../lib/local-upstream"; import { deepSeekImageFileBody, needsDeepSeekImageFiles } from "./deepseek-image-files"; import type { ProviderFetchOptions } from "./fetch-helpers"; import { captureConfigGeneration } from "../../lib/state-store-sweeper"; @@ -895,13 +897,15 @@ export async function prepareResponsesTransport( // The WS executor dials its own socket: HTTP connection policy and body coding do not apply. // WS eligibility (egress, pacing, URL) was checked for the provider it was built for; a // provider reselected above was not, so it takes the HTTP fallback under its own route. - response = ws?.provider === route.provider ? await execute(destination, dispatchInit) : await sendWithConnectionPolicy( + const physicalFetch = ((input, physicalInit) => ws?.provider === route.provider ? execute(input, physicalInit) : sendWithConnectionPolicy( distributedKeyPhysicalFetch(anthropicBillingPhysicalFetch(fetchImpl, route.providerName, route.provider), route.providerName, route.provider, target), - destination, - { ...dispatchInit, redirect: "manual", + input, + { ...physicalInit, redirect: "manual", ...physicalSendObserverInit(observeAutoPause, destination instanceof Request ? destination.url : String(destination)) }, { providerName: route.providerName, provider: route.provider }, - ); + )) as typeof globalThis.fetch; + response = await fetchWithHeaderTimeout(target, dispatchInit, dispatchInit.signal ?? options.abortSignal ?? req.signal, + resolveConnectTimeoutMs(config.connectTimeoutMs, { localUpstream: isLocalUpstream(target) }), requestParsed.stream, physicalFetch); } finally { releaseFamily(); } // Observe each physical response before retries replace it. The binding belongs to // this dispatch, so a manual switch cannot file A's headers against B. Header diff --git a/src/stall-timeout.ts b/src/stall-timeout.ts index f7256c612bd..fbf06147e34 100644 --- a/src/stall-timeout.ts +++ b/src/stall-timeout.ts @@ -43,3 +43,50 @@ export function resolveStallTimeoutSec(configuredSec: number | undefined, option export function resolveStallTimeoutMs(configuredSec: number | undefined, options: StallTimeoutOptions = {}): number { return resolveStallTimeoutSec(configuredSec, options) * 1_000; } + +/** Header-arrival (connect) deadline applied to public upstreams when the operator sets none. */ +export const DEFAULT_CONNECT_TIMEOUT_MS = 200_000; + +/** + * Resolve the per-request header-arrival (connect) deadline in milliseconds. + * + * This is the budget `fetchWithHeaderTimeout` applies from dispatch until the upstream returns its + * first response headers — it covers DNS, TCP, TLS, and the wait for response headers, but NOT the + * body/stream that follows (that is the stall budget, see {@link resolveStallTimeoutMs}). + * + * An explicit value always wins: `0` (or any non-positive) disables the deadline everywhere, and a + * positive value applies verbatim to local and public upstreams alike. Only when the operator left + * it unset does the destination matter. A local upstream (loopback / private / `.local` / `.lan`, + * via {@link StallTimeoutOptions.localUpstream}) resolves to disabled (`0`): LM Studio, Ollama, and + * similar CPU-bound backends can spend minutes in prompt prefill before the first response header + * arrives on a large prompt, and a flat ceiling would cut that healthy turn. A public upstream + * keeps the {@link DEFAULT_CONNECT_TIMEOUT_MS} safety deadline. + * + * Disabling is expressed as `0`: `fetchWithHeaderTimeout` arms no header clock for a non-positive + * budget. Client cancellation and any enclosing overall deadline remain authoritative. + */ +export function resolveConnectTimeoutMs(configuredMs: number | undefined, options: StallTimeoutOptions = {}): number { + const { localUpstream = false } = options; + if (typeof configuredMs === "number" && Number.isFinite(configuredMs)) { + if (configuredMs <= 0) return 0; + return Math.max(1, Math.ceil(configuredMs)); + } + return localUpstream ? 0 : DEFAULT_CONNECT_TIMEOUT_MS; +} + +/** Legacy minimum public JSON body-read allowance, independent of the local stall policy. */ +export const MIN_JSON_BODY_READ_MS = 5_000; + +/** + * Resolve the total/inactivity budget for reading a buffered non-streaming JSON response body. + * + * Local JSON reads use the stall budget for inactivity only; their enclosing signal owns any + * overall deadline. An unset local budget disables the body clock. Public reads preserve their + * legacy connect-derived total/inactivity allowance, including the five-second floor, independently + * of the new local policy. This does not extend the response-header timer into the body phase. + */ +export function resolveJsonBodyReadTimeoutMs(configuredSec: number | undefined, options: StallTimeoutOptions = {}, configuredConnectMs?: number): number { + return options.localUpstream + ? resolveStallTimeoutMs(configuredSec, options) + : Math.max(configuredConnectMs ?? DEFAULT_CONNECT_TIMEOUT_MS, MIN_JSON_BODY_READ_MS); +} diff --git a/src/types.ts b/src/types.ts index 99e08560f2c..50ffa1f8b47 100644 --- a/src/types.ts +++ b/src/types.ts @@ -79,6 +79,7 @@ export type { OcxCustomModel, OcxApiKeyEntry, OcxClientIntegrationsConfig, + OcxOpencodeClientExportConfig, OcxConfigRebaseProvenance, OcxHubConfig, OcxRemoteGuiConfig, diff --git a/src/types/config.ts b/src/types/config.ts index b8e2c0bcc21..30da5d1a18a 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -392,6 +392,28 @@ export interface OcxClientIntegrationsConfig { "claude-desktop"?: boolean; } +/** + * Opt-in opencode-side request timeouts written into the generated opencode provider block + * (both the V1 `options` and V2 `settings` connection objects). Each maps to the matching + * opencode `settings` field and accepts a number of milliseconds or `false` to disable that wait. + * + * Omitted by default: opencode keeps its native header and chunk waits for every upstream reached + * through the proxy (current V2 defaults the header wait to 5 minutes and the longer chunk wait to + * 15 minutes; this repo pins no opencode release). The knob exists because the generated + * `opencodex` provider multiplexes all upstreams under one block and opencode's timeouts are + * provider-wide, so a user fronting a slow local model (whose prompt prefill behind a non-streaming + * turn can exceed the header wait before the proxy emits a byte) can raise or disable the wait + * without touching the safety net for anyone else only by setting it here. + */ +export interface OcxOpencodeClientExportConfig { + /** opencode `settings.headerTimeout`: client wait for the proxy's first response header (ms), or false to disable. */ + headerTimeoutMs?: number | false; + /** opencode `settings.chunkTimeout`: wait between streamed response chunks (ms), or false to disable. */ + chunkTimeoutMs?: number | false; + /** opencode `settings.timeout`: whole-request bound (ms), or false. opencode applies no default. */ + timeoutMs?: number | false; +} + export interface OcxConfigRebaseProvenance { version: 1; deletedTopLevelKeys: string[]; @@ -673,6 +695,13 @@ export interface OcxConfig { * one key at a time rather than widening a shared union. */ clientIntegrations?: OcxClientIntegrationsConfig; + /** + * Opt-in opencode-side request timeouts emitted into the generated opencode provider block. + * Omitted by default (opencode keeps its native header/chunk waits). Set `headerTimeoutMs` to + * extend or disable the header wait that otherwise cuts a slow local model's prompt prefill. + * See {@link OcxOpencodeClientExportConfig}. + */ + opencodeClientExport?: OcxOpencodeClientExportConfig; /** Aside account-backed profile synchronization; individual overrides survive bulk refresh. */ asideProfileSync?: { allProfiles?: boolean; @@ -1032,7 +1061,14 @@ export interface OcxConfig { * response.incomplete. Default 300. Min 1. */ stallTimeoutSec?: number; - /** Connect timeout (ms) for upstream fetch — covers DNS, TCP, TLS, and response header. Default 200000. */ + /** + * Connect timeout (ms) for upstream fetch — the deadline from dispatch until the upstream + * returns its first response header (covers DNS, TCP, TLS, and the header wait; NOT the body + * stream that follows, which is governed by stallTimeoutSec). Set 0 (or any non-positive value) + * to disable it entirely. When unset, a public upstream defaults to 200000, while a local + * upstream (loopback / private / `.local` / `.lan`) disables it — a local model's prompt prefill + * can hold the connection silent for minutes on a large prompt before the first header arrives. + */ connectTimeoutMs?: number; /** Graceful shutdown drain timeout (ms). Active turns are aborted after this deadline. Default 5000. */ shutdownTimeoutMs?: number; diff --git a/src/web-search/header-deadline.ts b/src/web-search/header-deadline.ts index 5157614bae1..8db704a49b8 100644 --- a/src/web-search/header-deadline.ts +++ b/src/web-search/header-deadline.ts @@ -4,6 +4,7 @@ import type { ProviderFetch } from "../server/responses/fetch-helpers"; /** One iteration's cumulative header budget; only local pacing admission pauses its clock. */ export function pacingHeaderDeadline(timeoutMs: number, parent: AbortSignal) { + const disabled = timeoutMs <= 0; const controller = new AbortController(); const signal = AbortSignal.any([parent, controller.signal]); let remainingMs = timeoutMs; @@ -18,7 +19,7 @@ export function pacingHeaderDeadline(timeoutMs: number, parent: AbortSignal) { current.signal.addEventListener("abort", relayExpiry, { once: true }); const pause = () => { - if (cleared || pacingWaits++ > 0) return; + if (disabled || cleared || pacingWaits++ > 0) return; remainingMs = Math.max(0, remainingMs - (performance.now() - armedAt)); current.clear(); current.signal.removeEventListener("abort", relayExpiry); @@ -26,7 +27,7 @@ export function pacingHeaderDeadline(timeoutMs: number, parent: AbortSignal) { if (remainingMs === 0 && !signal.aborted) controller.abort(timeoutReason); }; const resume = () => { - if (cleared || --pacingWaits > 0 || signal.aborted) return; + if (disabled || cleared || --pacingWaits > 0 || signal.aborted) return; armedAt = performance.now(); current = clearableDeadline(remainingMs, parent); current.signal.addEventListener("abort", relayExpiry, { once: true }); diff --git a/structure/INDEX.md b/structure/INDEX.md index 971691c2222..be132400830 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -45,6 +45,7 @@ The wire surfaces a client actually talks to. | Doc | Scope | | --- | --- | +| [`transports/timeouts.md`](transports/timeouts.md) | Local header and JSON body deadlines, cancellation and optional OpenCode timeout export. | | [`remote-messaging.md`](remote-messaging.md) | Explicit peer enrollment, connection-bound capabilities, bounded foreground SSH ownership and queued remote replies. | | [`local-messaging.md`](local-messaging.md) | Command-local metadata discovery, Unix RPC queued submission, peer envelopes, receipts and owned lifecycle bounds. | | [`transports/byte-accounting.md`](transports/byte-accounting.md) | Request-copy and stream-buffer byte accounting shared by parsing, SSE rewriting, the adapters, and the translator budget, plus durable spill admission. | @@ -238,6 +239,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/client/link-parity-reporter.ts` | [`remote-link.md`](remote-link.md) | | `src/client/runtime-restart.ts` | [`remote-link.md`](remote-link.md) | | `src/clients/` | [`clients/integrations.md`](clients/integrations.md)
[`clients/qoder.md`](clients/qoder.md)
[`clients/t3code.md`](clients/t3code.md) | +| `src/clients/config-export.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `src/codex/` | [`runtime.md`](runtime.md)
[`config.md`](config.md)
[`codex-home.md`](codex-home.md)
[`codex-prompts.md`](codex-prompts.md)
[`catalog.md`](catalog.md)
[`subagents.md`](subagents.md)
[`subagent-capacity.md`](subagent-capacity.md)
[`transports/responses-failover.md`](transports/responses-failover.md)
[`providers/openai-tiers.md`](providers/openai-tiers.md)
[`providers/openai-accounts.md`](providers/openai-accounts.md)
[`gui-and-management-api.md`](gui-and-management-api.md)
[`dashboard-and-usage.md`](dashboard-and-usage.md)
[`fallback-policy.md`](fallback-policy.md)
[`clients/codex-desktop.md`](clients/codex-desktop.md)
[`clients/chatgpt-desktop.md`](clients/chatgpt-desktop.md)
[`ops/docs-and-release.md`](ops/docs-and-release.md) | | `src/codex/account-pause.ts` | [`account-auto-pause.md`](account-auto-pause.md) | | `src/codex/account-preservation.ts` | [`providers/openai-signed-in-preservation.md`](providers/openai-signed-in-preservation.md) | @@ -276,7 +278,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/config/live-reconcile.ts` | [`pricing.md`](pricing.md) | | `src/config/persisted-mutation.ts` | [`runtime.md`](runtime.md) | | `src/config/schema/account-auto-pause.ts` | [`account-auto-pause.md`](account-auto-pause.md) | -| `src/config/schema/leaf-validators.ts` | [`pricing.md`](pricing.md) | +| `src/config/schema/leaf-validators.ts` | [`transports/timeouts.md`](transports/timeouts.md)
[`pricing.md`](pricing.md) | | `src/generated/` | [`runtime.md`](runtime.md) | | `src/github/` | [`runtime.md`](runtime.md) | | `src/grok/` | [`runtime.md`](runtime.md) | @@ -285,10 +287,13 @@ A source area can be described by more than one doc, because these docs are orga | `src/integrations/` | [`clients/integrations.md`](clients/integrations.md)
[`clients/hermes.md`](clients/hermes.md)
[`clients/qoder.md`](clients/qoder.md)
[`clients/t3code.md`](clients/t3code.md) | | `src/lab/` | [`runtime.md`](runtime.md)
[`adapters/compatibility-lab.md`](adapters/compatibility-lab.md) | | `src/lib/` | [`overview.md`](overview.md)
[`runtime.md`](runtime.md)
[`transports/byte-accounting.md`](transports/byte-accounting.md)
[`transports/responses-wire-shapes.md`](transports/responses-wire-shapes.md)
[`transports/responses-failover.md`](transports/responses-failover.md)
[`transports/responses-spend.md`](transports/responses-spend.md)
[`transports/inventory.md`](transports/inventory.md)
[`gui-and-management-api.md`](gui-and-management-api.md)
[`dashboard-and-usage.md`](dashboard-and-usage.md)
[`clients/integrations.md`](clients/integrations.md)
[`ops/service-and-sidecars.md`](ops/service-and-sidecars.md)
[`ops/docs-and-release.md`](ops/docs-and-release.md) | +| `src/lib/abort.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `src/lib/account-auto-pause-response.ts` | [`account-auto-pause.md`](account-auto-pause.md) | | `src/lib/advisor-activation.ts` | [`advisor.md`](advisor.md) | +| `src/lib/bounded-body.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `src/lib/decision-quota-sink.ts` | [`providers/jev-decision.md`](providers/jev-decision.md) | | `src/lib/gui-pair-intent.ts` | [`remote-link.md`](remote-link.md) | +| `src/lib/local-upstream.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `src/lib/tool-argument-integers.ts` | [`providers-and-adapters.md`](providers-and-adapters.md) | | `src/lib/windows-owner-acl.ts` | [`remote-link.md`](remote-link.md) | | `src/link/` | [`remote-messaging.md`](remote-messaging.md)
[`remote-link.md`](remote-link.md) | @@ -336,7 +341,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/router.ts` | [`runtime.md`](runtime.md) | | `src/routing/` | [`catalog.md`](catalog.md)
[`fallback-policy.md`](fallback-policy.md) | | `src/routing/history/` | [`dashboard-and-usage.md`](dashboard-and-usage.md) | -| `src/server/` | [`providers/generic-account-pool.md`](providers/generic-account-pool.md)
[`runtime.md`](runtime.md)
[`codex-prompts.md`](codex-prompts.md)
[`catalog.md`](catalog.md)
[`subagents.md`](subagents.md)
[`transports/byte-accounting.md`](transports/byte-accounting.md)
[`transports/responses.md`](transports/responses.md)
[`transports/native-reasoning-retention.md`](transports/native-reasoning-retention.md)
[`transports/responses-wire-shapes.md`](transports/responses-wire-shapes.md)
[`transports/responses-failover.md`](transports/responses-failover.md)
[`transports/responses-combo-boundaries.md`](transports/responses-combo-boundaries.md)
[`transports/policy-fallback.md`](transports/policy-fallback.md)
[`transports/streaming-health.md`](transports/streaming-health.md)
[`transports/inventory.md`](transports/inventory.md)
[`data-planes/images.md`](data-planes/images.md)
[`data-planes/inbound-compat.md`](data-planes/inbound-compat.md)
[`providers/anthropic-progress.md`](providers/anthropic-progress.md)
[`providers-and-adapters.md`](providers-and-adapters.md)
[`providers/jev-decision.md`](providers/jev-decision.md)
[`providers/xai-grok.md`](providers/xai-grok.md)
[`adapters/registry.md`](adapters/registry.md)
[`gui-and-management-api.md`](gui-and-management-api.md)
[`provider-completion-diagnostic.md`](provider-completion-diagnostic.md)
[`dashboard-and-usage.md`](dashboard-and-usage.md)
[`fallback-policy.md`](fallback-policy.md)
[`clients/claude-desktop.md`](clients/claude-desktop.md)
[`clients/hermes.md`](clients/hermes.md)
[`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | +| `src/server/` | [`transports/timeouts.md`](transports/timeouts.md)
[`providers/generic-account-pool.md`](providers/generic-account-pool.md)
[`runtime.md`](runtime.md)
[`codex-prompts.md`](codex-prompts.md)
[`catalog.md`](catalog.md)
[`subagents.md`](subagents.md)
[`transports/byte-accounting.md`](transports/byte-accounting.md)
[`transports/responses.md`](transports/responses.md)
[`transports/native-reasoning-retention.md`](transports/native-reasoning-retention.md)
[`transports/responses-wire-shapes.md`](transports/responses-wire-shapes.md)
[`transports/responses-failover.md`](transports/responses-failover.md)
[`transports/responses-combo-boundaries.md`](transports/responses-combo-boundaries.md)
[`transports/policy-fallback.md`](transports/policy-fallback.md)
[`transports/streaming-health.md`](transports/streaming-health.md)
[`transports/inventory.md`](transports/inventory.md)
[`data-planes/images.md`](data-planes/images.md)
[`data-planes/inbound-compat.md`](data-planes/inbound-compat.md)
[`providers/anthropic-progress.md`](providers/anthropic-progress.md)
[`providers-and-adapters.md`](providers-and-adapters.md)
[`providers/jev-decision.md`](providers/jev-decision.md)
[`providers/xai-grok.md`](providers/xai-grok.md)
[`adapters/registry.md`](adapters/registry.md)
[`gui-and-management-api.md`](gui-and-management-api.md)
[`provider-completion-diagnostic.md`](provider-completion-diagnostic.md)
[`dashboard-and-usage.md`](dashboard-and-usage.md)
[`fallback-policy.md`](fallback-policy.md)
[`clients/claude-desktop.md`](clients/claude-desktop.md)
[`clients/hermes.md`](clients/hermes.md)
[`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/server/advisor-admission-scope.ts` | [`advisor.md`](advisor.md) | | `src/server/auth-cors.ts` | [`transports/response-turn-membership.md`](transports/response-turn-membership.md) | | `src/server/gui-pair-delivery.ts` | [`remote-link.md`](remote-link.md) | @@ -385,12 +390,13 @@ A source area can be described by more than one doc, because these docs are orga | `src/service/repair.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/sidecar/` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/sidecar/anthropic-binding.ts` | [`providers/anthropic-account-pool.md`](providers/anthropic-account-pool.md) | -| `src/stall-timeout.ts` | [`runtime.md`](runtime.md) | +| `src/stall-timeout.ts` | [`transports/timeouts.md`](transports/timeouts.md)
[`runtime.md`](runtime.md) | | `src/storage/` | [`runtime.md`](runtime.md) | | `src/tray/` | [`runtime.md`](runtime.md) | | `src/types.ts` | [`runtime.md`](runtime.md)
[`config.md`](config.md) | | `src/types/` | [`providers/generic-account-pool.md`](providers/generic-account-pool.md)
[`transports/native-reasoning-retention.md`](transports/native-reasoning-retention.md)
[`providers/anthropic-progress.md`](providers/anthropic-progress.md)
[`providers-and-adapters.md`](providers-and-adapters.md)
[`fallback-policy.md`](fallback-policy.md) | | `src/types/account-auto-pause.ts` | [`account-auto-pause.md`](account-auto-pause.md) | +| `src/types/config.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `src/types/provider.ts` | [`pricing.md`](pricing.md) | | `src/update/` | [`runtime.md`](runtime.md)
[`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/update/index.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | @@ -405,6 +411,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/usage/user-cost-validation.ts` | [`pricing.md`](pricing.md) | | `src/vision/` | [`runtime.md`](runtime.md)
[`gui-and-management-api.md`](gui-and-management-api.md) | | `src/web-search/` | [`runtime.md`](runtime.md)
[`providers/anthropic-progress.md`](providers/anthropic-progress.md)
[`providers-and-adapters.md`](providers-and-adapters.md) | +| `src/web-search/header-deadline.ts` | [`transports/timeouts.md`](transports/timeouts.md) | | `tests/` | [`ops/test-sandbox-cleanup.md`](ops/test-sandbox-cleanup.md) | ### Not described by any doc diff --git a/structure/clients/integrations.md b/structure/clients/integrations.md index 95a7d455236..e5b9668a93e 100644 --- a/structure/clients/integrations.md +++ b/structure/clients/integrations.md @@ -15,7 +15,7 @@ parsing and ownership rules below. | Module | Responsibility | | --- | --- | -| `src/clients/config-export.ts` | Pure per-client config builders and exact managed fragments; never writes files. The dependency-free `src/clients/client-ids.ts` roster defines the exhaustive `ExportClientId` union and `EXPORT_CLIENT_IDS`; CLI usage and its client count derive from it. `src/clients/client-path.ts` validates absolute selectors: Windows requires a drive letter or UNC share, refusing current-drive rooted paths. Qoder/Qoder CN/WorkBuddy share its tilde-aware resolver; DSH preserves raw whitespace and Gentle Shell refuses tilde while sharing its qualification predicate, as do ZCode bootstrap roots and Command Code's Windows home. `tests/clients/client-path.test.ts` exercises Windows/POSIX semantics on every host. | +| `src/clients/config-export.ts` | [OpenCode timeout export](../transports/timeouts.md#opencode-timeout-export). Pure per-client config builders and exact managed fragments; never writes files. The dependency-free `src/clients/client-ids.ts` roster defines the exhaustive `ExportClientId` union and `EXPORT_CLIENT_IDS`; CLI usage and its client count derive from it. `src/clients/client-path.ts` validates absolute selectors: Windows requires a drive letter or UNC share, refusing current-drive rooted paths. Qoder/Qoder CN/WorkBuddy share its tilde-aware resolver; DSH preserves raw whitespace and Gentle Shell refuses tilde while sharing its qualification predicate, as do ZCode bootstrap roots and Command Code's Windows home. `tests/clients/client-path.test.ts` exercises Windows/POSIX semantics on every host. | | `src/clients/reasoning-controls.ts` | Ingress compatibility for effort-enum clients: `resolveThinkingEnabled(provider, modelId, options)` decodes toggle aliases into `thinkingEnabled` before strength policy runs, preserving an explicit thinking choice and canonical provider defaults/maps. `normalizeThinkingOptions` removes the alias when real effort is unsupported. | | `src/integrations/registry.ts` | Canonical config/detection paths, current-provider-store declarations, source-preserving YAML declarations, writer-lock behavior, and client IDs. | | `src/integrations/target.ts` | Which file one operation reads, writes and records, and whether a write there reaches the client. | diff --git a/structure/config.md b/structure/config.md index 62997e44447..7e7d69607e4 100644 --- a/structure/config.md +++ b/structure/config.md @@ -1,5 +1,5 @@ # Config Surface -Pasted GCP JSON persists only as `gcp-sa:` references in one pool transaction in `src/providers/api-keys.ts`; post-commit deletion follows [Vertex credential markers](providers/google.md#vertex-credential-markers). +See [local timeout policy and OpenCode export](transports/timeouts.md). Pasted GCP JSON persists only as `gcp-sa:` references in one pool transaction in `src/providers/api-keys.ts`; post-commit deletion follows [Vertex credential markers](providers/google.md#vertex-credential-markers). `src/config/schema/leaf-validators.ts` accepts only boolean `apiKeys[].allowMeteredComboTargets`; malformed entries are dropped rather than promoted. `src/types/config.ts` and combo validation define optional boolean target `metered`, normalized false when omitted, and optional per-combo `allowMeteredTargets` (default false, literal boolean validation shared by config schema and management API). Explicit consent is required under the [combo runtime contract](runtime.md#request-local-target-compatibility). Model-price overrides follow the [pricing contract](pricing.md). Quota activation reuses the existing next-reset fields without adding a polling configuration key. See the [quota activation contract](providers/openai-tiers.md#public-provider-contract). `src/types/config.ts` defines `anthropicAccountPool.routes` as ordered model rules. `src/oauth/anthropic-model-routes.ts` validates bounded names, full case-sensitive globs and stored account IDs; `src/config/diagnostics.ts` rejects malformed candidate writes. Load retains a malformed hand edit so unrelated providers survive, while the enabled Anthropic selector refuses it locally before dispatch. Saved rules remain inert when the pool is disabled; affinity is process-local. Request logs use only the rule’s 1-based `route:#` position, never its configured name. `src/cli/hub-disable.ts` removes interdependent hub/loopback/role/hostname settings through `mutatePersistedConfig` and `validateConfigCandidate` in one commit. Top-level deletion intent uses `src/config/rebase-provenance.ts` and authoritative disk provenance in `src/config/live-reconcile.ts` to preserve removed hub bindings through later live saves, `src/config/live-topology.ts` keeps an independent active-topology baseline and merges desired next-start settings into a separate persistence projection, retaining actual socket identity and live hub/client topology during guarded saves and OAuth disk adoption. Client connections and runtimeRole remain one topology unit through connect/disconnect, including fresh disk-only connections. Disk additions/value updates and deletions all survive repeated saves without changing active topology. `src/config/api-key-reconcile.ts` merges key rows by stable ID under the existing save lock; either side's revocation defeats stale edits while disjoint names and rotations merge; unchanged live key membership/order adopts persisted ordering. `tests/config/hub-disable-live-save.test.ts` exercises disable followed by live key creation, rename, rotation, unrelated settings and repeated saves. Saved API keys have no immutable enrollment marker; explicit exact-ID revocation preserves all unselected keys. The WorkBuddy AI account row pins its adapter and origin under the [WorkBuddy contract](providers/workbuddy.md). JEV `decisionQuotaSignals` and partial `decisionQuotaTiers` use the shared validator and sparse preserve/clear/strategy-switch contract in [loaded advisory quota](providers/jev-decision.md#loaded-advisory-quota); the opt-in JEV `sessionPin` follows the same contract under [session pin](providers/jev-decision.md#session-pin). Native reasoning retention follows [its optional configuration contract](transports/native-reasoning-retention.md#configuration-and-management). Native function-result injection follows [the separate opt-in control contract](transports/streaming-health.md#experimental-native-function-result-injection); this surface does not infer upstream support or alter its defaults. diff --git a/structure/data-planes/inbound-compat.md b/structure/data-planes/inbound-compat.md index 24f050c6df6..00135946d9d 100644 --- a/structure/data-planes/inbound-compat.md +++ b/structure/data-planes/inbound-compat.md @@ -1,5 +1,10 @@ # Inbound Compatibility Surfaces +Native Chat/Messages and Claude passthrough use the +[local timeout contract](../transports/timeouts.md): +local JSON inactivity is independent of header arrival and total generation; public JSON defaults +retain their previous allowance. No disabled header clock detaches client cancellation. + Thinking enablement is separate from strength: `src/responses/schema.ts` accepts the OCX `reasoning.enabled` boolean and `src/responses/parser.ts` retains it as `options.thinkingEnabled`, independently of `options.reasoning`. Chat `thinking.type` and Anthropic `thinking.type` preserve this choice through their inbound translators; the existing Anthropic disabled-effort sentinel remains for legacy destinations. `src/clients/reasoning-controls.ts` decodes effort-only client aliases for declared toggle-only models before strength caps run. Route fallback restores the original choice alongside effort in `src/server/effort-policy.ts`; a cleared strength list does not clear a thinking switch. Regression coverage lives in `tests/responses/responses-parser.test.ts`, `tests/responses/chat-inbound-reasoning-none.test.ts`, `tests/claude-integration/claude-inbound.test.ts`, and `tests/codex-integration/effort-policy.test.ts`. The names used for these paths (native, translated, legacy bridge) and the declared per-feature diff --git a/structure/data-planes/protocol-paths.md b/structure/data-planes/protocol-paths.md index 0aaef5c5dea..68ae998da50 100644 --- a/structure/data-planes/protocol-paths.md +++ b/structure/data-planes/protocol-paths.md @@ -220,7 +220,7 @@ Messages wires without the internal Responses SSE. It is server side and not a l `adapter-events.ts` is the driver: it ports the item state machine of `bridgeToResponsesSSE`, including `tool_call_end.incomplete`, which closes an unfinished item without claiming argument completion (item boundaries, signature grouping, hidden and redacted reasoning envelopes, tool naming and argument gating, the integral-float repair, every terminal with its usage and durability rule, -the wire-silence heartbeat and stall watchdog, pull-based stepping, cancellation) and calls one +the wire-silence heartbeat and live-destination stall watchdog ([timeout policy](../transports/timeouts.md)), pull-based stepping, cancellation) and calls one `ClientWireWriter` method wherever the bridge would emit a frame a client converter reads. `chat.ts` (`encodeChatCompletionSse`, `foldChatCompletion`) and `messages.ts` (`encodeAnthropicMessageSse`, `foldAnthropicMessage`) are the writers; the Chat one also emits `reasoning_details` ([#7429](chat-reasoning-details.md)). They reuse the diff --git a/structure/manifest.json b/structure/manifest.json index aa42a4006f4..cdee9474d8e 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -47,6 +47,23 @@ } ], "docs": [ + { + "path": "transports/timeouts.md", + "tier": 3, + "title": "Upstream Timeouts", + "scope": "Local header and JSON body deadlines, cancellation and optional OpenCode timeout export.", + "documents": [ + "src/stall-timeout.ts", + "src/lib/abort.ts", + "src/lib/local-upstream.ts", + "src/lib/bounded-body.ts", + "src/server/", + "src/web-search/header-deadline.ts", + "src/clients/config-export.ts", + "src/config/schema/leaf-validators.ts", + "src/types/config.ts" + ] + }, { "path": "quota-snapshots.md", "tier": 4, diff --git a/structure/runtime.md b/structure/runtime.md index 31fa9dd7c8c..2fa9ccdb795 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -4,7 +4,7 @@ The minute sweep checks persisted activation deadlines locally; only missing dea ## Bun runtime and test runner -`package.json` pins both the shipped npm runtime (`dependencies.bun`) and the test runner (`testRunnerBun`) to 1.4.4; the roles remain independently validated and `@types/bun` follows the runtime pin. Runtime 1.4.2 fixed #6684's Windows Bun.fetch truncation after the first cleartext upstream body chunk; 1.4.4 keeps that fix and ships NSS 3.131.1 roots, which drop the compromised eMudhra CAs. +See [local timeout policy and OpenCode export](transports/timeouts.md). `package.json` pins both the shipped npm runtime (`dependencies.bun`) and the test runner (`testRunnerBun`) to 1.4.4; the roles remain independently validated and `@types/bun` follows the runtime pin. Runtime 1.4.2 fixed #6684's Windows Bun.fetch truncation after the first cleartext upstream body chunk; 1.4.4 keeps that fix and ships NSS 3.131.1 roots, which drop the compromised eMudhra CAs. The runner pin skipped 1.4.2's multi-file `bun test --isolate` preload reload segfault (`load_preloads -> JSModuleLoader::loadModule -> JSPromise::status`, address 0x10): #4064 introduced 1.4.2 on 2026-09-09 and #4821 reverted it on 2026-09-17. It left 1.4.0 for 1.4.3 because of oven-sh/bun#44581 (#4956). Bun 1.4.0's `Bun.spawnSync` pointed the VM at its private wait loop, so a finalizer that ran during one call (an earlier file's stdout/stderr sink, a stream reader) released its poll on that loop; the cached loop's poll count drifted to zero and a later call stopped polling for its child, spinning past its own `timeout` until the 120-second batch deadline. In a multi-file `--isolate` process that depends on what earlier files left to finalize, so single-file attribution stayed clean. `tests/ci-workflows/test-runner-spawnsync-finalizer.test.ts` reproduces the mechanism in a child under a bounded parent and fails on 1.4.0. #6713 run 37631558263 reproduces that 1.4.2 preload segfault on Linux 1/4, macOS 2/2 and Windows 9/9; single-file success does not prove the multi-file runner works. diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index 698b7872d4e..d0ea8de7b71 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -1,5 +1,9 @@ # Transport Inventory +Shared header helpers and bounded local JSON reads follow the +[local timeout contract](timeouts.md). Disabling +their clocks retains byte caps, cancellation, manual redirects and transport credential fences. + WorkBuddy AI uses bounded fixed-origin login requests and shared Chat inference transport; see the [WorkBuddy contract](../providers/workbuddy.md). OpenCode Go's Gemini model uses the [Go Gemini transport contract](../providers/google.md#opencode-go-gemini-transport) through the Google adapter and the default server fetch path. diff --git a/structure/transports/responses.md b/structure/transports/responses.md index a2070b14a44..810c8d782e4 100644 --- a/structure/transports/responses.md +++ b/structure/transports/responses.md @@ -11,7 +11,7 @@ When a successful streamed native response has a missing or unrecognized non-JSO ## Responses HTTP/SSE -HTTP ingress applies the [caller header contract](../data-planes/inbound-compat.md#http-caller-conversation-identity) before admission and dispatch. Optional Codex memory selection enters `src/server/responses/request-prepare.ts` for HTTP and WebSocket frames, as specified in [memory phase routing](responses-failover.md#memory-phase-routing); an unset phase keeps its existing route and starts no background work. Responses request preparation stabilizes incoming `` under `skills.catalog_refresh`: `per_session` (default) reuses the first received catalog for a conversation; `per_turn` leaves the supplied catalog unchanged. Other instruction sections and user/tool content remain untouched. Requests without a reliable conversation identity bypass snapshots; shared prompt-cache cohorts are not conversation identities. Only a body with exactly one catalog block across its instructions and developer/system content takes part; two or more pass through unchanged. A known snapshot is substituted before parsing, but a new catalog is stored only when preparation reaches its success return, so a request rejected by parsing or admission pins nothing. Without a named principal, snapshots are shared by conversation id only on a server that requires no data-plane auth. Snapshots are process-local, expire after four idle hours, and use bounded LRU retention; oversized blocks bypass caching. The dashboard's `src/codex/prompt-layers.ts` and `src/codex/prompt-text-probe.ts` continue observing current files for previews and do not own session snapshots. +See [local timeout policy and OpenCode export](timeouts.md). HTTP ingress applies the [caller header contract](../data-planes/inbound-compat.md#http-caller-conversation-identity) before admission and dispatch. Optional Codex memory selection enters `src/server/responses/request-prepare.ts` for HTTP and WebSocket frames, as specified in [memory phase routing](responses-failover.md#memory-phase-routing); an unset phase keeps its existing route and starts no background work. Responses request preparation stabilizes incoming `` under `skills.catalog_refresh`: `per_session` (default) reuses the first received catalog for a conversation; `per_turn` leaves the supplied catalog unchanged. Other instruction sections and user/tool content remain untouched. Requests without a reliable conversation identity bypass snapshots; shared prompt-cache cohorts are not conversation identities. Only a body with exactly one catalog block across its instructions and developer/system content takes part; two or more pass through unchanged. A known snapshot is substituted before parsing, but a new catalog is stored only when preparation reaches its success return, so a request rejected by parsing or admission pins nothing. Without a named principal, snapshots are shared by conversation id only on a server that requires no data-plane auth. Snapshots are process-local, expire after four idle hours, and use bounded LRU retention; oversized blocks bypass caching. The dashboard's `src/codex/prompt-layers.ts` and `src/codex/prompt-text-probe.ts` continue observing current files for previews and do not own session snapshots. `/v1/responses` is the main Codex-facing endpoint. The server parses Responses input, routes to a provider, lets the selected adapter speak the upstream protocol, then bridges adapter events back to Responses-compatible streaming output. Every Responses serializer projects unreferenced hosted-search history when the final body declares no hosted search tool; with search declared, the canonical Codex ordinary serializer retains structural hosted history by default, and only selected-provider `projectResponsesLiteHostedSearchHistory === true` plus exact canonical `/responses` and explicit Lite intent enables model-independent projection under the [hosted-search history boundary](../providers-and-adapters.md#hosted-search-continuation-binding), shared by HTTP/SSE and WS. The routed provider name follows every adapter build, including retries and continuations; only `github-copilot` injects a configured `modelContextTiers` value as upstream `contextTier`, while unconfigured passthrough retains the caller field. For an opted-in key-auth provider, a hosted-search continuation stays bound to the API-key selection that served the first leg; the contract is the [hosted-search continuation binding](../providers-and-adapters.md#hosted-search-continuation-binding). The native passthrough undeclared-tool guard fails closed on the Responses inbound wire in SSE and buffered JSON before continuation state is stored. Chat and Anthropic inbounds defer to the caller's own registry except an explicit empty catalog; the [declared-tool contract](responses-wire-shapes.md#declared-tool-membership-by-inbound-wire) also covers managed native, translated, directly encoded, and final media output while preserving internal hosted operations. GitHub Copilot Auto admission rechecks the negotiated provider/model against the key scope captured at request admission (a key edited or removed mid-request cannot widen it) before route mutation or dispatch, on initial and renewal negotiation; denial returns the standard `model_not_allowed_for_key` 403 naming the caller's inbound selector without disclosing the destination. The web-search and image sidecars answer a denial during their credential rotation with that 403 instead of the original 429, and a combo child's denial keeps the same envelope. Local model-scope refusals are non-replayable. A denial during Copilot dispatch renewal propagates as an admission exception to the request boundary, never as a provider fetch response; account recovery and combo/policy fallback cannot continue after that denial. A committed continuation, including OAuth account rotation after a 429, emits the same refusal type in-stream without dispatching another inference; ordinary negotiation failures still preserve the original provider refusal. diff --git a/structure/transports/streaming-health.md b/structure/transports/streaming-health.md index 9f005af61cb..b9ec2c7a0b8 100644 --- a/structure/transports/streaming-health.md +++ b/structure/transports/streaming-health.md @@ -21,7 +21,7 @@ Responses WebSocket terminal frames normalize known `incomplete_details.reason` ## Heartbeat and stall deadline -RunTurn search interception forwards adapter heartbeats immediately and emits progress for buffered semantic events. Its bridge uses the search plan's stall deadline so a bounded sidecar search fits the watchdog budget. +Header and JSON policy follow [local deadlines](timeouts.md). Responses and direct-client watchdogs read live serving locality across refreshed continuation legs and reset silence ticks on locality changes. RunTurn search interception forwards adapter heartbeats immediately and emits progress for buffered semantic events. Its bridge uses the search plan's stall deadline so a bounded sidecar search fits the watchdog budget. Native Chat uses the same resolved `stallTimeoutSec` with a pending-upstream-read allowance that pauses under downstream backpressure. Its Chat error and cancellation contract is documented in diff --git a/structure/transports/timeouts.md b/structure/transports/timeouts.md new file mode 100644 index 00000000000..4e77b7f02e6 --- /dev/null +++ b/structure/transports/timeouts.md @@ -0,0 +1,48 @@ +# Response-header and local JSON deadlines + +`src/stall-timeout.ts` resolves an unset inference header budget to 200,000 ms for public +upstreams and zero for local ones. Explicit positive `connectTimeoutMs` applies to either; +non-positive disables the header timer. Responses adapter/passthrough, native Chat/Messages, +Claude passthrough and direct compact use this policy. `src/lib/abort.ts` keeps parent cancellation +linked after clearing or disabling the header timer; `src/server/claude-messages.ts` retains dev's +`fetchWithHeaderDeadline` success/error cleanup and timeout provenance. Local pacing admission +still pauses positive header clocks; `src/web-search/header-deadline.ts` distinguishes disabled +budgets from exhausted positive budgets. Default image/search sidecar deadlines stay unchanged. +Native Chat/Messages, non-forward Responses (including adapter retries/continuations), and keyed +compact start header clocks at the final send after pacing and credential/destination refresh. +Header and subsequent body policy follow the serving URL across local/public transitions. +Responses SSE and direct Chat/Messages encoder watchdogs read that live locality each beat and +reset silence ticks when it changes, including terminal continuations and signature repair. +`src/adapters/command-code.ts` honors non-positive budgets via the clearable deadline; +`src/adapters/kiro/adapter.ts` preserves the fetch budget through its internal text fallback. +`src/server/responses/fetch-helpers.ts` awaits admission only when the executor has a pacing seam, +so wrapping an unpaced physical send adds no yield between final credential validation and dispatch. + +`src/lib/local-upstream.ts` classifies HTTP(S) destinations without DNS: exact case-insensitive +`localhost` (optional trailing dot), unspecified IPv4 `0.0.0.0`, IPv4 loopback/private/link-local, +IPv6 loopback/link-local/unique-local and IPv4-mapped equivalents, plus `.local`/`.lan` names. +Other names retain public deadlines even if DNS resolves them to loopback; malformed destinations +and unsupported schemes retain safety clocks. This is timeout policy, never network authorization. + +Native Chat/Messages local JSON reads use `stallTimeoutSec` only as an inactivity budget, with no +five-second floor or stall-derived total deadline. Client cancellation and enclosing overall +deadlines remain authoritative. Public JSON reads retain their legacy total/inactivity budget, +`max(connectTimeoutMs ?? 200000, 5000)`. `src/lib/bounded-body.ts` arms no non-positive body timer; +byte caps and cancellation remain active. Compact body reads retain their separate 300-second +default even locally. Unset local header and stall clocks do not create an overall bound before +headers. `tests/responses/local-upstream-deadlines.test.ts` covers dispatch, compact and native JSON +policy, public header expiry, client/overall cancellation and explicit local body bounds. +`tests/responses/refreshed-upstream-deadlines.test.ts` covers queued Responses destination changes; +`tests/providers/kiro/kiro-fallback-error-body.test.ts` checks disabled and positive fallback budgets. + +## OpenCode timeout export + +`src/types/config.ts` declares `opencodeClientExport`; `src/config/schema/leaf-validators.ts` +accepts positive milliseconds or `false` and drops malformed fields or blocks. + +`src/clients/config-export.ts` emits only configured `opencodeClientExport` fields into the +generated V1 provider `options` and V2 provider `settings`: `headerTimeoutMs` maps to +`headerTimeout`, `chunkTimeoutMs` to `chunkTimeout`, and `timeoutMs` to `timeout`. Positive +milliseconds or `false` pass through; unset fields retain the client's native defaults. These +are provider-wide settings, including public and local models multiplexed through that provider. +The builder remains pure; managed writes retain the existing ownership and snapshot rules. diff --git a/tests/config/client-config-export.test.ts b/tests/config/client-config-export.test.ts index 5cf380d8e33..da6d8969175 100644 --- a/tests/config/client-config-export.test.ts +++ b/tests/config/client-config-export.test.ts @@ -25,6 +25,7 @@ import { import { buildOpencodeProviderBlockFromCatalog, opencodeCatalogFromProxyRows, opencodeGlobalConfigPath } from "../../src/cli/opencode"; import { exportModelsFromProxyRows } from "../../src/cli/export-command"; import type { OcxConfig } from "../../src/types"; +import { configSchema } from "../../src/config/schema/config-schema"; import { removeTreeWithRetry } from "../helpers/remove-tree"; import * as facade from "../../src/clients/config-export"; import { OPENCODE_PROVIDER_BLOCK_DEFAULT_CONFIG as leafDefaultConfig } from "../../src/clients/config-export/constants"; @@ -794,6 +795,45 @@ describe("hub-resolved Fast exports", () => { expect(remote.v2.settings).toEqual(remote.v1.options); }); + test("opencodeClientExport timeouts are omitted by default and emitted into both blocks when set", () => { + const off = opencodeProviderBlocks(BASE_URL, [eligible], cfg({ fastRows: false })); + for (const conn of [off.v1.options, off.v2.settings]) { + expect(conn).not.toHaveProperty("headerTimeout"); + expect(conn).not.toHaveProperty("chunkTimeout"); + expect(conn).not.toHaveProperty("timeout"); + } + const full = opencodeProviderBlocks(BASE_URL, [eligible], cfg({ + fastRows: false, + opencodeClientExport: { headerTimeoutMs: false, chunkTimeoutMs: 1_800_000, timeoutMs: 3_600_000 }, + } as Partial)); + const expected = { baseURL: BASE_URL, apiKey: OPENCODE_API_KEY_ENV_REF, headerTimeout: false, chunkTimeout: 1_800_000, timeout: 3_600_000 }; + expect(full.v1.options).toEqual(expected); + expect(full.v2.settings).toEqual(expected); + + // A partial block emits only the fields the operator set. + const partial = opencodeProviderBlocks(BASE_URL, [eligible], cfg({ + fastRows: false, + opencodeClientExport: { headerTimeoutMs: false }, + } as Partial)); + expect(partial.v1.options).toEqual({ baseURL: BASE_URL, apiKey: OPENCODE_API_KEY_ENV_REF, headerTimeout: false }); + expect(partial.v2.settings).toEqual(partial.v1.options); + }); + + test("configSchema keeps valid opencodeClientExport timeouts and degrades malformed per-field values", () => { + const base = { port: 10100, defaultProvider: "mock", providers: { mock: { adapter: "openai-chat", baseUrl: "https://api.example.com/v1" } } }; + const valid = configSchema.parse({ ...base, opencodeClientExport: { headerTimeoutMs: false, chunkTimeoutMs: 1_800_000, timeoutMs: 3_600_000 } }); + expect(valid.opencodeClientExport).toEqual({ headerTimeoutMs: false, chunkTimeoutMs: 1_800_000, timeoutMs: 3_600_000 }); + + // A malformed per-field value is dropped (opencode keeps its native wait) without failing the + // whole parse or taking the sibling fields down with it. + const salvaged = configSchema.parse({ ...base, opencodeClientExport: { headerTimeoutMs: "nope", chunkTimeoutMs: -5, timeoutMs: 600_000 } }); + expect(salvaged.opencodeClientExport).toEqual({ timeoutMs: 600_000 }); + + // An entirely malformed block degrades to undefined rather than rejecting the config. + const dropped = configSchema.parse({ ...base, opencodeClientExport: 42 }); + expect(dropped.opencodeClientExport).toBeUndefined(); + }); + test("each generation owns its modalities map, so an edit to one cannot move the other", () => { const blocks = opencodeProviderBlocks(BASE_URL, [eligible], cfg({ fastRows: false })); blocks.v1.models["remote/model"]!.modalities!.input.push("audio"); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 3b3f402d598..f52d4c7e460 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1,4 +1,4 @@ -{ "service-desktop-recovery.test.ts": "service", "provider-input-modalities.test.ts": "providers", "turn-membership.test.ts": "responses", "turn-membership-ingress.test.ts": "responses", "workbuddy-quota.test.ts": "providers", "workbuddy-login.test.ts": "oauth", "workbuddy-adapter.test.ts": "adapters", "workbuddy-provider.test.ts": "providers", "link-tunnel-history.test.ts": "clients", "link-readiness.test.ts": "clients", "link-diagnostics.test.ts": "clients", "combo-description.test.ts": "routing", "combo-description-catalog.test.ts": "codex-integration", "catalog-auto-review-override.test.ts": "codex-integration", "management-auto-review-settings.test.ts": "server", "management-auto-review-auth.test.ts": "server", "prefer-model-alias-ids.test.ts": "server", "responses-mimo-envelope.test.ts": "responses", "responses-call-id-linkage.test.ts": "responses", "responses-phase-inference.test.ts": "responses", +{ "refreshed-upstream-deadlines.test.ts": "responses", "local-upstream-deadlines.test.ts": "responses", "service-desktop-recovery.test.ts": "service", "provider-input-modalities.test.ts": "providers", "turn-membership.test.ts": "responses", "turn-membership-ingress.test.ts": "responses", "workbuddy-quota.test.ts": "providers", "workbuddy-login.test.ts": "oauth", "workbuddy-adapter.test.ts": "adapters", "workbuddy-provider.test.ts": "providers", "link-tunnel-history.test.ts": "clients", "link-readiness.test.ts": "clients", "link-diagnostics.test.ts": "clients", "combo-description.test.ts": "routing", "combo-description-catalog.test.ts": "codex-integration", "catalog-auto-review-override.test.ts": "codex-integration", "management-auto-review-settings.test.ts": "server", "management-auto-review-auth.test.ts": "server", "prefer-model-alias-ids.test.ts": "server", "responses-mimo-envelope.test.ts": "responses", "responses-call-id-linkage.test.ts": "responses", "responses-phase-inference.test.ts": "responses", "t3code-client.test.ts": "clients", "t3code-client-lifecycle.test.ts": "clients", "fix-jsonc-cr-comments.test.ts": "lib", diff --git a/tests/lib/clearable-deadline.test.ts b/tests/lib/clearable-deadline.test.ts index 156f5ad0037..1a4b732c651 100644 --- a/tests/lib/clearable-deadline.test.ts +++ b/tests/lib/clearable-deadline.test.ts @@ -43,4 +43,23 @@ describe("clearableDeadline", () => { expect(deadline.didExpire()).toBe(false); deadline.clear(); }); + + test("a non-positive budget arms no header clock; only the parent can abort", async () => { + // Disabled header deadline for a local upstream: prefill may stay silent for minutes, so the + // deadline must never fire on its own. resolveConnectTimeoutMs hands 0 for this case. + for (const ms of [0, -1]) { + const parent = new AbortController(); + const deadline = clearableDeadline(ms, parent.signal); + await delay(20); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.didExpire()).toBe(false); + + const reason = new DOMException("client closed", "AbortError"); + parent.abort(reason); + expect(deadline.signal.aborted).toBe(true); + expect(deadline.signal.reason).toBe(reason); + expect(deadline.didExpire()).toBe(false); + deadline.clear(); + } + }); }); diff --git a/tests/lib/local-upstream.test.ts b/tests/lib/local-upstream.test.ts index d2110916b9c..22275823b6d 100644 --- a/tests/lib/local-upstream.test.ts +++ b/tests/lib/local-upstream.test.ts @@ -2,6 +2,25 @@ import { describe, expect, test } from "bun:test"; import { isLocalUpstream } from "../../src/lib/local-upstream"; describe("isLocalUpstream", () => { + test.each([ + "ftp://localhost/v1", "file://localhost/v1", "[::1]public.example", + "[::1]:invalid", "localhost:invalid", "localhost:99999", "localhost:", "0.1.2.3", + ])("malformed or unsupported destination keeps safety clocks: %s", destination => { + expect(isLocalUpstream(destination)).toBe(false); + }); + + test.each(["http://[::ffff:127.0.0.1]:8000", "http://[::ffff:192.168.1.2]/v1", "::ffff:7f00:1", "0:0:0:0:0:ffff:127.0.0.1"])( + "classifies normalized IPv4-mapped local IPv6: %s", destination => { + expect(isLocalUpstream(destination)).toBe(true); + }, + ); + + test.each(["http://[::ffff:8.8.8.8]", "127.0.0.1.nip.io", "localhost.example", "foo.local.example", "model.internal"])( + "does not infer locality from DNS or a misleading suffix: %s", destination => { + expect(isLocalUpstream(destination)).toBe(false); + }, + ); + describe("loopback", () => { test("classifies localhost (any case, with or without a port)", () => { expect(isLocalUpstream("localhost")).toBe(true); diff --git a/tests/lib/stall-timeout.test.ts b/tests/lib/stall-timeout.test.ts index fe21bc099c2..68639483746 100644 --- a/tests/lib/stall-timeout.test.ts +++ b/tests/lib/stall-timeout.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test"; -import { DEFAULT_STALL_TIMEOUT_SEC, resolveStallTimeoutMs, resolveStallTimeoutSec } from "../../src/stall-timeout"; +import { DEFAULT_CONNECT_TIMEOUT_MS, DEFAULT_STALL_TIMEOUT_SEC, MIN_JSON_BODY_READ_MS, resolveConnectTimeoutMs, resolveJsonBodyReadTimeoutMs, resolveStallTimeoutMs, resolveStallTimeoutSec } from "../../src/stall-timeout"; describe("resolveStallTimeoutSec", () => { test("defaults to 300 seconds for a public upstream when unset", () => { @@ -58,3 +58,71 @@ describe("resolveStallTimeoutMs", () => { expect(resolveStallTimeoutMs(undefined, { localUpstream: true })).toBe(0); }); }); + +describe("resolveConnectTimeoutMs", () => { + test("defaults to the 200s header deadline for a public upstream", () => { + expect(resolveConnectTimeoutMs(undefined)).toBe(DEFAULT_CONNECT_TIMEOUT_MS); + expect(resolveConnectTimeoutMs(undefined)).toBe(200_000); + expect(resolveConnectTimeoutMs(undefined, { localUpstream: false })).toBe(200_000); + }); + + test("disables the header deadline for a local upstream when unset", () => { + // A large prompt can hold a local model (LM Studio/Ollama) silent for minutes during prefill; + // an unset budget must resolve to disabled so the header clock never cuts a healthy turn. + expect(resolveConnectTimeoutMs(undefined, { localUpstream: true })).toBe(0); + }); + + test("an explicit value wins verbatim for local and public alike", () => { + expect(resolveConnectTimeoutMs(45_000)).toBe(45_000); + expect(resolveConnectTimeoutMs(45_000, { localUpstream: true })).toBe(45_000); + // Fractional budgets ceil to at least 1ms, matching the stall resolver's shape. + expect(resolveConnectTimeoutMs(0.4)).toBe(1); + expect(resolveConnectTimeoutMs(0.4, { localUpstream: true })).toBe(1); + }); + + test("an explicit non-positive value disables everywhere", () => { + expect(resolveConnectTimeoutMs(0)).toBe(0); + expect(resolveConnectTimeoutMs(0, { localUpstream: false })).toBe(0); + expect(resolveConnectTimeoutMs(-5)).toBe(0); + expect(resolveConnectTimeoutMs(-5, { localUpstream: true })).toBe(0); + }); + + test("non-finite falls back to the destination default", () => { + expect(resolveConnectTimeoutMs(Number.NaN)).toBe(200_000); + expect(resolveConnectTimeoutMs(Number.POSITIVE_INFINITY)).toBe(200_000); + expect(resolveConnectTimeoutMs(Number.NaN, { localUpstream: true })).toBe(0); + }); +}); + +describe("resolveJsonBodyReadTimeoutMs", () => { + test("disables the body-read budget for a local upstream when unset", () => { + // The regression this guards: a local upstream resolves connect to 0, and the JSON reader used + // to derive its budget from that as Math.max(0, 5000) = 5s, cutting a slow local body. The + // body-read budget must follow the stall clock instead and resolve to disabled for local. + expect(resolveJsonBodyReadTimeoutMs(undefined, { localUpstream: true })).toBe(0); + }); + + test("public JSON default ignores the separate streaming stall setting", () => { + expect(resolveJsonBodyReadTimeoutMs(undefined)).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(undefined, { localUpstream: false })).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(1)).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(2)).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(30)).toBe(200_000); + }); + + test("public JSON reads retain the legacy connect-derived budget", () => { + expect(resolveJsonBodyReadTimeoutMs(0)).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(0, { localUpstream: false })).toBe(200_000); + expect(resolveJsonBodyReadTimeoutMs(-1, { localUpstream: true })).toBe(0); + }); + + test("an explicit local stall budget applies without the public five-second floor", () => { + expect(resolveJsonBodyReadTimeoutMs(1, { localUpstream: true })).toBe(1_000); + }); + + test("public JSON custom connect budgets preserve the legacy floor", () => { + expect(resolveJsonBodyReadTimeoutMs(90, {}, 12_345)).toBe(12_345); + expect(resolveJsonBodyReadTimeoutMs(0, {}, 0)).toBe(MIN_JSON_BODY_READ_MS); + expect(resolveJsonBodyReadTimeoutMs(0, {}, -1)).toBe(MIN_JSON_BODY_READ_MS); + }); +}); diff --git a/tests/providers/kiro/kiro-fallback-error-body.test.ts b/tests/providers/kiro/kiro-fallback-error-body.test.ts index 588e0f3f2b3..48cadc14e28 100644 --- a/tests/providers/kiro/kiro-fallback-error-body.test.ts +++ b/tests/providers/kiro/kiro-fallback-error-body.test.ts @@ -1,4 +1,4 @@ -import { afterEach, beforeEach, expect, test } from "bun:test"; +import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; import { mkdtempSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -79,6 +79,31 @@ async function collectAdapterEvents(events: AsyncGenerator): Promi return out; } +test.each([0, -1, 17_013])("text fallback retains the current fetch header budget %s", async timeoutMs => { + const parent = new AbortController(); + const clocks: number[] = []; + const original = globalThis.setTimeout; + const timerSpy = spyOn(globalThis, "setTimeout").mockImplementation(((callback, delay, ...args) => { + if (delay === 200_000 || delay === 17_013) clocks.push(delay); + return original(callback, delay, ...args); + }) as typeof setTimeout); + let sends = 0; + const executor = (async (_input, init) => { + expect(init!.signal!.aborted).toBe(false); + sends++; + return new Response(streamOf(eventFrame({ content: sends === 1 ? "I am checking." : "fixture reply" }))); + }) as typeof fetch; + try { + const adapter = createKiroAdapter(provider); + const request = await adapter.buildRequest(parsedWith([{ role: "user", content: "do it" }], [bashTool])); + const response = await adapter.fetchResponse!(request, { timeoutMs, abortSignal: parent.signal, executor }); + const events = await collectAdapterEvents(adapter.parseStream(response)); + expect(sends).toBe(2); + expect(clocks).toEqual(timeoutMs > 0 ? [timeoutMs, timeoutMs] : []); + expect(events.some(event => event.type === "text_delta" && event.text.includes("fixture reply"))).toBe(true); + } finally { parent.abort(); timerSpy.mockRestore(); } +}); + test("fallback HTTP errors stop reading oversized upstream bodies", async () => { const chunk = new TextEncoder().encode("A".repeat(32 * 1024)); let pulls = 0; diff --git a/tests/responses/local-upstream-deadlines.test.ts b/tests/responses/local-upstream-deadlines.test.ts new file mode 100644 index 00000000000..ae80ee1d1d0 --- /dev/null +++ b/tests/responses/local-upstream-deadlines.test.ts @@ -0,0 +1,336 @@ +import { expect, spyOn, test } from "bun:test"; +import { fetchWithHeaderTimeout } from "../../src/server/responses/fetch-helpers"; +import { fetchWithHeaderDeadline } from "../../src/server/claude-messages"; +import { clearableDeadline } from "../../src/lib/abort"; +import { readBoundedResponseBody } from "../../src/lib/bounded-body"; +import { isLocalUpstream } from "../../src/lib/local-upstream"; +import { resolveConnectTimeoutMs, resolveJsonBodyReadTimeoutMs } from "../../src/stall-timeout"; +import { handleResponsesCompact } from "../../src/server/responses"; +import { handleChatCompletions } from "../../src/server/chat-completions"; +import { handleClaudeMessages } from "../../src/server/claude-messages"; +import type { OcxConfig } from "../../src/types"; +import { saveConfig } from "../../src/config"; +import { providerRequestPacingStatus, resetProviderRequestPacingForTest, waitForProviderRequestSlot } from "../../src/providers/request-pacing"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { compactionRequest, keyProviderConfig } from "../helpers/compaction-routing-fixtures"; +import { createCommandCodeAdapter } from "../../src/adapters/command-code"; + +// Advance only the policy's clock, without a 200-second sleep or host-speed assumptions. +function clockAt(ms: number) { + const callbacks: Array<() => void> = []; + const timers: Array> = []; + const original = globalThis.setTimeout; + const spy = spyOn(globalThis, "setTimeout").mockImplementation(((callback: () => void, delay?: number, ...args: unknown[]) => { + if (delay === ms) { + callbacks.push(callback); + const timer = original(() => {}, 2_000_000); + timers.push(timer); + return timer; + } + return original(callback, delay, ...args); + }) as typeof setTimeout); + return { callbacks, restore: () => { spy.mockRestore(); timers.forEach(clearTimeout); } }; +} + +test.each([0, -1])("Command Code disabled header budget %s keeps pending headers and parent cancellation", async timeoutMs => { + const clock = clockAt(timeoutMs); + const parent = new AbortController(); + const entered = Promise.withResolvers(); + const executor = (async (_input, init) => { + const signal = init!.signal!; + entered.resolve(signal); + return new Promise((_resolve, reject) => signal.addEventListener("abort", () => reject(signal.reason), { once: true })); + }) as typeof fetch; + const adapter = createCommandCodeAdapter({ baseUrl: "http://127.0.0.1:8000", apiKey: "fixture" }); + const pending = adapter.fetchResponse!({ url: "http://127.0.0.1:8000/alpha/generate", method: "POST", headers: {}, body: "{}" }, + { timeoutMs, abortSignal: parent.signal, executor }).catch(error => error); + try { + const signal = await entered.promise; + expect(clock.callbacks).toHaveLength(0); + expect(signal.aborted).toBe(false); + const reason = new DOMException("client cancelled", "AbortError"); + parent.abort(reason); + expect(await pending).toBe(reason); + } finally { parent.abort(); await pending; clock.restore(); } +}); + +test.each([undefined, 17_011])("Command Code public header budget %s rejects pending headers and clears on success", async timeoutMs => { + const clock = clockAt(timeoutMs ?? 200_000); + const entered = Promise.withResolvers(); + const parent = new AbortController(); + const executor = (async (_input, init) => { + const signal = init!.signal!; + entered.resolve(signal); + return new Promise((_resolve, reject) => signal.addEventListener("abort", () => reject(signal.reason), { once: true })); + }) as typeof fetch; + const adapter = createCommandCodeAdapter({ baseUrl: "https://gateway.example", apiKey: "fixture" }); + const request = { url: "https://gateway.example/alpha/generate", method: "POST", headers: {}, body: "{}" }; + const pending = adapter.fetchResponse!(request, { timeoutMs, abortSignal: parent.signal, executor }).catch(error => error); + try { + await entered.promise; + expect(clock.callbacks).toHaveLength(1); + clock.callbacks[0]!(); + expect(await pending).toMatchObject({ name: "TimeoutError" }); + let bodySignal!: AbortSignal; + const clear = spyOn(globalThis, "clearTimeout"); + try { + const result = await adapter.fetchResponse!(request, { timeoutMs, abortSignal: parent.signal, + executor: (async (_input, init) => { bodySignal = init!.signal!; expect(init!.redirect).toBe("manual"); return new Response("fixture"); }) as typeof fetch }); + expect(await result.text()).toBe("fixture"); + expect(clear).toHaveBeenCalled(); + expect(bodySignal.aborted).toBe(false); + parent.abort(); + expect(bodySignal.aborted).toBe(true); + } finally { clear.mockRestore(); } + } finally { parent.abort(); await pending; clock.restore(); } +}); + +test("local header wait retains an enclosing overall deadline", async () => { + const clock = clockAt(17_009); + const overall = clearableDeadline(17_009); + const entered = Promise.withResolvers(); + const executor = (async (_input, init) => { + const signal = init!.signal!; + entered.resolve(); + return new Promise((_resolve, reject) => signal.addEventListener("abort", () => reject(signal.reason), { once: true })); + }) as typeof fetch; + try { + const pending = fetchWithHeaderTimeout("http://127.0.0.1/v1", {}, overall.signal, + resolveConnectTimeoutMs(undefined, { localUpstream: true }), false, executor).catch(error => error); + await entered.promise; + expect(clock.callbacks).toHaveLength(1); + clock.callbacks[0]!(); + expect(await pending).toBe(overall.timeoutReason); + } finally { overall.clear(); clock.restore(); } +}); + +test.each(["responses", "claude"] as const)("%s public default rejects a pending header wait", async wire => { + const clock = clockAt(200_000); + const entered = Promise.withResolvers(); + const executor = (async (_input, init) => { + const signal = init!.signal!; + entered.resolve(); + return new Promise((_resolve, reject) => signal.addEventListener("abort", () => reject(signal.reason), { once: true })); + }) as typeof fetch; + const url = "https://public.example/v1"; + const ms = resolveConnectTimeoutMs(undefined, { localUpstream: isLocalUpstream(url) }); + try { + const pending = wire === "responses" + ? fetchWithHeaderTimeout(url, {}, new AbortController().signal, ms, false, executor).catch(error => error) + : fetchWithHeaderDeadline(url, {}, ms, undefined, undefined, executor); + await entered.promise; + expect(clock.callbacks).toHaveLength(1); + clock.callbacks[0]!(); + const result = await pending; + if (wire === "responses") expect(result).toMatchObject({ name: "TimeoutError" }); + else expect(result).toEqual({ kind: "timeout" }); + } finally { clock.restore(); } +}); + +test.each(["responses", "claude"] as const)("%s disabled local header clock retains cancellation", async wire => { + const parent = new AbortController(); + const clock = clockAt(200_000); + const entered = Promise.withResolvers(); + const reason = new DOMException("client cancelled", "AbortError"); + const executor = (async (_input, init) => { + const signal = init!.signal!; + entered.resolve(); + return new Promise((_resolve, reject) => signal.addEventListener("abort", () => reject(signal.reason), { once: true })); + }) as typeof fetch; + try { + const ms = resolveConnectTimeoutMs(undefined, { localUpstream: true }); + const pending = wire === "responses" + ? fetchWithHeaderTimeout("http://127.0.0.1/v1", {}, parent.signal, ms, false, executor).catch(error => error) + : fetchWithHeaderDeadline("http://127.0.0.1/v1", {}, ms, parent.signal, undefined, executor); + await entered.promise; + expect(clock.callbacks).toHaveLength(0); + parent.abort(reason); + const result = await pending; + if (wire === "responses") expect(result).toBe(reason); + else expect(result).toEqual({ kind: "error", error: reason }); + } finally { clock.restore(); } +}); + +test.each(["total", "stall", "client"] as const)("local body with no header deadline retains %s bound", async cause => { + const budget = resolveJsonBodyReadTimeoutMs(cause === "stall" ? 1 : undefined, { localUpstream: true }); + const clock = clockAt(cause === "total" ? 17_003 : 1_000); + const parent = new AbortController(); + let cancelled = false; + const body = new ReadableStream({ cancel() { cancelled = true; } }); + const reason = new DOMException("client cancelled", "AbortError"); + try { + const pending = readBoundedResponseBody(new Response(body), { + signal: parent.signal, totalTimeoutMs: cause === "total" ? 17_003 : 0, inactivityTimeoutMs: budget, + }).catch(error => error); + if (cause === "client") parent.abort(reason); + else { + expect(clock.callbacks).toHaveLength(1); + clock.callbacks[0]!(); + } + const result = await pending; + if (cause === "client") expect(result).toBe(reason); + else expect(result).toMatchObject({ timedOut: true, totalTimedOut: cause === "total", inactivityTimedOut: cause === "stall" }); + expect(cancelled).toBe(true); + expect(body.locked).toBe(false); + } finally { parent.abort(reason); clock.restore(); } +}); + +test.each(["http://127.0.0.1:8000/v1", "https://gateway.example/v1"])("compact applies destination-aware header default: %s", async baseUrl => { + const release = acquireOwnedSpendHome(); + const clock = clockAt(200_000); + const originalFetch = globalThis.fetch; + const sends: string[] = []; + globalThis.fetch = (async input => { + sends.push(String(input)); + return Response.json({ object: "response.compaction", output: [{ type: "compaction", encrypted_content: "fixture" }] }); + }) as typeof fetch; + try { + const config = keyProviderConfig({ baseUrl, supportsNativeCompactEndpoint: true, allowPrivateNetwork: true, models: ["test"] }); + const response = await handleResponsesCompact(compactionRequest({ model: "gw/test", input: [{ role: "user", content: "fixture" }] }), config, { model: "", provider: "" }); + expect(response.status).toBe(200); + await response.text(); + expect(sends).toEqual([`${baseUrl}/responses/compact`]); + expect(clock.callbacks).toHaveLength(isLocalUpstream(baseUrl) ? 0 : 1); + } finally { globalThis.fetch = originalFetch; clock.restore(); release(); } +}); + +test.each(["chat", "messages"] as const)("native %s local JSON arms inactivity without a stall-derived total deadline", async wire => { + const release = acquireOwnedSpendHome(); + const clock = clockAt(1_000); + const originalFetch = globalThis.fetch; + const reading = Promise.withResolvers(); + const parent = new AbortController(); + let source!: ReadableStreamDefaultController; + globalThis.fetch = (async () => new Response(new ReadableStream({ + start(controller) { source = controller; }, pull() { reading.resolve(); }, + }, { highWaterMark: 0 }), { headers: { "content-type": "application/json" } })) as typeof fetch; + const config = { defaultProvider: "fixture", stallTimeoutSec: 1, + protocols: { rollout: { managedMessagesNative: true } }, providers: { fixture: { + adapter: wire === "chat" ? "openai-chat" : "anthropic", authMode: "key", apiKey: "fixture-key", + baseUrl: "http://127.0.0.1:8000/v1", models: ["fixture-model"], allowPrivateNetwork: true, + } } } as OcxConfig; + const request = new Request(`http://localhost/v1/${wire === "chat" ? "chat/completions" : "messages"}`, { + method: "POST", headers: { "content-type": "application/json" }, signal: parent.signal, + body: JSON.stringify({ model: "fixture/fixture-model", max_tokens: 10, stream: false, messages: [{ role: "user", content: "fixture" }] }), + }); + try { + const pending = wire === "chat" ? handleChatCompletions(request, config, { model: "", provider: "" }) + : handleClaudeMessages(request, config, { model: "", provider: "" }); + // Failure before body consumption must surface rather than leave a hanging test. + await Promise.race([reading.promise, pending.then(() => { throw new Error("native body was not read"); })]); + expect(clock.callbacks).toHaveLength(1); + source.enqueue(new TextEncoder().encode(JSON.stringify(wire === "chat" + ? { id: "fixture", choices: [{ message: { role: "assistant", content: "fixture reply" }, finish_reason: "stop" }] } + : { id: "fixture", type: "message", role: "assistant", content: [{ type: "text", text: "fixture reply" }], stop_reason: "end_turn", usage: { input_tokens: 1, output_tokens: 1 } }))); + source.close(); + const response = await pending; + expect(response.status).toBe(200); + expect(await response.text()).toContain("fixture reply"); + } finally { parent.abort(); globalThis.fetch = originalFetch; clock.restore(); release(); } +}); + +test.each([ + { initial: "http://127.0.0.1:8000/v1", final: "https://gateway.example/v1", clocks: 1 }, + { initial: "https://gateway.example/v1", final: "http://127.0.0.1:8000/v1", clocks: 0 }, +])("queued compact refresh applies final destination's header clock: $initial -> $final", async row => { + const release = acquireOwnedSpendHome(); + resetProviderRequestPacingForTest(); + const clock = clockAt(200_000); + const originalFetch = globalThis.fetch; + const parent = new AbortController(); + const config = keyProviderConfig({ baseUrl: row.initial, supportsNativeCompactEndpoint: true, + allowPrivateNetwork: true, models: ["test"], apiKeySelectionRevision: "initial", + requestPacing: { enabled: true, maxConcurrentRequests: 1 }, + }); + saveConfig(config); + const provider = config.providers.gw!; + const held = await waitForProviderRequestSlot("gw", provider, "test"); + const sends: string[] = []; + globalThis.fetch = (async input => { + sends.push(String(input)); + return Response.json({ object: "response.compaction", output: [{ type: "compaction", encrypted_content: "fixture" }] }); + }) as typeof fetch; + try { + const pending = handleResponsesCompact(compactionRequest({ model: "gw/test", input: [{ role: "user", content: "fixture" }] }, parent.signal), config, { model: "", provider: "" }); + for (let turn = 0; providerRequestPacingStatus("gw", provider).queued === 0; turn++) { + if (turn > 1000) throw new Error("compact never queued"); + await new Promise(resolve => setImmediate(resolve)); + } + config.providers.gw = { ...provider, baseUrl: row.final, apiKey: "fixture-refreshed-key", apiKeySelectionRevision: "refreshed" }; + saveConfig(config); + held.release(); + const response = await pending; + expect(response.status).toBe(200); + await response.text(); + expect(sends).toEqual([`${row.final}/responses/compact`]); + expect(clock.callbacks).toHaveLength(row.clocks); + } finally { + parent.abort(); held.release(); globalThis.fetch = originalFetch; + clock.restore(); resetProviderRequestPacingForTest(); release(); + } +}); + +for (const wire of ["chat", "messages"] as const) test.each([ + { initial: "http://127.0.0.1:8000/v1", final: "https://gateway.example/v1", clocks: 1 }, + { initial: "https://gateway.example/v1", final: "http://127.0.0.1:8000/v1", clocks: 0 }, +])(`queued native ${wire} refresh uses final header and JSON policy: $initial -> $final`, async row => { + const release = acquireOwnedSpendHome(); + resetProviderRequestPacingForTest(); + const clock = clockAt(200_000); + const originalFetch = globalThis.fetch; + const entered = Promise.withResolvers(); + const headers = Promise.withResolvers(); + const reading = Promise.withResolvers(); + const parent = new AbortController(); + let source!: ReadableStreamDefaultController; + const sends: string[] = []; + globalThis.fetch = (async input => { + sends.push(String(input)); + entered.resolve(); + await headers.promise; + return new Response(new ReadableStream({ + start(controller) { source = controller; }, pull() { reading.resolve(); }, + }, { highWaterMark: 0 }), { headers: { "content-type": "application/json" } }); + }) as typeof fetch; + const config: OcxConfig = { defaultProvider: "fixture", stallTimeoutSec: 1, + protocols: { rollout: { managedMessagesNative: true } }, providers: { fixture: { + adapter: wire === "chat" ? "openai-chat" : "anthropic", authMode: "key", apiKey: "fixture-key", + baseUrl: row.initial, models: ["fixture-model"], allowPrivateNetwork: true, + apiKeySelectionRevision: "initial", requestPacing: { enabled: true, maxConcurrentRequests: 1 }, + } } }; + saveConfig(config); + const provider = config.providers.fixture!; + const held = await waitForProviderRequestSlot("fixture", provider, "fixture-model"); + const request = new Request(`http://localhost/v1/${wire === "chat" ? "chat/completions" : "messages"}`, { + method: "POST", headers: { "content-type": "application/json" }, signal: parent.signal, + body: JSON.stringify({ model: "fixture/fixture-model", max_tokens: 10, stream: false, messages: [{ role: "user", content: "fixture" }] }), + }); + const pending = wire === "chat" ? handleChatCompletions(request, config, { model: "", provider: "" }) + : handleClaudeMessages(request, config, { model: "", provider: "" }); + try { + for (let turn = 0; providerRequestPacingStatus("fixture", provider).queued === 0; turn++) { + if (turn > 1000) throw new Error("native request never queued"); + await new Promise(resolve => setImmediate(resolve)); + } + config.providers.fixture = { ...provider, baseUrl: row.final, apiKey: "fixture-refreshed-key", apiKeySelectionRevision: "refreshed" }; + saveConfig(config); + held.release(); + await Promise.race([entered.promise, pending.then(() => { throw new Error("native request never sent"); })]); + expect(sends).toEqual([`${row.final}/${wire === "chat" ? "chat/completions" : "messages"}`]); + expect(clock.callbacks).toHaveLength(row.clocks); + headers.resolve(); + await Promise.race([reading.promise, pending.then(() => { throw new Error("native body was not read"); })]); + // Public JSON owns total + inactivity clocks as well as the completed header clock. + expect(clock.callbacks).toHaveLength(row.clocks * 3); + source.enqueue(new TextEncoder().encode(JSON.stringify(wire === "chat" + ? { id: "fixture", choices: [{ message: { role: "assistant", content: "fixture reply" }, finish_reason: "stop" }] } + : { id: "fixture", type: "message", role: "assistant", content: [{ type: "text", text: "fixture reply" }], stop_reason: "end_turn", usage: { input_tokens: 1, output_tokens: 1 } }))); + source.close(); + const response = await pending; + expect(response.status).toBe(200); + expect(await response.text()).toContain("fixture reply"); + } finally { + parent.abort(); headers.resolve(); held.release(); await pending.catch(() => {}); + globalThis.fetch = originalFetch; clock.restore(); resetProviderRequestPacingForTest(); release(); + } +}); diff --git a/tests/responses/refreshed-upstream-deadlines.test.ts b/tests/responses/refreshed-upstream-deadlines.test.ts new file mode 100644 index 00000000000..cf4f10f6374 --- /dev/null +++ b/tests/responses/refreshed-upstream-deadlines.test.ts @@ -0,0 +1,145 @@ +import { expect, spyOn, test } from "bun:test"; +import { saveConfig } from "../../src/config"; +import { handleResponses } from "../../src/server/responses"; +import { providerRequestPacingStatus, resetProviderRequestPacingForTest, waitForProviderRequestSlot } from "../../src/providers/request-pacing"; +import type { OcxConfig } from "../../src/types"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { clearKeyCooldowns } from "../../src/providers/key-failover"; + +for (const trigger of ["queued", "key-rotation"] as const) for (const adapter of ["openai-chat", "openai-responses", "command-code"] as const) test.each([ + { initial: "http://127.0.0.1:8000/v1", final: "https://gateway.example/v1", clocks: 1 }, + { initial: "https://gateway.example/v1", final: "http://127.0.0.1:8000/v1", clocks: 0 }, +])(`${trigger} Responses ${adapter} uses final header policy: $initial -> $final`, async row => { + const release = acquireOwnedSpendHome(); + resetProviderRequestPacingForTest(); + clearKeyCooldowns(); + const originalTimer = globalThis.setTimeout; + const timers: Array> = []; + const callbacks: Array<() => void> = []; + const timerSpy = spyOn(globalThis, "setTimeout").mockImplementation(((callback, delay, ...args) => { + if (delay !== 200_000) return originalTimer(callback, delay, ...args); + callbacks.push(callback); + const timer = originalTimer(() => {}, 2_000_000); + timers.push(timer); + return timer; + }) as typeof setTimeout); + const originalFetch = globalThis.fetch; + const entered = Promise.withResolvers(); + const headers = Promise.withResolvers(); + const parent = new AbortController(); + const sends: string[] = []; + globalThis.fetch = (async input => { + sends.push(String(input)); + if (trigger === "key-rotation" && sends.length === 1) { + config.providers.fixture = { ...config.providers.fixture!, baseUrl: row.final }; + saveConfig(config); + return Response.json({ error: { message: "fixture rejected credential" } }, { status: adapter === "openai-responses" ? 429 : 401 }); + } + entered.resolve(); await headers.promise; + if (adapter === "command-code") return new Response('{"type":"text-delta","text":"fixture reply"}\n{"type":"finish","finishReason":"stop"}\n'); + if (adapter === "openai-chat") return Response.json({ id: "fixture", choices: [{ message: { role: "assistant", content: "fixture reply" }, finish_reason: "stop" }] }); + return Response.json({ id: "fixture", object: "response", status: "completed", output: [{ type: "message", role: "assistant", content: [{ type: "output_text", text: "fixture reply" }] }] }); + }) as typeof fetch; + const config: OcxConfig = { defaultProvider: "fixture", providers: { fixture: { + adapter, authMode: "key", apiKey: "fixture-key", baseUrl: row.initial, + models: ["fixture-model"], allowPrivateNetwork: true, apiKeySelectionRevision: "initial", + ...(trigger === "key-rotation" ? { apiKeyPool: [{ id: "first", key: "fixture-key" }, { id: "second", key: "fixture-refreshed-key" }] } : {}), + requestPacing: { enabled: true, maxConcurrentRequests: 1 }, + } } }; + saveConfig(config); + const provider = config.providers.fixture!; + const held = trigger === "queued" ? await waitForProviderRequestSlot("fixture", provider, "fixture-model") : undefined; + const pending = handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", headers: { "content-type": "application/json" }, signal: parent.signal, + body: JSON.stringify({ model: "fixture/fixture-model", input: "fixture", stream: false }), + }), config, { model: "", provider: "" }); + let settled: Response | undefined; + void pending.then(response => { settled = response; }); + try { + while (trigger === "queued" && providerRequestPacingStatus("fixture", provider).queued === 0) { + if (settled) throw new Error(`never queued: ${settled.status} ${await settled.text()}`); + await new Promise(resolve => setImmediate(resolve)); + } + if (trigger === "queued") { + config.providers.fixture = { ...provider, baseUrl: row.final, apiKey: "fixture-refreshed-key", apiKeySelectionRevision: "refreshed" }; + saveConfig(config); held!.release(); + } + await Promise.race([entered.promise, pending.then(result => { throw new Error(`never dispatched: ${result.status}`); })]); + const suffix = adapter === "openai-chat" ? "/chat/completions" : adapter === "command-code" ? "/alpha/generate" : "/responses"; + expect(sends).toEqual([...(trigger === "key-rotation" ? [`${row.initial}${suffix}`] : []), `${row.final}${suffix}`]); + expect(callbacks).toHaveLength(row.clocks + (trigger === "key-rotation" && row.initial.startsWith("https:") ? 1 : 0)); + headers.resolve(); + const response = await pending; + const body = await response.text(); + expect(response.status, body).toBe(200); + expect(body).toContain("fixture reply"); + } finally { + parent.abort(); headers.resolve(); held?.release(); await pending.catch(() => {}); + globalThis.fetch = originalFetch; timerSpy.mockRestore(); timers.forEach(clearTimeout); + resetProviderRequestPacingForTest(); clearKeyCooldowns(); release(); + } +}); + +for (const wire of ["responses", "chat", "messages"] as const) test.each([ + { initial: "https://gateway.example/v1", final: "http://127.0.0.1:8000/v1", explicit: undefined, expires: false }, + { initial: "http://127.0.0.1:8000/v1", final: "https://gateway.example/v1", explicit: undefined, expires: true }, + { initial: "https://gateway.example/v1", final: "http://127.0.0.1:8000/v1", explicit: 1, expires: true }, +])(`streaming ${wire} continuation follows final stall policy: $initial -> $final, explicit=$explicit`, async row => { + const release = acquireOwnedSpendHome(); + const originalFetch = globalThis.fetch; + const originalInterval = globalThis.setInterval; + const beats: Array<() => void> = []; + const intervals: Array> = []; + const clock = spyOn(globalThis, "setInterval").mockImplementation(((callback, delay, ...args) => { + if (delay !== 2_000) return originalInterval(callback, delay, ...args); + beats.push(callback as () => void); + const handle = originalInterval(() => {}, 2_000_000); + intervals.push(handle); return handle; + }) as typeof setInterval); + const parent = new AbortController(); + const reading = Promise.withResolvers(); + let secondSignal!: AbortSignal; + const sends: string[] = []; + const config: OcxConfig = { defaultProvider: "fixture", stallTimeoutSec: row.explicit, providers: { fixture: { + adapter: "openai-chat", authMode: "key", apiKey: "fixture-key", baseUrl: row.initial, + models: ["fixture-model"], allowPrivateNetwork: true, apiKeySelectionRevision: "initial", terminalContinuationGuard: true, + } } }; + saveConfig(config); + globalThis.fetch = (async (input, init) => { + sends.push(String(input)); + if (sends.length === 1) { + config.providers.fixture = { ...config.providers.fixture!, baseUrl: row.final, apiKeySelectionRevision: "refreshed" }; + saveConfig(config); + return new Response('data: {"id":"fixture","choices":[{"delta":{"content":"I will modify the file now."},"finish_reason":"stop"}]}\n\ndata: [DONE]\n\n', { headers: { "content-type": "text/event-stream" } }); + } + secondSignal = init!.signal!; + return new Response(new ReadableStream({ pull() { reading.resolve(); } }, { highWaterMark: 0 }), + { headers: { "content-type": "text/event-stream" } }); + }) as typeof fetch; + let body: Promise | undefined; + try { + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", headers: { "content-type": "application/json" }, signal: parent.signal, + body: JSON.stringify({ model: "fixture/fixture-model", input: "Please modify the file", stream: true, + tools: [{ type: "function", name: "read_file", description: "Read a file", parameters: { type: "object", properties: {} } }] }), + }), config, { model: "", provider: "" }, wire === "responses" ? {} : { clientEncoder: { protocol: wire === "chat" ? "chat" : "anthropic", model: "fixture-model", stream: true } }); + body = response.text(); + await Promise.race([reading.promise, body.then(text => { throw new Error(`no continuation: ${text}`); })]); + expect(sends).toEqual([`${row.initial}/chat/completions`, `${row.final}/chat/completions`]); + // Drain each beat's frames so client backpressure cannot suppress the watchdog. + for (let tick = 0; tick < 200; tick++) { + for (const beat of beats) beat(); + for (let turn = 0; turn < 30; turn++) await Promise.resolve(); + } + expect(secondSignal.aborted).toBe(row.expires); + if (row.expires) expect(await body).toContain("upstream_stall_timeout"); + else { + parent.abort(); + expect(secondSignal.aborted).toBe(true); + expect(await body).not.toContain("upstream_stall_timeout"); + } + } finally { + parent.abort(); await body?.catch(() => {}); + globalThis.fetch = originalFetch; clock.mockRestore(); intervals.forEach(clearInterval); release(); + } +}); diff --git a/tests/responses/responses-fetch-helpers-boundary.test.ts b/tests/responses/responses-fetch-helpers-boundary.test.ts index ce11c1b4dff..32a4c232426 100644 --- a/tests/responses/responses-fetch-helpers-boundary.test.ts +++ b/tests/responses/responses-fetch-helpers-boundary.test.ts @@ -9,6 +9,16 @@ import { repoRoot as resolveRepoRoot } from "../helpers/repo-root"; const repoRoot = resolveRepoRoot(); const helperPath = resolve(repoRoot, "src/server/responses/fetch-helpers.ts"); +test("an unpaced physical send does not yield after its caller's credential validation", async () => { + const order: string[] = []; + queueMicrotask(() => order.push("credential-change")); + const executor = (async () => { order.push("send"); return new Response("ok"); }) as typeof fetch; + const pending = fetchWithHeaderTimeout("https://fixture.example/v1", {}, new AbortController().signal, 0, false, executor); + expect(order).toEqual(["send"]); + await pending; + expect(order).toEqual(["send", "credential-change"]); +}); + describe("native Codex HTTP upload representation", () => { const endpoint = "https://chatgpt.com/backend-api/codex/responses"; @@ -313,6 +323,43 @@ describe("storedPoolReplayDispatchNotifier", () => { const executor = pacedExecutor({ pacing: async () => {} }); expect(storedPoolReplayDispatchNotifier(executor.wrapped, undefined)).toBe(executor.wrapped); }); + + // The header deadline that cut large local-model prompts: resolveConnectTimeoutMs hands a + // non-positive budget for a local upstream with no explicit connectTimeoutMs, and the helper + // must then arm no header clock so a slow prefill is bounded only by the caller's abort signal. + describe("non-positive header budget arms no deadline", () => { + const delay = (ms: number) => new Promise(resolve => setTimeout(resolve, ms)); + + async function capturedSignal(timeoutMs: number): Promise { + let seen!: AbortSignal; + const executor = (async (_input: Parameters[0], init?: RequestInit) => { + seen = init!.signal as AbortSignal; + // Stay open long enough that a positive budget would elapse mid-flight. + await delay(40); + return new Response("ok"); + }) as typeof fetch; + await fetchWithHeaderTimeout("https://local.test/v1/responses", { method: "POST" }, new AbortController().signal, timeoutMs, false, executor); + return seen; + } + + test.each([0, -1])("budget %s never aborts the composed signal on its own", async timeoutMs => { + const signal = await capturedSignal(timeoutMs); + await delay(30); + expect(signal.aborted).toBe(false); + }); + + test("a positive budget still arms the header clock", async () => { + let seen!: AbortSignal; + const executor = (async (_input: Parameters[0], init?: RequestInit) => { + seen = init!.signal as AbortSignal; + await delay(40); + return new Response("ok"); + }) as typeof fetch; + await fetchWithHeaderTimeout("https://public.test/v1/responses", { method: "POST" }, new AbortController().signal, 5, false, executor).catch(() => {}); + await delay(30); + expect(seen.aborted).toBe(true); + }); + }); test.each([false, true])("notification failure releases the slot (pre-acquired=%s)", async preAcquired => { let releases = 0, sends = 0; const slot = { leased: true, bodyTracked: false, released: false, release() { releases += 1; } }; diff --git a/tests/server/bounded-body.test.ts b/tests/server/bounded-body.test.ts index 1e875a41f2c..c5140839247 100644 --- a/tests/server/bounded-body.test.ts +++ b/tests/server/bounded-body.test.ts @@ -226,6 +226,38 @@ describe("readBoundedResponseBody", () => { expect(result.totalTimedOut).toBe(false); }); + test("a non-positive body-read budget arms no deadline and lets a slow body complete", async () => { + // Regression for the local-upstream JSON path: once headers arrive the body is governed by + // the stall clock, which resolves to disabled (0) for a local model. The old reader derived + // its budget as Math.max(connectMs, 5000) and a 0 budget armed setTimeout(fn, 0), firing on + // the next tick and cutting the body. A body that genuinely pauses between chunks must now + // complete under a disabled budget; a 20ms gap is well past the next-tick the old code armed. + let timer: ReturnType | undefined; + const response = new Response(new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode("first")); + timer = setTimeout(() => { + controller.enqueue(encoder.encode("-rest")); + controller.close(); + }, 20); + }, + cancel() { + if (timer) clearTimeout(timer); + }, + })); + + const result = await readBoundedResponseBody(response, { + totalTimeoutMs: 0, + inactivityTimeoutMs: 0, + }); + + expect(result.timedOut).toBe(false); + expect(result.totalTimedOut).toBe(false); + expect(result.inactivityTimedOut).toBe(false); + expect(result.text).toBe("first-rest"); + expect(result.truncated).toBe(false); + }); + test("a partial body followed by silence times out and flushes UTF-8", async () => { const response = new Response(new ReadableStream({ start(controller) { diff --git a/tests/web-search/web-search-pacing-deadline.test.ts b/tests/web-search/web-search-pacing-deadline.test.ts index e45d79b7014..23073d2feb9 100644 --- a/tests/web-search/web-search-pacing-deadline.test.ts +++ b/tests/web-search/web-search-pacing-deadline.test.ts @@ -145,6 +145,21 @@ async function waitUntilQueued(configured = provider) { } describe("web-search pacing and response-header deadline", () => { + test.each([0, -1])("disabled header budget %s survives pacing admission and retains cancellation", async connectTimeoutMs => { + let sends = 0; + const executor = fetcher(async () => { sends++; return new Response("ok"); }); + await expectSuccess(run(executor, { connectTimeoutMs })); + expect(sends).toBe(1); + const held = await waitForProviderRequestSlot(NAME, provider, MODEL); + const caller = new AbortController(); + const pending = run(executor, { connectTimeoutMs, abortSignal: caller.signal }); + await waitUntilQueued(); + caller.abort(new DOMException("client left", "AbortError")); + expect((await settle(pending)).value.status).toBe(499); + expect(sends).toBe(1); + held.release(); + }); + for (const adapterSend of [false, true]) { test(`${adapterSend ? "adapter fetchResponse" : "ordinary fetch"} excludes a queue wait longer than its header deadline`, async () => { const held = await waitForProviderRequestSlot(NAME, provider, MODEL);