From cf15b59f9376777dc58b9f7035af004b1e2669fd Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:09:39 +0900 Subject: [PATCH 01/21] wip(codex): lease releasing hooks, owned affinity paths, lineage anchor observer, plan unit --- .../261102_codex_affinity_persist/010_plan.md | 146 ++++++++++++++ .../020_plan_revision.md | 152 ++++++++++++++ .../030_plan_revision.md | 99 +++++++++ .../040_plan_revision.md | 95 +++++++++ .../050_plan_revision.md | 80 ++++++++ .../060_plan_revision.md | 69 +++++++ .../070_plan_revision.md | 45 +++++ .../080_implementation_spec.md | 188 ++++++++++++++++++ src/codex/lineage.ts | 20 +- src/lib/config-ownership.ts | 4 + src/lib/spend-ledger-owner.ts | 26 +++ structure/transports/responses-spend.md | 6 +- .../codex-lineage-placement.test.ts | 17 ++ tests/lib/spend-ledger-owner.test.ts | 37 ++++ 14 files changed, 976 insertions(+), 8 deletions(-) create mode 100644 devlog/_plan/261102_codex_affinity_persist/010_plan.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md create mode 100644 devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md diff --git a/devlog/_plan/261102_codex_affinity_persist/010_plan.md b/devlog/_plan/261102_codex_affinity_persist/010_plan.md new file mode 100644 index 00000000000..af1b9bdf773 --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/010_plan.md @@ -0,0 +1,146 @@ +# 261102 Codex pool affinity persistence (Lane B) — 010 plan + +Status: P (plan) — awaiting Terra audit. + +## Problem + +Codex pool session pins live only in memory, so a proxy restart re-places every ongoing conversation. +Two process-local facts cause it, and both must change: + +1. `src/codex/routing/thread-affinity.ts` keeps `threadAccountMap` (conversation key -> scope -> binding) and + `conversationStateIssuerMap` (binding key -> account that minted carried encrypted state) in memory only. +2. The key those maps are indexed by is `codexConversationKeyFor()` in `src/codex/lineage.ts`: + `app:HMAC(CODEX_LINEAGE_KEY, family, thread)` where `CODEX_LINEAGE_KEY = randomBytes(32)` at module load. + After a restart the same conversation derives a different key, so a persisted map alone could never be + looked up again. (The lineage scope key `auth:HMAC(key, Authorization)` uses the same secret.) + +Under round-robin most conversations move after a restart: the prompt cache goes cold and carried encrypted +state minted by the old account can fail with `invalid_encrypted_content`. + +Reference implementation to mirror: `src/oauth/anthropic-session-affinity-store.ts` (#7813 / #7828) and its +structure section `structure/providers/anthropic-account-pool.md#durable-session-affinity`. + +## Design + +### A. Durable conversation-key secret — new `src/codex/affinity-key.ts` + +- `codexAffinityKey(): Buffer` returns a 32-byte secret cached per resolved `getConfigDir()` (re-resolved when the + home changes, as the Claude store's scope does). +- Path: `/codex-affinity-key`. Content: one line `v1:` (validated by regex). +- Hydration: `readBoundedRegularFileSync(path, 128)` (rejects symlinks/junctions/non-regular files and oversize). + Valid -> use. ENOENT -> `randomBytes(32)`, cache it, and persist asynchronously: + `trackBackgroundWork("codex-affinity-key-write", withConfigDirHardeningSettled(home, () => atomicWriteFileNoFollow(path, text, undefined, {}, { exclusive: true })))`. + If the exclusive publish loses a race (another process created it first), keep the in-memory key for this process + (its pins are lost at the next restart, same as today) — no overwrite. Invalid or unreadable -> identifier-free + `console.warn("[codex-affinity] key unavailable or invalid; affinity is process-local until it is replaced")`, generate + a fresh key and replace the file with a non-exclusive atomic write (a corrupt key must not disable persistence forever). +- The file is a secret: private mode via the atomic publisher (0600 on POSIX) and the config-dir Windows ACL barrier. + Never logged, never included in diagnostics. +- `src/codex/lineage.ts`: drop `CODEX_LINEAGE_KEY`; both HMACs call `codexAffinityKey()`. Update the LIFETIME doc block: + keys are now reproducible across restarts of one config home; lineage records themselves stay process-local. + Keep lineage a leaf: the new module imports only config/lib modules (no routing import, no cycle). +- `src/adapters/google-antigravity-wire.ts` comment that calls the key "process-random ... changes across a proxy restart": + reword only (the derivation there stays independent; it must survive loss of the key file too). No behavior change. + +### B. Durable pin + issuer snapshot — new `src/codex/routing/thread-affinity-store.ts` + +- File `/codex-affinity.json`, `{ version: 1, threads: [...], issuers: [...] }`. + - thread row: `{ keyDigest, scope, accountId, generation, createdAt, lastUsedAt }` + - issuer row: `{ keyDigest, accountId, lastUsedAt }` + - `keyDigest = sha256hex("opencodex/codex-affinity//v1\0" + key)`. Raw conversation keys, raw parent ids + (legacy keys) and binding keys never reach disk. Account ids are opaque pool ids (`isValidCodexAccountId` or `__main__`). + - Transient fields (`transientHoldSince`, `transientDetourAccountId`) and `lastReevalAt` are not persisted; restored + entries get `lastReevalAt = restore time`. +- Bounds: `MAX_FILE_BYTES = 1 MiB` on read; newest `CODEX_THREAD_AFFINITY_MAX_ENTRIES` (2048) rows per table on write and read; + rows idle longer than `CODEX_THREAD_AFFINITY_IDLE_TTL_MS` (24 h), future-dated rows, malformed rows are dropped. + Scope strings validated (`spark`/quota scopes, `legacy`, `model-detour::`, <= 512 bytes). +- Hydration is lazy per config home (first affinity read/write), synchronous and bounded, like the Claude store. + Restored rows are held in digest-keyed `restoredThreads` / `restoredIssuers` maps; they never enter + `threadAccountMap` until a request looks them up. +- Adoption: in `getThreadAffinityForScope(threadId, scope)` on a memory miss, and in `peekConversationStateIssuer`, + compute the digest, take the restored row, validate, and insert it into the in-memory map (removing it from the restored + map). Validation for thread rows = not expired AND `isThreadAffinityGenerationLive({accountId, generation})` (account record + exists, has a credential, not deleted, same credential generation; `__main__` must be generation 0). A failing row is + discarded and the request falls back to normal selection. Usability (paused, reauth, plan exclusion, quota, cooldown) is + NOT checked at restore: the existing resolve path already re-evaluates a bound account every request and releases it + with the usual reason. Issuer rows are restored after TTL validation only: an issuer for a deleted account still correctly + forces the account-change scrub. +- Writes: the snapshot is serialized from the live maps plus unadopted restored rows at flush time, so in-place mutations + elsewhere are captured. Structural changes (new bind, account change, delete, clear, per-account clear, hand-off, + prune/evict, issuer change) flush promptly; timestamp-only activity (bind of the same account, a binding read, an issuer + peek) schedules one coalesced flush after 1 s (unref'd timer). Reading a binding schedules the coalesced flush because + `src/codex/routing.ts` updates `lastUsedAt` in place after the read and that file is at its file-size cap. +- Publisher: `trackBackgroundWork("codex-affinity-write", withConfigDirHardeningSettled(home, () => atomicWriteFileNoFollow(path, json)))`, + revision counter re-flush exactly like `flushScope` in the Claude store, best-effort `registerOptionalShutdownHook("codex-affinity", ...)`, + identifier-free warnings (`[codex-affinity] snapshot persistence failed; using runtime affinity` / + `snapshot unavailable or invalid; using runtime affinity`). An observed config-home change retires the scope (memory + restored maps + pending timer), as in the Claude store. +- Exports: `flushCodexThreadAffinity()` (tests and shutdown), `resetCodexAffinityStoreMemoryForTests()`. + +### C. Minimal hooks in `src/codex/routing/thread-affinity.ts` + +- `getThreadAffinityForScope`: on miss, adopt from the store; on hit, schedule the coalesced touch flush. +- `bindThreadAffinityForScope`, `deleteThreadAffinityForScope`, `deleteThreadAffinitiesForAccount`, + `clearThreadAccountMapForAccount` (also purge restored rows for that account), `handOffThreadAffinityGeneration` (also + move restored rows fromGeneration -> toGeneration), prune/LRU paths, `rememberConversationStateIssuer`, + `peekConversationStateIssuer`, `clearConversationStateIssuerMap`, `clearThreadAccountMap` (also clears restored rows and + persists the empty snapshot, matching the Claude "explicit runtime selection clears bindings and persists the reset"). +- Update the doc comments that promise process-local lifetime (issuer map comment, `adoptLegacyAffinityForScope` comment). +- Keep edits to call sites; do not touch the affinity log label (Lane D) or `src/codex/routing/selection.ts` (Lane C). +- Core-path boundary: the store imports only `src/config/*` and `src/lib/*`; no `src/lab/` reachability. + +## Tests (new files; register each in `scripts/test-layout/layout.json` or its domain shard AND a `tests/fixtures/test-layout-expected*.json` shard) + +`tests/codex-integration/codex-affinity-restart.test.ts` (temp `OPENCODEX_HOME` per test, `removeTreeWithRetry` cleanup): + +1. Restart keeps the pin: build headers with `session-id`/`thread-id`, key = `codexPoolAffinityKey(headers)`, seed two pool + accounts, `bindThreadAffinity(key, B, now)`, `rememberConversationStateIssuer(key, B)`, flush, simulate restart + (reset in-memory affinity, store scope and key cache — NOT the files), recompute the key from the same headers: it is + identical, `getThreadAffinity(key)` returns B, `peekConversationStateIssuer(key)` returns B. Prefer one assertion through + the real resolver (`resolveCodexAccountForThread` or the existing public resolve used in `tests/codex-integration/codex-routing.test.ts`) + under round-robin, showing the restored binding wins over the strategy cursor. + MUST FAIL on old code (record output): the key differs after restart and the map is empty. +2. Dead pins fall back: account removed, credential generation bumped (re-login), and expired (>24 h) rows are dropped; + resolve returns a normal selection and the dropped row is gone from the next snapshot. +3. Privacy/bounds: the snapshot contains no raw session id, thread id or conversation key; key file is 0600 on POSIX; + oversized, malformed, future-dated and symlinked snapshots are ignored with an identifier-free warning (assert the + warning text contains no account id or key); more than 2048 rows keeps the newest. +4. `clearThreadAccountMap()` and `clearThreadAccountMapForAccount()` persist their removals (after restart the pin is gone). +5. Key file: reused across restart; a corrupt key file is replaced and a warning is emitted. + +Existing suites to rerun: `tests/codex-integration/codex-routing*.test.ts`, `codex-affinity-debug.test.ts`, +`codex-idle-window.test.ts`, `codex-account-selection-atomicity.test.ts`, `codex-auth-release.test.ts`, +`tests/responses/responses-account-change-scrub.test.ts`, `tests/responses/responses-4546-incident-regression.test.ts`, +lineage tests (`rg -l lineage tests`), `tests/lab/core-lab-boundary.test.ts`, `tests/test-layout.test.ts`, file-size ratchet test. + +## Docs + +- `structure/providers/openai-tiers.md`: lineage section (lines ~105-160) — key is a per-config-home durable secret; add a + "Durable Codex pool affinity" paragraph parallel to the Anthropic one (file names, row shape, bounds, validation, flush policy, + test names). Run `bun run structure:check`; add the new src files to the owning manifest entry if the check asks. +- `docs-site/src/content/docs/reference/cli/providers-accounts.md` (~line 848): replace "proxy restart or affinity eviction can + also leave a task unbound" with: bindings survive a proxy restart for 24 h idle; eviction, account removal or re-login leave a + task unbound. Apply the equivalent edit in every locale copy that carries the sentence (`rg -l` across docs-site locales) so no + locale contradicts English; follow `docs-site/AGENTS.md`. +- `docs-site/src/content/docs/reference/configuration/providers.md` `accountPoolStrategy` row (~line 128) says a task "can become unbound after proxy restart or affinity reset"; update it (and locale copies) the same way. + +Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## Verification + +`bun test `, `bun run typecheck`, `bun run structure:check`, `bun run privacy:scan`, +`bun scripts/test.ts --changed=dev --parallel=1`. Red-on-old proof: run test 1 against `origin/dev` sources (stash src changes) and record the failure. +`ci:full` label: yes — the change reaches on-disk files and docs the import graph cannot see. + +## Out of scope + +`src/codex/routing/selection.ts` (Lane C), `src/codex/auth-context.ts` and the affinity log label (Lane D), +`src/server/claude-*`, `src/server/caller-session-identity.ts` (Lane A). No change to selection policy. + +## Risks + +- Persisting the HMAC secret changes the documented "keys are unreproducible after restart" posture. The secret stays private + (same protection class as `admin-api-token`), and the snapshot holds only digests of keys derived from it. +- Two processes sharing one config home: exclusive create means the loser keeps a process-local key; acceptable and documented. +- Write amplification: at most one coalesced write per second while traffic flows, same as the Claude store. + diff --git a/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md new file mode 100644 index 00000000000..d63a2316b8a --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md @@ -0,0 +1,152 @@ +# 261102 Codex pool affinity persistence — 020 plan revision (answers audit round 1 + coordinator note) + +Supersedes the Design, Tests and Verification sections of 010_plan.md. Problem statement, docs list and out-of-scope +boundaries in 010 stay. Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## Security posture (coordinator note; security-sensitive per MAINTAINERS.md) + +The existing conversation-key secret `CODEX_LINEAGE_KEY` in `src/codex/lineage.ts` STAYS process-local and unchanged. +Every consumer of the derived key `app:HMAC(CODEX_LINEAGE_KEY, anchor, anchor)` therefore reveals exactly what it reveals today: + +| Consumer of the ephemeral key | Where | Changes? | +|---|---|---| +| `threadAccountMap` / `conversationStateIssuerMap` keys, pending release reasons | `src/codex/routing/thread-affinity.ts` (memory) | no | +| `authCtx.affinityKey` passed as `threadId` to routing | `src/codex/auth-context.ts`, `src/server/responses/core-codex-account.ts`, `passthrough-*.ts`, `src/providers/openai-sidecar.ts` (memory) | no | +| `_cursorClientThreadId` (Cursor adapter conversation id, advisor state) | `src/server/responses/request-prepare.ts:417,563`, `src/adapters/cursor/request-builder.ts:385`, `src/advisor/state.ts` | no | +| lineage scope `auth:HMAC(key, Authorization)` and records | `src/codex/lineage.ts` (memory) | no | +| debug capture tags | `src/codex/affinity-debug.ts` (own process-local key, ADR-0089) | no | +| request log / usage ledger lanes | `src/server/request-log-conversation.ts` — uses its own digest of raw headers, not the affinity key | no | + +The ONLY new durable artifact is the snapshot. Its row keys are derived from a SEPARATE secret used for nothing else: +`rowKey = HMAC-SHA256(storeSecret, "opencodex/codex-affinity/conversation/v1\0" + anchor)` (hex), where `anchor` is the +cohort anchor already used by `codexConversationKeyFor(anchor, anchor)` (the session id, or the parent thread id for +session-less requests). Issuer rows use domain `.../issuer/v1`. The store secret never leaves the store module and the +snapshot file; it is not logged, not exposed by diagnostics or management APIs. What becomes linkable across restarts: +only that "the same anchor was bound to account X", and only to someone who can read both the private snapshot and the +private secret file (owner-only). Nothing written to logs gains cross-restart linkability. + +## Design + +### A. Store secret — inside `src/codex/routing/thread-affinity-store.ts` + +- File `/codex-affinity-key`, content `v1:\n`, read with `readBoundedRegularFileSync(path, 128)` + (rejects symlinks, junctions, non-regular files, oversize). +- States per config-home scope: `durable` (read valid file, or our exclusive create succeeded and re-read matches), + `pending` (generated, publish in flight), `disabled` (publish failed, lost the exclusive race, file unreadable). + **The snapshot is written only in state `durable`** (blocker 2): a process with an unpublished or losing key never touches + `codex-affinity.json`. Exclusive-race loser: re-read the winner's key; if valid, adopt it before any row key is derived + (rows are derived lazily at flush time from retained anchors — see B — so adoption is safe) and become `durable`; + otherwise `disabled`. +- ENOENT -> generate, exclusive publish via `withConfigDirHardeningSettled(home, () => atomicWriteFileNoFollow(path, text, undefined, {}, { exclusive: true }))` + inside `trackBackgroundWork`. A snapshot with no key file is ignored (cannot be verified) and replaced on the first durable flush. +- Present-but-invalid regular file -> identifier-free warning, replace with a fresh key (non-exclusive atomic write), drop any + existing snapshot (its rows are unverifiable). Unreadable/symlink/non-regular -> warning, `disabled`, never replaced blindly. +- Ownership: the key file and snapshot are recorded as owned config files (use the default non-exclusive path's ownership + recording, or call `recordOwnedConfigPath` after an exclusive publish) so uninstall/reset removes them. Implementer verifies + how `src/lib/config-ownership.ts` is consumed and states it in the PR. + +### B. Durable rows without changing the ephemeral key + +- `src/codex/lineage.ts` gains a one-slot observer: `setCodexConversationAnchorObserver(fn | undefined)`; `codexConversationKeyFor(familyId, threadId)` + calls `fn?.(key, familyId, threadId)` after deriving. No new imports in lineage.ts (stays a leaf). The store registers the + observer when `thread-affinity.ts` loads it. +- The store keeps, per config-home scope, a bounded `anchorByKey: Map` (same TTL/size as the binding map; + memory only — raw anchors already live in lineage records in memory). Row keys are computed from the anchor only when a + snapshot is serialized or a restored row is looked up. Keys with no recorded anchor (legacy raw-parent keys, test keys bound + directly) are memory-only, exactly as today. +- **Durability contract (blocker 1, narrowed and documented):** a pin survives restart when the request's key is derivable from + its headers alone: every request carrying `session-id` (all Codex CLI/App turns, roots and subagents), and session-less + requests whose anchor is their immediate parent. A session-less chain of depth >= 2 resolves through an in-memory lineage + record that a restart forgets, so it re-places after restart as it does today. Docs say so. Test both sides. + +### C. Snapshot store (`codex-affinity.json`) + +- `{ version: 1, threads: [{ rowKey, scope, accountId, fence, createdAt, lastUsedAt }], issuers: [{ rowKey, accountId, fence?, lastUsedAt }] }`. +- Fence (blocker 5): `fence = sha256hex("opencodex/codex-affinity/identity/v1\0" + record.quotaHistoryIdentity)`. `quotaHistoryIdentity` is a random + UUID minted on every new login/replacement and preserved across same-account token refresh (`src/codex/account-store.ts:264,493,554`), + i.e. the Codex twin of the Claude `loginFence`. Records without a valid `quotaHistoryIdentity` stay memory-only (Claude legacy rule). +- `__main__` pins are not persisted (no durable identity fence for the native login). `__main__` issuer rows and pool issuer rows + whose fence no longer matches restore as a foreign sentinel issuer (`"\u0000foreign"`), which never equals a serving account id + and therefore forces the account-change scrub instead of letting stale carried state reach a new account. +- Restore of a thread row: TTL ok, not future-dated, scope valid, account record has credential, not deleted, fence matches. On adoption + the entry takes the record's CURRENT generation (fence proves identity continuity; this also covers an offline `ocx account refresh`), + `lastReevalAt = now`, no transient fields. Otherwise discard -> normal selection. Usability (paused, reauth, cooldown, plan/model + exclusion, transient) is left to the existing resolver (`resolveCodexAccountForThread`, `src/codex/routing.ts:498`, reuse checks ~959-1021). +- Bounds (blocker 6): read cap and write cap both `MAX_FILE_BYTES = 2 MiB`; per table newest-first up to 2048 rows; on write, if the + serialized document exceeds the cap, drop oldest rows (detour scopes first, then by `lastUsedAt`, ties by rowKey) until it fits — + deterministic. Duplicate rowKey+scope on read: keep newest. Scope validation mirrors the real component rules: `legacy`, a known + `CodexQuotaScope`, or `model-detour::` where model passes the existing 512-byte component limit (no extra cap on the whole scope). +- Lazy adoption + reconciliation (blocker 3): restored rows live in `restoredThreads: Map>` and + `restoredIssuers: Map`. EVERY mutation for an ephemeral key K with a known anchor first reconciles the restored rows of + rowKey(K): bind (restored row for that scope dropped, live wins), delete scope, delete-for-account, issuer remember, and the getters + (adopt). Account-wide operations (`clearThreadAccountMapForAccount`, `handOffThreadAffinityGeneration` — fence unchanged so no row edit + needed) and `clearThreadAccountMap`/`clearConversationStateIssuerMap` act on restored rows too, even when the live map misses. TTL pruning and + LRU eviction count live + restored rows against the 2048 bound, detour-first as `pruneLruThreadAffinities` does. Adoption increments + `threadAffinityEntryTotal` and runs the existing prune. `getThreadAffinityScopes(threadId)` adopts all restored scopes for that key first. +- Flush policy: structural changes publish promptly; timestamp-only activity (same-account rebind, binding read, issuer peek) schedules one + coalesced flush after 1 s (unref'd), which captures the in-place `lastUsedAt` writes in `src/codex/routing.ts:911,942,985,1021` + (that file is at its size cap). Revision re-flush, `trackBackgroundWork("codex-affinity-write", withConfigDirHardeningSettled(...))`, + best-effort `registerOptionalShutdownHook("codex-affinity", ...)`. Clear racing a pending flush: the flush serializes at callback time + from current state under the active-scope check, and a clear bumps the revision so a re-flush follows. +- Config-home scope (blocker 4): scope = resolved `getConfigDir()`. On an observed change the old scope is retired (restored rows, anchor map, + timers, key state) and the live runtime affinity maps are cleared (mirrors the Claude store, whose rows ARE its runtime map), so home B + never inherits or persists home A's pins. Every deferred callback re-checks `scope.active && getConfigDir() === scope.home` INSIDE the + ACL-settled callback before writing (Claude `flushSync` pattern). Lineage records stay process-global but carry only ephemeral keys, + which home B cannot persist without its own anchor observation — implementer confirms no cross-home leak in the A->B->A test. +- Passive reads (blocker 7): when `isPassiveRead()` is true the store neither creates/replaces the key, nor hydrates, adopts, schedules, + or flushes; affinity behaves exactly as today for that call. +- Warnings are identifier-free: `[codex-affinity] snapshot persistence failed; using runtime affinity`, + `[codex-affinity] snapshot unavailable or invalid; using runtime affinity`, `[codex-affinity] key unavailable; affinity stays process-local`. + +### D. Hooks in `src/codex/routing/thread-affinity.ts` + +Call-site hooks only: the getters, bind/delete/clear paths, LRU/TTL prune, issuer remember/peek/clear, `getThreadAffinityScopes`. +Comment updates for the issuer map and legacy adoption lifetime notes. No change to the affinity log label (Lane D), `selection.ts` (Lane C), +`auth-context.ts` (Lane D), or `src/codex/routing.ts` (at cap). + +## Tests + +`tests/codex-integration/codex-affinity-restart.test.ts` (register in `scripts/test-layout/layout.json` next to line 709 and in +`tests/fixtures/test-layout-expected.json` next to line 697; run `tests/test-layout.test.ts` and `tests/test-layout-tooling.test.ts`). + +1. **Restart, two fresh subprocesses (red-on-old)**: a helper script (`tests/helpers/codex-affinity-restart-worker.ts`, registered if the + layout requires) run with `bun` twice against one temp `OPENCODEX_HOME`: process 1 seeds two pool accounts (round-robin), resolves a + session-bearing request through `resolveCodexAccountForThread` with the key from `codexPoolAffinityKey(headers)`, records the issuer, + then awaits the store flush if the export exists (dynamic import with catch, like `tests/adapters/anthropic/anthropic-affinity-restart.test.ts`) + and exits; process 2 advances the round-robin cursor so a fresh pick would choose the other account, resolves the same headers, and + prints the account and whether the account-change scrub fires for a body carrying `previous_response_id` when served by the other account. + Expect: same account; issuer known. Old code: different account / no issuer. Record the old-code failure from an isolated export of + `origin/dev` (`git archive origin/dev | tar -x -C ` + the test file + `bun install --frozen-lockfile` or a node_modules link); + no stash, no worktree. +2. Fallbacks: account removed; account re-logged (new `quotaHistoryIdentity`); legacy record without identity; expired row; future row -> + normal selection, row gone from the next snapshot. Usability cases (paused, needs reauth, cooldown, model exclusion, transient hold) assert + the existing resolver behavior after restore, case by case, not a blanket "deleted". +3. `__main__`: pin not persisted; issuer restores as foreign sentinel and forces scrub. +4. Session-less depth-1 survives; depth-2 session-less chain re-places (documented limitation). +5. Reconciliation before first lookup: bind, delete, delete-for-account, issuer remember, clear and clear-for-account on a key whose row + was restored but not adopted; pending flush racing clear. +6. Key states: reused across restart; exclusive race loser adopts the winner or stays disabled and never writes the snapshot; publish + failure leaves no snapshot; corrupt regular key replaced and old snapshot dropped; symlinked/unreadable key disabled, not replaced. + Key file and snapshot are 0600 on POSIX. +7. Privacy and bounds: snapshot contains no raw session/thread id, no ephemeral key; warnings contain no account id; two full tables with + worst-case UTF-8 detour scopes stay under the cap and re-read successfully; oversized/malformed/symlinked snapshot ignored. +8. Config home A -> B -> A with pending writes: no cross-home rows, no write after retirement. +9. Passive read: no files created, no pending work. +Drain tracked background work before temp-home cleanup. + +Existing suites to rerun: `rg -l` of the affinity entry points (the round-1 audit counted 38 files) plus +`tests/lab/core-lab-boundary.test.ts`, the file-size ratchet test, `tests/ci-workflows/test-home-guard.test.ts`, lineage tests. + +## Docs + +As in 010, plus: `structure/providers/openai-tiers.md` lineage section states the conversation key stays process-local and adds +"Durable Codex pool affinity" (secret, row derivation, fence, main rule, contract narrowing, bounds, flush, passive, home scope, tests); +`structure/providers/openai-accounts.md` only where it claims pins are process-local (do not touch the diagnostic tags or the idle-window +reservation map). Locale copies found per locale (fr, ko, zh-cn, and any others), not by English text. Build the docs site per docs-site/AGENTS.md. + +## Verification + +Focused new + affected tests, `bun run typecheck`, `bun run structure:check`, `bun run privacy:scan`, docs-site build, +`bun scripts/test.ts --changed=dev --parallel=1`, red-on-old recorded. Before the PR: a dedicated Terra security review (secret file +permissions/ACL, key scope, cross-restart linkability, snapshot contents, deletion on pool/config reset); its verdict goes in the PR description. + diff --git a/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md new file mode 100644 index 00000000000..0a163b5c671 --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md @@ -0,0 +1,99 @@ +# 261102 Codex pool affinity persistence — 030 plan revision (answers audit round 2) + +Amends 020_plan_revision.md; everything in 020 not changed here stands. Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## R2-B1 — fences captured at the credential event + +- `ThreadAffinityEntry` gains an optional `identityFence?: string` (sha256hex of `"opencodex/codex-affinity/identity/v1\0" + quotaHistoryIdentity`). + `bindThreadAffinityForScope` already reads the account record once to take `generation`; it takes the fence from that SAME + record read, so the fence always describes the credential the binding was made under. `handOffThreadAffinityGeneration` + keeps the fence (an owned refresh preserves `quotaHistoryIdentity`, account-store.ts:493,554). An entry without a fence + (legacy record, `__main__`) is memory-only. Serialization writes the captured fence; it never reads the current record. +- Issuers: `rememberConversationStateIssuer(bindingKey, accountId, now?, credentialGeneration?)`. The fence is captured only when + the record's current generation equals the dispatched `credentialGeneration` (one record read, under the same call); otherwise + the issuer row is written without a fence. `rememberServingConversationStateIssuer` in `src/server/responses/account-change-state.ts` + passes `authCtx.generation` for pool contexts (`CodexAuthContext` pool kind carries `generation`, auth-context.ts:300). Callers that + pass no generation (routing.ts, compact) produce fence-less rows. A re-login while a response is in flight therefore cannot + attach the new identity to state issued by the old credential. +- Restore: thread row adopted only if record live AND current `quotaHistoryIdentity` fence equals the stored fence; then it takes + the record's current generation. Issuer row: fence equal -> issuer = accountId; fence missing or different -> UNKNOWN issuer + (R2-B2). `__main__` issuer rows are not persisted at all (behaves exactly as today after restart). +- Orca/external rotation (account-store.ts:1157) preserves `quotaHistoryIdentity` only after verifying `chatgptAccountId` and + `sourceSubject` are unchanged, i.e. the same upstream account; restoring across it is correct. Tested. +- Tests: re-login between bind and flush (row restored? no -> dropped); re-login between dispatch and issuer completion (issuer row + has no fence -> unknown); owned refresh/handoff before flush (restored); Orca rotation (restored); delete + re-add same id (dropped). + +## R2-B2 — unknown issuer is not proof of an account change for files + +- `thread-affinity.ts` exports `CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER` (a value no account id can equal; account ids match + `^[A-Za-z0-9._-]{1,64}$` or `__main__`). `peekConversationStateIssuer` may return it only for a restored row. +- `applyAccountChangeConversationStateScrub`: unknown counts as changed (conservative continuation scrub; one cold turn). +- `accountChangeFileReferenceRefusal`: unknown issuer is ignored; refusal only when a KNOWN issuer differs or `priorAccountId` + proves an in-request move. So persistence adds no file refusal that did not exist before. +- Tests: pool re-login (fence mismatch) + file reference -> scrub of continuation state, no 400; known pool move + file -> 400 as today; + unchanged pool account -> neither. + +## R2-B3 — home switching and observer ordering + +- The anchor observer first resolves the active scope (`current()`: retire the old scope and clear live affinity maps if the home + changed) and only then records the anchor in the new scope. So the first request under home B keeps its own anchor. +- Anchor cache: per scope, LRU, refreshed on every derivation, bound 4096 (2x the binding bound), same 24 h TTL. Session-bearing + requests re-derive on every request, so their anchors are always present. A key that reaches routing without being observed in + the current scope (recorded session-less lineage reused after an anchor eviction or after a home change, legacy keys, directly + bound test keys) is memory-only: it is neither written nor matched. This is part of the documented narrowed contract. +- Tests: first B request persists under B; A -> B -> A with pending writes, both direct derivation and recorded lineage; no B row + ever appears in A's snapshot or vice versa; anchor eviction then recorded-lineage lookup -> memory-only. + +## R2-B4 — cross-process serialization + +All durable writes happen inside `withConfigDirHardeningSettled(home, () => withConfigMutationLockSync(() => ..., home))` +(`src/config/mutation-lock.ts:104`: OS-backed SQLite `BEGIN IMMEDIATE`, `busy_timeout=0`, released on process exit). Under the lock: + +1. Key: re-read the key file. Valid -> adopt it if it differs from ours (rows are derived lazily from anchors, so adoption is safe; + restored rows hydrated under a different key are discarded). Missing -> write ours. Invalid regular file -> replace with ours + and treat the existing snapshot as empty. Unreadable/symlink/non-regular -> `disabled`, no write. No `exclusive` publish is used, + so atomic-write records ownership as it does for other config files (verify; otherwise call `recordOwnedConfigPath` and treat a + `false` result as a one-time identifier-free warning, persistence continues). +2. Snapshot: read the on-disk snapshot (bounded) and MERGE: for each (rowKey, scope) keep the newest `lastUsedAt` of disk vs ours, + then apply our pending tombstones recorded since our last successful publish: per-row deletes, per-account deletes + (`{accountId, at}`: drop disk rows of that account with `lastUsedAt <= at`) and full clears (`{at}`: drop disk rows with + `lastUsedAt <= at`). Trim to the bounds. Write atomically. Clear tombstones only after the write succeeds. +3. Lock busy -> nothing written, state stays dirty, retried by the next coalesced flush (1 s, unref'd). Repeated failure warns once, + identifier-free. + +Two proxies on one config home therefore never lose each other's newer rows and never publish under a key the other replaced. +Tests: two processes creating the key concurrently end with one key and both see it; concurrent invalid-key recovery; two writers +with disjoint rows -> union; writer A deletes a row writer B still has older -> stays deleted; lock held by another process -> retry. + +## R2-B5 — reset and removal semantics + +| Entry point | Effect on disk | +|---|---| +| `clearThreadAccountMap()` — config, provider, model, combo, agent-settings, logs-usage, oauth-account and shared management routes, server index (`rg -n clearThreadAccountMap src`) | full-clear tombstone; next publish writes a snapshot without those rows (issuers cleared too, as today) | +| `clearConversationStateIssuerMap()` | issuer table cleared on disk | +| `clearThreadAccountMapForAccount(id)` — account removal, auto-pause, plan exclusion, quota refusal, account-preservation | per-account tombstone for pins only; issuer history is preserved, as today (thread-affinity.ts:223) | +| account deletion without a clear | row fails the fence on restore and is discarded | +| uninstall (`removeOwnedConfigState`) | both files are owned config paths and are removed | + +Clears bump the revision and fence pending writes (the publish serializes current state at callback time under the scope check). +Tests: restart after each row of the table. + +## R2-B7 (partial) — passive + +Under `isPassiveRead()`: no hydration, adoption, scope switch, anchor recording, scheduling or flushing. Tests with a populated +snapshot, an already scheduled write, passive getters and observer calls, and existing live maps. + +## Security table corrections (non-blocking 1) + +- Add rows: compact (`src/server/responses/compact.ts:1684` uses the ephemeral key as an in-memory issuer binding key only), + previous-response replay (keyed by response ids and its own scope, not the affinity key — implementer confirms with file:line), + advisor ledger (`src/advisor/state.ts:213` derives its own hash from `_cursorClientThreadId`, which stays ephemeral -> unchanged). +- Correct the linkability claim: anyone who can read two snapshots of the same home can link identical row keys and the plaintext + pool account ids across restarts without the secret; the secret is needed only to test a candidate session id against a row. + Both files are owner-only (0600 / config-dir ACL). The key file is itself a new durable artifact. + +## Security review + +Dedicated Terra security review before the PR; per MAINTAINERS.md security-sensitive changes also need explicit maintainer security +review — the PR says so and requests it. + diff --git a/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md new file mode 100644 index 00000000000..c7491bcfc2a --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md @@ -0,0 +1,95 @@ +# 261102 Codex pool affinity persistence — 040 plan revision (answers audit round 3) + +Replaces the STORAGE design of 010/020/030 (JSON snapshot, key file, merge, tombstones, config mutation lock). Everything the +round-3 audit marked RESOLVED stays as written in 020/030: identity fences captured at bind/issuance (030 R2-B1), unknown issuer +handling (030 R2-B2), observer/home ordering and the narrowed contract (030 R2-B3, 020 B), passive rule (030), security table (030), +regression-test approach (020 Tests 1). Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## Why the storage changes + +Round 3 showed that a whole-file snapshot needs cross-process deletion evidence and a lock that config writes also use. A small +dedicated SQLite database gives both for free: every mutation is a row operation inside a transaction, a delete is a DELETE that +no other writer can undo by republishing its memory, and the database's own locking replaces the shared config lock. The Claude +store's properties are kept: config-home location, atomic commits, bounded rows, 24 h idle expiry, digested keys, identifier-free +warnings, private file mode and Windows ACL hardening. + +## Storage: `/codex-affinity.sqlite` (module `src/codex/routing/thread-affinity-store.ts`) + +- Opened lazily per config-home scope with `bun:sqlite`. Setup mirrors `configMutationDatabasePath` in `src/config/mutation-lock.ts:38`: + `assertNotRealHomeUnderTest`, dir 0700, `hardenSecretDir` on Windows (failure -> one identifier-free warning, persistence disabled for + the scope), file chmod 0600. `PRAGMA journal_mode=WAL; synchronous=NORMAL; busy_timeout=0`. Its own file: no interaction with + `config-mutation.sqlite`, so config and credential mutations are never blocked by this store (round-3 blocker 1). +- Ownership (round-3 blocker 3): before the first open, `recordOwnedConfigPath` for the db and its `-wal`, `-shm`, `-journal` sidecars; + every call's boolean is checked. If any returns false the store does NOT create the database (warn once, identifier-free; affinity + stays process-local for that scope). So any file this store creates is an owned path that uninstall (`removeOwnedConfigState`) removes. + Test: existing owned home -> paths recorded; manifest recording failure -> no database created. +- Schema v1 (`PRAGMA user_version = 1`; unknown version -> warn, disabled, file untouched): + - `meta(name TEXT PRIMARY KEY, value BLOB)` — row `secret`: 32 random bytes, created with `INSERT OR IGNORE` then read back, so + concurrent processes converge on one secret atomically. Replaces the key file; no corrupt-key recovery path exists. + - `threads(row_key TEXT, scope TEXT, account_id TEXT, fence TEXT, created_at INT, last_used_at INT, PRIMARY KEY(row_key, scope))` + - `issuers(row_key TEXT PRIMARY KEY, account_id TEXT, fence TEXT NULL, last_used_at INT)` + - `row_key` = HMAC-SHA256(secret, domain + anchor) hex as in 020; fence as in 030. +- Corrupt or unreadable database (open/pragma/query throws SQLITE_CORRUPT/NOTADB, symlink at the path, non-regular file): warn once, + persistence disabled for the scope, the file is left in place. + +## Writes: an ordered, coalesced in-process operation queue + +- Every mutation of the live maps enqueues an operation keyed so that the last operation per key wins: + - `bind(rowKey, scope, accountId, fence, createdAt, lastUsedAt)` -> `INSERT ... ON CONFLICT DO UPDATE` (a real placement decision). + - `touch(rowKey, scope, accountId, fence, lastUsedAt)` -> `UPDATE threads SET last_used_at = max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?`. + A touch can never create or revive a row: if another process deleted or rebound it, the touch is a no-op (round-3 blocker 2: + a stale in-memory pin in process B cannot undo A's clear). + - `deleteScope`, `deleteKey` (all scopes), `issuerSet` (upsert), `issuerTouch` (UPDATE-only, same rule), `issuerDelete`. + - `clearAccountPins(accountId)` -> `DELETE FROM threads WHERE account_id=?` (issuer history preserved, as today). + - `clearAll` -> `DELETE FROM threads; DELETE FROM issuers`; `clearIssuers` -> `DELETE FROM issuers`. + - A `clearAll` discards every queued operation before it; a `clearAccountPins` discards queued thread ops for that account. +- Flush: structural ops (bind, deletes, clears, issuerSet) schedule an immediate microtask-deferred flush; touches schedule a coalesced + flush after 1 s (unref'd timer). Read-time touches capture the in-place `lastUsedAt` writes in `src/codex/routing.ts` (at its size cap) + because the touch reads the entry's `lastUsedAt` at flush time. A flush runs the whole queue in ONE `BEGIN IMMEDIATE` transaction inside + `withConfigDirHardeningSettled(home, syncCallback)` with an in-callback `scope.active && getConfigDir() === scope.home` check, then prunes: + `DELETE` rows idle > 24 h or future-dated, and rows beyond the newest 2048 per table (ORDER BY last_used_at DESC, row_key, scope — + deterministic, detour scopes evicted first for threads, mirroring `pruneLruThreadAffinities`). +- Contention: SQLITE_BUSY -> nothing committed, queue kept, retry on the next coalesced flush; one identifier-free warning after repeated + failure. Queue bound: one entry per key (coalesced) plus per-account and full clears; if it exceeds 16384 entries (persistent failure), + drop queued touches first, then convert the remainder to `clearAll` + binds of current live rows — the failure direction loses pins + (today's behavior), never revives a deleted one. +- Same-row conflicts between processes resolve in commit order (SQLite serializes writers); equal timestamps need no tie rule. +- Best-effort shutdown flush via `registerOptionalShutdownHook("codex-affinity", ...)`; `trackBackgroundWork` around flushes; the db handle is + closed on scope retirement and in a test reset export. + +## Reads: lazy adoption straight from the database + +- No hydrated restored map. On a live-map miss for key K whose anchor is known in the current scope: + 1. If the queue holds a delete/clear covering rowKey(K) (or a pending `clearAll`), treat as absent. + 2. Otherwise `SELECT` the rows for rowKey(K) (WAL readers are not blocked by writers; a BUSY/error read is treated as absent). + 3. Validate each: TTL, not future-dated, scope rules (020), account record live with matching `quotaHistoryIdentity` fence (030). Valid -> + adopt into `threadAccountMap` with the record's current generation, count it in `threadAffinityEntryTotal`, run the existing LRU prune. + Invalid -> enqueue `deleteScope` for it. + 4. Remember per scope that rowKey(K) was consulted (bounded set, same TTL) so a second miss does not query again. +- Issuers: same on `peekConversationStateIssuer` miss; fence mismatch or null -> `CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER` (030). +- Because writers go through the queue and reads check it first, a delete in this process can never be re-adopted from disk before it + is flushed, and a delete in another process is visible as soon as it commits. + +## Issuer fence capture (residual from round 3) + +`rememberServingConversationStateIssuer` passes `{ generation, chatgptAccountId }` from the pool auth context; the fence is captured only if the +record's current generation AND `credential.chatgptAccountId` both match (as `account-store.ts:382` does for dispatched credentials). + +## Tests (adds to/replaces 020+030 test list where storage-specific) + +- Two-process restart through `resolveCodexAccountForThread` (red-on-old from a `git archive origin/dev` export). +- Cross-process: A deletes/clears (row, account, issuer, full) and commits; B, holding the stale pin in memory, touches and flushes; a fresh + process restores nothing. B re-binds after A's clear (a real placement) -> row exists. Concurrent first open -> one secret. +- A process holding a write transaction on `codex-affinity.sqlite` does not block a real config mutation (`withConfigMutationLockSync`) or + a credential save. +- BUSY on flush -> retry succeeds later; queue overflow -> no deleted row revived. +- Ownership: paths recorded; recording failure -> no db created. Corrupt db / symlinked path / unknown user_version -> disabled, untouched. +- Bounds: >2048 rows per table keeps newest; idle and future rows pruned; worst-case UTF-8 detour scopes accepted. +- Everything else from 020/030 (fences, unknown issuer with files via responses and compact, priorAccountId retry, home A->B->A, passive, + privacy: no raw ids in db, warnings identifier-free, db 0600). + +## Docs + +structure/providers/openai-tiers.md "Durable Codex pool affinity" describes the SQLite store (file, schema, secret, fences, queue, touch +semantics, bounds, ownership, contract limits). Rest as in 010/030. + diff --git a/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md new file mode 100644 index 00000000000..c762f6e4f4d --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md @@ -0,0 +1,80 @@ +# 261102 Codex pool affinity persistence — 050 plan revision (answers audit round 4: single writer) + +Amends 040 (SQLite store). Items marked RESOLVED in rounds 3-4 stay: dedicated database separate from the config mutation lock, +fences captured at bind/issuance including `chatgptAccountId` (030/040), unknown issuer (030), observer/home ordering and narrowed +contract (020/030), passive rule, security table, deterministic pruning, regression-test approach. Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## Step back (loop-engineering, repeated reviewer FAIL) + +Rounds 2-4 failed on the same root: several processes writing one store and reconciling stale memory. Instead of more ordering +machinery, the store gets exactly ONE writer per config home, enforced by the operating system, and only the proxy server writes. + +## Arming: only the running proxy persists + +- `armCodexAffinityPersistence()` (store module) sets a process flag; the store opens nothing and writes nothing until armed. + `startServer` arms it synchronously (no await; preserves the Lab-activation window guard in `tests/lab/core-lab-boundary.test.ts`), + via an existing non-capped startup helper in `src/server/index/*` if one fits, otherwise the single line of headroom in + `src/server/index.ts` (cap 893, now 892). CLI processes, management passive reads and every test that does not arm behave + exactly as today: no files, no handles. This also removes Windows open-handle teardown risk from existing tests. +- Tests that exercise the store arm explicitly and close it in teardown via `resetCodexAffinityStoreForTests()` (flush + close + disarm). + +## Single writer: SQLite exclusive locking mode + +- Open sequence (all inside `withConfigDirHardeningSettled(home, sync)`, scope check inside the callback): `assertNotRealHomeUnderTest`, + ownership (below), dir 0700 + `hardenSecretDir` on Windows, `new Database(path, {create:true})`, chmod 0600, then + `PRAGMA busy_timeout=0`, `PRAGMA locking_mode=EXCLUSIVE`, `PRAGMA journal_mode=WAL` (exclusive mode uses no `-shm`), `BEGIN IMMEDIATE` + (acquires and, in exclusive mode, KEEPS the write lock until close), read `user_version`: 0 + empty -> create schema v1 and secret; + 1 -> continue; anything else -> ROLLBACK, close, disable (file untouched). COMMIT. Any failure -> rollback, close the handle, classify. +- Classification: SQLITE_BUSY/LOCKED at open = another process owns the store -> state `standby`; corrupt/NOTADB/symlink/non-regular/ + unknown version/ACL failure -> `disabled` for the scope (one identifier-free warning, file untouched). +- `standby`: affinity is process-local (today's behavior). The open is retried on a later affinity access at most every 2 s + (no timer). This covers a restart whose old process is still exiting. While in standby, mutations are recorded in the pending + log (below) bounded at 4096 key groups; on overflow the oldest key group (all ops for that rowKey) is dropped — the database then + keeps its pre-gap state for that key, which is the state persistence intends to restore and is fence-checked on adoption. +- The lock is released on close and on process exit (OS file lock), so a crash never wedges the next start. +- Two-process tests: B opening while A holds the store gets standby and never writes; A closes -> B's retry acquires; A crash + (kill) -> next process acquires. + +## In-process write log (answers round-4 blockers 1-3) + +- Ordered structural log: bind, deleteScope, deleteKey, deleteAccountPins, clearAll, clearIssuers, issuerSet, issuerDelete appended in + call order. No per-key coalescing of structural ops; `clearAll` may drop entries before it (they are dominated), nothing else is dropped + except the standby overflow rule above. +- Touch set (rowKey+scope -> latest, issuer rowKey -> latest), separate from the log. At flush, in ONE transaction: replay the log in + order, then apply touches as `UPDATE ... SET last_used_at = max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?`. + Bind-then-touch: the bind inserts, the touch updates it. Delete-then-touch: the touch matches nothing. A touch never inserts. +- With a single writer, commit order equals this process's call order, so stale cross-process writes cannot exist; validation-driven + deletes are unconditional because nothing else can have replaced the row. +- Flush failure while open (disk full, I/O): rollback, keep log and touches, retry on the next flush. After 3 consecutive failures: + close and unlink the database (an owned path) so no stale pins can be restored, disable for the process, warn once. If the unlink + fails too, warn; the next start only adopts rows that pass the fence and are then re-checked by the resolver. + +## Reads + +On a live-map miss with a known anchor and the store open: if the log holds a delete/clear covering rowKey(K) -> absent; else +`SELECT` (we hold the only connection, so no BUSY). Valid -> adopt (030 validation, current generation, entry count + LRU). Invalid -> +append `deleteScope`. A per-scope "consulted" set prevents re-querying the same rowKey; read errors are not cached and do not disable +unless classified corrupt. In standby, misses do not read. + +## Ownership (round-4 blocker 4) + +- Add `codex-affinity.sqlite`, `codex-affinity.sqlite-wal`, `codex-affinity.sqlite-journal` (and `-shm` defensively) to + `INITIAL_OWNED_PATHS` in `src/lib/config-ownership.ts`, next to the `config-mutation.sqlite*` entries. Every home created from now + on owns the files from creation, with no manifest update and no race. +- Existing homes: `recordOwnedConfigPath` for each path before first create; a `false` result -> do not create the database (warn once). + The residual race — another process holding an older cached manifest rewriting it — is a property of `recordOwnedConfigPath` shared by + every config writer today (`atomicWriteFileNoFollow` default path, `config-mutation.sqlite`, the Anthropic affinity file); fixing that + API is outside this lane and is noted in the PR as pre-existing. Tests: new home lists the paths; existing owned home records them; + recording failure creates nothing; `removeOwnedConfigState` removes the db and sidecars. + +## Handle lifetime + +Closed on: scope retirement (home change), `resetCodexAffinityStoreForTests`, the shutdown hook (flush then close), the failure path. +Windows test: arm, use, reset, `removeTreeWithRetry` succeeds. + +## Guards to extend/run + +`tests/ci-workflows/test-home-guard.test.ts` (the new writer refuses before any filesystem mutation when pointed at the real home), +`tests/lab/core-lab-boundary.test.ts` (new imports do not reach Lab; startServer window scan), config-ownership tests (INITIAL list), +file-size ratchet, test-layout guards, plus the 020/030/040 test list adjusted: cross-process cases become single-writer cases above. + diff --git a/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md new file mode 100644 index 00000000000..747af52310d --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md @@ -0,0 +1,69 @@ +# 261102 Codex pool affinity persistence — 060 plan revision (answers audit round 5: reuse the existing single-writer lease) + +Amends 050. Items resolved in rounds 3-5 stay (dedicated database, fences captured at bind/issuance incl. chatgptAccountId, unknown +issuer, observer/home ordering and narrowed contract, passive rule, security table, ordered structural log + UPDATE-only touch set, +deterministic pruning, regression-test approach). 050's own lease (locking_mode=EXCLUSIVE), standby queue, overflow drop rule and +failure unlink are REMOVED. Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## The product already has one serving writer per state directory + +`src/lib/spend-ledger-owner.ts` holds a process-lifetime SQLite write transaction on `spend-ledger-owner.sqlite` (acquireSpendLedgerOwner:144). +`startServer` acquires it first (`src/server/index.ts:210` via `acquireSpendLedgerServerLifecycle`); a second server on the same home fails with +`SPEND_LEDGER_OWNER_BUSY` and does not serve; a restart child waits up to 5 s for the parent's release (restartHandoffWaitMs). Leases are +reference-counted per process; release hooks (`onSpendLedgerOwnerReleased`) run BEFORE the owner transaction is rolled back and closed +(leaseFor:200-213), i.e. while the lock is still held. `mintSpendLedgerStorage(fileName)` issues a storage token bound to the current owner +generation and `assertStorageOwned(token)` refuses it after any release/reacquire (lines 255-326). The affinity store uses exactly this. + +## Activation and writes + +- The store is active for a config home only while `mintSpendLedgerStorage("codex-affinity.sqlite")` succeeds (this process owns the lease for + the current home). Otherwise affinity is process-local, exactly as today: CLI processes, passive reads, tests that never start a server. + No separate arming call; no change to `src/server/index.ts`. +- Every open and every flush first calls `assertStorageOwned(token)`; NOT_HELD or a generation change retires the scope (close the handle, + drop log/touches, memory-only) and writes nothing. A newer owner generation mints a new token and a new scope. +- One `onSpendLedgerOwnerReleased` hook (registered once): synchronously flush the pending log and touches, set `meta.clean = 1` in the same + transaction, close the handle, retire the scope. It runs while the lease is still held, so no next owner can open the database before the + final state is committed. If that flush fails, `clean` stays 0 (below). Mutations after release are memory-only (the server has stopped serving). +- Open (lazy, first affinity access while owned), synchronous like `prepareOwnerPath`: `assertNotRealHomeUnderTest` FIRST (before ownership + initialization), `recordOwnedConfigPath` for the db and its `-journal`/`-wal`/`-shm` sidecars (every boolean checked; any false -> do not create, + warn once), create with `openSync(path, "wx", 0o600)` tolerant of EEXIST, `assertPrivateFile`-equivalent (regular, not a symlink, nlink 1, + owned by uid on POSIX), `hardenSecretPath(path, { required: true })` on Windows. Then `new Database(path)`, `PRAGMA busy_timeout = 0` (the lease + guarantees no other writer; any BUSY is an anomaly -> disable), default rollback journal (no journal-mode change, so nothing is modified + before the version check), `BEGIN IMMEDIATE`, read `user_version` and `sqlite_master`: empty v0 -> create schema v1 + `meta.secret` + `meta.clean=0`; + v1 -> read `meta.clean`; if 0 (previous owner crashed or failed its final flush) DELETE all thread and issuer rows (fail closed: pins + fall back to normal selection, never revive a release that was not committed); then set `meta.clean = 0` for this ownership; any other + version/shape -> ROLLBACK, close, disable, file untouched. COMMIT. Any failure: rollback, close, classify, identifier-free warning. +- Flushes are synchronous SQLite transactions scheduled from a microtask (structural) or a 1 s unref'd timer (touches); no async ACL + barrier is needed because no file is created after open (the transient `-journal` inherits the hardened directory, as for the owner db). +- Flush failure: rollback, keep the log, retry next flush. If the log exceeds 16384 entries or 3 consecutive flushes fail, the scope is + POISONED: the log is dropped, nothing more is written, and `clean` is never set to 1 — the next owner discards every row. No unlink, no + handoff race: the file stays under the lease until this process releases or dies. + +## Why the round-5 blockers are closed + +1. Standby replay: there is no standby. Only the lease holder writes, and a new holder can open only after the previous holder's release + hook committed (or it died, in which case `clean = 0` discards everything). Decisions are never replayed across ownerships. +2. Close-then-unlink: removed. Invalidation is `clean = 0` + DELETE at the next open, serialized by the lease. +3. Dropped deletions: no drop rule. Overflow poisons the scope, which fails closed at the next start. +4. Ownership race for existing homes: REBUTTED as pre-existing and out of lane, with mitigation. New homes own the four paths from creation + via `INITIAL_OWNED_PATHS` (no manifest update, no race). Existing homes use `recordOwnedConfigPath` before creation — the identical + mechanism `spend-ledger-owner.sqlite` (prepareOwnerPath:93), `config-mutation.sqlite` (mutation-lock.ts:62) and every default + `atomicWriteFileNoFollow` write rely on today. The stale-cache rewrite is a defect of that shared API; fixing it would change every + writer and is outside this lane's scope. The PR states it as a residual: if another process's stale manifest rewrite drops the entry, + uninstall can leave `codex-affinity.sqlite` (row digests + opaque pool ids, no tokens) behind. A follow-up issue will be filed for the API. + +## Tests (storage-specific; replaces 050's cross-process list) + +- Two-process restart through `resolveCodexAccountForThread` with real `startServer`-equivalent lease acquisition (or `acquireSpendLedgerOwner` + directly) — red-on-old from a `git archive origin/dev` export. +- Not owned -> no file created, no handle; owned -> created 0600 with ownership recorded; INITIAL list contains the four names. +- Release hook: pending structural op made just before release is committed and `clean = 1`; next owner restores it. +- Crash: kill an owner subprocess with pending/committed rows -> next owner finds `clean = 0` and restores nothing. +- Poison: injected flush failures / log overflow -> next owner restores nothing. +- Release + reacquire in one process (generation change): old token refused, new scope, no writes from the old one. +- Second server on the same home is refused by the existing lease (document; existing test covers BUSY) and never opens the affinity db. +- Unknown version / non-v1 schema -> untouched bytes, disabled; symlink or non-regular path -> disabled. +- Windows: start, use, stop (release hook closes), `removeTreeWithRetry` succeeds. +- Everything else from 020/030/040 (fences, unknown issuer via responses/compact/retry, home switch, passive, bounds, privacy). +- Guards: test-home guard (real home refused before ownership init), core-lab boundary, config-ownership tests, file-size ratchet, test layout. + diff --git a/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md new file mode 100644 index 00000000000..a41b1448550 --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md @@ -0,0 +1,45 @@ +# 261102 Codex pool affinity persistence — 070 plan revision (answers audit round 6) + +Amends 060; everything else in 060 stands. Issue: https://github.com/lidge-jun/opencodex/issues/8091 + +## R6 blocker — final flush runs while ownership is still logically held + +`leaseFor` (src/lib/spend-ledger-owner.ts:200-213) decrements references and nulls `activeOwner` before running `releaseHooks`, so a hook +cannot pass `assertStorageOwned`. Change (minimal, in spend-ledger-owner.ts): + +- New `onSpendLedgerOwnerReleasing(hook: () => void): void` registry. In `release()`, when this is the FINAL reference + (`owner.references === 1`), set a module flag `releasing = true`, run the releasing hooks (each in try/catch), clear the flag, then + decrement, null `activeOwner`, run the existing `releaseHooks`, ROLLBACK and close as today. During releasing hooks the owner, its + generation and the physical lock are unchanged, so `assertStorageOwned(token)` passes for the current-generation token only. +- While `releasing` is true, `acquireSpendLedgerOwner` and `mintSpendLedgerStorage` throw NOT_HELD (no reentrant acquisition or new tokens + from inside a hook). Ordinary flushes after release fail `assertStorageOwned` exactly as before; stale tokens from an earlier generation + are refused as before. +- Optional neutral alias: export `mintStateDirectoryStorage`/`assertStateDirectoryStorageOwned` as thin aliases of the spend-named functions, + with a doc comment that the lease is the state directory's single-writer primitive. (Implementer may skip if maintainers' naming rules + object; behavior is identical.) +- The affinity store registers ONE releasing hook: sync flush of log + touches, `meta.clean = 1` in the same transaction, close, retire scope. + Hook failure -> rollback, `clean` stays 0, close, retire (fail closed). The existing `onSpendLedgerOwnerReleased` hook is used only to make + sure the scope is retired if the releasing hook did not run (defensive). +- Tests: pending structural op committed and restored across a real final release (two leases in one process: first release does not flush/close, + final does); stale-token refusal after release + reacquire; hook failure leaves `clean = 0` and the next owner restores nothing; acquire/mint + inside a releasing hook is refused; existing tests/lib/spend-ledger-owner.test.ts and tests/server/spend-ledger-owner-startup.test.ts still pass. + +## R6 residuals, accepted and specified + +- Stop failure keeps ownership until process exit; abrupt exit has no release hook -> `clean = 0` path (pins discarded). Documented. +- The final transaction is synchronous and bounded (at most the pending log, itself bounded at 16384 entries + touches); filesystem stalls + can still delay shutdown. Documented in structure. +- After final release, late async work is memory-only: the scope is retired, its token is invalid, nothing reopens under it and `clean` is never + touched again by that scope. +- Multiple servers in one process share the lease; the store stays live until the last reference releases. +- Test isolation (060 wording corrected): tests that start a server AND exercise affinity will create `codex-affinity.sqlite` in their isolated + home; the handle closes on the final server stop. Implementer runs every suite that starts a server and uses pool affinity + (`rg -l startServer tests` intersected with affinity entry points), checks teardown order, and adds a Windows-safe close in any helper that + resets state without stopping the server. +- Passive calls stay passive under a held lease: no open, adoption, scheduling or flush. Lazy open happens before any durable deletion that + must invalidate an unadopted row (a delete for a key with a known anchor opens the store first when owned). +- Ownership race for existing homes: residual stated in the PR; a follow-up issue for `recordOwnedConfigPath` is filed before the PR opens + and linked from it. +- Unsupported version/shape tests assert the application wrote nothing (no schema/meta change, no row change); they do not claim byte-for-byte + identity where SQLite's own hot-journal recovery may touch the file. + diff --git a/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md b/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md new file mode 100644 index 00000000000..afc55c18519 --- /dev/null +++ b/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md @@ -0,0 +1,188 @@ +# 261102 Codex pool affinity persistence — 080 consolidated implementation spec + +Single source of truth for implementation. Consolidates 010-070 and the round-7 residuals; where they differ, this file wins. +Issue: https://github.com/lidge-jun/opencodex/issues/8091. Follow-up (ownership manifest race): https://github.com/lidge-jun/opencodex/issues/8093. +Security-sensitive (MAINTAINERS.md): a dedicated Terra security review runs before the PR and its verdict goes in the PR description. + +## 1. Problem + +Codex pool pins (`threadAccountMap`) and conversation-state issuers (`conversationStateIssuerMap`) in `src/codex/routing/thread-affinity.ts` +are in memory only, and the key they use is `codexConversationKeyFor()` in `src/codex/lineage.ts`, an HMAC under a per-process random secret. +A restart re-places every conversation (round-robin moves most of them: cold prompt cache) and loses the issuer that triggers the +account-change scrub (carried encrypted state can fail with `invalid_encrypted_content`). + +## 2. Contract + +- A pin survives a graceful proxy restart for up to 24 h idle when the request's cohort anchor is derivable from its headers: every request + with `session-id` (Codex CLI/App roots and subagents) and session-less requests anchored on their immediate parent. Deeper session-less + chains, keys first derived under another config home, legacy raw-parent keys and keys whose anchor was evicted are memory-only. +- Pins whose account is gone, re-logged (new `quotaHistoryIdentity`), legacy (no identity) or expired are dropped: normal selection. + Usability (pause, reauth, cooldown, plan/model exclusion, transient hold) stays with the existing resolver (`resolveCodexAccountForThread`, src/codex/routing.ts:498). +- `__main__` pins and `__main__` issuers are not persisted. +- A crash, a failed final flush, or a poisoned store makes the next owner discard every row (fail closed). +- Only the process holding the state directory's single-writer lease (`src/lib/spend-ledger-owner.ts`, acquired by `startServer`) persists. + A server in another process on the same home is refused by that lease; servers in the same process share it (reference-counted). + +## 3. Security posture + +- `CODEX_LINEAGE_KEY` and every value derived from it stay process-local and unchanged: pin/issuer map keys, `authCtx.affinityKey`, + `_cursorClientThreadId` (request-prepare.ts:417,563; cursor request-builder.ts:385; advisor state.ts:213 derives its own hash from it), + compact's issuer binding key (compact.ts:1684), lineage scope keys, affinity-debug tags (own key, ADR-0089). Request-log lanes use their + own digest of raw headers. Previous-response replay is keyed by response ids (implementer cites file:line in the PR). None changes. +- New durable artifact: `/codex-affinity.sqlite` (0600, Windows ACL via `hardenSecretPath`). It holds a 32-byte store secret + (`meta.secret`), rows keyed by `HMAC-SHA256(secret, "opencodex/codex-affinity/conversation/v1\0" + anchor)` (issuers: `.../issuer/v1`), + opaque pool account ids, identity fences `sha256("opencodex/codex-affinity/identity/v1\0" + quotaHistoryIdentity)`, scopes and timestamps. + No raw session/thread ids, no ephemeral keys, no tokens, no emails. Anyone who can read two copies of the file can link identical row keys + and pool ids across restarts; the secret is needed only to test a guessed anchor. Never logged; warnings are identifier-free. + +## 4. Modules and changes + +### 4.1 `src/codex/lineage.ts` +- Add `setCodexConversationAnchorObserver(fn?: (key: string, anchor: string) => void)`; `codexConversationKeyFor` calls it after deriving + (`anchor` = the family/thread value; for the cohort form both are the anchor). No new imports. Update the LIFETIME doc block: keys stay + process-local; the affinity store persists only digests under its own secret. + +### 4.2 `src/lib/spend-ledger-owner.ts` +- `onSpendLedgerOwnerReleasing(hook)`: in `release()`, after the per-lease `released` idempotence guard and the existing owner/refcount checks, + when this is the final reference (`owner.references === 1`): set `releasing = true`; run each releasing hook in its own try/catch; + clear `releasing` in `finally`; then continue exactly as today (decrement, null `activeOwner`, `boundLedgerHome = null`, release hooks, + ROLLBACK/close, throw on rollback/close failure). +- While `releasing`, `acquireSpendLedgerOwner` and `mintSpendLedgerStorage` throw `SPEND_LEDGER_OWNER_NOT_HELD`. +- Optional neutral aliases (`mintStateDirectoryStorage`, `assertStateDirectoryStorageOwned`) with a doc comment; behavior identical. +- Tests: throwing releasing hook does not block release; repeated release is a no-op; first of two leases does not run hooks; acquire/mint inside + a hook refused; existing `tests/lib/spend-ledger-owner.test.ts`, `tests/server/spend-ledger-owner-startup.test.ts` pass. + +### 4.3 `src/lib/config-ownership.ts` +- Add `codex-affinity.sqlite`, `-journal`, `-wal`, `-shm` to `INITIAL_OWNED_PATHS` next to the `config-mutation.sqlite*` entries. + +### 4.4 New `src/codex/routing/thread-affinity-store.ts` +Imports only `src/config/*`, `src/lib/*`, `bun:sqlite`, node builtins and the account-store reader (no routing value import cycle, no Lab). + +Scope = (config home, owner generation, storage token). Created lazily on the first affinity access when `mintSpendLedgerStorage("codex-affinity.sqlite")` +succeeds and `isPassiveRead()` is false. A home change or generation change retires the scope (close handle, drop log/touches/anchors/consulted set) +and clears the live affinity maps (as the Claude store drops its rows on home change). The anchor observer resolves the active scope FIRST, +then records `anchorByKey` (LRU, 4096, 24 h TTL, refreshed per derivation). Passive calls do nothing (no open, observe, adopt, schedule, flush). + +Open (synchronous; any failing step -> rollback/close, identifier-free warning, scope disabled for its lifetime. Earlier steps may already have +created the empty file or set its mode/ACL; for an unsupported version or shape the app makes no schema, metadata or row change, though SQLite's +own hot-journal recovery may still alter bytes): +1. `assertNotRealHomeUnderTest(home)` before anything else. +2. `recordOwnedConfigPath` for the db and its three sidecars (`-journal`, `-wal`, `-shm`); any `false` -> do not create. +3. `openSync(path, "wx", 0o600)` tolerating EEXIST; regular file, not a symlink, nlink 1, uid owner on POSIX; chmod 0600; `hardenSecretPath(path, {required:true})`; re-check. +4. `assertStorageOwned(token)`; `new Database(path)`; `PRAGMA busy_timeout = 0`; default rollback journal (no journal-mode change). +5. `BEGIN IMMEDIATE`; read `user_version` and `sqlite_master`. Empty v0 -> create schema v1, `meta.secret` (32 random bytes), `meta.clean = 0`. + v1 with the exact expected tables -> if `meta.clean != 1`, DELETE all thread and issuer rows; set `meta.clean = 0`. Anything else -> ROLLBACK, close, disable. + COMMIT. BUSY at any point is an anomaly under the lease -> disable. + +Schema v1: `meta(name TEXT PRIMARY KEY, value BLOB)`; `threads(row_key TEXT, scope TEXT, account_id TEXT, fence TEXT, created_at INTEGER, last_used_at INTEGER, PRIMARY KEY(row_key, scope))`; +`issuers(row_key TEXT PRIMARY KEY, account_id TEXT, fence TEXT, last_used_at INTEGER)` (fence nullable). + +Writes: +- Ordered structural log in call order: bind(rowKey, scope, accountId, fence, createdAt, lastUsedAt) = upsert; deleteScope; deleteKey; + deleteAccountPins(accountId) (pins only; issuers kept, as today); issuerSet (upsert); issuerDelete; clearIssuers; clearAll. Only `clearAll` + may drop earlier entries. Ops are logged only for keys with a known anchor in the scope (account-wide and clear ops always). +- Touch set (rows touched since the last flush) applied after the log in the same transaction. The timestamp is read from the LIVE in-memory entry + AT FLUSH TIME, never captured when the touch is scheduled: the getter schedules the touch before `src/codex/routing.ts` mutates the returned entry + (routing.ts:903-911), so a captured value would persist the old time and could expire an active pin. A touched row whose live entry is gone is skipped. Statement: + `UPDATE ... SET last_used_at = max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?` (issuers: `account_id` and `fence IS ?`). Never inserts. +- Structural ops schedule a microtask flush; touches (same-account rebind, binding read, issuer peek) a 1 s unref'd timer. A binding read schedules a + touch so the in-place `lastUsedAt` writes in `src/codex/routing.ts:911,942,985,1021` are captured (that file is at its size cap and is not edited). +- Flush = `assertStorageOwned(token)`, one `BEGIN IMMEDIATE` transaction: log, touches, prune (idle > 24 h or future-dated; newest 2048 per table ordered by + `last_used_at DESC, row_key, scope`, model-detour scopes evicted first). Failure -> ROLLBACK, keep log, retry next flush. 3 consecutive failures or log + > 16384 entries -> POISONED: drop log/touches, never write again, `clean` stays 0. +- Releasing hook (registered once via `onSpendLedgerOwnerReleasing`): if the scope is open and not poisoned, one transaction = log + touches + prune + + `meta.clean = 1`. Transaction failure -> ROLLBACK (clean stays 0). Then close; a close error after a successful COMMIT leaves `clean = 1` committed + (state is consistent: everything was committed) and is reported as an identifier-free warning. Retire the scope. `onSpendLedgerOwnerReleased` retires + any scope still open (defensive). Late async work after release is memory-only and never reopens under the old token. + +Reads (adoption): +- On a live-map miss for key K with a known anchor in an open scope: a pending delete/clear in the log covering rowKey(K) -> absent. Otherwise SELECT. + Thread row valid when: not idle > 24 h, not future-dated, scope valid (`legacy`, a known `CodexQuotaScope`, or `model-detour::` whose model + passes the existing 512-byte component rule), account record has a credential, not deleted, current `quotaHistoryIdentity` fence equals the stored fence. + Valid -> insert into `threadAccountMap` with the record's CURRENT generation, `lastReevalAt = now`, no transient fields, increment + `threadAffinityEntryTotal`, run the existing LRU prune. Invalid -> log deleteScope. +- Issuer row: idle > 24 h or future-dated -> absent and log issuerDelete (time passes after the last flush prune, and the live issuer map also + expires on peek, thread-affinity.ts:238,274). Otherwise fence equal -> issuer = account id; fence null or different -> `CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER`. +- A bounded per-scope consulted set avoids re-querying; read errors are not cached. +- `getThreadAffinityScopes(K)` adopts all restored scopes for K first. + +### 4.5 `src/codex/routing/thread-affinity.ts` (call-site hooks only; not the affinity log label) +- `ThreadAffinityEntry.identityFence?: string`, captured in `bindThreadAffinityForScope` from the same record read that supplies `generation`; + kept by `handOffThreadAffinityGeneration`. Entries without a fence (legacy record, `__main__`) are memory-only. +- `rememberConversationStateIssuer(bindingKey, accountId, now?, credential?: { generation, chatgptAccountId })`: fence captured only when the record's current + generation AND `credential.chatgptAccountId` match; else the row is fence-less (restores as unknown). `__main__` not persisted. +- Export `CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER` (a string no account id can equal). +- Hooks in: scoped getters (adopt on miss; touch on hit), `getThreadAffinityScopes`, bind/delete/delete-for-account, `clearThreadAccountMapForAccount`, + TTL/LRU prune, `handOffThreadAffinityGeneration`, issuer remember/peek/clear, `clearThreadAccountMap` (clearAll). +- Update the lifetime comments (issuer map, legacy adoption). + +### 4.6 `src/server/responses/account-change-state.ts` +- `rememberServingConversationStateIssuer` passes `{ generation, chatgptAccountId }` for pool contexts. +- `applyAccountChangeConversationStateScrub`: unknown issuer counts as changed. +- `accountChangeFileReferenceRefusal`: unknown issuer ignored; refuse only for a known different issuer or a `priorAccountId` move. + +### 4.7 Comment-only +`src/adapters/google-antigravity-wire.ts` comment about the process-random key: keep the meaning (the key still changes across restarts), adjust wording only if needed. + +## 5. Reset and removal + +| Entry point | Durable effect | +|---|---| +| `clearThreadAccountMap()` (config/provider/model/combo/agent-settings/logs-usage/oauth-account/shared management routes, server index) | clearAll | +| `clearConversationStateIssuerMap()` | clearIssuers | +| `clearThreadAccountMapForAccount(id)` (removal, auto-pause, plan exclusion, quota refusal, preservation) | deleteAccountPins (issuers kept) | +| account deleted/re-logged without a clear | fence fails on adoption -> row deleted | +| uninstall | owned paths removed (new homes always; existing homes subject to #8093) | + +## 6. Tests + +New file `tests/codex-integration/codex-affinity-restart.test.ts` (+ helper worker under `tests/helpers/` if needed); register in +`scripts/test-layout/layout.json` (near line 709) and `tests/fixtures/test-layout-expected.json` (near line 697). Lease tests may live in +`tests/lib/spend-ledger-owner.test.ts` if under its size cap, else a sibling file (registered). Cases: + +1. Red-on-old: two fresh `bun` subprocesses on one temp home, each acquiring the lease, seeding two pool accounts with round-robin, resolving a + session-bearing request via `resolveCodexAccountForThread` with `codexPoolAffinityKey(headers)`, recording the issuer, releasing. Process 2 advances + the cursor so a fresh pick chooses the other account. Expect same account and known issuer (scrub not triggered for the same account; triggered for + a different one). Old code (isolated `git archive origin/dev` export with the test + helper copied, only the optional store import caught) fails on + the assertion; record output. +2. Fallbacks: removed, re-logged, legacy, expired, future -> normal selection, row deleted. Usability cases asserted against the resolver one by one. +3. `__main__`: pin and issuer not persisted. +4. Session-less depth-1 survives; depth-2 re-places; anchor eviction -> memory-only. +5. Reconciliation before first lookup: bind/delete/delete-for-account/issuer set/clear/clear-for-account on a key with a restored, unadopted row. +6. Fence races: re-login between bind and release; re-login between dispatch and issuer completion; owned refresh/handoff; Orca rotation; delete + re-add. +7. Unknown issuer with file references via responses and compact; `priorAccountId` retry still refuses; known different issuer still refuses. +8. Lease: not owned -> no file; release flush commits pending op and `clean = 1`; two leases -> only final release flushes; crash (kill subprocess) -> next owner + restores nothing; poison (injected failures, overflow) -> next owner restores nothing; transaction failure in the releasing hook -> clean 0; close failure + after commit -> rows restored; release + reacquire -> old token refused. +9. Database shape: unknown version/shape -> app makes no schema, metadata or row change (SQLite hot-journal recovery may still alter bytes) and disables; symlink/non-regular -> disabled; file 0600; INITIAL list has the four names; + existing owned home records them; recording failure -> no db. +10. Bounds and privacy: > 2048 rows per table keeps newest; worst-case UTF-8 detour scope accepted; no raw ids/ephemeral keys in the db; warnings identifier-free. + Touch timing: resolver reuse (which mutates `lastUsedAt` after the getter) then release and restart persists the NEW timestamp; an issuer row that + expired or is future-dated at read time is treated as absent and deleted. +11. Home A -> B -> A, first B request, recorded lineage; passive calls with a populated db and a held lease. +12. Windows: start, use, stop, `removeTreeWithRetry` succeeds. + +Existing suites to run: every test file using the affinity entry points or `startServer` + pool affinity (the round-1 audit counted 38), lineage tests, +`tests/responses/responses-account-change-scrub.test.ts`, `tests/responses/responses-4546-incident-regression.test.ts`, owner tests, config-ownership tests, +`tests/lab/core-lab-boundary.test.ts`, `tests/ci-workflows/test-home-guard.test.ts` (new writer refused before ownership init), file-size ratchet, +`tests/test-layout.test.ts`, `tests/test-layout-tooling.test.ts`. + +## 7. Docs + +- `structure/providers/openai-tiers.md`: lineage section (key stays process-local) + new "Durable Codex pool affinity" subsection (contract, file, schema, + secret, fences, lease, clean flag, write log, touches, bounds, ownership residual #8093, tests). +- `structure/providers/openai-accounts.md`: only claims that pins are process-local (not the diagnostic tags or idle-window reservations). +- `structure/` owner of `src/lib/spend-ledger-owner.ts` (find via structure/manifest.json): document releasing hooks. +- docs-site: `reference/cli/providers-accounts.md` (~848) and `reference/configuration/providers.md` (`accountPoolStrategy` row) restart wording, in English + and every locale copy (search per locale); build the docs site per docs-site/AGENTS.md. + +## 8. Verification + +Focused tests above, `bun run typecheck`, `bun run structure:check`, `bun run privacy:scan`, docs-site build, `bun scripts/test.ts --changed=dev --parallel=1`, +red-on-old recorded. PR gets `ci:full` (subprocess tests, docs, on-disk contracts). + +## 9. Out of scope + +`src/codex/routing/selection.ts` (Lane C), `src/codex/auth-context.ts` and the affinity log label (Lane D), `src/server/claude-*` and +`src/server/caller-session-identity.ts` (Lane A), `src/codex/routing.ts` and `src/server/index.ts` (size caps), the ownership API fix (#8093). + diff --git a/src/codex/lineage.ts b/src/codex/lineage.ts index 4788efbe274..821b0f74b6f 100644 --- a/src/codex/lineage.ts +++ b/src/codex/lineage.ts @@ -31,12 +31,10 @@ * posture as the affinity key itself. Two callers presenting identical thread ids can never read * each other's lineage, and no raw identifier or durable hash is stored. * - * LIFETIME, stated plainly because the word "affinity" invites the opposite assumption: none of - * this survives the process. The binding map is in memory, and the HMAC key above is fresh random - * bytes taken at module load, so a restart does not merely forget the table -- it makes yesterday's - * keys unreproducible. This is a warm-start hint for the life of one proxy process, never durable - * account ownership, and nothing here should be read as a promise to a conversation that outlives - * a restart. + * LIFETIME: lineage records and their keys remain process-local. The HMAC key is fresh random + * bytes taken at module load, so yesterday's keys cannot be reproduced after a restart. An + * affinity store may persist separate digests of cohort anchors under its own secret; it must + * never persist these process-local keys. * * The one upgrade that is neither a fresh start nor an untouched process is a code swap under a * live conversation, where the binding map is still populated with entries made under the @@ -50,6 +48,12 @@ import { retainedUtf8Bytes } from "../lib/admission"; const CODEX_LINEAGE_COMPONENT_MAX_BYTES = 512; const CODEX_LINEAGE_KEY = randomBytes(32); +let conversationAnchorObserver: ((key: string, anchor: string) => void) | undefined; + +/** Register the consumer of derived cohort anchors, or remove it. */ +export function setCodexConversationAnchorObserver(fn?: (key: string, anchor: string) => void): void { + conversationAnchorObserver = fn; +} /** * Mirrors `CODEX_THREAD_AFFINITY_IDLE_TTL_MS` in ./routing. Deliberately duplicated rather than @@ -149,12 +153,14 @@ function boundedLineageComponent(value: string | null): string | undefined { * byte-identical to the key the thread actually binds under. */ export function codexConversationKeyFor(familyId: string, threadId: string): string { - return `app:${createHmac("sha256", CODEX_LINEAGE_KEY) + const key = `app:${createHmac("sha256", CODEX_LINEAGE_KEY) .update("opencodex-app-pool-affinity-v1\0") .update(familyId) .update("\0") .update(threadId) .digest("base64url")}`; + conversationAnchorObserver?.(key, familyId); + return key; } /** diff --git a/src/lib/config-ownership.ts b/src/lib/config-ownership.ts index a3494002b34..9b799fa9a83 100644 --- a/src/lib/config-ownership.ts +++ b/src/lib/config-ownership.ts @@ -53,6 +53,10 @@ const INITIAL_OWNED_PATHS = [ "config-mutation.sqlite-journal", "config-mutation.sqlite-shm", "config-mutation.sqlite-wal", + "codex-affinity.sqlite", + "codex-affinity.sqlite-journal", + "codex-affinity.sqlite-shm", + "codex-affinity.sqlite-wal", "local-credentials.json", "local-credential-observers", "local-credential-publication.sqlite", diff --git a/src/lib/spend-ledger-owner.ts b/src/lib/spend-ledger-owner.ts index 40cf6dd07b4..00c6f9a0187 100644 --- a/src/lib/spend-ledger-owner.ts +++ b/src/lib/spend-ledger-owner.ts @@ -47,6 +47,20 @@ let boundLedgerHome: string | null = null; */ let ownerGeneration = 0; const releaseHooks: Array<() => void> = []; +const releasingHooks: Array<() => void> = []; +let releasing = false; + +/** Run before the final lease is released, while existing storage tokens remain valid. */ +export function onSpendLedgerOwnerReleasing(hook: () => void): void { + releasingHooks.push(hook); +} + +function assertNotReleasing(): void { + if (releasing) throw new SpendLedgerOwnerError( + "SPEND_LEDGER_OWNER_NOT_HELD", + "Spend-ledger ownership is required before the shared ledger can be used.", + ); +} /** * Run when the last lease on a state directory goes away. @@ -142,6 +156,7 @@ export function spendLedgerRestartEnvironment( * can evict a live owner or mistake a reused process identity for this lease. */ export function acquireSpendLedgerOwner(configDir = getConfigDir()): SpendLedgerOwnerLease { + assertNotReleasing(); const requestedHome = stateDirectoryIdentity(configDir); const busyTimeout = restartHandoffWaitMs(); if (boundLedgerHome !== null && boundLedgerHome !== requestedHome) { @@ -204,6 +219,16 @@ function leaseFor(owner: ActiveOwner): SpendLedgerOwnerLease { if (released) return; released = true; if (activeOwner !== owner || owner.references < 1) return; + if (owner.references === 1) { + releasing = true; + try { + for (const hook of releasingHooks) { + try { hook(); } catch { /* a flush hook must not prevent release */ } + } + } finally { + releasing = false; + } + } owner.references -= 1; if (owner.references > 0) return; activeOwner = null; @@ -265,6 +290,7 @@ function trustedStorage(storage: SpendLedgerStorage): { readonly path: string; r * choose the destination, which is the thing ownership is supposed to decide. */ export function mintSpendLedgerStorage(fileName: string): SpendLedgerStorage { + assertNotReleasing(); if (fileName.includes("/") || fileName.includes("\\") || fileName === "." || fileName === "..") { throw new SpendLedgerOwnerError( "SPEND_LEDGER_OWNER_UNAVAILABLE", diff --git a/structure/transports/responses-spend.md b/structure/transports/responses-spend.md index fd0cbc511aa..9545a45c199 100644 --- a/structure/transports/responses-spend.md +++ b/structure/transports/responses-spend.md @@ -368,7 +368,11 @@ identically with and without configured ceilings: observe-only still appends, se Two servers in one process and one directory share a reference-counted lease; that process cannot hold two directories at once. Sequential ownership is allowed and concurrent ownership is not: releasing the final reference discards the singleton, so a later directory replays its own -journal instead of inheriting figures. A ledger records the ownership it was built under and +journal instead of inheriting figures. +Final-release hooks run before that discard while existing storage tokens remain valid; each hook +is isolated from errors in the others. Acquiring another lease or minting storage during these +hooks is refused. Releasing a non-final reference or releasing the same lease twice runs no hooks. +The ledger records the ownership it was built under and proves that exact identity on every accounting read and change, so a handle kept across a release and a reacquire of the same directory is refused rather than resuming over writes another owner may have made. File-backed journal and salt writers are owner-bound at construction, and a diff --git a/tests/codex-integration/codex-lineage-placement.test.ts b/tests/codex-integration/codex-lineage-placement.test.ts index 496c49e2929..9494bca51be 100644 --- a/tests/codex-integration/codex-lineage-placement.test.ts +++ b/tests/codex-integration/codex-lineage-placement.test.ts @@ -30,11 +30,13 @@ import { CODEX_LINEAGE_MAX_ENTRIES, CODEX_LINEAGE_MAX_SCOPES, clearCodexThreadLineageForTests, + codexConversationKeyFor, codexLineageRootForRequest, codexLineageScopeKey, codexLineageWorkflowLane, codexThreadLineageLookup, recordCodexThreadLineage, + setCodexConversationAnchorObserver, } from "../../src/codex/lineage"; import { clearPoolRotationState } from "../../src/codex/pool-rotation"; import { saveCodexAccountCredential } from "../../src/codex/account-store"; @@ -52,6 +54,21 @@ const ICACLS_OK = { success: true, exitCode: 0, timedOut: false, stdout: "" }; const ACCOUNT_IDS = ["a", "b", "c"] as const; const NOW = 1_700_000_000_000; +test("lineage observer receives the derived key and cohort anchor", () => { + const seen: Array<[string, string]> = []; + setCodexConversationAnchorObserver((key, anchor) => seen.push([key, anchor])); + try { + const key = codexConversationKeyFor("cohort", "cohort"); + expect(seen).toEqual([[key, "cohort"]]); + expect(codexConversationKeyFor("cohort", "cohort")).toBe(key); + expect(seen).toHaveLength(2); + } finally { + setCodexConversationAnchorObserver(); + } + codexConversationKeyFor("cohort", "cohort"); + expect(seen).toHaveLength(2); +}); + function installScratchHome(): void { previousOpencodexHome = process.env.OPENCODEX_HOME; previousCodexHome = process.env.CODEX_HOME; diff --git a/tests/lib/spend-ledger-owner.test.ts b/tests/lib/spend-ledger-owner.test.ts index 25584deb8bb..49690a34ec0 100644 --- a/tests/lib/spend-ledger-owner.test.ts +++ b/tests/lib/spend-ledger-owner.test.ts @@ -7,6 +7,8 @@ import { acquireSpendLedgerOwner, SPEND_LEDGER_OWNER_FILENAME, mintSpendLedgerStorage, + onSpendLedgerOwnerReleasing, + assertStorageOwned, SPEND_LEDGER_RESTART_PARENT_ENV, SpendLedgerOwnerError, spendLedgerRestartEnvironment, @@ -239,6 +241,38 @@ describe("in-process references and privacy", () => { for (const lease of leases.splice(0).reverse()) lease.release(); }); + test("final release runs isolated hooks with existing storage but refuses new ownership", () => { + const first = acquireSpendLedgerOwner(); + const second = acquireSpendLedgerOwner(); + const token = mintSpendLedgerStorage("hook-test.sqlite"); + let calls = 0; + let enabled = true; + let tokenAccepted = false; + const refusalCodes: unknown[] = []; + onSpendLedgerOwnerReleasing(() => { + if (!enabled) return; + calls++; + try { assertStorageOwned(token); tokenAccepted = true; } catch { /* checked below */ } + for (const action of [() => acquireSpendLedgerOwner(), () => mintSpendLedgerStorage("new.sqlite")]) { + try { action(); refusalCodes.push("accepted"); } + catch (error) { refusalCodes.push((error as SpendLedgerOwnerError).code); } + } + throw new Error("hook failure"); + }); + let subsequent = 0; + onSpendLedgerOwnerReleasing(() => { if (enabled) subsequent++; }); + first.release(); + expect(calls).toBe(0); + second.release(); + second.release(); + expect(calls).toBe(1); + expect(subsequent).toBe(1); + expect(tokenAccepted).toBe(true); + expect(refusalCodes).toEqual(["SPEND_LEDGER_OWNER_NOT_HELD", "SPEND_LEDGER_OWNER_NOT_HELD"]); + expect(spendLedgerOwnerSnapshot().ownership).toBe("unheld"); + enabled = false; + }); + test("two leases share ownership and one release does not free it", async () => { const first = acquireSpendLedgerOwner(); const second = acquireSpendLedgerOwner(); @@ -435,6 +469,9 @@ describe("backing file identity", () => { readFileSync(join(home, CONFIG_UNINSTALL_MANIFEST), "utf8"), ) as { paths: string[] }; expect(manifest.paths).toContain(SPEND_LEDGER_OWNER_FILENAME); + for (const suffix of ["", "-journal", "-wal", "-shm"]) { + expect(manifest.paths).toContain(`codex-affinity.sqlite${suffix}`); + } for (const sidecar of ["-journal", "-wal", "-shm"]) { expect(manifest.paths).toContain(`${SPEND_LEDGER_OWNER_FILENAME}${sidecar}`); } From 84fc188953df7b848bb673d6b079879389cf1fa9 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:13:48 +0900 Subject: [PATCH 02/21] wip(codex): durable affinity store module --- scripts/test-layout/layout.json | 3 +- src/codex/lineage.ts | 6 +- src/codex/routing/thread-affinity-store.ts | 177 ++++++++++++++++++ .../codex-affinity-store.test.ts | 105 +++++++++++ .../codex-lineage-placement.test.ts | 10 +- tests/fixtures/test-layout-expected.json | 1 + 6 files changed, 294 insertions(+), 8 deletions(-) create mode 100644 src/codex/routing/thread-affinity-store.ts create mode 100644 tests/codex-integration/codex-affinity-store.test.ts diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 80b5b0185ef..48d2835d1a3 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -645,7 +645,8 @@ "codex-integration-record.test.ts": "codex-integration", "codex-journal-recovery.test.ts": "codex-integration", "codex-journal.test.ts": "codex-integration", "codex-legacy-config-keys.test.ts": "codex-integration", - "codex-lineage-placement.test.ts": "codex-integration", + "codex-affinity-store.test.ts": "codex-integration", + "codex-lineage-placement.test.ts": "codex-integration", "codex-log-guard-coderabbit.test.ts": "codex-integration", "codex-log-guard-doctor-coderabbit.test.ts": "codex-integration", "codex-log-guard-doctor-protection.test.ts": "codex-integration", diff --git a/src/codex/lineage.ts b/src/codex/lineage.ts index 821b0f74b6f..a755e8ac690 100644 --- a/src/codex/lineage.ts +++ b/src/codex/lineage.ts @@ -48,10 +48,10 @@ import { retainedUtf8Bytes } from "../lib/admission"; const CODEX_LINEAGE_COMPONENT_MAX_BYTES = 512; const CODEX_LINEAGE_KEY = randomBytes(32); -let conversationAnchorObserver: ((key: string, anchor: string) => void) | undefined; +let conversationAnchorObserver: ((key: string, familyId: string, threadId: string) => void) | undefined; /** Register the consumer of derived cohort anchors, or remove it. */ -export function setCodexConversationAnchorObserver(fn?: (key: string, anchor: string) => void): void { +export function setCodexConversationAnchorObserver(fn?: (key: string, familyId: string, threadId: string) => void): void { conversationAnchorObserver = fn; } @@ -159,7 +159,7 @@ export function codexConversationKeyFor(familyId: string, threadId: string): str .update("\0") .update(threadId) .digest("base64url")}`; - conversationAnchorObserver?.(key, familyId); + conversationAnchorObserver?.(key, familyId, threadId); return key; } diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts new file mode 100644 index 00000000000..11860a948b6 --- /dev/null +++ b/src/codex/routing/thread-affinity-store.ts @@ -0,0 +1,177 @@ +import { Database } from "bun:sqlite"; +import { createHash, createHmac, randomBytes } from "node:crypto"; +import { chmodSync, closeSync, lstatSync, openSync, realpathSync } from "node:fs"; +import { resolve } from "node:path"; +import { getConfigDir } from "../../config/paths"; +import { recordOwnedConfigPath } from "../../lib/config-ownership"; +import { isPassiveRead } from "../../lib/passive-read"; +import { assertStorageOwned, currentSpendLedgerOwnerGeneration, mintSpendLedgerStorage, onSpendLedgerOwnerReleased, onSpendLedgerOwnerReleasing, spendLedgerStoragePath, type SpendLedgerStorage } from "../../lib/spend-ledger-owner"; +import { assertNotRealHomeUnderTest } from "../../lib/test-home-guard"; +import { hardenSecretPath } from "../../lib/windows-secret-acl"; +import { readCodexAccountRecord } from "../account-store"; +import { setCodexConversationAnchorObserver } from "../lineage"; + +export type StoredThreadRow = { scope: string; accountId: string; fence: string; createdAt: number; lastUsedAt: number }; +export const CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER = "__unknown_codex_conversation_state_issuer__"; +type LiveReader = { thread(key: string, scope: string): { accountId: string; identityFence?: string; createdAt: number; lastUsedAt: number } | undefined; issuer(key: string): { accountId: string; fence?: string; lastUsedAt: number } | undefined; clearLive(): void }; +type Op = { sql: string; args: (string | number | null)[]; key?: string; scope?: string; kind: string }; +type Scope = { home: string; generation: number; token: SpendLedgerStorage; db?: Database; secret?: Uint8Array; disabled: boolean; poisoned: boolean; failures: number; log: Op[]; threads: Map; issuers: Map; anchors: Map; consulted: Set; timer?: ReturnType; scheduled: boolean }; +const TTL = 24 * 60 * 60_000; +let current: Scope | undefined; +let reader: LiveReader | undefined; + +export function registerCodexAffinityLiveState(value: LiveReader): void { reader = value; } +function warning(): void { console.warn("Codex affinity persistence unavailable; continuing in memory."); } +function retire(): void { + const s = current; + if (!s) return; + current = undefined; + if (s.timer) clearTimeout(s.timer); + try { s.db?.close(); } catch { warning(); } + reader?.clearLive(); +} +function checkFile(path: string): void { + const stat = lstatSync(path); + if (!stat.isFile() || stat.isSymbolicLink() || stat.nlink !== 1 || (process.platform !== "win32" && stat.uid !== process.getuid!())) throw new Error("unsafe file"); +} +function open(s: Scope): void { + const path = spendLedgerStoragePath(s.token); + let db: Database | undefined; + try { + assertNotRealHomeUnderTest(s.home); + for (const suffix of ["", "-journal", "-wal", "-shm"]) if (!recordOwnedConfigPath(s.home, path + suffix)) throw new Error("ownership"); + try { closeSync(openSync(path, "wx", 0o600)); } catch (e) { if ((e as NodeJS.ErrnoException).code !== "EEXIST") throw e; } + checkFile(path); + if (process.platform !== "win32") chmodSync(path, 0o600); + if (!hardenSecretPath(path, { required: true }).ok) throw new Error("acl"); + checkFile(path); + assertStorageOwned(s.token); + db = new Database(path); + db.exec("PRAGMA busy_timeout = 0; BEGIN IMMEDIATE"); + const version = (db.query("PRAGMA user_version").get() as { user_version: number }).user_version; + const objects = db.query("SELECT type, name FROM sqlite_master ORDER BY name").all() as { type: string; name: string }[]; + const tables = objects.filter(o => o.type === "table"); + if (version === 0 && objects.length === 0) { + db.exec("CREATE TABLE meta(name TEXT PRIMARY KEY, value BLOB); CREATE TABLE threads(row_key TEXT, scope TEXT, account_id TEXT, fence TEXT, created_at INTEGER, last_used_at INTEGER, PRIMARY KEY(row_key, scope)); CREATE TABLE issuers(row_key TEXT PRIMARY KEY, account_id TEXT, fence TEXT, last_used_at INTEGER); PRAGMA user_version = 1"); + db.query("INSERT INTO meta VALUES ('secret', ?), ('clean', 0)").run(randomBytes(32)); + } else { + if (version !== 1 || tables.map(t => t.name).join(",") !== "issuers,meta,threads") throw new Error("version"); + const expected: Record = { meta: "name:TEXT:1,value:BLOB:0", threads: "row_key:TEXT:1,scope:TEXT:2,account_id:TEXT:0,fence:TEXT:0,created_at:INTEGER:0,last_used_at:INTEGER:0", issuers: "row_key:TEXT:1,account_id:TEXT:0,fence:TEXT:0,last_used_at:INTEGER:0" }; + for (const [table, columns] of Object.entries(expected)) { + const actual = db.query(`PRAGMA table_info(${table})`).all() as { name: string; type: string; pk: number; notnull: number; dflt_value: unknown }[]; + if (actual.map(c => `${c.name}:${c.type}:${c.pk}`).join(",") !== columns || actual.some(c => c.notnull !== 0 || c.dflt_value !== null)) throw new Error("shape"); + } + const indexes = objects.filter(o => o.type === "index").map(o => o.name).join(","); + if (objects.length !== 6 || indexes !== "sqlite_autoindex_issuers_1,sqlite_autoindex_meta_1,sqlite_autoindex_threads_1") throw new Error("shape"); + const rows = db.query("SELECT name, value FROM meta").all() as { name: string; value: unknown }[]; + const secret = rows.find(r => r.name === "secret")?.value; + const clean = rows.find(r => r.name === "clean")?.value; + if (!(secret instanceof Uint8Array) || secret.length !== 32 || rows.length !== 2 || !rows.some(r => r.name === "clean")) throw new Error("meta"); + if (clean !== 1) db.exec("DELETE FROM threads; DELETE FROM issuers"); + db.exec("UPDATE meta SET value=0 WHERE name='clean'"); + } + const secret = (db.query("SELECT value FROM meta WHERE name='secret'").get() as { value: Uint8Array }).value; + db.exec("COMMIT"); + s.db = db; s.secret = secret; + } catch { + try { db?.exec("ROLLBACK"); } catch { /* no transaction */ } + try { db?.close(); } catch { /* preserve open failure */ } + s.disabled = true; warning(); + } +} +function scope(): Scope | undefined { + if (isPassiveRead()) return undefined; + let home = resolve(getConfigDir()); + try { home = realpathSync.native(home); } catch { /* ownership check below refuses it */ } + if (process.platform === "win32") home = home.toLowerCase(); + const generation = currentSpendLedgerOwnerGeneration(); + if (current && (current.home !== home || current.generation !== generation)) retire(); + if (current) { try { assertStorageOwned(current.token); return current; } catch { retire(); return undefined; } } + let token: SpendLedgerStorage; + try { token = mintSpendLedgerStorage("codex-affinity.sqlite"); assertStorageOwned(token); } catch { return undefined; } + const s: Scope = { home, generation, token, disabled: false, poisoned: false, failures: 0, log: [], threads: new Map(), issuers: new Map(), anchors: new Map(), consulted: new Set(), scheduled: false }; + current = s; open(s); return s; +} +function usable(): Scope | undefined { const s = scope(); return s && !s.disabled && !s.poisoned && s.db ? s : undefined; } +setCodexConversationAnchorObserver((key, family, thread) => { + const s = usable(); if (!s) return; + const now = Date.now(); + for (const [k, a] of s.anchors) { if (now - a.at <= TTL) break; s.anchors.delete(k); } + s.anchors.delete(key); s.anchors.set(key, { family, thread, at: now }); + while (s.anchors.size > 4096) s.anchors.delete(s.anchors.keys().next().value!); +}); +function rowKey(s: Scope, key: string, kind: "conversation" | "issuer"): string | undefined { + const a = s.anchors.get(key); + if (!a || Date.now() - a.at > TTL) { s.anchors.delete(key); return undefined; } + return createHmac("sha256", s.secret!).update(`opencodex/codex-affinity/${kind}/v1\0`).update(a.family).update("\0").update(a.thread).digest("hex"); +} +export function codexAffinityIdentityFence(accountId: string): string | undefined { + if (accountId === "__main__") return undefined; + const record = readCodexAccountRecord(accountId); + if (!record?.credential || record.deletedAt != null || !record.quotaHistoryIdentity) return undefined; + return createHash("sha256").update("opencodex/codex-affinity/identity/v1\0").update(record.quotaHistoryIdentity).digest("hex"); +} +function poison(s: Scope): void { s.poisoned = true; s.log = []; s.threads.clear(); s.issuers.clear(); if (s.timer) clearTimeout(s.timer); warning(); } +function enqueue(s: Scope, op: Op): void { + if (op.kind === "clearAll") s.log = []; + s.log.push(op); + if (s.log.length > 16384) { poison(s); return; } + if (!s.scheduled) { s.scheduled = true; queueMicrotask(() => { s.scheduled = false; if (current === s && !isPassiveRead()) flush(s, false); }); } +} +function scheduleTouch(s: Scope): void { + if (s.timer) return; + s.timer = setTimeout(() => { s.timer = undefined; if (current === s && !isPassiveRead()) flush(s, false); }, 1000); + s.timer.unref(); +} +function flush(s: Scope, clean: boolean): boolean { + if (!s.db || s.disabled || s.poisoned || isPassiveRead()) return false; + const db = s.db; + try { + assertStorageOwned(s.token); db.exec("BEGIN IMMEDIATE"); + for (const op of s.log) { if (op.args.length) db.query(op.sql).run(...op.args); else db.exec(op.sql); } + for (const t of s.threads.values()) { const row = reader?.thread(t.key, t.scope); if (row?.identityFence) db.query("UPDATE threads SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?").run(row.lastUsedAt, t.rowKey, t.scope, row.accountId, row.identityFence); } + for (const t of s.issuers.values()) { const row = reader?.issuer(t.key); if (row) db.query("UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?").run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); } + const now = Date.now(); + for (const table of ["threads", "issuers"]) db.query(`DELETE FROM ${table} WHERE last_used_at < ? OR last_used_at > ?`).run(now - TTL, now); + db.query("DELETE FROM threads WHERE created_at > ?").run(now); + db.exec("DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)"); + if (clean) db.exec("UPDATE meta SET value=1 WHERE name='clean'"); + db.exec("COMMIT"); s.log = []; s.threads.clear(); s.issuers.clear(); s.failures = 0; return true; + } catch { try { db.exec("ROLLBACK"); } catch { /* preserve failure */ } if (++s.failures >= 3) poison(s); else warning(); return false; } +} +onSpendLedgerOwnerReleasing(() => { const s = current; if (s) { flush(s, true); retire(); } }); +onSpendLedgerOwnerReleased(() => retire()); +function pending(s: Scope, row: string, issuer: boolean): boolean { return s.log.some(op => op.kind === "clearAll" || (issuer ? op.kind === "clearIssuers" || (op.kind === "issuerDelete" && op.key === row) : op.kind === "deleteKey" && op.key === row)); } +function consulted(s: Scope, key: string): boolean { if (s.consulted.has(key)) return true; s.consulted.add(key); if (s.consulted.size > 8192) s.consulted.delete(s.consulted.values().next().value!); return false; } +function validScope(value: string): boolean { if (["legacy", "shared", "reserve"].includes(value)) return true; const m = /^model-detour:(legacy|shared|reserve):(.+)$/s.exec(value); return !!m && Buffer.byteLength(m[2]!, "utf8") <= 512; } +export function adoptStoredThreadRows(key: string, now: number): StoredThreadRow[] { + const s = usable(); if (!s) return []; const rk = rowKey(s, key, "conversation"); if (!rk || pending(s, rk, false) || s.consulted.has(`t:${rk}`)) return []; + try { + const rows = s.db!.query("SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=?").all(rk) as { scope: string; account_id: string; fence: string; created_at: number; last_used_at: number }[]; + const result: StoredThreadRow[] = []; + for (const r of rows) { + if (s.log.some(op => op.kind === "deleteScope" && op.key === rk && op.scope === r.scope) || s.log.some(op => op.kind === "deleteAccount" && op.args[0] === r.account_id)) continue; + if (!validScope(r.scope) || !Number.isFinite(r.created_at) || !Number.isFinite(r.last_used_at) || r.created_at > now || r.last_used_at > now || now - r.last_used_at > TTL || !r.fence || codexAffinityIdentityFence(r.account_id) !== r.fence) { enqueue(s, { kind: "deleteScope", key: rk, scope: r.scope, sql: "DELETE FROM threads WHERE row_key=? AND scope=?", args: [rk, r.scope] }); continue; } + result.push({ scope: r.scope, accountId: r.account_id, fence: r.fence, createdAt: r.created_at, lastUsedAt: r.last_used_at }); + } + consulted(s, `t:${rk}`); return result; + } catch { warning(); return []; } +} +export function adoptStoredIssuer(key: string, now: number): string | undefined { + const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (!rk || pending(s, rk, true) || s.consulted.has(`i:${rk}`)) return; + try { const r = s.db!.query("SELECT account_id, fence, last_used_at FROM issuers WHERE row_key=?").get(rk) as { account_id: string; fence: string | null; last_used_at: number } | null; if (!r) { consulted(s, `i:${rk}`); return; } + if (!Number.isFinite(r.last_used_at) || r.last_used_at > now || now - r.last_used_at > TTL) { enqueue(s, { kind: "issuerDelete", key: rk, sql: "DELETE FROM issuers WHERE row_key=?", args: [rk] }); return; } + const result = r.fence && codexAffinityIdentityFence(r.account_id) === r.fence ? r.account_id : CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER; consulted(s, `i:${rk}`); return result; + } catch { warning(); return; } +} +export function recordThreadBind(key: string, scopeName: string, row: StoredThreadRow): void { const s = usable(); if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; enqueue(s, { kind: "bind", key: rk, scope: scopeName, sql: "INSERT INTO threads VALUES (?,?,?,?,?,?) ON CONFLICT(row_key,scope) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,created_at=excluded.created_at,last_used_at=excluded.last_used_at", args: [rk, scopeName, row.accountId, row.fence, row.createdAt, row.lastUsedAt] }); } +export function recordThreadTouch(key: string, scopeName: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; s.threads.set(`${rk}\0${scopeName}`, { key, scope: scopeName, rowKey: rk }); scheduleTouch(s); } +export function recordThreadDelete(key: string, scopeName?: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; enqueue(s, scopeName === undefined ? { kind: "deleteKey", key: rk, sql: "DELETE FROM threads WHERE row_key=?", args: [rk] } : { kind: "deleteScope", key: rk, scope: scopeName, sql: "DELETE FROM threads WHERE row_key=? AND scope=?", args: [rk, scopeName] }); } +export function recordAccountPinsCleared(id: string): void { const s = usable(); if (s) enqueue(s, { kind: "deleteAccount", sql: "DELETE FROM threads WHERE account_id=?", args: [id] }); } +export function recordAllCleared(): void { const s = usable(); if (s) enqueue(s, { kind: "clearAll", sql: "DELETE FROM threads; DELETE FROM issuers", args: [] }); } +export function recordIssuerSet(key: string, id: string, fence: string | null, lastUsedAt: number): void { const s = usable(); if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; enqueue(s, { kind: "issuerSet", key: rk, sql: "INSERT INTO issuers VALUES (?,?,?,?) ON CONFLICT(row_key) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,last_used_at=excluded.last_used_at", args: [rk, id, fence, lastUsedAt] }); } +export function recordIssuerTouch(key: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (!rk) return; s.issuers.set(rk, { key, rowKey: rk }); scheduleTouch(s); } +export function recordIssuerDelete(key: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (rk) enqueue(s, { kind: "issuerDelete", key: rk, sql: "DELETE FROM issuers WHERE row_key=?", args: [rk] }); } +export function recordIssuersCleared(): void { const s = usable(); if (s) enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); } +export function flushCodexAffinityStoreForTests(): void { if (current && !isPassiveRead()) flush(current, false); } +export function resetCodexAffinityStoreForTests(): void { if (current) { flush(current, false); retire(); } } diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts new file mode 100644 index 00000000000..80a8bded598 --- /dev/null +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -0,0 +1,105 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { Database } from "bun:sqlite"; +import { existsSync, mkdtempSync, readFileSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveCodexAccountCredential } from "../../src/codex/account-store"; +import { codexConversationKeyFor } from "../../src/codex/lineage"; +import { acquireSpendLedgerOwner, type SpendLedgerOwnerLease } from "../../src/lib/spend-ledger-owner"; +import { runPassiveRead } from "../../src/lib/passive-read"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { adoptStoredIssuer, adoptStoredThreadRows, codexAffinityIdentityFence, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, flushCodexAffinityStoreForTests, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordThreadBind, recordThreadTouch, registerCodexAffinityLiveState, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; + +let home: string; +let prior: string | undefined; +let lease: SpendLedgerOwnerLease | undefined; +let live: { accountId: string; identityFence: string; createdAt: number; lastUsedAt: number } | undefined; +const path = () => join(home, "codex-affinity.sqlite"); +const key = () => codexConversationKeyFor("private-family", "private-thread"); +function seed(): string { + saveCodexAccountCredential("account-a", { accessToken: "access", refreshToken: "refresh", expiresAt: Date.now() + 60_000, chatgptAccountId: "chat-account" }); + return codexAffinityIdentityFence("account-a")!; +} +function bind(fence: string): string { + const k = key(); const now = Date.now(); live = { accountId: "account-a", identityFence: fence, createdAt: now - 1000, lastUsedAt: now - 1000 }; + recordThreadBind(k, "legacy", { scope: "legacy", accountId: "account-a", fence, createdAt: live.createdAt, lastUsedAt: live.lastUsedAt }); return k; +} +beforeEach(() => { + prior = process.env.OPENCODEX_HOME; home = mkdtempSync(join(tmpdir(), "ocx-affinity-")); process.env.OPENCODEX_HOME = home; live = undefined; + registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, clearLive: () => { live = undefined; } }); +}); +afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); + +test("inactive and passive operations do not create a database", async () => { + key(); expect(existsSync(path())).toBe(false); + lease = acquireSpendLedgerOwner(); + await runPassiveRead(async () => { key(); recordAllCleared(); }); + expect(existsSync(path())).toBe(false); +}); +test("bind survives release and touch reads the live timestamp at flush", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); const newer = Date.now(); + recordThreadTouch(k, "legacy"); live!.lastUsedAt = newer; flushCodexAffinityStoreForTests(); + lease.release(); lease = acquireSpendLedgerOwner(); const restored = adoptStoredThreadRows(key(), Date.now()); + expect(restored).toHaveLength(1); expect(restored[0]!.lastUsedAt).toBe(newer); + const bytes = readFileSync(path()); expect(bytes.includes(Buffer.from("private-family"))).toBe(false); expect(bytes.includes(Buffer.from("private-thread"))).toBe(false); expect(bytes.includes(Buffer.from(k))).toBe(false); + if (process.platform !== "win32") expect(statSync(path()).mode & 0o777).toBe(0o600); +}); +test("unclean database is discarded and unknown version remains unchanged", () => { + lease = acquireSpendLedgerOwner(); bind(seed()); lease.release(); + let db = new Database(path()); db.exec("UPDATE meta SET value=0 WHERE name='clean'"); db.close(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); lease.release(); + db = new Database(path()); db.exec("PRAGMA user_version=99"); db.close(); const before = readFileSync(path()); + lease = acquireSpendLedgerOwner(); key(); lease.release(); expect(readFileSync(path()).equals(before)).toBe(true); +}); +test("issuer fence mismatch is unknown and account clearing preserves issuer", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); recordIssuerSet(k, "account-a", fence, Date.now()); recordAccountPinsCleared("account-a"); lease.release(); + seed(); lease = acquireSpendLedgerOwner(); const restoredKey = key(); expect(adoptStoredThreadRows(restoredKey, Date.now())).toEqual([]); expect(adoptStoredIssuer(restoredKey, Date.now())).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); +}); +test("clear all followed by bind preserves the later bind", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); recordAllCleared(); bind(fence); lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); + +test("expiry, future timestamps and identity replacement reject restored rows", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); lease.release(); + let db = new Database(path()); db.query("UPDATE threads SET last_used_at=?").run(Date.now() - 25 * 60 * 60_000); db.close(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); lease.release(); + lease = acquireSpendLedgerOwner(); bind(fence); lease.release(); + db = new Database(path()); db.query("UPDATE threads SET last_used_at=?").run(Date.now() + 60_000); db.close(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); lease.release(); + lease = acquireSpendLedgerOwner(); bind(fence); lease.release(); seed(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); +test("issuer expiration and future timestamps are absent", () => { + for (const timestamp of [Date.now() - 25 * 60 * 60_000, Date.now() + 60_000]) { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = key(); recordIssuerSet(k, "account-a", fence, Date.now()); lease.release(); + const db = new Database(path()); db.query("UPDATE issuers SET last_used_at=?").run(timestamp); db.close(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); lease.release(); lease = undefined; + } +}); +test("releasing transaction failure leaves the store unclean and emits identifier-free warnings", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); flushCodexAffinityStoreForTests(); + const blocker = new Database(path()); blocker.exec("BEGIN IMMEDIATE"); + const warnings: string[] = []; const original = console.warn; console.warn = (...args: unknown[]) => { warnings.push(args.join(" ")); }; + try { lease.release(); } finally { console.warn = original; blocker.exec("ROLLBACK"); blocker.close(); } + expect(warnings.length).toBeGreaterThan(0); expect(warnings.join(" ")).not.toContain(home); expect(warnings.join(" ")).not.toContain("account-a"); expect(warnings.join(" ")).not.toContain("private-family"); + const db = new Database(path()); expect(db.query("SELECT value FROM meta WHERE name='clean'").get()).toEqual({ value: 0 }); db.close(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); +test("three failed flushes poison persistence until the next owner discards rows", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); flushCodexAffinityStoreForTests(); + const blocker = new Database(path()); blocker.exec("BEGIN IMMEDIATE"); + const original = console.warn; console.warn = () => {}; + try { flushCodexAffinityStoreForTests(); flushCodexAffinityStoreForTests(); flushCodexAffinityStoreForTests(); } finally { console.warn = original; blocker.exec("ROLLBACK"); blocker.close(); } + lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); + +test("known and fence-less issuers restore with distinct outcomes", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const known = key(); const unknown = codexConversationKeyFor("other-family", "other-thread"); + recordIssuerSet(known, "account-a", fence, Date.now()); recordIssuerSet(unknown, "account-a", null, Date.now()); lease.release(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredIssuer(key(), Date.now())).toBe("account-a"); + expect(adoptStoredIssuer(codexConversationKeyFor("other-family", "other-thread"), Date.now())).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); +}); +test("account clear followed by bind retains the later pin", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); recordAccountPinsCleared("account-a"); bind(fence); lease.release(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); diff --git a/tests/codex-integration/codex-lineage-placement.test.ts b/tests/codex-integration/codex-lineage-placement.test.ts index 9494bca51be..5b492ffdbd7 100644 --- a/tests/codex-integration/codex-lineage-placement.test.ts +++ b/tests/codex-integration/codex-lineage-placement.test.ts @@ -55,18 +55,20 @@ const ACCOUNT_IDS = ["a", "b", "c"] as const; const NOW = 1_700_000_000_000; test("lineage observer receives the derived key and cohort anchor", () => { - const seen: Array<[string, string]> = []; - setCodexConversationAnchorObserver((key, anchor) => seen.push([key, anchor])); + const seen: Array<[string, string, string]> = []; + setCodexConversationAnchorObserver((key, family, thread) => seen.push([key, family, thread])); try { const key = codexConversationKeyFor("cohort", "cohort"); - expect(seen).toEqual([[key, "cohort"]]); + expect(seen).toEqual([[key, "cohort", "cohort"]]); expect(codexConversationKeyFor("cohort", "cohort")).toBe(key); expect(seen).toHaveLength(2); + const other = codexConversationKeyFor("family", "thread"); + expect(seen[2]).toEqual([other, "family", "thread"]); } finally { setCodexConversationAnchorObserver(); } codexConversationKeyFor("cohort", "cohort"); - expect(seen).toHaveLength(2); + expect(seen).toHaveLength(3); }); function installScratchHome(): void { diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index a839b41d9d3..cdaca219ef0 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -657,6 +657,7 @@ "codex-inject-write-lock.test.ts": "codex-integration", "codex-inject.test.ts": "codex-integration", "codex-injected-marker.test.ts": "codex-integration", "codex-integration-record.test.ts": "codex-integration", "codex-journal-recovery.test.ts": "codex-integration", "codex-journal.test.ts": "codex-integration", "codex-legacy-config-keys.test.ts": "codex-integration", + "codex-affinity-store.test.ts": "codex-integration", "codex-lineage-placement.test.ts": "codex-integration", "codex-log-guard-coderabbit.test.ts": "codex-integration", "codex-log-guard-doctor-coderabbit.test.ts": "codex-integration", "codex-log-guard-doctor-protection.test.ts": "codex-integration", From 9f0c104bd0e979381801fa311bcb15bb608fd667 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:16:47 +0900 Subject: [PATCH 03/21] wip(codex): harden affinity store (readability, release vs home retirement, scope exhaustiveness, gap tests) --- src/codex/routing/thread-affinity-store.ts | 714 +++++++++++++++--- .../codex-affinity-store.test.ts | 78 ++ 2 files changed, 696 insertions(+), 96 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index 11860a948b6..c87f0aecb0f 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -1,46 +1,139 @@ import { Database } from "bun:sqlite"; import { createHash, createHmac, randomBytes } from "node:crypto"; -import { chmodSync, closeSync, lstatSync, openSync, realpathSync } from "node:fs"; +import { + chmodSync, + closeSync, + lstatSync, + openSync, + realpathSync, +} from "node:fs"; import { resolve } from "node:path"; import { getConfigDir } from "../../config/paths"; import { recordOwnedConfigPath } from "../../lib/config-ownership"; import { isPassiveRead } from "../../lib/passive-read"; -import { assertStorageOwned, currentSpendLedgerOwnerGeneration, mintSpendLedgerStorage, onSpendLedgerOwnerReleased, onSpendLedgerOwnerReleasing, spendLedgerStoragePath, type SpendLedgerStorage } from "../../lib/spend-ledger-owner"; +import { + assertStorageOwned, + currentSpendLedgerOwnerGeneration, + mintSpendLedgerStorage, + onSpendLedgerOwnerReleased, + onSpendLedgerOwnerReleasing, + spendLedgerStoragePath, + type SpendLedgerStorage, +} from "../../lib/spend-ledger-owner"; import { assertNotRealHomeUnderTest } from "../../lib/test-home-guard"; import { hardenSecretPath } from "../../lib/windows-secret-acl"; import { readCodexAccountRecord } from "../account-store"; import { setCodexConversationAnchorObserver } from "../lineage"; +import type { CodexQuotaScope } from "./health-store"; -export type StoredThreadRow = { scope: string; accountId: string; fence: string; createdAt: number; lastUsedAt: number }; -export const CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER = "__unknown_codex_conversation_state_issuer__"; -type LiveReader = { thread(key: string, scope: string): { accountId: string; identityFence?: string; createdAt: number; lastUsedAt: number } | undefined; issuer(key: string): { accountId: string; fence?: string; lastUsedAt: number } | undefined; clearLive(): void }; -type Op = { sql: string; args: (string | number | null)[]; key?: string; scope?: string; kind: string }; -type Scope = { home: string; generation: number; token: SpendLedgerStorage; db?: Database; secret?: Uint8Array; disabled: boolean; poisoned: boolean; failures: number; log: Op[]; threads: Map; issuers: Map; anchors: Map; consulted: Set; timer?: ReturnType; scheduled: boolean }; +export type StoredThreadRow = { + scope: string; + accountId: string; + fence: string; + createdAt: number; + lastUsedAt: number; +}; +export const CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER = + "__unknown_codex_conversation_state_issuer__"; +type LiveReader = { + thread( + key: string, + scope: string, + ): + | { + accountId: string; + identityFence?: string; + createdAt: number; + lastUsedAt: number; + } + | undefined; + issuer( + key: string, + ): { accountId: string; fence?: string; lastUsedAt: number } | undefined; + clearLive(): void; +}; +type Op = { + sql: string; + args: (string | number | null)[]; + key?: string; + scope?: string; + kind: string; +}; +type Scope = { + home: string; + generation: number; + token: SpendLedgerStorage; + db?: Database; + secret?: Uint8Array; + disabled: boolean; + poisoned: boolean; + failures: number; + log: Op[]; + threads: Map; + issuers: Map; + anchors: Map; + consulted: Set; + timer?: ReturnType; + scheduled: boolean; +}; const TTL = 24 * 60 * 60_000; +const BASE_SCOPES = [ + "shared", + "reserve", +] as const satisfies readonly CodexQuotaScope[]; +const BASE_SCOPES_EXHAUSTIVE: Exclude< + CodexQuotaScope, + (typeof BASE_SCOPES)[number] +> extends never + ? true + : never = true; +void BASE_SCOPES_EXHAUSTIVE; let current: Scope | undefined; +let observedHome: string | undefined; let reader: LiveReader | undefined; -export function registerCodexAffinityLiveState(value: LiveReader): void { reader = value; } -function warning(): void { console.warn("Codex affinity persistence unavailable; continuing in memory."); } +export function registerCodexAffinityLiveState(value: LiveReader): void { + reader = value; + setCodexConversationAnchorObserver(observeAnchor); +} +function warning(): void { + console.warn("Codex affinity persistence unavailable; continuing in memory."); +} +/** Retire the lease-scoped handle without discarding in-memory affinity. */ function retire(): void { const s = current; if (!s) return; current = undefined; if (s.timer) clearTimeout(s.timer); - try { s.db?.close(); } catch { warning(); } - reader?.clearLive(); + try { + s.db?.close(); + } catch { + warning(); + } } function checkFile(path: string): void { const stat = lstatSync(path); - if (!stat.isFile() || stat.isSymbolicLink() || stat.nlink !== 1 || (process.platform !== "win32" && stat.uid !== process.getuid!())) throw new Error("unsafe file"); + if ( + !stat.isFile() || + stat.isSymbolicLink() || + stat.nlink !== 1 || + (process.platform !== "win32" && stat.uid !== process.getuid!()) + ) + throw new Error("unsafe file"); } function open(s: Scope): void { const path = spendLedgerStoragePath(s.token); let db: Database | undefined; try { assertNotRealHomeUnderTest(s.home); - for (const suffix of ["", "-journal", "-wal", "-shm"]) if (!recordOwnedConfigPath(s.home, path + suffix)) throw new Error("ownership"); - try { closeSync(openSync(path, "wx", 0o600)); } catch (e) { if ((e as NodeJS.ErrnoException).code !== "EEXIST") throw e; } + for (const suffix of ["", "-journal", "-wal", "-shm"]) + if (!recordOwnedConfigPath(s.home, path + suffix)) + throw new Error("ownership"); + try { + closeSync(openSync(path, "wx", 0o600)); + } catch (e) { + if ((e as NodeJS.ErrnoException).code !== "EEXIST") throw e; + } checkFile(path); if (process.platform !== "win32") chmodSync(path, 0o600); if (!hardenSecretPath(path, { required: true }).ok) throw new Error("acl"); @@ -48,130 +141,559 @@ function open(s: Scope): void { assertStorageOwned(s.token); db = new Database(path); db.exec("PRAGMA busy_timeout = 0; BEGIN IMMEDIATE"); - const version = (db.query("PRAGMA user_version").get() as { user_version: number }).user_version; - const objects = db.query("SELECT type, name FROM sqlite_master ORDER BY name").all() as { type: string; name: string }[]; - const tables = objects.filter(o => o.type === "table"); + const version = ( + db.query("PRAGMA user_version").get() as { user_version: number } + ).user_version; + const objects = db + .query("SELECT type, name FROM sqlite_master ORDER BY name") + .all() as { type: string; name: string }[]; + const tables = objects.filter((o) => o.type === "table"); if (version === 0 && objects.length === 0) { - db.exec("CREATE TABLE meta(name TEXT PRIMARY KEY, value BLOB); CREATE TABLE threads(row_key TEXT, scope TEXT, account_id TEXT, fence TEXT, created_at INTEGER, last_used_at INTEGER, PRIMARY KEY(row_key, scope)); CREATE TABLE issuers(row_key TEXT PRIMARY KEY, account_id TEXT, fence TEXT, last_used_at INTEGER); PRAGMA user_version = 1"); - db.query("INSERT INTO meta VALUES ('secret', ?), ('clean', 0)").run(randomBytes(32)); + db.exec( + "CREATE TABLE meta(name TEXT PRIMARY KEY, value BLOB); CREATE TABLE threads(row_key TEXT, scope TEXT, account_id TEXT, fence TEXT, created_at INTEGER, last_used_at INTEGER, PRIMARY KEY(row_key, scope)); CREATE TABLE issuers(row_key TEXT PRIMARY KEY, account_id TEXT, fence TEXT, last_used_at INTEGER); PRAGMA user_version = 1", + ); + db.query("INSERT INTO meta VALUES ('secret', ?), ('clean', 0)").run( + randomBytes(32), + ); } else { - if (version !== 1 || tables.map(t => t.name).join(",") !== "issuers,meta,threads") throw new Error("version"); - const expected: Record = { meta: "name:TEXT:1,value:BLOB:0", threads: "row_key:TEXT:1,scope:TEXT:2,account_id:TEXT:0,fence:TEXT:0,created_at:INTEGER:0,last_used_at:INTEGER:0", issuers: "row_key:TEXT:1,account_id:TEXT:0,fence:TEXT:0,last_used_at:INTEGER:0" }; + if ( + version !== 1 || + tables.map((t) => t.name).join(",") !== "issuers,meta,threads" + ) + throw new Error("version"); + const expected: Record = { + meta: "name:TEXT:1,value:BLOB:0", + threads: + "row_key:TEXT:1,scope:TEXT:2,account_id:TEXT:0,fence:TEXT:0,created_at:INTEGER:0,last_used_at:INTEGER:0", + issuers: + "row_key:TEXT:1,account_id:TEXT:0,fence:TEXT:0,last_used_at:INTEGER:0", + }; for (const [table, columns] of Object.entries(expected)) { - const actual = db.query(`PRAGMA table_info(${table})`).all() as { name: string; type: string; pk: number; notnull: number; dflt_value: unknown }[]; - if (actual.map(c => `${c.name}:${c.type}:${c.pk}`).join(",") !== columns || actual.some(c => c.notnull !== 0 || c.dflt_value !== null)) throw new Error("shape"); + const actual = db.query(`PRAGMA table_info(${table})`).all() as { + name: string; + type: string; + pk: number; + notnull: number; + dflt_value: unknown; + }[]; + if ( + actual.map((c) => `${c.name}:${c.type}:${c.pk}`).join(",") !== + columns || + actual.some((c) => c.notnull !== 0 || c.dflt_value !== null) + ) + throw new Error("shape"); } - const indexes = objects.filter(o => o.type === "index").map(o => o.name).join(","); - if (objects.length !== 6 || indexes !== "sqlite_autoindex_issuers_1,sqlite_autoindex_meta_1,sqlite_autoindex_threads_1") throw new Error("shape"); - const rows = db.query("SELECT name, value FROM meta").all() as { name: string; value: unknown }[]; - const secret = rows.find(r => r.name === "secret")?.value; - const clean = rows.find(r => r.name === "clean")?.value; - if (!(secret instanceof Uint8Array) || secret.length !== 32 || rows.length !== 2 || !rows.some(r => r.name === "clean")) throw new Error("meta"); + const indexes = objects + .filter((o) => o.type === "index") + .map((o) => o.name) + .join(","); + if ( + objects.length !== 6 || + indexes !== + "sqlite_autoindex_issuers_1,sqlite_autoindex_meta_1,sqlite_autoindex_threads_1" + ) + throw new Error("shape"); + const rows = db.query("SELECT name, value FROM meta").all() as { + name: string; + value: unknown; + }[]; + const secret = rows.find((r) => r.name === "secret")?.value; + const clean = rows.find((r) => r.name === "clean")?.value; + if ( + !(secret instanceof Uint8Array) || + secret.length !== 32 || + rows.length !== 2 || + !rows.some((r) => r.name === "clean") + ) + throw new Error("meta"); + // Only a completed final flush permits the next owner to restore rows. if (clean !== 1) db.exec("DELETE FROM threads; DELETE FROM issuers"); db.exec("UPDATE meta SET value=0 WHERE name='clean'"); } - const secret = (db.query("SELECT value FROM meta WHERE name='secret'").get() as { value: Uint8Array }).value; + const secret = ( + db.query("SELECT value FROM meta WHERE name='secret'").get() as { + value: Uint8Array; + } + ).value; db.exec("COMMIT"); - s.db = db; s.secret = secret; + s.db = db; + s.secret = secret; } catch { - try { db?.exec("ROLLBACK"); } catch { /* no transaction */ } - try { db?.close(); } catch { /* preserve open failure */ } - s.disabled = true; warning(); + try { + db?.exec("ROLLBACK"); + } catch { + /* no transaction */ + } + try { + db?.close(); + } catch { + /* preserve open failure */ + } + s.disabled = true; + warning(); } } function scope(): Scope | undefined { if (isPassiveRead()) return undefined; let home = resolve(getConfigDir()); - try { home = realpathSync.native(home); } catch { /* ownership check below refuses it */ } + try { + home = realpathSync.native(home); + } catch { + /* ownership check below refuses it */ + } if (process.platform === "win32") home = home.toLowerCase(); + // A lease ends a store scope; only a change of home invalidates live maps. + if (observedHome !== undefined && observedHome !== home) { + retire(); + reader?.clearLive(); + } + observedHome = home; const generation = currentSpendLedgerOwnerGeneration(); - if (current && (current.home !== home || current.generation !== generation)) retire(); - if (current) { try { assertStorageOwned(current.token); return current; } catch { retire(); return undefined; } } + if (current && (current.home !== home || current.generation !== generation)) + retire(); + if (current) { + try { + assertStorageOwned(current.token); + return current; + } catch { + retire(); + return undefined; + } + } let token: SpendLedgerStorage; - try { token = mintSpendLedgerStorage("codex-affinity.sqlite"); assertStorageOwned(token); } catch { return undefined; } - const s: Scope = { home, generation, token, disabled: false, poisoned: false, failures: 0, log: [], threads: new Map(), issuers: new Map(), anchors: new Map(), consulted: new Set(), scheduled: false }; - current = s; open(s); return s; + try { + token = mintSpendLedgerStorage("codex-affinity.sqlite"); + assertStorageOwned(token); + } catch { + return undefined; + } + const s: Scope = { + home, + generation, + token, + disabled: false, + poisoned: false, + failures: 0, + log: [], + threads: new Map(), + issuers: new Map(), + anchors: new Map(), + consulted: new Set(), + scheduled: false, + }; + current = s; + open(s); + return s; +} +function usable(): Scope | undefined { + const s = scope(); + return s && !s.disabled && !s.poisoned && s.db ? s : undefined; } -function usable(): Scope | undefined { const s = scope(); return s && !s.disabled && !s.poisoned && s.db ? s : undefined; } -setCodexConversationAnchorObserver((key, family, thread) => { - const s = usable(); if (!s) return; +function observeAnchor(key: string, family: string, thread: string): void { + const s = usable(); + if (!s) return; const now = Date.now(); - for (const [k, a] of s.anchors) { if (now - a.at <= TTL) break; s.anchors.delete(k); } - s.anchors.delete(key); s.anchors.set(key, { family, thread, at: now }); - while (s.anchors.size > 4096) s.anchors.delete(s.anchors.keys().next().value!); -}); -function rowKey(s: Scope, key: string, kind: "conversation" | "issuer"): string | undefined { + for (const [k, a] of s.anchors) { + if (now - a.at <= TTL) break; + s.anchors.delete(k); + } + s.anchors.delete(key); + s.anchors.set(key, { family, thread, at: now }); + while (s.anchors.size > 4096) + s.anchors.delete(s.anchors.keys().next().value!); +} +setCodexConversationAnchorObserver(observeAnchor); +function rowKey( + s: Scope, + key: string, + kind: "conversation" | "issuer", +): string | undefined { const a = s.anchors.get(key); - if (!a || Date.now() - a.at > TTL) { s.anchors.delete(key); return undefined; } - return createHmac("sha256", s.secret!).update(`opencodex/codex-affinity/${kind}/v1\0`).update(a.family).update("\0").update(a.thread).digest("hex"); + if (!a || Date.now() - a.at > TTL) { + s.anchors.delete(key); + return undefined; + } + return createHmac("sha256", s.secret!) + .update(`opencodex/codex-affinity/${kind}/v1\0`) + .update(a.family) + .update("\0") + .update(a.thread) + .digest("hex"); } -export function codexAffinityIdentityFence(accountId: string): string | undefined { +export function codexAffinityIdentityFence( + accountId: string, +): string | undefined { if (accountId === "__main__") return undefined; const record = readCodexAccountRecord(accountId); - if (!record?.credential || record.deletedAt != null || !record.quotaHistoryIdentity) return undefined; - return createHash("sha256").update("opencodex/codex-affinity/identity/v1\0").update(record.quotaHistoryIdentity).digest("hex"); + if ( + !record?.credential || + record.deletedAt != null || + !record.quotaHistoryIdentity + ) + return undefined; + return createHash("sha256") + .update("opencodex/codex-affinity/identity/v1\0") + .update(record.quotaHistoryIdentity) + .digest("hex"); +} +/** A poisoned scope cannot safely declare itself clean on release. */ +function poison(s: Scope): void { + s.poisoned = true; + s.log = []; + s.threads.clear(); + s.issuers.clear(); + if (s.timer) clearTimeout(s.timer); + warning(); } -function poison(s: Scope): void { s.poisoned = true; s.log = []; s.threads.clear(); s.issuers.clear(); if (s.timer) clearTimeout(s.timer); warning(); } function enqueue(s: Scope, op: Op): void { + // Preserve structural ordering; only clearAll supersedes the whole log. if (op.kind === "clearAll") s.log = []; s.log.push(op); - if (s.log.length > 16384) { poison(s); return; } - if (!s.scheduled) { s.scheduled = true; queueMicrotask(() => { s.scheduled = false; if (current === s && !isPassiveRead()) flush(s, false); }); } + if (s.log.length > 16384) { + poison(s); + return; + } + if (!s.scheduled) { + s.scheduled = true; + queueMicrotask(() => { + s.scheduled = false; + if (current === s && !isPassiveRead()) flush(s, false); + }); + } } function scheduleTouch(s: Scope): void { if (s.timer) return; - s.timer = setTimeout(() => { s.timer = undefined; if (current === s && !isPassiveRead()) flush(s, false); }, 1000); + s.timer = setTimeout(() => { + s.timer = undefined; + if (current === s && !isPassiveRead()) flush(s, false); + }, 1000); s.timer.unref(); } function flush(s: Scope, clean: boolean): boolean { if (!s.db || s.disabled || s.poisoned || isPassiveRead()) return false; const db = s.db; try { - assertStorageOwned(s.token); db.exec("BEGIN IMMEDIATE"); - for (const op of s.log) { if (op.args.length) db.query(op.sql).run(...op.args); else db.exec(op.sql); } - for (const t of s.threads.values()) { const row = reader?.thread(t.key, t.scope); if (row?.identityFence) db.query("UPDATE threads SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?").run(row.lastUsedAt, t.rowKey, t.scope, row.accountId, row.identityFence); } - for (const t of s.issuers.values()) { const row = reader?.issuer(t.key); if (row) db.query("UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?").run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); } + assertStorageOwned(s.token); + db.exec("BEGIN IMMEDIATE"); + for (const op of s.log) { + if (op.args.length) db.query(op.sql).run(...op.args); + else db.exec(op.sql); + } + // Read touches at flush time, after callers have updated the live entries. + for (const t of s.threads.values()) { + const row = reader?.thread(t.key, t.scope); + if (row?.identityFence) + db.query( + "UPDATE threads SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?", + ).run( + row.lastUsedAt, + t.rowKey, + t.scope, + row.accountId, + row.identityFence, + ); + } + for (const t of s.issuers.values()) { + const row = reader?.issuer(t.key); + if (row) + db.query( + "UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?", + ).run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); + } const now = Date.now(); - for (const table of ["threads", "issuers"]) db.query(`DELETE FROM ${table} WHERE last_used_at < ? OR last_used_at > ?`).run(now - TTL, now); + for (const table of ["threads", "issuers"]) + db.query( + `DELETE FROM ${table} WHERE last_used_at < ? OR last_used_at > ?`, + ).run(now - TTL, now); db.query("DELETE FROM threads WHERE created_at > ?").run(now); - db.exec("DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)"); + db.exec( + "DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)", + ); if (clean) db.exec("UPDATE meta SET value=1 WHERE name='clean'"); - db.exec("COMMIT"); s.log = []; s.threads.clear(); s.issuers.clear(); s.failures = 0; return true; - } catch { try { db.exec("ROLLBACK"); } catch { /* preserve failure */ } if (++s.failures >= 3) poison(s); else warning(); return false; } + db.exec("COMMIT"); + s.log = []; + s.threads.clear(); + s.issuers.clear(); + s.failures = 0; + return true; + } catch { + try { + db.exec("ROLLBACK"); + } catch { + /* preserve failure */ + } + if (++s.failures >= 3) poison(s); + else warning(); + return false; + } } -onSpendLedgerOwnerReleasing(() => { const s = current; if (s) { flush(s, true); retire(); } }); +// Commit clean before closing; a later close failure cannot undo that commit. +onSpendLedgerOwnerReleasing(() => { + const s = current; + if (s) { + flush(s, true); + retire(); + } +}); onSpendLedgerOwnerReleased(() => retire()); -function pending(s: Scope, row: string, issuer: boolean): boolean { return s.log.some(op => op.kind === "clearAll" || (issuer ? op.kind === "clearIssuers" || (op.kind === "issuerDelete" && op.key === row) : op.kind === "deleteKey" && op.key === row)); } -function consulted(s: Scope, key: string): boolean { if (s.consulted.has(key)) return true; s.consulted.add(key); if (s.consulted.size > 8192) s.consulted.delete(s.consulted.values().next().value!); return false; } -function validScope(value: string): boolean { if (["legacy", "shared", "reserve"].includes(value)) return true; const m = /^model-detour:(legacy|shared|reserve):(.+)$/s.exec(value); return !!m && Buffer.byteLength(m[2]!, "utf8") <= 512; } -export function adoptStoredThreadRows(key: string, now: number): StoredThreadRow[] { - const s = usable(); if (!s) return []; const rk = rowKey(s, key, "conversation"); if (!rk || pending(s, rk, false) || s.consulted.has(`t:${rk}`)) return []; +function pending(s: Scope, row: string, issuer: boolean): boolean { + return s.log.some( + (op) => + op.kind === "clearAll" || + (issuer + ? op.kind === "clearIssuers" || + (op.kind === "issuerDelete" && op.key === row) + : op.kind === "deleteKey" && op.key === row), + ); +} +function consulted(s: Scope, key: string): boolean { + if (s.consulted.has(key)) return true; + s.consulted.add(key); + if (s.consulted.size > 8192) + s.consulted.delete(s.consulted.values().next().value!); + return false; +} +function validScope(value: string): boolean { + const bases: readonly string[] = ["legacy", ...BASE_SCOPES]; + if (bases.includes(value)) return true; + const m = /^model-detour:([^:]+):(.+)$/s.exec(value); + return ( + !!m && bases.includes(m[1]!) && Buffer.byteLength(m[2]!, "utf8") <= 512 + ); +} +export function adoptStoredThreadRows( + key: string, + now: number, +): StoredThreadRow[] { + const s = usable(); + if (!s) return []; + const rk = rowKey(s, key, "conversation"); + if (!rk || pending(s, rk, false) || s.consulted.has(`t:${rk}`)) return []; try { - const rows = s.db!.query("SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=?").all(rk) as { scope: string; account_id: string; fence: string; created_at: number; last_used_at: number }[]; + const rows = s + .db!.query( + "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=?", + ) + .all(rk) as { + scope: string; + account_id: string; + fence: string; + created_at: number; + last_used_at: number; + }[]; const result: StoredThreadRow[] = []; for (const r of rows) { - if (s.log.some(op => op.kind === "deleteScope" && op.key === rk && op.scope === r.scope) || s.log.some(op => op.kind === "deleteAccount" && op.args[0] === r.account_id)) continue; - if (!validScope(r.scope) || !Number.isFinite(r.created_at) || !Number.isFinite(r.last_used_at) || r.created_at > now || r.last_used_at > now || now - r.last_used_at > TTL || !r.fence || codexAffinityIdentityFence(r.account_id) !== r.fence) { enqueue(s, { kind: "deleteScope", key: rk, scope: r.scope, sql: "DELETE FROM threads WHERE row_key=? AND scope=?", args: [rk, r.scope] }); continue; } - result.push({ scope: r.scope, accountId: r.account_id, fence: r.fence, createdAt: r.created_at, lastUsedAt: r.last_used_at }); + if ( + s.log.some( + (op) => + op.kind === "deleteScope" && op.key === rk && op.scope === r.scope, + ) || + s.log.some( + (op) => op.kind === "deleteAccount" && op.args[0] === r.account_id, + ) + ) + continue; + if ( + !validScope(r.scope) || + !Number.isFinite(r.created_at) || + !Number.isFinite(r.last_used_at) || + r.created_at > now || + r.last_used_at > now || + now - r.last_used_at > TTL || + !r.fence || + codexAffinityIdentityFence(r.account_id) !== r.fence + ) { + enqueue(s, { + kind: "deleteScope", + key: rk, + scope: r.scope, + sql: "DELETE FROM threads WHERE row_key=? AND scope=?", + args: [rk, r.scope], + }); + continue; + } + result.push({ + scope: r.scope, + accountId: r.account_id, + fence: r.fence, + createdAt: r.created_at, + lastUsedAt: r.last_used_at, + }); } - consulted(s, `t:${rk}`); return result; - } catch { warning(); return []; } -} -export function adoptStoredIssuer(key: string, now: number): string | undefined { - const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (!rk || pending(s, rk, true) || s.consulted.has(`i:${rk}`)) return; - try { const r = s.db!.query("SELECT account_id, fence, last_used_at FROM issuers WHERE row_key=?").get(rk) as { account_id: string; fence: string | null; last_used_at: number } | null; if (!r) { consulted(s, `i:${rk}`); return; } - if (!Number.isFinite(r.last_used_at) || r.last_used_at > now || now - r.last_used_at > TTL) { enqueue(s, { kind: "issuerDelete", key: rk, sql: "DELETE FROM issuers WHERE row_key=?", args: [rk] }); return; } - const result = r.fence && codexAffinityIdentityFence(r.account_id) === r.fence ? r.account_id : CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER; consulted(s, `i:${rk}`); return result; - } catch { warning(); return; } -} -export function recordThreadBind(key: string, scopeName: string, row: StoredThreadRow): void { const s = usable(); if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; enqueue(s, { kind: "bind", key: rk, scope: scopeName, sql: "INSERT INTO threads VALUES (?,?,?,?,?,?) ON CONFLICT(row_key,scope) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,created_at=excluded.created_at,last_used_at=excluded.last_used_at", args: [rk, scopeName, row.accountId, row.fence, row.createdAt, row.lastUsedAt] }); } -export function recordThreadTouch(key: string, scopeName: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; s.threads.set(`${rk}\0${scopeName}`, { key, scope: scopeName, rowKey: rk }); scheduleTouch(s); } -export function recordThreadDelete(key: string, scopeName?: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; enqueue(s, scopeName === undefined ? { kind: "deleteKey", key: rk, sql: "DELETE FROM threads WHERE row_key=?", args: [rk] } : { kind: "deleteScope", key: rk, scope: scopeName, sql: "DELETE FROM threads WHERE row_key=? AND scope=?", args: [rk, scopeName] }); } -export function recordAccountPinsCleared(id: string): void { const s = usable(); if (s) enqueue(s, { kind: "deleteAccount", sql: "DELETE FROM threads WHERE account_id=?", args: [id] }); } -export function recordAllCleared(): void { const s = usable(); if (s) enqueue(s, { kind: "clearAll", sql: "DELETE FROM threads; DELETE FROM issuers", args: [] }); } -export function recordIssuerSet(key: string, id: string, fence: string | null, lastUsedAt: number): void { const s = usable(); if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; enqueue(s, { kind: "issuerSet", key: rk, sql: "INSERT INTO issuers VALUES (?,?,?,?) ON CONFLICT(row_key) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,last_used_at=excluded.last_used_at", args: [rk, id, fence, lastUsedAt] }); } -export function recordIssuerTouch(key: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (!rk) return; s.issuers.set(rk, { key, rowKey: rk }); scheduleTouch(s); } -export function recordIssuerDelete(key: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (rk) enqueue(s, { kind: "issuerDelete", key: rk, sql: "DELETE FROM issuers WHERE row_key=?", args: [rk] }); } -export function recordIssuersCleared(): void { const s = usable(); if (s) enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); } -export function flushCodexAffinityStoreForTests(): void { if (current && !isPassiveRead()) flush(current, false); } -export function resetCodexAffinityStoreForTests(): void { if (current) { flush(current, false); retire(); } } + consulted(s, `t:${rk}`); + return result; + } catch { + warning(); + return []; + } +} +export function adoptStoredIssuer( + key: string, + now: number, +): string | undefined { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "issuer"); + if (!rk || pending(s, rk, true) || s.consulted.has(`i:${rk}`)) return; + try { + const r = s + .db!.query( + "SELECT account_id, fence, last_used_at FROM issuers WHERE row_key=?", + ) + .get(rk) as { + account_id: string; + fence: string | null; + last_used_at: number; + } | null; + if (!r) { + consulted(s, `i:${rk}`); + return; + } + if ( + !Number.isFinite(r.last_used_at) || + r.last_used_at > now || + now - r.last_used_at > TTL + ) { + enqueue(s, { + kind: "issuerDelete", + key: rk, + sql: "DELETE FROM issuers WHERE row_key=?", + args: [rk], + }); + return; + } + const result = + r.fence && codexAffinityIdentityFence(r.account_id) === r.fence + ? r.account_id + : CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER; + consulted(s, `i:${rk}`); + return result; + } catch { + warning(); + return; + } +} +export function recordThreadBind( + key: string, + scopeName: string, + row: StoredThreadRow, +): void { + const s = usable(); + if (!s || row.accountId === "__main__" || !row.fence) return; + const rk = rowKey(s, key, "conversation"); + if (!rk) return; + enqueue(s, { + kind: "bind", + key: rk, + scope: scopeName, + sql: "INSERT INTO threads VALUES (?,?,?,?,?,?) ON CONFLICT(row_key,scope) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,created_at=excluded.created_at,last_used_at=excluded.last_used_at", + args: [ + rk, + scopeName, + row.accountId, + row.fence, + row.createdAt, + row.lastUsedAt, + ], + }); +} +export function recordThreadTouch(key: string, scopeName: string): void { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "conversation"); + if (!rk) return; + s.threads.set(`${rk}\0${scopeName}`, { key, scope: scopeName, rowKey: rk }); + scheduleTouch(s); +} +export function recordThreadDelete(key: string, scopeName?: string): void { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "conversation"); + if (!rk) return; + enqueue( + s, + scopeName === undefined + ? { + kind: "deleteKey", + key: rk, + sql: "DELETE FROM threads WHERE row_key=?", + args: [rk], + } + : { + kind: "deleteScope", + key: rk, + scope: scopeName, + sql: "DELETE FROM threads WHERE row_key=? AND scope=?", + args: [rk, scopeName], + }, + ); +} +export function recordAccountPinsCleared(id: string): void { + const s = usable(); + if (s) + enqueue(s, { + kind: "deleteAccount", + sql: "DELETE FROM threads WHERE account_id=?", + args: [id], + }); +} +export function recordAllCleared(): void { + const s = usable(); + if (s) + enqueue(s, { + kind: "clearAll", + sql: "DELETE FROM threads; DELETE FROM issuers", + args: [], + }); +} +export function recordIssuerSet( + key: string, + id: string, + fence: string | null, + lastUsedAt: number, +): void { + const s = usable(); + if (!s || id === "__main__") return; + const rk = rowKey(s, key, "issuer"); + if (!rk) return; + enqueue(s, { + kind: "issuerSet", + key: rk, + sql: "INSERT INTO issuers VALUES (?,?,?,?) ON CONFLICT(row_key) DO UPDATE SET account_id=excluded.account_id,fence=excluded.fence,last_used_at=excluded.last_used_at", + args: [rk, id, fence, lastUsedAt], + }); +} +export function recordIssuerTouch(key: string): void { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "issuer"); + if (!rk) return; + s.issuers.set(rk, { key, rowKey: rk }); + scheduleTouch(s); +} +export function recordIssuerDelete(key: string): void { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "issuer"); + if (rk) + enqueue(s, { + kind: "issuerDelete", + key: rk, + sql: "DELETE FROM issuers WHERE row_key=?", + args: [rk], + }); +} +export function recordIssuersCleared(): void { + const s = usable(); + if (s) + enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); +} +export function flushCodexAffinityStoreForTests(): void { + if (current && !isPassiveRead()) flush(current, false); +} +export function resetCodexAffinityStoreForTests(): void { + if (current) { + flush(current, false); + retire(); + } +} diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 80a8bded598..c9290ae25e9 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -103,3 +103,81 @@ test("account clear followed by bind retains the later pin", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); recordAccountPinsCleared("account-a"); bind(fence); lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); }); + +test("overflow poisons the store and the next owner discards committed rows", () => { + lease = acquireSpendLedgerOwner(); + const fence = seed(); + bind(fence); + flushCodexAffinityStoreForTests(); + const db = new Database(path()); + expect(db.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 1 }); + db.close(); + const original = console.warn; + console.warn = () => {}; + try { + for (let i = 0; i < 16385; i++) recordAccountPinsCleared("other-account"); + } finally { + console.warn = original; + } + lease.release(); + lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); + +test("a close error after final commit leaves rows clean and restorable", () => { + lease = acquireSpendLedgerOwner(); + bind(seed()); + // Throw after the real close: the final transaction has already committed. + const originalClose = Database.prototype.close; + const originalWarn = console.warn; + let injected = false; + Database.prototype.close = function (...args) { + originalClose.apply(this, args); + if (!injected) { + injected = true; + throw new Error("injected close failure"); + } + }; + console.warn = () => {}; + try { + lease.release(); + } finally { + Database.prototype.close = originalClose; + console.warn = originalWarn; + } + expect(injected).toBe(true); + const db = new Database(path()); + expect(db.query("SELECT value FROM meta WHERE name='clean'").get()).toEqual({ value: 1 }); + db.close(); + lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); + +test("an evicted anchor makes a later bind memory-only", () => { + lease = acquireSpendLedgerOwner(); + const fence = seed(); + const evicted = key(); + for (let i = 0; i < 4096; i++) codexConversationKeyFor(`family-${i}`, `thread-${i}`); + const now = Date.now(); + recordThreadBind(evicted, "legacy", { scope: "legacy", accountId: "account-a", fence, createdAt: now, lastUsedAt: now }); + lease.release(); + lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); + +test("release preserves live state, while an observed home change clears it", () => { + lease = acquireSpendLedgerOwner(); + bind(seed()); + lease.release(); + lease = undefined; + expect(live).toBeDefined(); + const otherHome = mkdtempSync(join(tmpdir(), "ocx-affinity-other-")); + try { + process.env.OPENCODEX_HOME = otherHome; + key(); + expect(live).toBeUndefined(); + } finally { + process.env.OPENCODEX_HOME = home; + removeTreeWithRetry(otherHome); + } +}); From 5132f928869f85df1198f041a1b5da40da97138f Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:18:44 +0900 Subject: [PATCH 04/21] wip(codex): wire durable affinity into thread affinity and account-change state; restart tests --- scripts/test-layout/layout.json | 3 +- src/codex/routing/thread-affinity.ts | 78 +++++++++++++++++-- src/server/responses/account-change-state.ts | 6 +- .../codex-affinity-restart.test.ts | 72 +++++++++++++++++ tests/fixtures/test-layout-expected.json | 2 +- .../helpers/codex-affinity-restart-worker.ts | 36 +++++++++ 6 files changed, 184 insertions(+), 13 deletions(-) create mode 100644 tests/codex-integration/codex-affinity-restart.test.ts create mode 100644 tests/helpers/codex-affinity-restart-worker.ts diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 48d2835d1a3..05c6820f89d 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -706,8 +706,7 @@ "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", - "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index 643d64fc5c4..c22e1463a9b 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -6,10 +6,14 @@ import type { CodexThreadLineage } from "../lineage"; import type { CodexQuotaScope } from "./health-store"; import type { TransientProbeLease } from "../../routing/probe-lease"; import { clearPoolRecoveryState } from "../../routing/probe-lease"; +import { createHash } from "node:crypto"; +import { adoptStoredThreadRows, adoptStoredIssuer, codexAffinityIdentityFence, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; +export { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "./thread-affinity-store"; export type ThreadAffinityEntry = { accountId: string; generation: number; + identityFence?: string; createdAt: number; lastUsedAt: number; // Last time the bound account's quota threshold was re-evaluated for this @@ -200,15 +204,27 @@ let threadAffinityEntryTotal = 0; * Which pool account minted the conversation's carried OpenAI state * (`previous_response_id`, encrypted reasoning, provider conversation/file ids). * Keyed by the same affinity key as {@link threadAccountMap}, bounded the same - * way, and process-local — raw account ids never reach a log. + * way. Durable rows use a separate keyed digest; raw account ids never reach a log. */ type ConversationStateIssuerEntry = { accountId: string; lastUsedAt: number; + fence?: string; }; const conversationStateIssuerMap = new Map(); +function identityFence(identity: string | undefined): string | undefined { + return identity ? createHash("sha256").update("opencodex/codex-affinity/identity/v1\0").update(identity).digest("hex") : undefined; +} + +registerCodexAffinityLiveState({ + thread: (key, scope) => threadAccountMap.get(key)?.get(scope as ThreadAffinityScope), + issuer: key => conversationStateIssuerMap.get(key), + clearLive: () => { threadAccountMap.clear(); threadAffinityEntryTotal = 0; conversationStateIssuerMap.clear(); }, +}); + export function clearThreadAccountMap(options: { preserveConversationStateIssuers?: boolean } = {}): void { + recordAllCleared(); threadAccountMap.clear(); threadAffinityEntryTotal = 0; // A refresh cooldown is per-account runtime state learned alongside these bindings. Leaving it @@ -225,6 +241,7 @@ export function clearThreadAccountMap(options: { preserveConversationStateIssuer } export function clearConversationStateIssuerMap(): void { + recordIssuersCleared(); conversationStateIssuerMap.clear(); } @@ -232,6 +249,7 @@ export function clearThreadAccountMapForAccount( accountId: string, reason: CodexAffinityReason = "unusable", ): void { + recordAccountPinsCleared(accountId); for (const [threadId, affinities] of threadAccountMap) { for (const [scope, entry] of affinities) { if (entry.accountId === accountId && affinities.delete(scope)) { @@ -247,6 +265,7 @@ function pruneConversationStateIssuers(now: number): void { for (const [key, entry] of conversationStateIssuerMap) { if (now - entry.lastUsedAt > CODEX_THREAD_AFFINITY_IDLE_TTL_MS) { conversationStateIssuerMap.delete(key); + recordIssuerDelete(key); } } while (conversationStateIssuerMap.size > CODEX_THREAD_AFFINITY_MAX_ENTRIES) { @@ -260,22 +279,30 @@ function pruneConversationStateIssuers(now: number): void { } if (!oldestKey) break; conversationStateIssuerMap.delete(oldestKey); + recordIssuerDelete(oldestKey); } } /** * Record the pool account that just issued carried conversation state for this - * binding key. In-memory only; the id is never written to a request log. + * binding key. Persisted when an anchor is available; the id is never logged. */ export function rememberConversationStateIssuer( bindingKey: string, accountId: string, now = Date.now(), + credential?: { generation: number; chatgptAccountId: string }, ): void { if (!bindingKey.trim() || !accountId.trim()) return; if (!admissibleAffinityComponent(bindingKey) || !admissibleAffinityComponent(accountId)) return; pruneConversationStateIssuers(now); - conversationStateIssuerMap.set(bindingKey, { accountId, lastUsedAt: now }); + const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); + const fence = record?.credential && record.deletedAt == null && credential + && record.generation === credential.generation && record.credential.chatgptAccountId === credential.chatgptAccountId + ? identityFence(record.quotaHistoryIdentity) : undefined; + if (accountId === MAIN_CODEX_ACCOUNT_ID) recordIssuerDelete(bindingKey); + else if (accountId !== CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER) recordIssuerSet(bindingKey, accountId, fence ?? null, now); + conversationStateIssuerMap.set(bindingKey, { accountId, lastUsedAt: now, fence }); pruneConversationStateIssuers(now); } @@ -286,7 +313,16 @@ export function peekConversationStateIssuer( ): string | undefined { if (!bindingKey.trim() || !admissibleAffinityComponent(bindingKey)) return undefined; pruneConversationStateIssuers(now); - const entry = conversationStateIssuerMap.get(bindingKey); + recordIssuerTouch(bindingKey); + let entry = conversationStateIssuerMap.get(bindingKey); + if (!entry) { + const accountId = adoptStoredIssuer(bindingKey, now); + if (accountId !== undefined) { + entry = { accountId, lastUsedAt: now, fence: accountId === CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER ? undefined : codexAffinityIdentityFence(accountId) }; + conversationStateIssuerMap.set(bindingKey, entry); + pruneConversationStateIssuers(now); + } + } if (!entry) return undefined; entry.lastUsedAt = now; return entry.accountId; @@ -349,9 +385,25 @@ function getThreadAffinityForScope( scope: ThreadAffinityScope, ): ThreadAffinityEntry | undefined { if (!admissibleAffinityComponent(threadId)) return undefined; + recordThreadTouch(threadId, scope); + if (!threadAccountMap.get(threadId)?.get(scope)) adoptThreadAffinities(threadId, Date.now()); return threadAccountMap.get(threadId)?.get(scope); } +function adoptThreadAffinities(threadId: string, now: number): void { + for (const row of adoptStoredThreadRows(threadId, now)) { + const record = readCodexAccountRecord(row.accountId); + if (!record?.credential || record.deletedAt != null || identityFence(record.quotaHistoryIdentity) !== row.fence) continue; + const scope = row.scope as ThreadAffinityScope; + const affinities = threadAccountMap.get(threadId) ?? new Map(); + if (affinities.has(scope)) continue; + affinities.set(scope, { accountId: row.accountId, generation: record.generation, identityFence: row.fence, createdAt: row.createdAt, lastUsedAt: row.lastUsedAt, lastReevalAt: now }); + threadAccountMap.set(threadId, affinities); + threadAffinityEntryTotal += 1; + } + pruneLruThreadAffinities(); +} + export function getThreadAffinity(threadId: string, quotaScope?: CodexQuotaScope): ThreadAffinityEntry | undefined { return getThreadAffinityForScope(threadId, threadAffinityScope(quotaScope)); } @@ -367,6 +419,7 @@ export function getModelDetourAffinity( function deleteThreadAffinityForScope(threadId: string, scope: ThreadAffinityScope): void { if (!admissibleAffinityComponent(threadId)) return; + recordThreadDelete(threadId, scope); const affinities = threadAccountMap.get(threadId); if (!affinities) return; if (affinities.delete(scope)) { @@ -391,10 +444,12 @@ export function deleteModelDetourAffinity( /** Remove only the matching failed account's affinities for one thread. */ export function deleteThreadAffinitiesForAccount(threadId: string, accountId: string): void { if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; + adoptThreadAffinities(threadId, Date.now()); const affinities = threadAccountMap.get(threadId); if (!affinities) return; for (const [scope, entry] of affinities) { if (entry.accountId === accountId && affinities.delete(scope)) { + recordThreadDelete(threadId, scope); threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); } } @@ -467,6 +522,7 @@ function pruneExpiredThreadAffinities(now: number): void { for (const [threadId, affinities] of threadAccountMap) { for (const [scope, entry] of affinities) { if (isThreadAffinityExpired(entry, now) && affinities.delete(scope)) { + recordThreadDelete(threadId, scope); threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); } } @@ -510,15 +566,21 @@ function bindThreadAffinityForScope( const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); if (accountId !== MAIN_CODEX_ACCOUNT_ID && (!record?.credential || record.deletedAt != null)) return; pruneExpiredThreadAffinities(now); + recordThreadTouch(threadId, scope); const affinities = threadAccountMap.get(threadId) ?? new Map(); const previous = affinities.get(scope); - affinities.set(scope, { + const fence = identityFence(record?.quotaHistoryIdentity); + const entry: ThreadAffinityEntry = { accountId, generation: accountId === MAIN_CODEX_ACCOUNT_ID ? 0 : record!.generation, + identityFence: fence, createdAt: previous?.createdAt ?? now, lastUsedAt: now, lastReevalAt: now, - }); + }; + if (fence) recordThreadBind(threadId, scope, { scope, accountId, fence, createdAt: entry.createdAt, lastUsedAt: now }); + else recordThreadDelete(threadId, scope); + affinities.set(scope, entry); if (!previous) threadAffinityEntryTotal += 1; threadAccountMap.set(threadId, affinities); pruneLruThreadAffinities(); @@ -548,14 +610,14 @@ export function bindModelDetourAffinity( export function getThreadAffinityScopes( threadId: string, ): ReadonlyMap | undefined { + adoptThreadAffinities(threadId, Date.now()); return threadAccountMap.get(threadId); } /** * Move one scope's binding from the pre-#4546 RAW parent key onto the key this thread uses now. * - * Bindings and the key that derives them are process-local, so an ordinary restart already - * discards every binding and there is nothing to migrate. The case this exists for is the + * Legacy raw-parent keys are process-local and are not persisted. The case this exists for is the * narrow one: a code swap under a live conversation, where the map still holds entries made by * the old rule. Rebinding those cold is precisely the defect the lineage work exists to prevent, * so the conversation keeps its account and the legacy entry is retired in the same step. diff --git a/src/server/responses/account-change-state.ts b/src/server/responses/account-change-state.ts index c76130dfb21..71f0906ab42 100644 --- a/src/server/responses/account-change-state.ts +++ b/src/server/responses/account-change-state.ts @@ -12,6 +12,7 @@ import { conversationStateAccountId } from "../../codex/request-owned-affinity"; import type { CodexAuthContext } from "../../codex/auth-context"; +import { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "../../codex/routing/thread-affinity"; import { peekConversationStateIssuer, rememberConversationStateIssuer, @@ -239,7 +240,8 @@ export function rememberServingConversationStateIssuer( ): void { const binding = conversationStateBindingFromAuth(authCtx, fallbackAffinityKey); if (!binding) return; - rememberConversationStateIssuer(binding.bindingKey, binding.accountId); + rememberConversationStateIssuer(binding.bindingKey, binding.accountId, undefined, + authCtx.kind === "pool" ? { generation: authCtx.generation, chatgptAccountId: authCtx.chatgptAccountId } : undefined); } export interface ApplyAccountChangeConversationStateScrubArgs { @@ -321,7 +323,7 @@ export function accountChangeFileReferenceRefusal( const { body, bindingKey, servingAccountId, priorAccountId } = args; if (!servingAccountId || !bindingKey) return undefined; const issuer = peekConversationStateIssuer(bindingKey); - const accountChanged = (issuer != null && issuer !== servingAccountId) + const accountChanged = (issuer != null && issuer !== CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER && issuer !== servingAccountId) || (priorAccountId != null && priorAccountId !== servingAccountId); if (!accountChanged) return undefined; // Checked against the carriers directly rather than through the portability verdict: that diff --git a/tests/codex-integration/codex-affinity-restart.test.ts b/tests/codex-integration/codex-affinity-restart.test.ts new file mode 100644 index 00000000000..04234520927 --- /dev/null +++ b/tests/codex-integration/codex-affinity-restart.test.ts @@ -0,0 +1,72 @@ +import { expect, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { helperPath } from "../helpers/repo-root"; + +function run(home: string, mode: string) { + const result = Bun.spawnSync([process.execPath, helperPath("codex-affinity-restart-worker.ts"), mode], { + env: { ...process.env, OPENCODEX_HOME: home, CODEX_HOME: home }, stdout: "pipe", stderr: "pipe", + }); + return { exit: result.exitCode, stdout: result.stdout.toString(), stderr: result.stderr.toString() }; +} + +test("pool binding and issuer survive a clean restart", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-restart-")); + try { + const first = run(home, "write"); + expect(first.exit, first.stderr).toBe(0); + const second = run(home, "read"); + expect(second.exit, second.stderr).toBe(0); + const before = JSON.parse(first.stdout); + const after = JSON.parse(second.stdout); + expect(after.account).toBe(before.account); + expect(after.issuer).toBe(before.account); + expect(after.sameScrub).toBe(false); + expect(after.otherScrub).toBe(true); + expect(after.otherFileRefusal).toBe(400); + } finally { removeTreeWithRetry(home); } +}); + +test("an unclean owner discards persisted affinity", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-crash-")); + try { + const first = run(home, "crash"); + expect(first.exit).not.toBe(0); + const second = run(home, "read"); + expect(second.exit, second.stderr).toBe(0); + expect(JSON.parse(second.stdout).issuer).toBeUndefined(); + expect(JSON.parse(second.stdout).account).toBe("b"); + } finally { removeTreeWithRetry(home); } +}); + +test("unknown issuer scrubs continuation without refusing files, while retry moves are refused", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-unknown-")); + try { + const first = run(home, "unknown-write"); + expect(first.exit, first.stderr).toBe(0); + const second = run(home, "unknown-read"); + expect(second.exit, second.stderr).toBe(0); + const result = JSON.parse(second.stdout); + expect(result.issuer).toBe("__unknown_codex_conversation_state_issuer__"); + expect(result.sameScrub).toBe(true); + expect(result.fileRefusal).toBeUndefined(); + expect(result.retryRefusal).toBe(400); + expect(result.otherFileRefusal).toBeUndefined(); + } finally { removeTreeWithRetry(home); } +}); + +test("a replacement identity invalidates the old pin and makes its issuer unknown", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-relogin-")); + try { + const first = run(home, "relogin"); + expect(first.exit, first.stderr).toBe(0); + const second = run(home, "read"); + expect(second.exit, second.stderr).toBe(0); + const before = JSON.parse(first.stdout); + const after = JSON.parse(second.stdout); + expect(after.account).not.toBe(before.account); + expect(after.issuer).toBe("__unknown_codex_conversation_state_issuer__"); + } finally { removeTreeWithRetry(home); } +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index cdaca219ef0..f5ed42e01fa 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -698,7 +698,7 @@ "codex-restart-contract-parity.test.ts": "codex-integration", "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", "codex-service-manager-probe.test.ts": "codex-integration", "codex-systemd-env-probe.test.ts": "codex-integration", diff --git a/tests/helpers/codex-affinity-restart-worker.ts b/tests/helpers/codex-affinity-restart-worker.ts new file mode 100644 index 00000000000..dd115ccddcf --- /dev/null +++ b/tests/helpers/codex-affinity-restart-worker.ts @@ -0,0 +1,36 @@ +import { acquireSpendLedgerOwner } from "../../src/lib/spend-ledger-owner"; +import { readCodexAccountRecord, saveCodexAccountCredential } from "../../src/codex/account-store"; +import { codexPoolAffinityKey } from "../../src/codex/auth-context"; +import { resolveCodexAccountForThread, rememberConversationStateIssuer, peekConversationStateIssuer } from "../../src/codex/routing"; +import { applyAccountChangeConversationStateScrub, accountChangeFileReferenceRefusal } from "../../src/server/responses/account-change-state"; +import type { OcxConfig } from "../../src/types"; + +const lease = acquireSpendLedgerOwner(); +for (const id of ["a", "b"]) if (!readCodexAccountRecord(id)) saveCodexAccountCredential(id, { + accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `acct-${id}`, +}); +const config = { providers: {}, codexAccounts: ["a", "b"].map(id => ({ id, email: `${id}@test`, isMain: false })), activeCodexAccountId: "a", accountPoolStrategy: "round-robin", autoSwitchThreshold: 80, upstreamFailoverThreshold: 3 } as OcxConfig; +if (process.argv[2] === "read") resolveCodexAccountForThread("advance-cursor", config); +const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; +const issuer = peekConversationStateIssuer(key); +const account = resolveCodexAccountForThread(key, config)!; +const sameBody = { previous_response_id: "previous" }; +const otherBody = { previous_response_id: "previous" }; +const sameScrub = applyAccountChangeConversationStateScrub({ body: sameBody, bindingKey: key, servingAccountId: account }); +const otherScrub = applyAccountChangeConversationStateScrub({ body: otherBody, bindingKey: key, servingAccountId: account === "a" ? "b" : "a" }); +const fileBody = { previous_response_id: "previous", input: [{ type: "input_file", file_id: "file-test" }] }; +const fileRefusal = accountChangeFileReferenceRefusal({ body: fileBody, bindingKey: key, servingAccountId: account })?.status; +const otherFileRefusal = accountChangeFileReferenceRefusal({ body: fileBody, bindingKey: key, servingAccountId: account === "a" ? "b" : "a" })?.status; +const retryRefusal = accountChangeFileReferenceRefusal({ body: fileBody, bindingKey: key, servingAccountId: account, priorAccountId: account === "a" ? "b" : "a" })?.status; +const record = readCodexAccountRecord(account)!; +rememberConversationStateIssuer(key, account, undefined, process.argv[2] === "unknown-write" ? undefined : { generation: record.generation, chatgptAccountId: record.credential!.chatgptAccountId }); +if (process.argv[2] === "relogin") saveCodexAccountCredential(account, { + accessToken: "new-access", refreshToken: "new-refresh", expiresAt: Date.now() + 600_000, chatgptAccountId: "new-identity", +}); +if (process.argv[2] === "crash") { + const store = await import("../../src/codex/routing/thread-affinity-store").catch(() => undefined); + store?.flushCodexAffinityStoreForTests(); + process.kill(process.pid, "SIGKILL"); +} +lease.release(); +console.log(JSON.stringify({ account, issuer, sameScrub, otherScrub, fileRefusal, otherFileRefusal, retryRefusal })); From 76b9ffc30f200acd77e8372c67c94966818f0a14 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:23:23 +0900 Subject: [PATCH 05/21] docs(codex): durable pool affinity contract and restart wording --- .../docs/fr/reference/cli/providers-accounts.md | 5 ++--- .../docs/fr/reference/configuration/providers.md | 5 ++--- .../docs/ja/reference/cli/providers-accounts.md | 2 +- .../docs/ja/reference/configuration/providers.md | 5 ++--- .../docs/ko/reference/cli/providers-accounts.md | 2 +- .../docs/ko/reference/configuration/providers.md | 5 ++--- .../docs/reference/cli/providers-accounts.md | 5 ++--- .../docs/reference/configuration/providers.md | 5 ++--- .../docs/ru/reference/cli/providers-accounts.md | 2 +- .../docs/ru/reference/configuration/providers.md | 5 ++--- .../docs/tr/reference/cli/providers-accounts.md | 6 ++---- .../docs/tr/reference/configuration/providers.md | 5 ++--- .../zh-cn/reference/cli/providers-accounts.md | 2 +- .../zh-cn/reference/configuration/providers.md | 4 ++-- .../zh-tw/reference/cli/providers-accounts.md | 2 +- .../zh-tw/reference/configuration/providers.md | 4 ++-- structure/INDEX.md | 4 ++++ structure/manifest.json | 11 +++++++++++ structure/providers/codex-pool-affinity.md | 15 +++++++++++++++ structure/providers/openai-tiers.md | 4 ++-- 20 files changed, 59 insertions(+), 39 deletions(-) create mode 100644 structure/providers/codex-pool-affinity.md diff --git a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md index acdc51645c5..185723f8efb 100644 --- a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md @@ -229,9 +229,8 @@ cet état et quitte toujours 0. `--json` renvoie : `auto` efface la sélection manuelle pour que le pool place à nouveau le travail selon sa propre stratégie — sauf si un compte Codex porte littéralement l'id `auto`, qui l'emporte par correspondance exacte d'id ; `ocx account clear ` rétablit toujours la sélection automatique. Un compte Codex peut être désigné par l'alias défini avec `ocx account alias` au lieu de son id ; cela vaut aussi pour `priority`, `pause`, `resume`, `clear-cooldown`, `remove` et `alias`. Pour les comptes Codex, `auto`, `main` et `__main__` sont réservés sans distinction de casse et ne peuvent pas être attribués comme alias. Les noms affichés des comptes OAuth et des clés API conservent leurs règles existantes. Sélectionne un compte Codex, un compte OAuth ou une clé API existant. Pour `openai`, `main` sélectionne la -connexion Codex App. Une sélection en mode Codex Pool efface l'affinité locale du processus et s'applique à la requête suivante, -dont un provenant d'une tâche visible existante ; le redémarrage du proxy ou l'expulsion par affinité peuvent également laisser une tâche -non lié, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; +connexion Codex App. Une sélection en mode Codex Pool efface l'affinité et s'applique à la requête suivante, +dont un provenant d'une tâche visible existante . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; Le mode direct continue d’utiliser les informations d’identification principales de l’appelant. Commutation proactive basée sur l'utilisation, 401/403 réauthentification, 429/retry-after temps de recharge, exclusion et pré-sortie 429/402 échec la récupération peut sélectionner ultérieurement un autre compte Pool éligible. Ces chemins de récupération restent actifs lorsque diff --git a/docs-site/src/content/docs/fr/reference/configuration/providers.md b/docs-site/src/content/docs/fr/reference/configuration/providers.md index c2ba9e8836a..7ef4a918b3b 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/fr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Après une inscription ou une connexion OAuth dans l’interface, une boîte de | `activeCodexAccountPinned?` | `string` | — | Identifiant du compte du dernier opérateur sélectionné manuellement. Lorsqu'il est défini, un niveau `codexAccountPriorities` supérieur ne peut pas le préempter jusqu'à ce que la broche soit libérée par drainage, exclusion, suppression ou un failover/promotion explicite. Un mouvement circulaire ordinaire à l’intérieur du niveau plafonné ne le libère pas. L'écriture d'une entrée `codexAccountPriorities` libère également le pin, donc un pin créé avant qu'un ordre n'existe ne peut pas surpasser un ensemble par la suite. `GET /api/codex-auth/active` indique à la fois si le compte effectif est épinglé (`pinned`) et le compte portant le plafond (`pinnedAccountId`). | | `autoSwitchThreshold?` | `number` | `80` | Seuil d'utilisation pour la commutation proactive. `quota` peut réévaluer les requêtes non liées lors de leur prochaine requête. Les tâches liées conservent leur compte au-delà du seuil par défaut (`pool.cacheAffinity`) jusqu'à ce que ce compte soit épuisé ou ne puisse plus servir, et ne basculent alors que vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour réévaluer aussi les tâches liées à ce seuil. `fill-first` ne l'utilise que comme seuil d'évacuation pour l'affectation des requêtes non liées ; `round-robin` ne l'utilise que pour les nouvelles affectations non liées : tant qu'un compte admissible reste sous son seuil effectif, il ignore ceux qui l'atteignent ou le dépassent, sans jamais déplacer une tâche liée pour cette raison. Le score retient la plus élevée des fenêtres de quota connues sur 5 heures, une semaine ou 30 jours. `0` désactive uniquement la commutation proactive fondée sur l'utilisation, pas l'affectation des requêtes non liées ni la récupération après incident. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Seuils par compte remplaçant `autoSwitchThreshold` : identifiant → entier de `0` à `100`. Une entrée absente hérite du seuil global ; `0` désactive uniquement les déplacements fondés sur l’utilisation depuis ce compte. Le compte principal `__main__` est pris en charge. La carte du compte dans Codex Auth gère cette valeur. Activer la valeur spécifique au compte copie le seuil global actuel dans une valeur fixe. Cette valeur, y compris `0`, reste prioritaire après toute modification du seuil global. La désactiver envoie `threshold: null`, supprime l'entrée et rétablit l'héritage du seuil global actuel et de ses modifications futures. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota ; une tâche existante visible peut perdre son lien après le redémarrage du proxy ou la réinitialisation de l'affinité. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | | `pool.cacheAffinity?` | `boolean` | `true` | Ordre d'affinité de cache pour les threads Codex liés, indépendant de `pool.kernel`. Activé par défaut ; omettre la clé ou la définir à `true` conserve la liaison, et une valeur autre que `false` est lue comme activée. Une liaison active prime sur la marge de quota : `quota` ne déplace pas le thread simplement parce que l'utilisation a franchi `autoSwitchThreshold`, car déplacer une conversation liée jette le cache d'invites isolé par compte. Le thread quitte encore le compte s'il ne peut plus servir — suspendu, inutilisable, exclu du plan, identifiants invalides, génération remplacée, TTL expiré, refus de quota 429/402, ou réellement épuisé (utilisation connue à 100 %) — et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Un compte dont l'utilisation est inconnue n'est jamais choisi comme destination d'une tâche liée. Si tous les comptes dépassent le seuil, la tâche liée reste, car aucune destination n'est meilleure. Définissez `false` pour rétablir la réaffectation au seuil, toujours sous la même contrainte de destination. L'affinité est un réordonnancement, pas un verrouillage. | | `accountPoolStickyLimit?` | `number` | `1` | Nombre d'affectations de tâches nouvelles ou non liées conservées sur une même sélection tournante avant de passer à la suivante ; le compteur avance lorsqu'une tâche est liée, et non après une réponse réussie en amont. Plage : 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Nombre d'échecs transitoires consécutifs avant le basculement des futures nouvelles sessions. Réglez `0` pour désactiver ce mécanisme. Pour les requêtes Responses ordinaires et les envois compacts natifs, les échecs avérés d'accessibilité DNS/TCP avant connexion sont suivis au niveau du couple fournisseur-hôte : ils n'affectent jamais l'état ni la temporisation du compte, l'affinité de tâche ou de session, la sélection du compte actif ou le routage du pool, et ne sont jamais comptabilisés dans ce seuil. | @@ -235,8 +235,7 @@ Utilisez **Codex Auth** dans le tableau de bord pour ajouter des comptes au grou métadonnées non secrètes ; les jetons d'accès et d'actualisation utilisent le magasin d'identifiants renforcé. Le routage du pool distingue l'affectation des requêtes nouvelles ou non liées, la commutation proactive fondée sur l'utilisation et la récupération après incident. Une tâche liée conserve normalement son affinité. Par défaut (`pool.cacheAffinity`), cette réaffectation attend que le compte lié soit épuisé ou ne puisse plus servir, et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour laisser `quota` la relier dès la requête suivante une fois le seuil franchi, toujours sous la même contrainte de destination. Un refus de quota 429/402, une pause, une invalidation des identifiants ou l'expiration du TTL libèrent encore la liaison immédiatement ; une série d'échecs transitoires (5xx et autres échecs hors quota) sert la requête sur un autre compte sans supprimer la liaison active. -Une requête non liée ne possède aucune liaison active à un compte ; il peut s'agir d'une tâche existante visible après le redémarrage -du proxy ou la réinitialisation de l'affinité. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un +Une requête non liée ne possède aucune liaison active à un compte . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un autre compte admissible au sein de la même requête, même lorsque la commutation proactive fondée sur l'utilisation est désactivée. Les changements de compte préservent et relisent le contexte de la conversation, mais la réutilisation du cache d'invites du fournisseur entre plusieurs comptes n'est pas garantie et le cache peut devoir être réchauffé. diff --git a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md index b199c822ffc..3d36b06cc96 100644 --- a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md @@ -180,7 +180,7 @@ Codex クライアント連携を無効にしても、既存のネイティブ `auto` は手動の選択を解除し、プールが自身の戦略で再び配置するようにします — ただし id が `auto` の Codex アカウントが存在する場合は完全一致の id が優先され、`ocx account clear ` が常に自動選択を復元します。Codex アカウントは id の代わりに `ocx account alias` で付けたエイリアスでも指定でき、`priority`、`pause`、`resume`、`clear-cooldown`、`remove`、`alias` でも同様です。Codex アカウントでは `auto`、`main`、`__main__` は大文字・小文字を区別せず予約語として扱われるため、エイリアスとして設定できません。OAuth アカウントと API キーの表示名には従来のルールが適用されます。 既存の Codex アカウント、OAuth アカウント、または API key を選びます。`openai` で `main` は Codex App ログインを -選択します。Codex Pool の選択は process-local affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。プロキシ再起動や affinity eviction 後もタスクは未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 +選択します。Codex Pool の選択は affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 不明なプロバイダーや id は終了コード 1 です。`--json` は次を返します。 **401/403** では、そのアカウントへのプロセスローカルな affinity を解除し、再認証を要求します。 **429** では `Retry-After` を尊重してアカウントの cooldown を開始し、affinity を解除したうえで、 diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index 3e9e5085980..cf71d84dc52 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI で登録または OAuth ログインが完了すると、Models ページ | `codexAccountPriorities?` | `Record` | — | Codex pool のアカウント別選択順。アカウント ID → `-100` から `100` の整数で、**大きいほど先に使われ**、未設定は `0` です。これは eligibility ではなく順序の境界です。選択は適格なアカウントを、まだ quota に余裕がある最上位 tier に絞り込み、その tier の中を `accountPoolStrategy` が選びます。tier が飛ばされるのは、そのメンバー全員が自身の 0 以外の実効しきい値（アカウント別上書き、未設定ならグローバル値）以上、cooldown 中、soft-avoid、一時停止、または再認証待ちのときだけで、usage 不明が tier を drain させることはありません。順序付けが不適格なアカウントを選択可能にすることはなく、すでにアカウントが結び付いた thread を再 bind することもありません。メインの `__main__` も同じ条件で参加するため、Codex Desktop ログインを最後に使わせられます。エントリが 1 つもなければ挙動は従来どおりです。map が不正な場合は警告を出して順序付けを無効にします（config の修復処理は走りません）。`ocx account priority` と Codex Auth ページで管理します。 | | `autoSwitchThreshold?` | `number` | `80` | 使用量ベースのプロアクティブ切り替えしきい値。`quota` は未紐付けタスクの次のリクエストを再評価できます。紐付け済みタスクは既定（`pool.cacheAffinity`）ではしきい値を超えても同じアカウントを維持し、アカウントが使い切られるか処理できなくなったときだけ離れ、その場合も実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`pool.cacheAffinity: false` にするとしきい値で紐付け済みタスクも再評価します。`fill-first` は未紐付け割り当ての使い切り基準としてのみ使用し、`round-robin` は新規/未紐付けの割り当て時にのみ使用し、しきい値未満の適格アカウントがある間は実効しきい値以上のアカウントを除外し、この理由で紐付け済みタスクを移動しません。既知の 5 時間、週次、30 日 quota window の最大スコアを使います。`0` は使用量ベースの切り替えだけを無効にし、未紐付け割り当てや障害回復は無効にしません。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | `autoSwitchThreshold` のアカウント別上書き（アカウント ID → `0`〜`100` の整数）。未設定はグローバル値を継承し、`0` はそのアカウントからの使用量ベース切り替えだけを無効にします。メインの `__main__` も指定できます。Codex Auth の各アカウントカードで管理します。 上書きを有効にすると、現在のグローバルしきい値が固定のアカウント別値としてコピーされます。`0` を含む上書き値は、その後グローバル値が変更されても優先されます。無効にすると `threshold: null` を送信してエントリを削除し、現在のグローバルしきい値と今後の変更を継承する状態に戻ります。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けで、プロキシ再起動や affinity リセット後は既存の表示タスクも未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けです。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | | `pool.cacheAffinity?` | `boolean` | `true` | 紐付け済み Codex スレッド向けの cache-affinity 順序。`pool.kernel` とは独立で、既定はオンです。不正な値はオンとして読みます。live な紐付けが quota 余裕より優先され、`quota` は使用量が `autoSwitchThreshold` を超えたという理由だけではスレッドを移しません。一時停止、使用不可、または実際に使い切られたアカウント（既知 usage 100%）では離れますが、実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`false` にするとしきい値での再紐付けに戻ります。affinity は固定ではなく並べ替えです。 | | `accountPoolStickyLimit?` | `number` | `1` | 1 回の round-robin 選択で次へ進む前に保持する新規/未紐付けタスク割り当て数。カウンターは上流の成功後ではなくタスクの紐付け時に増えます。範囲 1–100。`accountPoolStrategy` が `round-robin` のときのみ。 | | `upstreamFailoverThreshold?` | `number` | `3` |今後の新しいセッションがフェイルオーバーする前に一時的なエラーが連続して発生する。 `0` を無効に設定します。通常のResponses送信とネイティブcompact送信では、実証済みの接続前DNS/TCP到達不能障害はprovider-host単位で記録され、アカウントの健全性、アカウントのクールダウン、スレッド/セッションの親和性、アクティブアカウントの選択、Poolルーティングには影響せず、この閾値にもカウントされません。 | @@ -220,8 +220,7 @@ pool アカウントの追加と quota 更新はダッシュボードの **Codex 使い切られるか処理できなくなるまで再紐付けを延期し、離れるときは実際に quota 余裕があり usage が より低いアカウントへだけ移ります。フラグをオフにすると、`quota` はしきい値超過後の次のリクエストで 再紐付けできます。 -pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには -プロキシ再起動や affinity リセット後の既存タスクも含まれます。出力前の **429/402** は使用量ベースの +pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには有効なアカウントの紐付けがありません。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。出力前の **429/402** は使用量ベースの 切り替えがオフでも同じリクエストで適格な代替アカウントへ 1 回再試行できます。アカウント変更後も会話 コンテキストは保持・再生されますが、アカウント間の provider prompt cache 再利用は保証されません。 一時停止したアカウントと quota metadata は表示されたままですが、自動切り替え、再試行/failover 選択、cooldown 復旧プローブ、手動有効化の対象外です。 diff --git a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md index 4355de7071b..dfdb335f3cd 100644 --- a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md @@ -298,7 +298,7 @@ Codex JSON 행에는 등록된 `id`와 같은 `slotId`, 그리고 `native-main` `auto`는 수동 선택을 지워 풀이 다시 자체 전략으로 작업을 배치하게 합니다 — 단 id가 `auto`인 Codex 계정이 있으면 정확한 id 일치가 우선되며 `ocx account clear `는 항상 자동 선택을 복원합니다. Codex 계정은 id 대신 `ocx account alias`로 지정한 별칭으로도 가리킬 수 있으며, `priority`, `pause`, `resume`, `clear-cooldown`, `remove`, `alias`에서도 마찬가지입니다. Codex 계정에서 `auto`, `main`, `__main__`은 대소문자 구분 없이 예약어이므로 별칭으로 지정할 수 없습니다. OAuth 계정과 API 키의 표시 이름에는 기존 규칙이 그대로 적용됩니다. 기존 Codex 계정, OAuth 계정 또는 API key를 선택합니다. `openai`에서 `main`은 Codex App 로그인을 -선택합니다. Codex Pool 선택은 프로세스 로컬 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 프록시 재시작이나 affinity eviction 뒤에도 작업이 바인딩 없는 상태가 될 수 있지만, 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. +선택합니다. Codex Pool 선택은 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. 알 수 없는 프로바이더나 id는 종료 코드 1입니다. `--json`은 다음을 반환합니다. **401/403**이 발생하면 해당 계정의 프로세스 로컬 affinity를 해제하고 재인증을 요구합니다. **429**에서는 `Retry-After`를 준수해 계정 cooldown을 시작하고 affinity를 해제한 뒤, diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index 4dfefc64d14..d94d07da409 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI에서 등록이나 OAuth 로그인을 마치면 Models 페이지로 이동 | `codexAccountPriorities?` | `Record` | — | Codex pool의 계정별 선택 순서. 계정 ID → `-100`부터 `100`까지의 정수이며 **값이 클수록 먼저** 쓰이고, 항목이 없으면 `0`입니다. 이는 eligibility 경계가 아니라 순서 경계입니다. 선택은 이미 적격한 계정들을 quota 여유가 남은 최상위 tier로 좁히고, 그 tier 안에서 `accountPoolStrategy`가 계정을 고릅니다. tier를 건너뛰는 경우는 그 구성원 전부가 자신의 0이 아닌 유효 임계값(계정별 재정의 또는 전역 기본값) 이상, cooldown, soft-avoid, 일시 중지 또는 재인증 대기일 때뿐이며, usage를 알 수 없다고 해서 tier가 소진되지는 않습니다. 순서는 부적격 계정을 선택 가능하게 만들지 않고, 이미 계정에 묶인 thread를 다시 bind하지도 않습니다. 메인 `__main__` 계정도 동일한 조건으로 참여하므로 Codex Desktop 로그인을 마지막에 쓰도록 둘 수 있습니다. 항목이 하나도 없으면 동작은 이전과 같습니다. map이 잘못된 경우 경고를 출력하고 순서 지정을 끕니다(config 복구는 하지 않습니다). `ocx account priority`와 Codex Auth 페이지에서 관리합니다. | | `autoSwitchThreshold?` | `number` | `80` | 사용량 기반 선제 전환 임계값입니다. `quota`는 바인딩 없는 작업의 다음 요청을 재평가할 수 있습니다. 바인딩된 작업은 기본값(`pool.cacheAffinity`)에서 이 임계값을 넘어도 계정을 유지하며, 해당 계정이 소진되었거나 더 이상 처리할 수 없을 때에만 떠나고, 그때도 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `pool.cacheAffinity: false`로 두면 임계값에서 바인딩된 작업도 재평가합니다. `fill-first`는 바인딩 없는 작업 배정의 소진 기준으로만 사용하며, `round-robin`은 새/바인딩 없는 작업 배정에만 사용하며 임계값 미만인 적격 계정이 남아 있는 동안 유효 임계값 이상인 계정을 건너뛰고 이 때문에 바인딩된 작업을 옮기지 않습니다. 알려진 5시간, 주간, 30일 quota window 중 가장 높은 점수를 씁니다. `0`은 사용량 기반 전환만 끄며 바인딩 없는 작업 배정이나 실패 복구는 끄지 않습니다. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 계정별 `autoSwitchThreshold` 재정의입니다(계정 ID → `0`~`100` 정수). 항목이 없으면 전역 값을 상속하고, `0`은 해당 계정에서 시작하는 사용량 기반 전환만 끕니다. 메인 `__main__` 계정도 지원합니다. Codex Auth의 계정 카드에서 관리합니다. 재정의를 켜면 현재 전역 임계값을 고정된 계정별 값으로 복사합니다. `0`을 포함한 재정의 값은 이후 전역 값이 바뀌어도 우선합니다. 끄면 `threshold: null`을 보내 항목을 삭제하고 현재 전역 임계값과 향후 변경 사항을 상속하는 상태로 돌아갑니다. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청이며, 프록시 재시작이나 affinity 초기화 뒤에는 기존에 보이던 작업도 바인딩이 없어질 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청입니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | | `pool.cacheAffinity?` | `boolean` | `true` | 바인딩된 Codex 스레드의 cache-affinity 순서입니다. `pool.kernel`과는 별개이며 기본값은 켜짐입니다. 잘못된 값은 켜진 것으로 읽습니다. live 바인딩이 quota 여유보다 우선하므로 `quota`는 사용량이 `autoSwitchThreshold`를 넘었다는 이유만으로 스레드를 옮기지 않습니다. 해당 계정이 일시 중지되었거나 사용할 수 없거나 실제로 소진된 경우(알려진 usage 100%)에는 떠나되, 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `false`로 두면 임계값 재바인딩이 복원됩니다. affinity는 고정이 아니라 재정렬입니다. | | `accountPoolStickyLimit?` | `number` | `1` | 한 round-robin 선택이 다음으로 넘어가기 전에 유지하는 새 작업/바인딩 없는 작업 배정 수입니다. 카운터는 업스트림 성공 뒤가 아니라 작업을 바인딩할 때 증가합니다. 범위 1–100이며 `accountPoolStrategy`가 `round-robin`일 때만 적용됩니다. | | `upstreamFailoverThreshold?` | `number` | `3` | 연속된 일시적 실패가 이 횟수에 도달하면 이후 새 세션은 failover됩니다. `0`으로 두면 비활성화됩니다. 일반 Responses와 네이티브 compact 전송에서 입증된 연결 전 DNS/TCP 도달 불가 실패는 provider-host 범위로 기록되며 계정 상태, 계정 쿨다운, 스레드/세션 선호도, 활성 계정 선택 또는 Pool 라우팅에 영향을 주지 않고 이 임계값에도 집계되지 않습니다. | @@ -221,8 +221,7 @@ pool 계정 추가와 quota 갱신은 대시보드의 **Codex Auth** 페이지 구분됩니다. 바인딩된 작업은 보통 affinity를 유지합니다. 기본값(`pool.cacheAffinity`)에서는 바인딩된 계정이 소진되었거나 더 이상 처리할 수 없을 때까지 재바인딩을 미루고, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 `quota`가 사용량 임계값을 넘은 뒤 다음 요청에서 재바인딩할 수 있습니다. 일시 중지, cooldown, 재인증, 실패 처리도 -독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청은 live 계정 바인딩이 없는 요청이며, 프록시 재시작이나 -affinity 초기화 뒤의 기존 작업도 포함될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 +독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청에는 live 계정 바인딩이 없습니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 전환이 꺼져 있어도 같은 요청에서 적격 대체 계정으로 한 번 재시도할 수 있습니다. 계정이 바뀌어도 대화 문맥은 보존·재생되지만 계정 간 프로바이더 측 prompt cache 재사용은 보장되지 않아 다시 예열해야 할 수 있습니다. diff --git a/docs-site/src/content/docs/reference/cli/providers-accounts.md b/docs-site/src/content/docs/reference/cli/providers-accounts.md index 3100c891189..5401508adc8 100644 --- a/docs-site/src/content/docs/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/reference/cli/providers-accounts.md @@ -845,9 +845,8 @@ that state and still exits 0. `--json` returns: `auto` clears the manual selection so the pool places work by its own strategy again — unless a Codex account literally carries the id `auto`, which wins by exact-id precedence; `ocx account clear ` always restores automatic selection. Any Codex account can be named by the alias set with `ocx account alias` instead of its id; that holds for `priority`, `pause`, `resume`, `clear-cooldown`, `remove` and `alias` too. For Codex accounts, `auto`, `main` and `__main__` are reserved regardless of case and cannot be assigned as aliases. OAuth and API-key display names keep their existing rules. Selects an existing Codex account, OAuth account, or API key. For `openai`, `main` selects the Codex -App login. A Codex Pool selection clears process-local affinity and applies to the next request, -including one from an existing visible task; proxy restart or affinity eviction can also leave a task -unbound, while in-flight requests keep their captured account. This controls Pool routing only; +App login. A Codex Pool selection clears affinity and applies to the next request, +including one from an existing visible task. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound, while in-flight requests keep their captured account. This controls Pool routing only; Direct mode keeps using the caller-owned/native main credential. Usage-based proactive switching, 401/403 reauthentication, 429/retry-after cooldowns, exclusion, and pre-output 429/402 failure recovery may later select another eligible Pool account. Those recovery paths remain active when diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 7190882103a..7a43a0013b7 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -125,7 +125,7 @@ separate. Full request URLs such as `/api/v1/responses` are not provider base UR | `codexAccountPriorityFailback?` | `boolean` | `false` | Opt-in return of ongoing Codex Pool tasks to a usable higher-priority account under the `quota` strategy. Requests trigger a background quota refresh at most once every five minutes, including failed refresh attempts. When recent quota observations show headroom, the next request may rebind even if its current account is below the threshold. This explicit preference can discard a warm prompt cache; it does not change default cache affinity. No requests means no polling. Manual pins, model access, cooldowns, pauses and reauthentication exclusions still apply; in-flight requests keep their captured account. Requires the current source account's effective threshold to be positive (an override of 0 disables it; a positive override works with global 0); exact account routes and Direct mode are unchanged. | | `autoSwitchThreshold?` | `number` | `80` | Usage threshold for placing new/unbound work. `quota` can re-evaluate unbound tasks on their next request once usage crosses this threshold. Bound tasks keep their account past the threshold by default (`pool.cacheAffinity`); they leave only when that account is exhausted or otherwise cannot serve, and then only for an account with genuine quota headroom and strictly lower usage. Set `pool.cacheAffinity: false` to re-evaluate bound tasks at this threshold, still only onto such a destination. `fill-first` uses it only as the drain point for unbound assignment; `round-robin` uses it only when placing new/unbound work: it skips accounts at or over their effective threshold while any eligible account is still under it, and never moves a bound task for it. The score uses the hottest known 5h, weekly, or 30d quota window. `0` disables usage-based proactive switching only, not unbound assignment or failure recovery. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Per-account overrides for `autoSwitchThreshold`: account id → integer from `0` to `100`. An absent entry inherits the global value; `0` disables usage-driven switching only when that account is the source. `quota` uses the current source account's effective threshold for bound-task re-evaluation and unbound switching. Fill-first drain and selection-order tier headroom are evaluated against each account's own effective threshold; main-account pins and subagent fallback also use the relevant account's effective threshold. Supports the main `__main__` account. A malformed map is ignored with a warning. Managed by each account card on the Codex Auth page; disabling an override removes its entry. Enabling the override copies the current global threshold into a fixed account value. The override, including `0`, takes precedence over later global changes. Disabling it sends `threshold: null`, removes the entry, and restores inheritance of the current global threshold and future global changes. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity; a visible existing task can become unbound after proxy restart or affinity reset. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | | `pool.cacheAffinity?` | `boolean` | `true` | Cache-affinity ordering for bound Codex threads, independent of `pool.kernel`. On by default; omitting the key or setting `true` keeps a bound task on its account until that account genuinely cannot serve. Only an explicit `false` restores threshold-based rebinding of bound tasks. A live binding outranks quota headroom: `quota` does not move the thread merely because usage crossed `autoSwitchThreshold`. The thread still leaves if that account cannot serve — paused, unusable, or genuinely exhausted (known usage at 100%) — and then only to an account with genuine quota headroom and strictly lower usage. Under either setting, an account with unknown usage is never chosen as a destination for a bound task, so when every account sits above the threshold the task stays put. Affinity is a reordering, not a pin. | | `pool.credentialGroups?` | `Array<{ id: string; credentials: string[]; note?: string }>` | `[]` | Accepted and validated, but not yet consumed by routing: declaring a group changes no routing decision until a consuming layer lands. Operator-declared quota domains: groups of credentials that demonstrably share one upstream usage limit. Members of one group count once toward available capacity, and a quota refusal inside a group is not answered by rotating to another member — the limit is the same, so the move would pay a cold prefix for zero new capacity. Declared groups speak only to quota; sharing a limit says nothing about prompt-cache compatibility, which is classified separately. Each member is written provider-qualified as `":"`, because a credential id means something only inside its provider; the provider segment accepts the usual aliases (`chatgpt:` and `codex:` both mean OpenAI). Group ids must be unique, `credentials` must be non-empty, and a credential may appear in at most one group — an ambiguous declaration is rejected on write and dropped with a warning on load rather than resolved by whichever group is listed first, since that would merge two unrelated quota domains. A malformed list costs only the grouping: `pool.kernel` and `pool.cacheAffinity` are preserved. Absent or empty means no declared grouping, so an unconfigured install behaves exactly as before. | | `accountPoolStickyLimit?` | `number` | `1` | New/unbound task assignments retained on one round-robin selection before advancing; the counter advances when a task is bound, not after an upstream success. Range 1–100. | @@ -1022,8 +1022,7 @@ Set `pool.cacheAffinity: false` to restore threshold rebinding of bound tasks, s a destination. Pause, cooldown, reauthentication, and quota refusals still release a live binding immediately. A transient-failure streak serves the request from another account while keeping the binding, and the task returns once that account serves again; after 10 minutes the binding is -released normally. An unbound request has no live account binding; this can include an existing visible -task after proxy restart or affinity reset. A pre-stream 429 or 402, or a 5xx response whose bounded +released normally. An unbound request has no live account binding. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. A pre-stream 429 or 402, or a 5xx response whose bounded body explicitly reports quota exhaustion, retries once on an eligible alternate account in the same request, even when usage-based proactive switching is off. The ordinary transient-5xx policy runs first, so a wrapped quota response may make up to three sends on the exhausted account before pool diff --git a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md index 80861b69725..abf6cf8b6ff 100644 --- a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md @@ -211,7 +211,7 @@ credential'а, это состояние тоже печатается, но к `auto` снимает ручной выбор, и пул снова распределяет работу по своей стратегии — если только аккаунт Codex буквально не имеет id `auto`: точное совпадение id выигрывает, а `ocx account clear ` всегда восстанавливает автоматический выбор. Аккаунт Codex можно указать по псевдониму, заданному через `ocx account alias`, вместо id; это относится и к `priority`, `pause`, `resume`, `clear-cooldown`, `remove` и `alias`. Для аккаунтов Codex значения `auto`, `main` и `__main__` зарезервированы независимо от регистра и не могут назначаться как псевдонимы. Для отображаемых имён аккаунтов OAuth и API-ключей действуют прежние правила. Выбирает существующий аккаунт Codex, OAuth-аккаунт или API-ключ. Для `openai` значение `main` -выбирает вход Codex App. Выбор Codex Pool очищает process-local affinity и применяется к следующему запросу, включая запрос существующей видимой задачи; после перезапуска прокси или affinity eviction задача также может стать непривязанной, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры +выбирает вход Codex App. Выбор Codex Pool очищает affinity и применяется к следующему запросу, включая запрос существующей видимой задачи. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры или id завершаются с кодом 1. `--json` возвращает: При **401/403** локальная для процесса привязка к аккаунту сбрасывается и требуется повторная аутентификация. При **429** учитывается `Retry-After`, для аккаунта запускается cooldown, привязка сбрасывается, diff --git a/docs-site/src/content/docs/ru/reference/configuration/providers.md b/docs-site/src/content/docs/ru/reference/configuration/providers.md index 22a4986e82d..91c2d47f0e5 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ru/reference/configuration/providers.md @@ -56,7 +56,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Порядок выбора для каждого аккаунта пула Codex: id аккаунта → целое число от `-100` до `100`, **больше — используется раньше**, отсутствие означает `0`. Это граница порядка, а не пригодности: выбор сужает уже подходящие аккаунты до самого высокого уровня, у которого ещё есть запас квоты, а внутри этого уровня аккаунт выбирает `accountPoolStrategy`. Уровень пропускается, только когда все его аккаунты достигли собственного ненулевого эффективного порога (переопределение аккаунта или глобальное значение), находятся в cooldown, под soft-avoid, на паузе или требуют повторной аутентификации; неизвестный usage никогда не исчерпывает уровень. Порядок не делает выбираемым непригодный аккаунт и не перепривязывает поток, у которого аккаунт уже есть. Основной аккаунт `__main__` участвует на равных — именно так логин Codex Desktop можно оставить на самый конец. Без записей поведение остаётся прежним. Некорректная map игнорируется с предупреждением в консоли (порядок отключается, восстановление config не запускается). Управляется через `ocx account priority` и страницу Codex Auth. | | `autoSwitchThreshold?` | `number` | `80` | Порог проактивного переключения по использованию. `quota` может повторно оценить следующий непривязанный запрос. Привязанная задача по умолчанию (`pool.cacheAffinity`) сохраняет аккаунт после порога, пока он не исчерпан и ещё может обслуживать запрос, и переносится только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы повторно оценивать привязанные задачи на этом пороге. `fill-first` использует его только как точку исчерпания для непривязанных назначений; `round-robin` применяет его только к новым назначениям: пока есть подходящий аккаунт ниже своего порога, аккаунты на пороге или выше пропускаются; привязанная задача из-за этого не переносится. Оценка берёт самое горячее из окон 5 часов, недели и 30 дней. `0` отключает только переключение по использованию, но не назначение непривязанных задач и не восстановление после сбоев. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Переопределения `autoSwitchThreshold` по аккаунтам: id → целое число `0`–`100`. Без записи наследуется глобальное значение; `0` отключает только переключения по использованию с этого аккаунта. Поддерживается основной аккаунт `__main__`. Управляется в карточке аккаунта на странице Codex Auth. Включение переопределения копирует текущий глобальный порог в фиксированное значение аккаунта. Переопределение, включая `0`, имеет приоритет и после изменения глобального порога. Отключение отправляет `threshold: null`, удаляет запись и восстанавливает наследование текущего глобального порога и его будущих изменений. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`; видимая существующая задача может стать непривязанной после перезапуска прокси или сброса affinity. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | | `pool.cacheAffinity?` | `boolean` | `true` | Порядок cache-affinity для привязанных потоков Codex, независимый от `pool.kernel`. По умолчанию включён; пропуск ключа или `true` сохраняет привязку, а любое значение кроме `false` читается как включённое. Живая привязка важнее запаса квоты: `quota` не переносит поток только потому, что usage пересёк `autoSwitchThreshold` — перенос живого разговора сбрасывает изолированный по аккаунту prompt cache. Поток всё равно уходит, если аккаунт не может обслуживать запрос — на паузе, непригоден, исключён планом, с недействительными credential'ами, со сменённой generation, с истёкшим TTL, при отказе квоты 429/402 или реально исчерпан (известный usage 100%) — и только на аккаунт со строго меньшим usage и реальным запасом квоты. Аккаунт с неизвестным usage никогда не выбирается как назначение для привязанной задачи. Если все аккаунты выше порога, привязанная задача остаётся: лучшего назначения нет. `false` возвращает перепривязку по порогу, всё равно только при той же проверке назначения. Affinity меняет порядок, а не закрепляет учётные данные. | | `accountPoolStickyLimit?` | `number` | `1` | Число назначений новых/непривязанных задач на одном выборе round-robin перед переходом дальше. Счётчик растёт при привязке задачи, а не после успеха upstream. Диапазон 1–100; только при `accountPoolStrategy` = `round-robin`. | | `upstreamFailoverThreshold?` | `number` | `3` | Сколько подряд transient failure допустить, прежде чем новые сессии начнут делать failover. `0` отключает эту логику. Для обычных Responses-запросов и нативных compact-отправок доказанные ошибки доступности DNS/TCP до соединения учитываются на уровне пары «провайдер, хост» и не влияют на здоровье аккаунта, кулдауны аккаунта, привязку потока/сессии, выбор активного аккаунта или маршрутизацию пула, а также не учитываются в этом пороге. | @@ -249,8 +249,7 @@ redirect'ов для обычных provider-request'ов реализована защищённом хранилище учётных данных аккаунтов Codex. Pool routing разделяет назначение новых/непривязанных задач, проактивное переключение по использованию и восстановление после сбоев. Привязанная задача обычно сохраняет affinity. По умолчанию (`pool.cacheAffinity`) эта перепривязка ждёт, пока -привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Непривязанным может стать и существующая задача -после перезапуска прокси или сброса affinity. Отказ **429/402** до вывода допускает одну попытку +привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. Отказ **429/402** до вывода допускает одну попытку на подходящем альтернативном аккаунте даже при выключенном переключении по использованию. Контекст разговора сохраняется и воспроизводится, но prompt cache провайдера между аккаунтами может не переиспользоваться и потребовать прогрева. diff --git a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md index eec9aabd8d2..1e8a5307792 100644 --- a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md @@ -243,10 +243,8 @@ yine de 0 ile çıkar. `--json` şunu döndürür: `auto` elle yapılan seçimi temizler; havuz işi yeniden kendi stratejisiyle yerleştirir — ancak id'si `auto` olan bir Codex hesabı varsa tam id eşleşmesi kazanır ve `ocx account clear ` her zaman otomatik seçimi geri yükler. Bir Codex hesabı, id yerine `ocx account alias` ile verilen takma adla da belirtilebilir; bu `priority`, `pause`, `resume`, `clear-cooldown`, `remove` ve `alias` için de geçerlidir. Codex hesaplarında `auto`, `main` ve `__main__` büyük/küçük harf fark etmeksizin ayrılmış sözcüklerdir ve takma ad olarak atanamaz. OAuth hesaplarının ve API anahtarlarının görünen adları için mevcut kurallar geçerlidir. Mevcut bir Codex hesabını, OAuth hesabını veya API anahtarını seçer. `openai` -için `main` Codex App girişini seçer. Bir Codex Havuzu seçimi süreç içi yerel -bağlılığı temizler ve mevcut görünür bir görevden gelen istekler de dahil olmak -üzere bir sonraki isteğe uygulanır; proxy yeniden başlatması veya bağlılık -çıkarma işlemi de bir görevi bağımsız bırakabilirken devam eden istekler +için `main` Codex App girişini seçer. Bir Codex Havuzu seçimi bağlılığı temizler ve mevcut görünür bir görevden gelen istekler de dahil olmak +üzere bir sonraki isteğe uygulanır. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilirken devam eden istekler yakaladıkları hesabı tutar. Bu yalnızca Havuz yönlendirmesini denetler; Direct modu arayana ait/yerel ana kimlik bilgisini kullanmaya devam eder. Kullanıma dayalı proaktif geçiş, 401/403 yeniden kimlik doğrulaması, 429/retry-after diff --git a/docs-site/src/content/docs/tr/reference/configuration/providers.md b/docs-site/src/content/docs/tr/reference/configuration/providers.md index 1e155303344..a9040c5f9da 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/tr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Arayüzde kayıt veya OAuth girişi tamamlanınca Models sayfasını açan bir b | `activeCodexAccountPinned?` | `string` | — | Operatörün en son elle seçtiği hesap kimliği. Ayarlandığı sürece, pin tükenme, hariç tutma, silme veya açık bir yük devretme/yükseltme ile serbest bırakılana kadar daha yüksek bir `codexAccountPriorities` katmanı onu öncelikleyemez. Sınırlı katman içindeki sıradan round-robin hareketi onu serbest bırakmaz. Herhangi bir `codexAccountPriorities` girdisi yazmak da pini serbest bırakır, böylece bir sıra var olmadan önce yapılan bir pin daha sonra ayarlanan bir pinin önüne geçemez. `GET /api/codex-auth/active`, hem geçerli hesabın sabitlenip sabitlenmediğini (`pinned`) hem de tavanı taşıyan hesabı (`pinnedAccountId`) bildirir. | | `autoSwitchThreshold?` | `number` | `80` | Proaktif geçiş için kullanım eşiği. `quota`, bağımsız görevlerin bir sonraki isteğini yeniden değerlendirebilir. Bağlı görevler varsayılan olarak (`pool.cacheAffinity`) eşiğin ötesinde hesabını korur; hesap tükenene veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile bağlı görevler de bu eşikte yeniden değerlendirilir. `fill-first` bunu yalnızca bağımsız atama için tükenme noktası olarak kullanır; `round-robin` bunu yalnızca yeni/bağımsız atamalarda kullanır: etkin eşiğinin altında uygun bir hesap kaldıkça eşiğine ulaşmış veya aşmış hesapları atlar ve bağlı bir görevi bu nedenle taşımaz. Puan, bilinen en sıcak 5 saatlik, haftalık veya 30 günlük kota penceresini kullanır. `0`, yalnızca kullanıma dayalı proaktif geçişi devre dışı bırakır, bağımsız atamayı veya arıza kurtarmayı devre dışı bırakmaz. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Hesap başına `autoSwitchThreshold` geçersiz kılmaları: hesap kimliği → `0`–`100` tam sayısı. Kayıt yoksa genel değer devralınır; `0` yalnızca bu hesaptan kullanıma dayalı geçişi kapatır. Ana `__main__` hesabı desteklenir. Codex Auth hesap kartından yönetilir. Hesaba özel değeri etkinleştirmek, mevcut genel eşiği sabit bir hesap değerine kopyalar. `0` dahil bu değer, genel eşik daha sonra değişse de önceliğini korur. Devre dışı bırakmak `threshold: null` gönderir, kaydı siler ve mevcut genel eşikle gelecekteki değişikliklerini devralmayı yeniden etkinleştirir. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır; görünür mevcut bir görev, proxy yeniden başlatmasından veya bağlılık sıfırlamasından sonra bağımsız hale gelebilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | | `pool.cacheAffinity?` | `boolean` | `true` | Bağlı Codex iş parçacıkları için önbellek bağlılığı sıralaması; `pool.kernel`'dan bağımsızdır. Varsayılan olarak açıktır; anahtarı atlamak veya `true` vermek bağlamayı korur ve `false` dışındaki bir değer açık okunur. Canlı bağlama kota payından öndedir: `quota`, kullanımın `autoSwitchThreshold`'u geçmesi nedeniyle iş parçacığını taşımaz, çünkü canlı bir konuşmayı taşımak hesaba özel istem önbelleğini atar. Hesap duraklatılmış, kullanılamaz, plandan dışlanmış, kimlik bilgisi geçersiz, generation değişmiş, TTL dolmuş, 429/402 kota reddi almış veya gerçekten tükenmişse (bilinen kullanım %100) iş parçacığı yine ayrılır ve yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Kullanımı bilinmeyen bir hesap bağlı bir görev için hedef olarak asla seçilmez. Tüm hesaplar eşiğin üzerindeyse bağlı görev yerinde kalır; daha iyi bir hedef yoktur. `false` eşiğe göre yeniden bağlamayı geri getirir; hedef kısıtı yine geçerlidir. Bağlılık bir sabitleme değil yeniden sıralamadır. | | `accountPoolStickyLimit?` | `number` | `1` | İlerlemeden önce bir round-robin seçiminde tutulan yeni/bağımsız görev atamaları; sayaç yukarı akış başarısından sonra değil, bir görev bağlandığında ilerler. Aralık 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Gelecekteki yeni oturumların yük devretmesinden önceki ardışık geçici arızalar. Devre dışı bırakmak için `0` ayarlayın. Düzenli Responses ve yerel sıkıştırma gönderimleri için kanıtlanmış bağlantı öncesi DNS/TCP erişilebilirlik arızaları sağlayıcı-ana bilgisayar düzeyinde izlenir: hesap sağlığını, hesap soğuma sürelerini, iş parçacığı/oturum bağlılığını, aktif hesap seçimini veya Havuz yönlendirmesini asla etkilemez ve bu eşiğe asla sayılmaz. | @@ -253,8 +253,7 @@ yönlendirmesi yeni/bağımsız atamayı, kullanıma dayalı proaktif geçişi v kurtarmayı ayırır. Bağlı bir görev normalde bağlılığı korur. Varsayılan olarak (`pool.cacheAffinity`) bu yeniden bağlama, bağlı hesap tükenene veya hizmet veremez hale gelene kadar bekler ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile `quota`, kullanım eşiği aşıldıktan sonraki isteğinde onu yeniden bağlayabilir; hedef kısıtı yine geçerlidir. 429/402 kota reddi, duraklatma, kimlik bilgisi geçersizliği ve TTL dolması bağlamayı hemen serbest bırakır; geçici arıza serisi (5xx ve diğer kota dışı arızalar) isteği başka bir hesapta sunar ama canlı bağlamayı silmez. Bağımsız bir -isteğin canlı hesap bağlaması yoktur; bu, proxy yeniden başlatmasından veya -bağlılık sıfırlamasından sonra mevcut görünür bir görevi içerebilir. Akış öncesi +isteğin canlı hesap bağlaması yoktur. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. Akış öncesi bir 429 veya 402, kullanıma dayalı proaktif geçiş kapalı olsa bile aynı istekte uygun alternatif bir hesapta bir kez yeniden dener. Hesap değişiklikleri görüşme bağlamını korur ve yeniden oynatır, ancak hesaplar arasında sağlayıcı tarafı diff --git a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md index 6c7a259ad2c..5de702e9901 100644 --- a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md @@ -200,7 +200,7 @@ OAuth 账号会显示为 `Account N`，而 plan/label 列会在 plan、屏蔽后 `auto` 会清除手动选择，让 Pool 重新按自身策略分配工作 — 但如果某个 Codex 账号的 id 恰为 `auto`，则精确 id 匹配优先；`ocx account clear ` 始终恢复自动选择。Codex 账号可以用 `ocx account alias` 设置的别名代替 id 来指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 和 `alias` 同样如此。对于 Codex 账号，`auto`、`main` 和 `__main__` 为保留字（不区分大小写），不能设为别名。OAuth 账号和 API 密钥的显示名称仍遵循原有规则。 选择已有的 Codex 账号、OAuth 账号或 API key。对 `openai` 而言，`main` 选择 Codex App 登录。 -Codex Pool 选择会清除进程本地 affinity，并从下一次请求开始生效，包括已有可见任务的请求；代理重启或 affinity eviction 后，任务也可能变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： +Codex Pool 选择会清除 affinity，并从下一次请求开始生效，包括已有可见任务的请求；符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： 遇到 **401/403** 时，App 登录会清除该账户的进程内 affinity 并要求重新认证。 遇到 **429** 时，它会遵循 `Retry-After`、启动账户 cooldown、清除 affinity， 并可将请求切换到另一个符合条件的 Pool 账户。即使 `autoSwitchThreshold: 0`， diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index 49c984a6a29..43a48aa4935 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -55,7 +55,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Codex pool 各账号的选择顺序：账号 ID → `-100` 到 `100` 的整数，**数值越大越先使用**，未设置即为 `0`。这是顺序边界而非资格边界：选择会把已经合格的账号收窄到仍有 quota 余量的最高 tier，再由 `accountPoolStrategy` 在该 tier 内挑选。只有当某个 tier 的所有成员都达到各自非零的有效阈值（账号覆盖值，未设置则使用全局值）、处于 cooldown、被 soft-avoid、已暂停或需要重新认证时，该 tier 才会被跳过；usage 未知不会让 tier 耗尽。顺序不会让不合格的账号变得可选，也不会重新绑定已经绑定账号的 thread。主账号 `__main__` 同样参与排序，因此可以让 Codex Desktop 登录账号最后才被用到。没有任何条目时，行为与以往完全一致。映射格式非法时会打印警告并关闭排序（不会触发 config 修复）。可通过 `ocx account priority` 和 Codex Auth 页面管理。 | | `autoSwitchThreshold?` | `number` | `80` | 基于用量的主动切换阈值。`quota` 可在下一次请求中重新评估未绑定任务。已绑定任务默认（`pool.cacheAffinity`）在越过阈值后仍保留账号，直到该账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号。将 `pool.cacheAffinity` 设为 `false` 才会在该阈值重新评估已绑定任务。`fill-first` 仅把它用作未绑定分配的耗尽点；`round-robin` 仅在新建/未绑定任务分配时使用它：只要仍有低于自身有效阈值的合格账号，就跳过达到或超过阈值的账号，且不会因此移动已绑定任务。分数取已知 5 小时、周或 30 天 quota window 的最高值。`0` 只关闭基于用量的主动切换，不关闭未绑定任务分配或故障恢复。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各账号对 `autoSwitchThreshold` 的覆盖：账号 ID → `0`–`100` 的整数。没有条目时继承全局值；`0` 只关闭从该账号发起的基于用量的切换。支持主账号 `__main__`。可在 Codex Auth 的账号卡片中管理。 启用覆盖时，会将当前全局阈值复制为固定的账号值。覆盖值（包括 `0`）在之后修改全局阈值时仍优先。禁用时发送 `threshold: null`，删除条目，并恢复继承当前全局阈值及其未来的更改。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；代理重启或 affinity 重置后，已有可见任务也可能未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | | `pool.cacheAffinity?` | `boolean` | `true` | 已绑定 Codex 线程的 cache-affinity 排序，独立于 `pool.kernel`。默认开启；省略该键或设为 `true` 即为开启，非法值视为开启。live 绑定优先于 quota 余量：`quota` 不会仅因用量越过 `autoSwitchThreshold` 就移动线程。账号暂停、不可用或真正耗尽（已知 usage 为 100%）时仍会离开，且只改绑到确有额度余量且 usage 严格更低的账号。用量未知的账号不会作为已绑定任务的改绑目标。设为 `false` 可恢复按阈值改绑。affinity 是重排而非钉死。 | | `accountPoolStickyLimit?` | `number` | `1` | 一次 round-robin 选择在推进前保留的新建/未绑定任务分配数。计数在任务绑定时增加，而不是在上游成功后增加。范围 1–100；仅当 `accountPoolStrategy` 为 `round-robin` 时生效。 | | `upstreamFailoverThreshold?` | `number` | `3` | 连续发生多少次瞬态故障后，后续新会话会切换到备用上游。设为 `0` 可禁用。对于常规 Responses 和原生 compact 发送，已证明的连接前 DNS/TCP 不可达故障按 provider-host 粒度记录，不影响账户健康、账户冷却、线程/会话亲和性、活动账户选择或 Pool 路由，也不会计入此阈值。 | @@ -249,7 +249,7 @@ OpenCodex 不会依据网关名称或 URL 猜测可信度。如果同一网关 请在仪表盘 **Codex Auth** 页面添加 pool account 并刷新 quota。配置只保存非 secret account metadata；access/refresh token 存放在加固的 Codex account credential store 中。Pool routing 分为新建/未绑定任务分配、基于用量的主动切换和故障恢复。已绑定任务通常保持 affinity。默认（`pool.cacheAffinity`）下，该重新绑定会等到绑定账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号；所有账号都高于阈值时，已绑定任务留在原账号。关闭该标志后，`quota` 可在超过阈值后的下一次请求中重新绑定，但仍只改绑到确有额度余量且 usage 严格更低的账号。暂停、cooldown、重新认证和故障处理也能独立清除或改变 -routing。未绑定请求没有 live 账号绑定，也可能是代理重启或 affinity 重置后的已有任务。输出前的 +routing。未绑定请求没有 live 账号绑定。符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。输出前的 **429/402** 即使在关闭基于用量的主动切换时，也可在同一请求中对合格替代账号重试一次。 账号变化后会保留并重放对话上下文，但账号间的 provider prompt cache 不保证复用，可能需要重新预热。 暂停后仍会显示账号及其 quota metadata，但不会参与自动切换、重试/failover 选择、cooldown 恢复探测或手动激活。 diff --git a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md index a743fe07a5d..9bcc719a758 100644 --- a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md @@ -158,7 +158,7 @@ Codex 池選擇套用於清除既有親和性後的下一個請求；進行中 `auto` 會清除手動選擇，讓池重新依自身策略分配工作 — 但若 Codex 帳號的 id 恰為 `auto`，則精確 id 比對優先；`ocx account clear ` 一律還原自動選擇。Codex 帳號可以用 `ocx account alias` 設定的別名代替 id 來指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 與 `alias` 亦然。對於 Codex 帳號，`auto`、`main` 和 `__main__` 為保留字（不區分大小寫），不能設為別名。OAuth 帳號與 API 金鑰的顯示名稱仍遵循原有規則。 -選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除行程本地親和性並套用於下一個請求，包含來自既有可見任務的請求；代理重啟或親和性驅逐也可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 +選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除親和性並套用於下一個請求，包含來自既有可見任務的請求；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 `--json` 回傳： diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index b3e75e1da34..33bc739cc02 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -53,7 +53,7 @@ ocx models provider openrouter on | `activeCodexAccountId?` | `string` | — | 為下一個請求手動選擇的池帳號。選擇清除執行緒親和性；進行中的請求保留擷取的憑證。 | | `autoSwitchThreshold?` | `number` | `80` | 主動切換的用量閾值。`quota` 可在下一個請求時重新評估未綁定任務。綁定任務預設（`pool.cacheAffinity`）在越過閾值後仍會保留帳號，直到該帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號。將 `pool.cacheAffinity` 設為 `false` 才會在此閾值重新評估綁定任務。`fill-first` 僅將其用作未綁定指派的排空點；`round-robin` 僅在新／未綁定工作指派時使用它：只要仍有低於自身有效閾值的合格帳號，就略過達到或超過閾值的帳號，且不會因此移動已綁定任務。分數使用最熱的已知 5h、週或 30d 配額視窗。`0` 僅停用基於用量的主動切換，而非未綁定指派或失敗復原。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各帳號對 `autoSwitchThreshold` 的覆寫：帳號 ID → `0`–`100` 的整數。沒有項目時繼承全域值；`0` 只停用從該帳號發起的使用量主動切換。支援主帳號 `__main__`。可在 Codex Auth 的帳號卡片中管理。 啟用覆寫時，會將目前全域閾值複製為固定的帳號值。覆寫值（包括 `0`）在之後修改全域閾值時仍優先。停用時傳送 `threshold: null`，刪除項目，並恢復繼承目前全域閾值及其未來的變更。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；可見的既有任務在代理重啟或親和性重置後可變為未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | | `pool.cacheAffinity?` | `boolean` | `true` | 綁定 Codex 執行緒的 cache-affinity 排序，獨立於 `pool.kernel`。預設開啟；省略該鍵或設為 `true` 即為開啟，格式錯誤視為開啟。即時綁定優先於配額餘裕：`quota` 不會只因用量越過 `autoSwitchThreshold` 就移動執行緒。帳號暫停、無法使用或真正耗盡（已知用量 100%）時仍會離開，且只改綁到確有額度餘裕且用量嚴格更低的帳號。用量未知的帳號不會作為綁定任務的改綁目標。設為 `false` 可恢復依閾值重新綁定。親和性是重排而非釘死。 | | `accountPoolStickyLimit?` | `number` | `1` | 在前進一個 round-robin 選擇前保留的新／未綁定任務指派；計數器在任務綁定時前進，而非在上游成功後。範圍 1–100。 | | `upstreamFailoverThreshold?` | `number` | `3` | 未來新 session 容錯移轉前的連續暫時性失敗。設 `0` 停用。 | @@ -186,7 +186,7 @@ API-key 供應商可持有字面值金鑰或環境參考。OAuth 供應商使用 ## Codex 帳號池 -在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；這可包含代理重啟或親和性重置後的既有可見任務。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 +在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 diff --git a/structure/INDEX.md b/structure/INDEX.md index 5e9f69f973e..b368a06639d 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -98,6 +98,7 @@ Per-vendor contracts and the adapter authority that constructs them. | [`providers/anthropic-account-pool.md`](providers/anthropic-account-pool.md) | Anthropic OAuth instances (primary and Pool 2), account pause, model routes, quota labels, and pool-bound helpers, quota, reset grants and management surfaces. | | [`providers/oauth-code-display-login.md`](providers/oauth-code-display-login.md) | Headless code-display OAuth login: flow ownership, expiry, unsupported providers, single-operator store. | | [`providers/openai-tiers.md`](providers/openai-tiers.md) | Pool/Direct account modes, API-key separation, and the public provider and quota contract. | +| [`providers/codex-pool-affinity.md`](providers/codex-pool-affinity.md) | Restart persistence of Codex pool pins and conversation-state issuers: store, fences, single-writer lease, fail-closed recovery. | | [`providers/pool-credit-renewal.md`](providers/pool-credit-renewal.md) | Independent credit-evidence renewal for opted-in exhausted Codex pool accounts. | | [`providers/openai-accounts.md`](providers/openai-accounts.md) | Migration and restore, wire identity, store concurrency, pool ordering and exclusions, quota observations, and account-bound retention. | | [`providers/codex-round-robin-placement.md`](providers/codex-round-robin-placement.md) | Round-robin new-conversation placement preferring accounts under their effective auto-switch threshold, with all-over fallback and bound, failover and lineage boundaries. | @@ -294,6 +295,8 @@ A source area can be described by more than one doc, because these docs are orga | `src/codex/request-owned-affinity.ts` | [`providers/codex-request-owned-affinity.md`](providers/codex-request-owned-affinity.md) | | `src/codex/request-owned-main-pin.ts` | [`providers/codex-request-owned-affinity.md`](providers/codex-request-owned-affinity.md) | | `src/codex/routing/selection.ts` | [`providers/codex-round-robin-placement.md`](providers/codex-round-robin-placement.md) | +| `src/codex/routing/thread-affinity-store.ts` | [`providers/codex-pool-affinity.md`](providers/codex-pool-affinity.md) | +| `src/codex/routing/thread-affinity.ts` | [`providers/codex-pool-affinity.md`](providers/codex-pool-affinity.md) | | `src/codex/runtime.ts` | [`catalog-effort-clamp.md`](catalog-effort-clamp.md) | | `src/codex/shim.ts` | [`codex-cli-update.md`](codex-cli-update.md) | | `src/codex/signed-in-identity.ts` | [`providers/openai-signed-in-preservation.md`](providers/openai-signed-in-preservation.md) | @@ -327,6 +330,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/lib/account-auto-pause-response.ts` | [`account-auto-pause.md`](account-auto-pause.md) | | `src/lib/advisor-activation.ts` | [`advisor.md`](advisor.md) | | `src/lib/bounded-body.ts` | [`transports/timeouts.md`](transports/timeouts.md) | +| `src/lib/config-ownership.ts` | [`providers/codex-pool-affinity.md`](providers/codex-pool-affinity.md) | | `src/lib/decision-quota-sink.ts` | [`providers/jev-decision.md`](providers/jev-decision.md) | | `src/lib/gui-pair-intent.ts` | [`remote-link.md`](remote-link.md) | | `src/lib/local-upstream.ts` | [`transports/timeouts.md`](transports/timeouts.md) | diff --git a/structure/manifest.json b/structure/manifest.json index 494781fd41e..741745dc1d7 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -697,6 +697,17 @@ "src/providers/codex-capacity.ts" ] }, + { + "path": "providers/codex-pool-affinity.md", + "tier": 4, + "title": "Durable Codex Pool Affinity", + "scope": "Restart persistence of Codex pool pins and conversation-state issuers: store, fences, single-writer lease, fail-closed recovery.", + "documents": [ + "src/codex/routing/thread-affinity-store.ts", + "src/codex/routing/thread-affinity.ts", + "src/lib/config-ownership.ts" + ] + }, { "path": "providers/pool-credit-renewal.md", "tier": 4, diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md new file mode 100644 index 00000000000..005ddbdacf2 --- /dev/null +++ b/structure/providers/codex-pool-affinity.md @@ -0,0 +1,15 @@ +# Durable Codex Pool Affinity + +This doc owns how Codex pool session pins and conversation-state issuers survive a proxy restart. +`src/codex/routing/thread-affinity.ts` keeps the live maps and calls the store at every bind, delete, clear and lookup. +The conversation key, cohort anchors and lineage are owned by [OpenAI Provider Account Modes](openai-tiers.md); +the single-writer lease is owned by [Responses spend](../transports/responses-spend.md). + +`src/codex/routing/thread-affinity-store.ts` restores eligible pool bindings after a graceful proxy restart for up to 24 hours idle. Session-bearing requests and session-less requests anchored on their immediate parent have derivable anchors. Deeper session-less chains, keys derived under another config home, legacy raw-parent keys and evicted anchors remain memory-only. Removed, re-logged, expired or identity-less accounts are not restored; normal selection and usability rules still apply. + +The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state, but alone never triggers the uploaded-file refusal; a known account change or prior-account move retains that refusal. + +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. Pruning removes expired or future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A config-home change retires the store scope and clears live affinity. + +`src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. + diff --git a/structure/providers/openai-tiers.md b/structure/providers/openai-tiers.md index 0eafaec4b20..724149eed58 100644 --- a/structure/providers/openai-tiers.md +++ b/structure/providers/openai-tiers.md @@ -116,7 +116,7 @@ parent derives for itself, so a chain of parent-only turns converges rather than every depth. No caller-supplied identifier reaches Pool state. Which requests bind at all is unchanged -- a bare `thread-id` with neither a session nor a parent has no family anchor and stays unbound. Components are trimmed and bounded at 512 bytes, missing or -oversized values stay unbound, raw identifiers and durable hashes are never stored, and +oversized values stay unbound, raw identifiers never enter Pool state, and account-qualified selectors skip both lookup and mutation. Selection, subagent fallback preview, and terminal outcome accounting carry the same key so route planning cannot preview one account and authenticate another. A transient-failure streak does not delete the live binding that @@ -141,7 +141,7 @@ a behaviour change, not a refinement. It also interacts with the send-budget and thread's own conversation key, its immediate parent's thread id, and the transitive root, so a grandchild resolves to the same root as its parent. Records are held per authenticated scope (an HMAC of the caller's Authorization under the same process-local key), and bounded in both -dimensions -- idle TTL and an LRU cap on records per scope, and an LRU cap on scopes. +dimensions -- idle TTL and an LRU cap on records per scope, and an LRU cap on scopes. The key stays process-local; restart persistence uses its own secret ([Durable Codex Pool Affinity](codex-pool-affinity.md)). First placement is largely subsumed by cohort keying: a member of a tree any other member has already bound resolves to that same binding, so there is nothing to place. `pickLineageServingAccount` From 1c22623c09c36ee501c396bf2dcbf7a5a5330e72 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:24:50 +0900 Subject: [PATCH 06/21] test(codex): restart matrix for durable pool affinity --- .../codex-affinity-restart.test.ts | 160 +++++++++++++++++- tests/helpers/codex-affinity-matrix-worker.ts | 125 ++++++++++++++ 2 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 tests/helpers/codex-affinity-matrix-worker.ts diff --git a/tests/codex-integration/codex-affinity-restart.test.ts b/tests/codex-integration/codex-affinity-restart.test.ts index 04234520927..a056c5204b8 100644 --- a/tests/codex-integration/codex-affinity-restart.test.ts +++ b/tests/codex-integration/codex-affinity-restart.test.ts @@ -1,9 +1,10 @@ import { expect, test } from "bun:test"; -import { mkdtempSync } from "node:fs"; +import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { removeTreeWithRetry } from "../helpers/remove-tree"; import { helperPath } from "../helpers/repo-root"; +import { Database } from "bun:sqlite"; function run(home: string, mode: string) { const result = Bun.spawnSync([process.execPath, helperPath("codex-affinity-restart-worker.ts"), mode], { @@ -70,3 +71,160 @@ test("a replacement identity invalidates the old pin and makes its issuer unknow expect(after.issuer).toBe("__unknown_codex_conversation_state_issuer__"); } finally { removeTreeWithRetry(home); } }); + +function matrix(home: string, action: string) { + const result = Bun.spawnSync([process.execPath, helperPath("codex-affinity-matrix-worker.ts"), action], { + env: { ...process.env, OPENCODEX_HOME: home, CODEX_HOME: home }, stdout: "pipe", stderr: "pipe", + }); + expect(result.exitCode, result.stderr.toString()).toBe(0); + return result.stdout.length ? JSON.parse(result.stdout.toString()) : undefined; +} + +function withSeededHome(body: (home: string) => void) { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-matrix-")); + try { + const seed = run(home, "write"); + expect(seed.exit, seed.stderr).toBe(0); + expect(JSON.parse(seed.stdout).account).toBe("a"); + body(home); + } finally { removeTreeWithRetry(home); } +} + +for (const action of ["delete", "delete-account", "clear-account", "clear"] as const) { + test(`${action} reconciles a restored pin before its first lookup`, () => withSeededHome(home => { + const result = matrix(home, action); + expect(result.before).toBeUndefined(); + expect(result.selected).toBe("b"); + expect(result.rows).toBe(0); + expect(result.issuer).toBe(action === "clear" ? undefined : "a"); + })); +} + +test("a new bind and issuer replace unadopted restored rows", () => { + withSeededHome(home => { + const result = matrix(home, "bind"); + expect(result.before).toBe("b"); + expect(result.selected).toBe("b"); + expect(result.rows).toBe(0); + }); + withSeededHome(home => expect(matrix(home, "issuer").issuer).toBe("b")); + withSeededHome(home => { + const result = matrix(home, "clear-issuer"); + expect(result.issuer).toBeUndefined(); + expect(result.issuers).toBe(0); + }); +}); + +for (const action of ["removed", "relogin", "paused", "plan", "reauth", "cooldown"] as const) { + test(`${action} makes the resolver leave a restored account`, () => withSeededHome(home => { + const result = matrix(home, action); + expect(result.selected).toBe("b"); + expect(result.rows).toBe(0); + })); +} + +test("transient failure detours a restored pin while retaining its owner", () => withSeededHome(home => { + const result = matrix(home, "transient"); + expect(result.before).toBe("a"); + expect(result.selected).toBe("b"); + expect(result.after).toBe("a"); +})); + +test("a legacy account without an identity falls back and removes its stored row", () => withSeededHome(home => { + const file = join(home, "codex-accounts.json"); + const records = JSON.parse(readFileSync(file, "utf8")); + delete records.a.quotaHistoryIdentity; + writeFileSync(file, JSON.stringify(records)); + const result = matrix(home, "read"); + expect(result.before).toBeUndefined(); + expect(result.selected).toBe("b"); + expect(result.rows).toBe(0); +})); + +for (const [label, timestamp] of [["expired", Date.now() - 25 * 60 * 60_000], ["future", Date.now() + 60_000]] as const) { + test(`${label} restored pin falls back and is deleted`, () => withSeededHome(home => { + const db = new Database(join(home, "codex-affinity.sqlite")); + db.query("UPDATE threads SET last_used_at=?").run(timestamp); + db.close(); + const result = matrix(home, "read"); + expect(result.before).toBeUndefined(); + expect(result.selected).toBe("b"); + expect(result.rows).toBe(0); + })); +} + +test("main account bindings and issuers do not survive release", () => withSeededHome(home => { + matrix(home, "write-main"); + const result = matrix(home, "read"); + expect(result.before).toBeUndefined(); + expect(result.issuer).toBeUndefined(); +})); + +test("session-less depth one restores while depth two places independently", () => { + for (const [action, expected] of [["depth-one", "a"], ["depth-two", "b"]] as const) withSeededHome(home => { + matrix(home, "depth-write"); + expect(matrix(home, action).selected).toBe(expected); + }); +}); + +test("passive affinity operations leave a populated database untouched under a lease", () => withSeededHome(home => { + expect(matrix(home, "passive").unchanged).toBe(true); + expect(matrix(home, "read").before).toBe("a"); +})); + +test.skipIf(process.platform !== "win32")("Windows releases affinity handles before directory removal", () => { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-windows-")); + const first = run(home, "write"); + expect(first.exit, first.stderr).toBe(0); + removeTreeWithRetry(home); +}); + +for (const action of ["race-relogin", "race-delete-add"] as const) { + test(`${action} invalidates a binding written before the identity changed`, () => withSeededHome(home => { + matrix(home, action); + const result = matrix(home, "read"); + expect(result.before).toBeUndefined(); + expect(result.selected).toBe("b"); + })); +} + +for (const action of ["race-refresh", "race-external"] as const) { + test(`${action} keeps a binding when the account identity is unchanged`, () => withSeededHome(home => { + matrix(home, action); + expect(matrix(home, "read").before).toBe("a"); + })); +} + +test("issuer completion with a superseded credential restores as unknown", () => withSeededHome(home => { + matrix(home, "race-issuer"); + expect(matrix(home, "read").issuer).toBe("__unknown_codex_conversation_state_issuer__"); +})); + +test("only the last lease flushes and a token from the prior lease is refused", () => withSeededHome(home => { + // Remove the seeded binding through the public API before testing a pending bind. + matrix(home, "clear"); + const db = new Database(join(home, "codex-affinity.sqlite")); + db.exec("DELETE FROM threads"); + db.close(); + expect(matrix(home, "lease")).toEqual({ before: 0, rejected: true }); + expect(matrix(home, "read").before).toBe("a"); +})); + +test("resolver reuse persists its updated last-used time on release", () => withSeededHome(home => { + const db = new Database(join(home, "codex-affinity.sqlite")); + db.query("UPDATE threads SET last_used_at=?").run(Date.now() - 10_000); + db.close(); + const result = matrix(home, "touch"); + expect(result.row.timestamp).toBe(result.timestamp); +})); + +for (const [label, timestamp] of [["expired", Date.now() - 25 * 60 * 60_000], ["future", Date.now() + 60_000]] as const) { + test(`${label} issuer is removed when read`, () => withSeededHome(home => { + const db = new Database(join(home, "codex-affinity.sqlite")); + db.query("UPDATE issuers SET last_used_at=?").run(timestamp); + db.close(); + const result = matrix(home, "read"); + expect(result.issuer).toBeUndefined(); + expect(result.issuers).toBe(0); + })); +} diff --git a/tests/helpers/codex-affinity-matrix-worker.ts b/tests/helpers/codex-affinity-matrix-worker.ts new file mode 100644 index 00000000000..1f1ccb18c63 --- /dev/null +++ b/tests/helpers/codex-affinity-matrix-worker.ts @@ -0,0 +1,125 @@ +import { Database } from "bun:sqlite"; +import { join } from "node:path"; +import { readFileSync } from "node:fs"; +import { acquireSpendLedgerOwner, assertStorageOwned, mintSpendLedgerStorage } from "../../src/lib/spend-ledger-owner"; +import { readCodexAccountRecord, removeCodexAccountCredential, saveCodexAccountCredential, saveCodexAccountCredentialIfGeneration } from "../../src/codex/account-store"; +import { codexPoolAffinityKey } from "../../src/codex/auth-context"; +import { recordCodexThreadLineage } from "../../src/codex/lineage"; +import { runPassiveRead } from "../../src/lib/passive-read"; +import { recordCodexUpstreamOutcome, resolveCodexAccountForThread } from "../../src/codex/routing"; +import { bindThreadAffinity, clearConversationStateIssuerMap, clearThreadAccountMap, clearThreadAccountMapForAccount, deleteThreadAffinity, deleteThreadAffinitiesForAccount, getThreadAffinity, handOffThreadAffinityGeneration, peekConversationStateIssuer, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; +import type { OcxConfig } from "../../src/types"; + +const action = process.argv[2]!; +const lease = acquireSpendLedgerOwner(); +const config = { providers: {}, codexAccounts: ["a", "b"].map(id => ({ id, email: `${id}@test`, isMain: false })), activeCodexAccountId: "a", accountPoolStrategy: "round-robin", autoSwitchThreshold: 80, upstreamFailoverThreshold: 3 } as OcxConfig; +if (action === "passive") { + const file = join(process.env.OPENCODEX_HOME!, "codex-affinity.sqlite"); + const before = readFileSync(file); + await runPassiveRead(async () => { + const passiveKey = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; + getThreadAffinity(passiveKey); + peekConversationStateIssuer(passiveKey); + clearThreadAccountMap(); + }); + lease.release(); + console.log(JSON.stringify({ unchanged: before.equals(readFileSync(file)) })); + process.exit(0); +} +const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; +const cred = (id: string) => ({ accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `acct-${id}` }); + +if (action.startsWith("depth-")) { + const child = new Headers({ "thread-id": "child", "x-codex-parent-thread-id": "root" }); + const grandchild = new Headers({ "thread-id": "grandchild", "x-codex-parent-thread-id": "child" }); + if (action === "depth-write") { + const lineage = recordCodexThreadLineage(child)!; + bindThreadAffinity(lineage.conversationKey, "a", Date.now()); + if (recordCodexThreadLineage(grandchild)!.conversationKey !== lineage.conversationKey) throw new Error("lineage did not converge"); + lease.release(); + } else { + resolveCodexAccountForThread("advance-cursor", config); + const headers = action === "depth-one" ? child : grandchild; + const selected = resolveCodexAccountForThread(codexPoolAffinityKey(headers)!, config); + lease.release(); + console.log(JSON.stringify({ selected })); + } + process.exit(0); +} else if (action.startsWith("race-")) { + const record = readCodexAccountRecord("a")!; + bindThreadAffinity(key, "a", Date.now()); + if (action === "race-relogin" || action === "race-issuer") saveCodexAccountCredential("a", cred("a")); + if (action === "race-delete-add") { + removeCodexAccountCredential("a"); + saveCodexAccountCredential("a", cred("a")); + } + if (action === "race-refresh" || action === "race-external") { + if (!saveCodexAccountCredentialIfGeneration("a", record.generation, { ...cred("a"), accessToken: "rotated" })) throw new Error("refresh failed"); + if (action === "race-refresh" && !handOffThreadAffinityGeneration("a", record.generation, record.generation + 1)) throw new Error("handoff failed"); + } + if (action === "race-issuer") rememberConversationStateIssuer(key, "a", undefined, { generation: record.generation, chatgptAccountId: record.credential!.chatgptAccountId }); +} else if (action === "write-main") { + bindThreadAffinity(key, "__main__", Date.now()); + rememberConversationStateIssuer(key, "__main__"); +} else if (action === "lease") { + const second = acquireSpendLedgerOwner(); + const token = mintSpendLedgerStorage("codex-affinity.sqlite"); + bindThreadAffinity(key, "a", Date.now()); + lease.release(); + const db = new Database(join(process.env.OPENCODEX_HOME!, "codex-affinity.sqlite")); + const before = db.query("SELECT count(*) AS n FROM threads").get() as { n: number }; + db.close(); + second.release(); + const again = acquireSpendLedgerOwner(); + let rejected = false; + try { assertStorageOwned(token); } catch { rejected = true; } + again.release(); + console.log(JSON.stringify({ before: before.n, rejected })); + process.exit(0); +} else { + if (action === "removed") removeCodexAccountCredential("a"); + if (action === "relogin") saveCodexAccountCredential("a", cred("a")); + if (action === "paused") config.pausedCodexAccountIds = ["a"]; + if (action === "plan") { + config.codexAccounts = [{ id: "a", email: "a@test", isMain: false, plan: "free" }, { id: "b", email: "b@test", isMain: false, plan: "plus" }]; + config.codexPool = { excludedPlans: ["free"] }; + } + if (action === "reauth" || action === "cooldown" || action === "transient") { + if (getThreadAffinity(key)?.accountId !== "a") throw new Error("expected restored affinity"); + const status = action === "reauth" ? 401 : action === "cooldown" ? 429 : 503; + for (let i = 0; i < (action === "transient" ? 3 : 1); i++) recordCodexUpstreamOutcome(config, "a", status, { now: Date.now() }); + } + if (action === "bind") bindThreadAffinity(key, "b", Date.now()); + if (action === "delete") deleteThreadAffinity(key); + if (action === "delete-account") deleteThreadAffinitiesForAccount(key, "a"); + if (action === "clear-account") clearThreadAccountMapForAccount("a"); + if (action === "clear") clearThreadAccountMap(); + if (action === "clear-issuer") clearConversationStateIssuerMap(); + if (action === "issuer") { + const record = readCodexAccountRecord("b")!; + rememberConversationStateIssuer(key, "b", undefined, { generation: record.generation, chatgptAccountId: record.credential!.chatgptAccountId }); + } + if (action === "touch") { + const timestamp = Date.now(); + resolveCodexAccountForThread(key, config, timestamp); + lease.release(); + const db = new Database(join(process.env.OPENCODEX_HOME!, "codex-affinity.sqlite")); + const row = db.query("SELECT last_used_at AS timestamp FROM threads").get(); + db.close(); + console.log(JSON.stringify({ timestamp, row })); + process.exit(0); + } + const before = getThreadAffinity(key)?.accountId; + const issuer = peekConversationStateIssuer(key); + resolveCodexAccountForThread("advance-cursor", config); + const selected = resolveCodexAccountForThread(key, config); + const after = getThreadAffinity(key)?.accountId; + lease.release(); + const db = new Database(join(process.env.OPENCODEX_HOME!, "codex-affinity.sqlite")); + const rows = db.query("SELECT account_id FROM threads WHERE account_id='a'").all().length; + const issuers = db.query("SELECT count(*) AS n FROM issuers").get() as { n: number }; + db.close(); + console.log(JSON.stringify({ before, issuer, selected, after, rows, issuers: issuers.n })); + process.exit(0); +} +lease.release(); From 523ea3793e7b0ad602c9b5edc8b49d573340845a Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:29:15 +0900 Subject: [PATCH 07/21] test(codex): remaining restart matrix cases (compact turn-state case currently red) --- scripts/test-layout/layout.json | 2 +- .../codex-affinity-restart-matrix.test.ts | 152 ++++++++++++++++++ tests/fixtures/test-layout-expected.json | 2 +- .../helpers/codex-affinity-handler-worker.ts | 27 ++++ .../codex-affinity-home-switch-worker.ts | 40 +++++ tests/helpers/codex-affinity-matrix-worker.ts | 29 +++- 6 files changed, 248 insertions(+), 4 deletions(-) create mode 100644 tests/codex-integration/codex-affinity-restart-matrix.test.ts create mode 100644 tests/helpers/codex-affinity-handler-worker.ts create mode 100644 tests/helpers/codex-affinity-home-switch-worker.ts diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 05c6820f89d..fa8a4f43414 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -706,7 +706,7 @@ "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts new file mode 100644 index 00000000000..f5c3282a7d1 --- /dev/null +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -0,0 +1,152 @@ +import { expect, test } from "bun:test"; +import { Database } from "bun:sqlite"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { initializeConfigOwnership, removeOwnedConfigState } from "../../src/lib/config-ownership"; +import { helperPath } from "../helpers/repo-root"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +const names = ["codex-affinity.sqlite", "codex-affinity.sqlite-journal", "codex-affinity.sqlite-wal", "codex-affinity.sqlite-shm"]; +function worker(home: string, file: string, action: string): any { + const result = Bun.spawnSync([process.execPath, helperPath(file), action], { env: { ...process.env, OPENCODEX_HOME: home, CODEX_HOME: home }, stdout: "pipe", stderr: "pipe" }); + expect(result.exitCode, result.stderr.toString()).toBe(0); + return result.stdout.length ? JSON.parse(result.stdout.toString()) : undefined; +} +function seed(home: string) { return worker(home, "codex-affinity-restart-worker.ts", "write"); } +function matrix(home: string, action: string) { return worker(home, "codex-affinity-matrix-worker.ts", action); } +function inHome(fn: (home: string) => void) { + const home = mkdtempSync(join(tmpdir(), "ocx-affinity-extra-")); + try { fn(home); } finally { if (existsSync(home)) removeTreeWithRetry(home); } +} + +test("a restored shared binding survives a model eligibility detour", () => inHome(home => { + expect(seed(home).account).toBe("a"); + matrix(home, "shared-write"); + const result = matrix(home, "model-excluded"); + expect(result.before).toBe("a"); + expect(result.result).toMatchObject({ status: "selected", accountId: "b" }); + expect(result.shared).toBe("a"); +})); + +test("fresh and existing owned homes list and remove all affinity paths", () => inHome(home => { + expect(initializeConfigOwnership(home)).toBe(true); + let manifest = JSON.parse(readFileSync(join(home, ".opencodex-uninstall.json"), "utf8")); + for (const name of names) expect(manifest.paths).toContain(name); + // Simulate an older owned home that predates these paths. + manifest.paths = manifest.paths.filter((path: string) => !names.includes(path)); + writeFileSync(join(home, ".opencodex-uninstall.json"), JSON.stringify(manifest)); + matrix(home, "probe"); + manifest = JSON.parse(readFileSync(join(home, ".opencodex-uninstall.json"), "utf8")); + for (const name of names) expect(manifest.paths).toContain(name); + for (const name of names.slice(1)) writeFileSync(join(home, name), "sidecar"); + expect(removeOwnedConfigState(home).status).toBe("removed"); + for (const name of names) expect(existsSync(join(home, name))).toBe(false); +})); + +for (const kind of ["symlink", "directory"] as const) { + test(`${kind} at the database path disables the store`, () => inHome(home => { + expect(initializeConfigOwnership(home)).toBe(true); + const target = join(home, "outside-target"); + if (kind === "symlink") symlinkSync(target, join(home, names[0]!)); + else mkdirSync(join(home, names[0]!)); + matrix(home, "probe"); + expect(existsSync(target)).toBe(false); + })); +} + +test("an unowned nonempty home cannot create an affinity database", () => inHome(home => { + writeFileSync(join(home, "unowned"), "keep"); + matrix(home, "probe"); + expect(existsSync(join(home, names[0]!))).toBe(false); +})); + +for (const kind of ["extra-table", "wrong-column"] as const) { + test(`v1 ${kind} is rejected without changing schema, metadata, or rows`, () => inHome(home => { + seed(home); + const file = join(home, names[0]!); + let db = new Database(file); + if (kind === "extra-table") db.exec("CREATE TABLE extra(value TEXT)"); + else db.exec("ALTER TABLE threads RENAME COLUMN account_id TO wrong_account"); + db.close(); + const before = readFileSync(file); + matrix(home, "probe"); + expect(readFileSync(file).equals(before)).toBe(true); + })); +} + +test("pruning keeps newest issuers and prioritizes ordinary bindings over detours", () => inHome(home => { + seed(home); + const file = join(home, names[0]!); + let db = new Database(file); + const now = Date.now() - 5000; + db.exec("BEGIN"); + const insertThread = db.query("INSERT INTO threads VALUES (?, ?, 'a', 'fence', ?, ?)"); + const insertIssuer = db.query("INSERT INTO issuers VALUES (?, 'a', 'fence', ?)"); + for (let i = 0; i < 2052; i++) { + const key = `bulk-${String(i).padStart(4, "0")}`; + insertThread.run(key, i < 4 ? "model-detour:shared:gpt-test" : "shared", now, now + i); + insertIssuer.run(key, now + i); + } + db.exec("COMMIT"); db.close(); + matrix(home, "probe"); + db = new Database(file); + expect(db.query("SELECT count(*) AS n FROM threads").get()).toEqual({ n: 2048 }); + expect(db.query("SELECT count(*) AS n FROM issuers").get()).toEqual({ n: 2048 }); + expect(db.query("SELECT count(*) AS n FROM threads WHERE scope LIKE 'model-detour:%'").get()).toEqual({ n: 0 }); + expect(db.query("SELECT count(*) AS n FROM issuers WHERE row_key LIKE 'bulk-%' AND row_key < 'bulk-0004'").get()).toEqual({ n: 0 }); + db.close(); +})); + +test("restoration accepts a 512-byte model component and rejects 513 bytes", () => inHome(home => { + seed(home); + const file = join(home, names[0]!); + let db = new Database(file); + const row = db.query("SELECT row_key, account_id, fence, created_at, last_used_at FROM threads LIMIT 1").get() as { row_key: string; account_id: string; fence: string; created_at: number; last_used_at: number }; + const insert = db.query("INSERT INTO threads VALUES (?, ?, ?, ?, ?, ?)"); + const valid = `model-detour:shared:${"é".repeat(256)}`; + const invalid = `${valid}x`; + insert.run(row.row_key, valid, row.account_id, row.fence, row.created_at, row.last_used_at); + insert.run(row.row_key, invalid, row.account_id, row.fence, row.created_at, row.last_used_at); + db.close(); + const result = matrix(home, "scopes"); + expect(result.scopes).toContain(valid); + expect(result.scopes).not.toContain(invalid); + db = new Database(file); + expect(db.query("SELECT count(*) AS n FROM threads WHERE scope=?").get(invalid)).toEqual({ n: 0 }); + db.close(); +})); + +test("home switching drops pending writes and does not carry recorded anchors across homes", () => inHome(homeA => inHome(homeB => { + seed(homeA); seed(homeB); + for (const home of [homeA, homeB]) { + const db = new Database(join(home, names[0]!)); + db.exec("DELETE FROM threads; DELETE FROM issuers"); + db.close(); + } + const result = Bun.spawnSync([process.execPath, helperPath("codex-affinity-home-switch-worker.ts"), homeB], { + env: { ...process.env, OPENCODEX_HOME: homeA, CODEX_HOME: homeA }, stdout: "pipe", stderr: "pipe", + }); + expect(result.exitCode, result.stderr.toString()).toBe(0); + expect(JSON.parse(result.stdout.toString())).toEqual({ first: "a", rowsB: 1, restoredB: "a" }); +}))); + +for (const path of ["responses", "compact"] as const) { + for (const issuer of ["unknown", "known"] as const) { + test(`${path} handler ${issuer} restored issuer with an uploaded file`, () => inHome(home => { + worker(home, "codex-affinity-restart-worker.ts", issuer === "unknown" ? "unknown-write" : "write"); + const result = worker(home, "codex-affinity-handler-worker.ts", path); + if (issuer === "known") { + expect(result.status).toBe(400); + expect(result.bodies).toHaveLength(0); + } else { + expect(result.status, result.responseBody).toBe(200); + expect(result.bodies).toHaveLength(1); + expect(result.bodies[0].previous_response_id).toBeUndefined(); + expect(result.turnStates).toEqual([null]); + expect(result.bodies[0].client_metadata?.["x-codex-turn-state"]).toBeUndefined(); + expect(JSON.stringify(result.bodies[0].input)).toContain("file-test"); + } + })); + } +} diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index f5ed42e01fa..c1a4d9c35cd 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -698,7 +698,7 @@ "codex-restart-contract-parity.test.ts": "codex-integration", "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", "codex-service-manager-probe.test.ts": "codex-integration", "codex-systemd-env-probe.test.ts": "codex-integration", diff --git a/tests/helpers/codex-affinity-handler-worker.ts b/tests/helpers/codex-affinity-handler-worker.ts new file mode 100644 index 00000000000..af1ad97b2cb --- /dev/null +++ b/tests/helpers/codex-affinity-handler-worker.ts @@ -0,0 +1,27 @@ +import { acquireSpendLedgerOwner } from "../../src/lib/spend-ledger-owner"; +import { handleResponses, handleResponsesCompact } from "../../src/server/responses"; +import { compactionRequest, completedPayload, jsonResponse, sseResponse, twoAccountPoolConfig } from "./compaction-routing-fixtures"; + +const lease = acquireSpendLedgerOwner(); +const config = twoAccountPoolConfig(); +config.codexAccounts = ["a", "b"].map(id => ({ id, email: `${id}@test`, isMain: false, chatgptAccountId: `acct-${id}` })); +config.activeCodexAccountId = "a"; +config.codexAccountNamespaces = { side: "b" }; +const bodies: Array> = []; +const turnStates: Array = []; +const mock = (async (input: string | URL | Request, init?: RequestInit) => { + const request = new Request(input, init); + turnStates.push(request.headers.get("x-codex-turn-state")); + bodies.push(await request.json() as Record); + return request.url.endsWith("/compact") ? jsonResponse({ output: [] }) : sseResponse([{ type: "response.completed", response: completedPayload("ok") }]); +}) as typeof fetch; +globalThis.fetch = mock; +for (const provider of Object.values(config.providers)) Object.assign(provider, { fetch: mock, upstreamWebsocket: false }); +const body = { model: "side/gpt-5.5", stream: false, client_metadata: { "x-codex-turn-state": "old-state" }, input: [{ type: "message", role: "user", content: [{ type: "input_file", file_id: "file-test" }] }] }; +const request = compactionRequest(body, undefined, { "session-id": "restart-session", "thread-id": "restart-thread", "x-codex-turn-state": "old-state" }); +const response = process.argv[2] === "compact" + ? await handleResponsesCompact(request, config, { model: "", provider: "" }) + : await handleResponses(request, config, { model: "", provider: "" }); +const responseBody = await response.text(); +lease.release(); +console.log(JSON.stringify({ status: response.status, bodies, responseBody, turnStates })); diff --git a/tests/helpers/codex-affinity-home-switch-worker.ts b/tests/helpers/codex-affinity-home-switch-worker.ts new file mode 100644 index 00000000000..5f95f1cd7bc --- /dev/null +++ b/tests/helpers/codex-affinity-home-switch-worker.ts @@ -0,0 +1,40 @@ +import { Database } from "bun:sqlite"; +import { join } from "node:path"; +import { codexConversationKeyFor, recordCodexThreadLineage } from "../../src/codex/lineage"; +import { acquireSpendLedgerOwner } from "../../src/lib/spend-ledger-owner"; +import { bindThreadAffinity, getThreadAffinity } from "../../src/codex/routing/thread-affinity"; +import { resolveCodexAccountForThread } from "../../src/codex/routing"; +import type { OcxConfig } from "../../src/types"; + +const homeA = process.env.OPENCODEX_HOME!; +const homeB = process.argv[2]!; +const config = { providers: {}, codexAccounts: ["a", "b"].map(id => ({ id, email: `${id}@test`, isMain: false })), activeCodexAccountId: "a", accountPoolStrategy: "round-robin" } as OcxConfig; +let lease = acquireSpendLedgerOwner(); +const oldHeaders = new Headers({ "thread-id": "old-thread", "x-codex-parent-thread-id": "old-family" }); +const oldKey = recordCodexThreadLineage(oldHeaders)!.conversationKey; +bindThreadAffinity(oldKey, "a", Date.now()); // deliberately unflushed +process.env.OPENCODEX_HOME = homeB; +const freshKey = codexConversationKeyFor("new-family", "new-thread"); // retires A while its write is pending +lease.release(); +lease = acquireSpendLedgerOwner(); +// Derive again under B's owner, then route B's first request. +codexConversationKeyFor("new-family", "new-thread"); +const first = resolveCodexAccountForThread(freshKey, config); +bindThreadAffinity(oldKey, "b", Date.now()); // no anchor for A's key in B +lease.release(); +const dbB = new Database(join(homeB, "codex-affinity.sqlite")); +const rowsB = (dbB.query("SELECT count(*) AS n FROM threads").get() as { n: number }).n; +dbB.close(); +process.env.OPENCODEX_HOME = homeA; +codexConversationKeyFor("switch", "switch"); +process.env.OPENCODEX_HOME = homeB; +lease = acquireSpendLedgerOwner(); +const restoredB = getThreadAffinity(codexConversationKeyFor("new-family", "new-thread"))?.accountId; +const oldInB = getThreadAffinity(recordCodexThreadLineage(oldHeaders)!.conversationKey)?.accountId; +lease.release(); +process.env.OPENCODEX_HOME = homeA; +lease = acquireSpendLedgerOwner(); +const oldInA = getThreadAffinity(recordCodexThreadLineage(oldHeaders)!.conversationKey)?.accountId; +const newInA = getThreadAffinity(codexConversationKeyFor("new-family", "new-thread"))?.accountId; +lease.release(); +console.log(JSON.stringify({ first, rowsB, restoredB, oldInB, oldInA, newInA })); diff --git a/tests/helpers/codex-affinity-matrix-worker.ts b/tests/helpers/codex-affinity-matrix-worker.ts index 1f1ccb18c63..fe9a8507bc9 100644 --- a/tests/helpers/codex-affinity-matrix-worker.ts +++ b/tests/helpers/codex-affinity-matrix-worker.ts @@ -6,7 +6,8 @@ import { readCodexAccountRecord, removeCodexAccountCredential, saveCodexAccountC import { codexPoolAffinityKey } from "../../src/codex/auth-context"; import { recordCodexThreadLineage } from "../../src/codex/lineage"; import { runPassiveRead } from "../../src/lib/passive-read"; -import { recordCodexUpstreamOutcome, resolveCodexAccountForThread } from "../../src/codex/routing"; +import { adoptStoredThreadRows } from "../../src/codex/routing/thread-affinity-store"; +import { recordCodexUpstreamOutcome, resolveCodexAccountForThread, resolveCodexAccountForThreadDetailed } from "../../src/codex/routing"; import { bindThreadAffinity, clearConversationStateIssuerMap, clearThreadAccountMap, clearThreadAccountMapForAccount, deleteThreadAffinity, deleteThreadAffinitiesForAccount, getThreadAffinity, handOffThreadAffinityGeneration, peekConversationStateIssuer, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; import type { OcxConfig } from "../../src/types"; @@ -29,6 +30,23 @@ if (action === "passive") { const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; const cred = (id: string) => ({ accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `acct-${id}` }); +if (action === "probe") { + getThreadAffinity(key); + lease.release(); + process.exit(0); +} +if (action === "shared-write") { + bindThreadAffinity(key, "a", Date.now(), "shared"); + lease.release(); + process.exit(0); +} +if (action === "scopes") { + const scopes = adoptStoredThreadRows(key, Date.now()).map(row => row.scope); + lease.release(); + console.log(JSON.stringify({ scopes })); + process.exit(0); +} + if (action.startsWith("depth-")) { const child = new Headers({ "thread-id": "child", "x-codex-parent-thread-id": "root" }); const grandchild = new Headers({ "thread-id": "grandchild", "x-codex-parent-thread-id": "child" }); @@ -109,9 +127,16 @@ if (action.startsWith("depth-")) { console.log(JSON.stringify({ timestamp, row })); process.exit(0); } - const before = getThreadAffinity(key)?.accountId; + const before = getThreadAffinity(key, action === "model-excluded" ? "shared" : undefined)?.accountId; const issuer = peekConversationStateIssuer(key); resolveCodexAccountForThread("advance-cursor", config); + if (action === "model-excluded") { + const result = resolveCodexAccountForThreadDetailed(key, config, Date.now(), "shared", { modelEligibleAccountIds: new Set(["b"]) }, "gpt-test"); + const shared = resolveCodexAccountForThread(key, config, Date.now(), "shared"); + lease.release(); + console.log(JSON.stringify({ before, result, shared })); + process.exit(0); + } const selected = resolveCodexAccountForThread(key, config); const after = getThreadAffinity(key)?.accountId; lease.release(); From 70a5a7b9cddd237c9522af51b9af21a78199eadb Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:33:44 +0900 Subject: [PATCH 08/21] fix(codex): keep row keys for persisted deletes, durable touches for unknown issuers, compact turn-state scrub --- src/codex/routing/thread-affinity-store.ts | 52 ++++++++++++++++--- src/codex/routing/thread-affinity.ts | 18 +++++-- src/server/responses/compact.ts | 16 ++++-- .../codex-affinity-restart-matrix.test.ts | 6 +++ .../codex-affinity-store.test.ts | 35 +++++++++++-- .../helpers/codex-affinity-restart-worker.ts | 10 ++++ 6 files changed, 116 insertions(+), 21 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index c87f0aecb0f..aae19e2bd8d 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -72,6 +72,8 @@ type Scope = { threads: Map; issuers: Map; anchors: Map; + threadKeys: Map }>; + issuerKeys: Map; consulted: Set; timer?: ReturnType; scheduled: boolean; @@ -278,6 +280,8 @@ function scope(): Scope | undefined { threads: new Map(), issuers: new Map(), anchors: new Map(), + threadKeys: new Map(), + issuerKeys: new Map(), consulted: new Set(), scheduled: false, }; @@ -311,7 +315,7 @@ function rowKey( const a = s.anchors.get(key); if (!a || Date.now() - a.at > TTL) { s.anchors.delete(key); - return undefined; + return kind === "conversation" ? s.threadKeys.get(key)?.rowKey : s.issuerKeys.get(key); } return createHmac("sha256", s.secret!) .update(`opencodex/codex-affinity/${kind}/v1\0`) @@ -336,12 +340,24 @@ export function codexAffinityIdentityFence( .update(record.quotaHistoryIdentity) .digest("hex"); } +function rememberThreadKey(s: Scope, key: string, rk: string, scopeName: string, accountId: string): void { + let entry = s.threadKeys.get(key); + if (!entry) s.threadKeys.set(key, entry = { rowKey: rk, scopes: new Map() }); + entry.scopes.set(scopeName, accountId); + if (s.threadKeys.size + s.issuerKeys.size > 8192) poison(s); +} +function rememberIssuerKey(s: Scope, key: string, rk: string): void { + s.issuerKeys.set(key, rk); + if (s.threadKeys.size + s.issuerKeys.size > 8192) poison(s); +} /** A poisoned scope cannot safely declare itself clean on release. */ function poison(s: Scope): void { s.poisoned = true; s.log = []; s.threads.clear(); s.issuers.clear(); + s.threadKeys.clear(); + s.issuerKeys.clear(); if (s.timer) clearTimeout(s.timer); warning(); } @@ -512,6 +528,8 @@ export function adoptStoredThreadRows( }); continue; } + rememberThreadKey(s, key, rk, r.scope, r.account_id); + if (s.poisoned) return []; result.push({ scope: r.scope, accountId: r.account_id, @@ -530,7 +548,7 @@ export function adoptStoredThreadRows( export function adoptStoredIssuer( key: string, now: number, -): string | undefined { +): { issuer: string; storedAccountId: string; storedFence: string | null } | undefined { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); @@ -566,8 +584,10 @@ export function adoptStoredIssuer( r.fence && codexAffinityIdentityFence(r.account_id) === r.fence ? r.account_id : CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER; + rememberIssuerKey(s, key, rk); + if (s.poisoned) return; consulted(s, `i:${rk}`); - return result; + return { issuer: result, storedAccountId: r.account_id, storedFence: r.fence }; } catch { warning(); return; @@ -582,6 +602,8 @@ export function recordThreadBind( if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; + rememberThreadKey(s, key, rk, scopeName, row.accountId); + if (s.poisoned) return; enqueue(s, { kind: "bind", key: rk, @@ -610,6 +632,9 @@ export function recordThreadDelete(key: string, scopeName?: string): void { if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; + const known = s.threadKeys.get(key); + if (scopeName === undefined) s.threadKeys.delete(key); + else if (known) { known.scopes.delete(scopeName); if (!known.scopes.size) s.threadKeys.delete(key); } enqueue( s, scopeName === undefined @@ -630,21 +655,28 @@ export function recordThreadDelete(key: string, scopeName?: string): void { } export function recordAccountPinsCleared(id: string): void { const s = usable(); - if (s) + if (s) { + for (const [key, entry] of s.threadKeys) { + for (const [scopeName, account] of entry.scopes) if (account === id) entry.scopes.delete(scopeName); + if (!entry.scopes.size) s.threadKeys.delete(key); + } enqueue(s, { kind: "deleteAccount", sql: "DELETE FROM threads WHERE account_id=?", args: [id], }); + } } export function recordAllCleared(): void { const s = usable(); - if (s) + if (s) { + s.threadKeys.clear(); s.issuerKeys.clear(); enqueue(s, { kind: "clearAll", sql: "DELETE FROM threads; DELETE FROM issuers", args: [], }); + } } export function recordIssuerSet( key: string, @@ -656,6 +688,8 @@ export function recordIssuerSet( if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; + rememberIssuerKey(s, key, rk); + if (s.poisoned) return; enqueue(s, { kind: "issuerSet", key: rk, @@ -675,18 +709,22 @@ export function recordIssuerDelete(key: string): void { const s = usable(); if (!s) return; const rk = rowKey(s, key, "issuer"); - if (rk) + if (rk) { + s.issuerKeys.delete(key); enqueue(s, { kind: "issuerDelete", key: rk, sql: "DELETE FROM issuers WHERE row_key=?", args: [rk], }); + } } export function recordIssuersCleared(): void { const s = usable(); - if (s) + if (s) { + s.issuerKeys.clear(); enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); + } } export function flushCodexAffinityStoreForTests(): void { if (current && !isPassiveRead()) flush(current, false); diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index c22e1463a9b..faf10fcf62b 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -7,7 +7,7 @@ import type { CodexQuotaScope } from "./health-store"; import type { TransientProbeLease } from "../../routing/probe-lease"; import { clearPoolRecoveryState } from "../../routing/probe-lease"; import { createHash } from "node:crypto"; -import { adoptStoredThreadRows, adoptStoredIssuer, codexAffinityIdentityFence, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; +import { adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; export { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "./thread-affinity-store"; export type ThreadAffinityEntry = { @@ -210,6 +210,8 @@ type ConversationStateIssuerEntry = { accountId: string; lastUsedAt: number; fence?: string; + storedAccountId?: string; + storedFence?: string | null; }; const conversationStateIssuerMap = new Map(); @@ -217,11 +219,17 @@ function identityFence(identity: string | undefined): string | undefined { return identity ? createHash("sha256").update("opencodex/codex-affinity/identity/v1\0").update(identity).digest("hex") : undefined; } +export function registerThreadAffinityLiveState(): void { registerCodexAffinityLiveState({ thread: (key, scope) => threadAccountMap.get(key)?.get(scope as ThreadAffinityScope), - issuer: key => conversationStateIssuerMap.get(key), + issuer: key => { + const entry = conversationStateIssuerMap.get(key); + return entry && { accountId: entry.storedAccountId ?? entry.accountId, fence: entry.storedAccountId ? entry.storedFence ?? undefined : entry.fence, lastUsedAt: entry.lastUsedAt }; + }, clearLive: () => { threadAccountMap.clear(); threadAffinityEntryTotal = 0; conversationStateIssuerMap.clear(); }, }); +} +registerThreadAffinityLiveState(); export function clearThreadAccountMap(options: { preserveConversationStateIssuers?: boolean } = {}): void { recordAllCleared(); @@ -316,9 +324,9 @@ export function peekConversationStateIssuer( recordIssuerTouch(bindingKey); let entry = conversationStateIssuerMap.get(bindingKey); if (!entry) { - const accountId = adoptStoredIssuer(bindingKey, now); - if (accountId !== undefined) { - entry = { accountId, lastUsedAt: now, fence: accountId === CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER ? undefined : codexAffinityIdentityFence(accountId) }; + const stored = adoptStoredIssuer(bindingKey, now); + if (stored !== undefined) { + entry = { accountId: stored.issuer, lastUsedAt: now, storedAccountId: stored.storedAccountId, storedFence: stored.storedFence }; conversationStateIssuerMap.set(bindingKey, entry); pruneConversationStateIssuers(now); } diff --git a/src/server/responses/compact.ts b/src/server/responses/compact.ts index 01f90d542be..212e3325fb6 100644 --- a/src/server/responses/compact.ts +++ b/src/server/responses/compact.ts @@ -985,6 +985,7 @@ export async function handleResponsesCompact( if (refusal) return refusal; applyAccountChangeConversationStateScrub({ body: raw, + headers, bindingKey: binding.bindingKey, servingAccountId: binding.accountId, logCtx, @@ -1003,7 +1004,11 @@ export async function handleResponsesCompact( && compactProvider.supportsNativeCompactEndpoint === true ? buildResponsesTransport({ ...compactProvider, apiKey: compactProvider.apiKey ? resolveProviderApiKey(compactProvider.apiKey) : undefined }, { headers: req.headers }) : undefined; - if (gatewayTransport) headers = new Headers(gatewayTransport.headers); + if (gatewayTransport) { + headers = new Headers(gatewayTransport.headers); + const binding = conversationStateBindingFromAuth(authCtx, codexPoolAffinityKey(req.headers)); + if (binding) applyAccountChangeConversationStateScrub({ body: compactBody, headers, bindingKey: binding.bindingKey, servingAccountId: binding.accountId, logCtx }); + } // Native v1 bypasses the adapter serializer. Bind the same replay provenance as ordinary // turns, but leave canonical compact's established account-state handling unchanged. const gatewayReplayParsed = gatewayTransport ? parseRequest(raw) : undefined; @@ -1159,7 +1164,9 @@ export async function handleResponsesCompact( onPhysicalDispatch?: () => void, ): Promise => { // A gateway send carries only its own transport headers, whatever recovery handed in. + // An account-change scrub that removed the turn state from the send headers still applies. let wireHeaders = gatewayTransport ? new Headers(gatewayTransport.headers) : sendHeaders; + if (gatewayTransport && !sendHeaders.has("x-codex-turn-state")) wireHeaders.delete("x-codex-turn-state"); sendHeaders = wireHeaders; const buildCompactSendBody = () => { if (gatewayReplayParsed) bindRouteReasoningReplayScope({ @@ -1431,7 +1438,7 @@ export async function handleResponsesCompact( recordCompactPoolOutcome(outcomeCtx, outcome); return formatErrorResponse(502, "upstream_error", "Failed to connect to compact upstream"); }; - const scrubCompactStateForAlternate = (alternateAuthCtx: CodexAuthContext): void => { + const scrubCompactStateForAlternate = (alternateAuthCtx: CodexAuthContext, alternateHeaders: Headers): void => { const binding = conversationStateBindingFromAuth( alternateAuthCtx, compactAlternateAffinityKey(authCtx, () => codexPoolAffinityKey(req.headers)), @@ -1440,6 +1447,7 @@ export async function handleResponsesCompact( for (const body of [raw, compactBody]) { applyAccountChangeConversationStateScrub({ body, + headers: alternateHeaders, bindingKey: binding.bindingKey, servingAccountId: binding.accountId, priorAccountId: conversationStateAccountId(authCtx), @@ -1545,7 +1553,7 @@ export async function handleResponsesCompact( await upstream.body?.cancel().catch(() => undefined); outcomeCtx = alternate.authCtx; logCtx.accountLogLabel = codexAuthContextLogLabel(alternate.authCtx, config); - scrubCompactStateForAlternate(alternate.authCtx); + scrubCompactStateForAlternate(alternate.authCtx, alternate.headers); try { upstream = await sendCompactAttempt(alternate.provider, alternate.headers, "single", alternate.authCtx); } catch (err) { @@ -1602,7 +1610,7 @@ export async function handleResponsesCompact( // B's identity before B's admission, so every budget and spend check runs against B. A's // response stays intact until B is actually sent. logCtx.accountLogLabel = codexAuthContextLogLabel(alternate.authCtx, config); - scrubCompactStateForAlternate(alternate.authCtx); + scrubCompactStateForAlternate(alternate.authCtx, alternate.headers); const recordFirstMove = () => { recordCompactPoolOutcome(authCtx, firstStatus, { retryAfter: firstRetryAfter, resetAt: firstResetAt }); void upstream.body?.cancel().catch(() => undefined); diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts index f5c3282a7d1..6a6dd70e006 100644 --- a/tests/codex-integration/codex-affinity-restart-matrix.test.ts +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -150,3 +150,9 @@ for (const path of ["responses", "compact"] as const) { })); } } + +test("touching an unknown restored issuer extends its durable lifetime", () => inHome(home => { + worker(home, "codex-affinity-restart-worker.ts", "unknown-write"); + expect(worker(home, "codex-affinity-restart-worker.ts", "touch-unknown").issuer).toBe("__unknown_codex_conversation_state_issuer__"); + expect(worker(home, "codex-affinity-restart-worker.ts", "read-unknown").issuer).toBe("__unknown_codex_conversation_state_issuer__"); +})); diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index c9290ae25e9..c77369e70e0 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -1,3 +1,4 @@ +import { registerThreadAffinityLiveState } from "../../src/codex/routing/thread-affinity"; import { afterEach, beforeEach, expect, test } from "bun:test"; import { Database } from "bun:sqlite"; import { existsSync, mkdtempSync, readFileSync, statSync } from "node:fs"; @@ -8,7 +9,7 @@ import { codexConversationKeyFor } from "../../src/codex/lineage"; import { acquireSpendLedgerOwner, type SpendLedgerOwnerLease } from "../../src/lib/spend-ledger-owner"; import { runPassiveRead } from "../../src/lib/passive-read"; import { removeTreeWithRetry } from "../helpers/remove-tree"; -import { adoptStoredIssuer, adoptStoredThreadRows, codexAffinityIdentityFence, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, flushCodexAffinityStoreForTests, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordThreadBind, recordThreadTouch, registerCodexAffinityLiveState, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; +import { adoptStoredIssuer, adoptStoredThreadRows, codexAffinityIdentityFence, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, flushCodexAffinityStoreForTests, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerDelete, recordThreadDelete, recordThreadBind, recordThreadTouch, registerCodexAffinityLiveState, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; let home: string; let prior: string | undefined; @@ -28,7 +29,7 @@ beforeEach(() => { prior = process.env.OPENCODEX_HOME; home = mkdtempSync(join(tmpdir(), "ocx-affinity-")); process.env.OPENCODEX_HOME = home; live = undefined; registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, clearLive: () => { live = undefined; } }); }); -afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); +afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); registerThreadAffinityLiveState(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); test("inactive and passive operations do not create a database", async () => { key(); expect(existsSync(path())).toBe(false); @@ -53,7 +54,7 @@ test("unclean database is discarded and unknown version remains unchanged", () = }); test("issuer fence mismatch is unknown and account clearing preserves issuer", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); recordIssuerSet(k, "account-a", fence, Date.now()); recordAccountPinsCleared("account-a"); lease.release(); - seed(); lease = acquireSpendLedgerOwner(); const restoredKey = key(); expect(adoptStoredThreadRows(restoredKey, Date.now())).toEqual([]); expect(adoptStoredIssuer(restoredKey, Date.now())).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); + seed(); lease = acquireSpendLedgerOwner(); const restoredKey = key(); expect(adoptStoredThreadRows(restoredKey, Date.now())).toEqual([]); expect(adoptStoredIssuer(restoredKey, Date.now())?.issuer).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); }); test("clear all followed by bind preserves the later bind", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); recordAllCleared(); bind(fence); lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); @@ -96,8 +97,8 @@ test("three failed flushes poison persistence until the next owner discards rows test("known and fence-less issuers restore with distinct outcomes", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); const known = key(); const unknown = codexConversationKeyFor("other-family", "other-thread"); recordIssuerSet(known, "account-a", fence, Date.now()); recordIssuerSet(unknown, "account-a", null, Date.now()); lease.release(); - lease = acquireSpendLedgerOwner(); expect(adoptStoredIssuer(key(), Date.now())).toBe("account-a"); - expect(adoptStoredIssuer(codexConversationKeyFor("other-family", "other-thread"), Date.now())).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); + lease = acquireSpendLedgerOwner(); expect(adoptStoredIssuer(key(), Date.now())?.issuer).toBe("account-a"); + expect(adoptStoredIssuer(codexConversationKeyFor("other-family", "other-thread"), Date.now())?.issuer).toBe(CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER); }); test("account clear followed by bind retains the later pin", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); recordAccountPinsCleared("account-a"); bind(fence); lease.release(); @@ -181,3 +182,27 @@ test("release preserves live state, while an observed home change clears it", () removeTreeWithRetry(otherHome); } }); + +for (const kind of ["thread", "issuer"] as const) { + test(`persisted ${kind} delete survives anchor eviction`, () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); + recordIssuerSet(k, "account-a", fence, Date.now()); flushCodexAffinityStoreForTests(); + for (let i = 0; i < 4096; i++) codexConversationKeyFor(`evict-family-${i}`, `evict-thread-${i}`); + if (kind === "thread") recordThreadDelete(k, "legacy"); else recordIssuerDelete(k); + lease.release(); lease = acquireSpendLedgerOwner(); + if (kind === "thread") expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); + else expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); + }); +} + +test("adopted rows retain delete keys after anchor eviction", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); + recordIssuerSet(k, "account-a", fence, Date.now()); lease.release(); + lease = acquireSpendLedgerOwner(); const adopted = key(); + expect(adoptStoredThreadRows(adopted, Date.now())).toHaveLength(1); + expect(adoptStoredIssuer(adopted, Date.now())?.issuer).toBe("account-a"); + for (let i = 0; i < 4096; i++) codexConversationKeyFor(`adopt-family-${i}`, `adopt-thread-${i}`); + recordThreadDelete(adopted, "legacy"); recordIssuerDelete(adopted); lease.release(); + lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); + expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); +}); diff --git a/tests/helpers/codex-affinity-restart-worker.ts b/tests/helpers/codex-affinity-restart-worker.ts index dd115ccddcf..8ae88a73858 100644 --- a/tests/helpers/codex-affinity-restart-worker.ts +++ b/tests/helpers/codex-affinity-restart-worker.ts @@ -5,6 +5,11 @@ import { resolveCodexAccountForThread, rememberConversationStateIssuer, peekConv import { applyAccountChangeConversationStateScrub, accountChangeFileReferenceRefusal } from "../../src/server/responses/account-change-state"; import type { OcxConfig } from "../../src/types"; +const action = process.argv[2]; +if (action === "touch-unknown" || action === "read-unknown") { + const realNow = Date.now; + Date.now = () => realNow() + (action === "touch-unknown" ? 23 : 25) * 60 * 60_000; +} const lease = acquireSpendLedgerOwner(); for (const id of ["a", "b"]) if (!readCodexAccountRecord(id)) saveCodexAccountCredential(id, { accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `acct-${id}`, @@ -13,6 +18,11 @@ const config = { providers: {}, codexAccounts: ["a", "b"].map(id => ({ id, email if (process.argv[2] === "read") resolveCodexAccountForThread("advance-cursor", config); const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; const issuer = peekConversationStateIssuer(key); +if (action === "touch-unknown" || action === "read-unknown") { + lease.release(); + console.log(JSON.stringify({ issuer })); + process.exit(0); +} const account = resolveCodexAccountForThread(key, config)!; const sameBody = { previous_response_id: "previous" }; const otherBody = { previous_response_id: "previous" }; From 7bd2fcb274829e3f4bf6bd74b2cc1eb8d42a98d4 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:43:43 +0900 Subject: [PATCH 09/21] fix(codex): reconcile live affinity on new lease, bounded and pruned store, file refusal for moved unknown issuers, gateway compact tests --- src/codex/routing/thread-affinity-store.ts | 49 ++++++++--- src/codex/routing/thread-affinity.ts | 11 ++- src/server/responses/account-change-state.ts | 6 +- structure/providers/codex-pool-affinity.md | 4 +- .../codex-affinity-restart-matrix.test.ts | 85 ++++++++++++++++--- .../codex-affinity-restart.test.ts | 4 +- .../helpers/codex-affinity-handler-worker.ts | 2 +- tests/helpers/codex-affinity-matrix-worker.ts | 40 +++++++++ ...ive-compaction-gateway-credentials.test.ts | 22 +++++ .../native-compaction-gateway.test.ts | 13 +++ 10 files changed, 197 insertions(+), 39 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index aae19e2bd8d..af094b08a79 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -6,6 +6,7 @@ import { lstatSync, openSync, realpathSync, + type Stats, } from "node:fs"; import { resolve } from "node:path"; import { getConfigDir } from "../../config/paths"; @@ -76,6 +77,7 @@ type Scope = { issuerKeys: Map; consulted: Set; timer?: ReturnType; + pruneTimer?: ReturnType; scheduled: boolean; }; const TTL = 24 * 60 * 60_000; @@ -92,6 +94,7 @@ const BASE_SCOPES_EXHAUSTIVE: Exclude< void BASE_SCOPES_EXHAUSTIVE; let current: Scope | undefined; let observedHome: string | undefined; +let hadScope = false; let reader: LiveReader | undefined; export function registerCodexAffinityLiveState(value: LiveReader): void { @@ -107,13 +110,14 @@ function retire(): void { if (!s) return; current = undefined; if (s.timer) clearTimeout(s.timer); + if (s.pruneTimer) clearInterval(s.pruneTimer); try { s.db?.close(); } catch { warning(); } } -function checkFile(path: string): void { +function checkFile(path: string): Stats { const stat = lstatSync(path); if ( !stat.isFile() || @@ -122,6 +126,7 @@ function checkFile(path: string): void { (process.platform !== "win32" && stat.uid !== process.getuid!()) ) throw new Error("unsafe file"); + return stat; } function open(s: Scope): void { const path = spendLedgerStoragePath(s.token); @@ -136,11 +141,14 @@ function open(s: Scope): void { } catch (e) { if ((e as NodeJS.ErrnoException).code !== "EEXIST") throw e; } - checkFile(path); + const before = checkFile(path); + if (before.size > 8 * 1024 * 1024) throw new Error("oversized"); if (process.platform !== "win32") chmodSync(path, 0o600); if (!hardenSecretPath(path, { required: true }).ok) throw new Error("acl"); checkFile(path); assertStorageOwned(s.token); + // SQLite creates sidecars in the private, hardened config directory, as with + // spend-ledger-owner.sqlite and config-mutation.sqlite. db = new Database(path); db.exec("PRAGMA busy_timeout = 0; BEGIN IMMEDIATE"); const version = ( @@ -208,18 +216,29 @@ function open(s: Scope): void { !rows.some((r) => r.name === "clean") ) throw new Error("meta"); + db.exec("PRAGMA secure_delete = ON"); // Only a completed final flush permits the next owner to restore rows. if (clean !== 1) db.exec("DELETE FROM threads; DELETE FROM issuers"); db.exec("UPDATE meta SET value=0 WHERE name='clean'"); } + db.exec("PRAGMA secure_delete = ON"); + const pageSize = (db.query("PRAGMA page_size").get() as { page_size: number }).page_size; + db.exec(`PRAGMA max_page_count = ${Math.floor(8 * 1024 * 1024 / pageSize)}`); + prune(db, Date.now()); const secret = ( db.query("SELECT value FROM meta WHERE name='secret'").get() as { value: Uint8Array; } ).value; db.exec("COMMIT"); + const after = checkFile(path); + if (before.dev !== after.dev || before.ino !== after.ino) throw new Error("replaced"); s.db = db; s.secret = secret; + s.pruneTimer = setInterval(() => { + if (current === s && !isPassiveRead()) flush(s, false); + }, 60 * 60_000); + s.pruneTimer.unref(); } catch { try { db?.exec("ROLLBACK"); @@ -269,6 +288,8 @@ function scope(): Scope | undefined { } catch { return undefined; } + if (hadScope) reader?.clearLive(); + hadScope = true; const s: Scope = { home, generation, @@ -335,9 +356,13 @@ export function codexAffinityIdentityFence( !record.quotaHistoryIdentity ) return undefined; + return identityFence(record.quotaHistoryIdentity); +} +export function identityFence(identity: string | undefined): string | undefined { + if (!identity) return undefined; return createHash("sha256") .update("opencodex/codex-affinity/identity/v1\0") - .update(record.quotaHistoryIdentity) + .update(identity) .digest("hex"); } function rememberThreadKey(s: Scope, key: string, rk: string, scopeName: string, accountId: string): void { @@ -385,6 +410,12 @@ function scheduleTouch(s: Scope): void { }, 1000); s.timer.unref(); } +function prune(db: Database, now: number): void { + for (const table of ["threads", "issuers"]) + db.query(`DELETE FROM ${table} WHERE last_used_at < ? OR last_used_at > ?`).run(now - TTL, now); + db.query("DELETE FROM threads WHERE created_at > ?").run(now); + db.exec("DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)"); +} function flush(s: Scope, clean: boolean): boolean { if (!s.db || s.disabled || s.poisoned || isPassiveRead()) return false; const db = s.db; @@ -416,15 +447,7 @@ function flush(s: Scope, clean: boolean): boolean { "UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?", ).run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); } - const now = Date.now(); - for (const table of ["threads", "issuers"]) - db.query( - `DELETE FROM ${table} WHERE last_used_at < ? OR last_used_at > ?`, - ).run(now - TTL, now); - db.query("DELETE FROM threads WHERE created_at > ?").run(now); - db.exec( - "DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)", - ); + prune(db, Date.now()); if (clean) db.exec("UPDATE meta SET value=1 WHERE name='clean'"); db.exec("COMMIT"); s.log = []; @@ -488,7 +511,7 @@ export function adoptStoredThreadRows( try { const rows = s .db!.query( - "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=?", + "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? ORDER BY last_used_at DESC, scope LIMIT 64", ) .all(rk) as { scope: string; diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index faf10fcf62b..92342f2d7e3 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -6,8 +6,7 @@ import type { CodexThreadLineage } from "../lineage"; import type { CodexQuotaScope } from "./health-store"; import type { TransientProbeLease } from "../../routing/probe-lease"; import { clearPoolRecoveryState } from "../../routing/probe-lease"; -import { createHash } from "node:crypto"; -import { adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; +import { identityFence, adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; export { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "./thread-affinity-store"; export type ThreadAffinityEntry = { @@ -215,10 +214,6 @@ type ConversationStateIssuerEntry = { }; const conversationStateIssuerMap = new Map(); -function identityFence(identity: string | undefined): string | undefined { - return identity ? createHash("sha256").update("opencodex/codex-affinity/identity/v1\0").update(identity).digest("hex") : undefined; -} - export function registerThreadAffinityLiveState(): void { registerCodexAffinityLiveState({ thread: (key, scope) => threadAccountMap.get(key)?.get(scope as ThreadAffinityScope), @@ -336,6 +331,10 @@ export function peekConversationStateIssuer( return entry.accountId; } +export function peekConversationStateStoredIssuerAccount(bindingKey: string): string | undefined { + return conversationStateIssuerMap.get(bindingKey)?.storedAccountId; +} + /** * Why a binding was released, held until that thread's next resolve can report it (#4546). * diff --git a/src/server/responses/account-change-state.ts b/src/server/responses/account-change-state.ts index 71f0906ab42..d594e3362dd 100644 --- a/src/server/responses/account-change-state.ts +++ b/src/server/responses/account-change-state.ts @@ -12,7 +12,7 @@ import { conversationStateAccountId } from "../../codex/request-owned-affinity"; import type { CodexAuthContext } from "../../codex/auth-context"; -import { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "../../codex/routing/thread-affinity"; +import { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, peekConversationStateStoredIssuerAccount } from "../../codex/routing/thread-affinity"; import { peekConversationStateIssuer, rememberConversationStateIssuer, @@ -323,7 +323,9 @@ export function accountChangeFileReferenceRefusal( const { body, bindingKey, servingAccountId, priorAccountId } = args; if (!servingAccountId || !bindingKey) return undefined; const issuer = peekConversationStateIssuer(bindingKey); - const accountChanged = (issuer != null && issuer !== CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER && issuer !== servingAccountId) + const accountChanged = (issuer === CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER + ? peekConversationStateStoredIssuerAccount(bindingKey) !== servingAccountId + : issuer != null && issuer !== servingAccountId) || (priorAccountId != null && priorAccountId !== servingAccountId); if (!accountChanged) return undefined; // Checked against the carriers directly rather than through the portability verdict: that diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 005ddbdacf2..e7ed5d31f0c 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -7,9 +7,9 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp `src/codex/routing/thread-affinity-store.ts` restores eligible pool bindings after a graceful proxy restart for up to 24 hours idle. Session-bearing requests and session-less requests anchored on their immediate parent have derivable anchors. Deeper session-less chains, keys derived under another config home, legacy raw-parent keys and evicted anchors remain memory-only. Removed, re-logged, expired or identity-less accounts are not restored; normal selection and usability rules still apply. -The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state, but alone never triggers the uploaded-file refusal; a known account change or prior-account move retains that refusal. +The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. Pruning removes expired or future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A config-home change retires the store scope and clears live affinity. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse; idle rows are deleted within about an hour while the proxy runs and at the next start after an offline period. SQLite secure_delete overwrites freed pages; its transient rollback journal is removed after each commit. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts index 6a6dd70e006..45b2901f4cb 100644 --- a/tests/codex-integration/codex-affinity-restart-matrix.test.ts +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -1,6 +1,6 @@ import { expect, test } from "bun:test"; import { Database } from "bun:sqlite"; -import { existsSync, mkdirSync, mkdtempSync, readFileSync, symlinkSync, writeFileSync } from "node:fs"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, symlinkSync, truncateSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { initializeConfigOwnership, removeOwnedConfigState } from "../../src/lib/config-ownership"; @@ -98,6 +98,53 @@ test("pruning keeps newest issuers and prioritizes ordinary bindings over detour db.close(); })); +test("oversized database is disabled without changing its bytes", () => inHome(home => { + seed(home); + const file = join(home, names[0]!); + truncateSync(file, 8 * 1024 * 1024 + 1); + const before = readFileSync(file); + matrix(home, "probe"); + expect(readFileSync(file).equals(before)).toBe(true); +})); + +test("open immediately removes expired and excess rows", () => inHome(home => { + seed(home); + const db = new Database(join(home, names[0]!)); + const now = Date.now() - 5000; + db.exec("BEGIN"); + const thread = db.query("INSERT INTO threads VALUES (?, 'shared', 'a', 'fence', ?, ?)"); + const issuer = db.query("INSERT INTO issuers VALUES (?, 'a', 'fence', ?)"); + for (let i = 0; i < 2050; i++) { thread.run(`bulk-${i}`, now, now); issuer.run(`bulk-${i}`, now); } + thread.run("expired", now - 90_000_000, now - 90_000_000); + issuer.run("expired", now - 90_000_000); + db.exec("COMMIT"); db.close(); + const result = matrix(home, "open-count"); + expect(result.threads.n).toBe(2048); + expect(result.issuers.n).toBe(2048); + expect(result.expired.n).toBe(0); +})); + +test("hourly timer prunes an idle open store", () => inHome(home => { + seed(home); + const result = matrix(home, "hourly"); + expect(result.scheduled).toBe(true); + expect(result.threads.n).toBe(0); + expect(result.issuers.n).toBe(0); +})); + +test("adoption loads at most 64 newest scopes for a row key", () => inHome(home => { + seed(home); + const db = new Database(join(home, names[0]!)); + const row = db.query("SELECT row_key, account_id, fence, created_at, last_used_at FROM threads LIMIT 1").get() as { row_key: string; account_id: string; fence: string; created_at: number; last_used_at: number }; + const insert = db.query("INSERT INTO threads VALUES (?, ?, ?, ?, ?, ?)"); + for (let i = 0; i < 80; i++) insert.run(row.row_key, `model-detour:shared:model-${i}`, row.account_id, row.fence, row.created_at, row.last_used_at - i - 1); + db.close(); + const scopes = matrix(home, "scopes").scopes; + expect(scopes).toHaveLength(64); + expect(scopes).toContain("model-detour:shared:model-0"); + expect(scopes).not.toContain("model-detour:shared:model-79"); +})); + test("restoration accepts a 512-byte model component and rejects 513 bytes", () => inHome(home => { seed(home); const file = join(home, names[0]!); @@ -129,28 +176,40 @@ test("home switching drops pending writes and does not carry recorded anchors ac }); expect(result.exitCode, result.stderr.toString()).toBe(0); expect(JSON.parse(result.stdout.toString())).toEqual({ first: "a", rowsB: 1, restoredB: "a" }); -}))); +}))); + +for (const mutation of ["bind", "delete", "issuer"]) { + test(`late ${mutation} after release is discarded on reacquisition`, () => inHome(home => { + seed(home); + expect(matrix(home, `late-${mutation}`)).toEqual({ pin: "a", issuer: "a" }); + const restored = matrix(home, "read"); + expect(restored.before).toBe("a"); + expect(restored.issuer).toBe("a"); + })); +} for (const path of ["responses", "compact"] as const) { for (const issuer of ["unknown", "known"] as const) { test(`${path} handler ${issuer} restored issuer with an uploaded file`, () => inHome(home => { worker(home, "codex-affinity-restart-worker.ts", issuer === "unknown" ? "unknown-write" : "write"); const result = worker(home, "codex-affinity-handler-worker.ts", path); - if (issuer === "known") { - expect(result.status).toBe(400); - expect(result.bodies).toHaveLength(0); - } else { - expect(result.status, result.responseBody).toBe(200); - expect(result.bodies).toHaveLength(1); - expect(result.bodies[0].previous_response_id).toBeUndefined(); - expect(result.turnStates).toEqual([null]); - expect(result.bodies[0].client_metadata?.["x-codex-turn-state"]).toBeUndefined(); - expect(JSON.stringify(result.bodies[0].input)).toContain("file-test"); - } + expect(result.status).toBe(400); + expect(result.bodies).toHaveLength(0); })); } } +for (const path of ["responses", "compact"] as const) { + test(`${path} permits a file for the same stored account with unknown issuer`, () => inHome(home => { + worker(home, "codex-affinity-restart-worker.ts", "unknown-write"); + const result = Bun.spawnSync([process.execPath, helperPath("codex-affinity-handler-worker.ts"), path, "same"], { env: { ...process.env, OPENCODEX_HOME: home, CODEX_HOME: home }, stdout: "pipe", stderr: "pipe" }); + expect(result.exitCode, result.stderr.toString()).toBe(0); + const parsed = JSON.parse(result.stdout.toString()); + expect(parsed.status, parsed.responseBody).toBe(200); + expect(parsed.bodies).toHaveLength(1); + })); +} + test("touching an unknown restored issuer extends its durable lifetime", () => inHome(home => { worker(home, "codex-affinity-restart-worker.ts", "unknown-write"); expect(worker(home, "codex-affinity-restart-worker.ts", "touch-unknown").issuer).toBe("__unknown_codex_conversation_state_issuer__"); diff --git a/tests/codex-integration/codex-affinity-restart.test.ts b/tests/codex-integration/codex-affinity-restart.test.ts index a056c5204b8..40d2450e8ab 100644 --- a/tests/codex-integration/codex-affinity-restart.test.ts +++ b/tests/codex-integration/codex-affinity-restart.test.ts @@ -42,7 +42,7 @@ test("an unclean owner discards persisted affinity", () => { } finally { removeTreeWithRetry(home); } }); -test("unknown issuer scrubs continuation without refusing files, while retry moves are refused", () => { +test("unknown issuer scrubs continuation and refuses files on account moves", () => { const home = mkdtempSync(join(tmpdir(), "ocx-affinity-unknown-")); try { const first = run(home, "unknown-write"); @@ -54,7 +54,7 @@ test("unknown issuer scrubs continuation without refusing files, while retry mov expect(result.sameScrub).toBe(true); expect(result.fileRefusal).toBeUndefined(); expect(result.retryRefusal).toBe(400); - expect(result.otherFileRefusal).toBeUndefined(); + expect(result.otherFileRefusal).toBe(400); } finally { removeTreeWithRetry(home); } }); diff --git a/tests/helpers/codex-affinity-handler-worker.ts b/tests/helpers/codex-affinity-handler-worker.ts index af1ad97b2cb..3e0e59182e9 100644 --- a/tests/helpers/codex-affinity-handler-worker.ts +++ b/tests/helpers/codex-affinity-handler-worker.ts @@ -17,7 +17,7 @@ const mock = (async (input: string | URL | Request, init?: RequestInit) => { }) as typeof fetch; globalThis.fetch = mock; for (const provider of Object.values(config.providers)) Object.assign(provider, { fetch: mock, upstreamWebsocket: false }); -const body = { model: "side/gpt-5.5", stream: false, client_metadata: { "x-codex-turn-state": "old-state" }, input: [{ type: "message", role: "user", content: [{ type: "input_file", file_id: "file-test" }] }] }; +const body = { model: process.argv[3] === "same" ? "gpt-5.5" : "side/gpt-5.5", stream: false, client_metadata: { "x-codex-turn-state": "old-state" }, input: [{ type: "message", role: "user", content: [{ type: "input_file", file_id: "file-test" }] }] }; const request = compactionRequest(body, undefined, { "session-id": "restart-session", "thread-id": "restart-thread", "x-codex-turn-state": "old-state" }); const response = process.argv[2] === "compact" ? await handleResponsesCompact(request, config, { model: "", provider: "" }) diff --git a/tests/helpers/codex-affinity-matrix-worker.ts b/tests/helpers/codex-affinity-matrix-worker.ts index fe9a8507bc9..2feae2ff07d 100644 --- a/tests/helpers/codex-affinity-matrix-worker.ts +++ b/tests/helpers/codex-affinity-matrix-worker.ts @@ -12,6 +12,14 @@ import { bindThreadAffinity, clearConversationStateIssuerMap, clearThreadAccount import type { OcxConfig } from "../../src/types"; const action = process.argv[2]!; +let hourly: (() => void) | undefined; +if (action === "hourly") { + const original = globalThis.setInterval; + globalThis.setInterval = ((callback: TimerHandler, delay?: number) => { + if (delay === 3_600_000) hourly = callback as () => void; + return original(callback, delay); + }) as typeof setInterval; +} const lease = acquireSpendLedgerOwner(); const config = { providers: {}, codexAccounts: ["a", "b"].map(id => ({ id, email: `${id}@test`, isMain: false })), activeCodexAccountId: "a", accountPoolStrategy: "round-robin", autoSwitchThreshold: 80, upstreamFailoverThreshold: 3 } as OcxConfig; if (action === "passive") { @@ -30,11 +38,43 @@ if (action === "passive") { const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; const cred = (id: string) => ({ accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `acct-${id}` }); +if (action === "open-count" || action === "hourly") { + const db = new Database(join(process.env.OPENCODEX_HOME!, "codex-affinity.sqlite")); + if (action === "hourly") { + const realNow = Date.now; + Date.now = () => realNow() + 25 * 60 * 60_000; + hourly?.(); + } + const threads = db.query("SELECT count(*) AS n FROM threads").get(); + const issuers = db.query("SELECT count(*) AS n FROM issuers").get(); + const expired = db.query("SELECT count(*) AS n FROM threads WHERE row_key='expired'").get(); + db.close(); + lease.release(); + console.log(JSON.stringify({ threads, issuers, expired, scheduled: !!hourly })); + process.exit(0); +} + if (action === "probe") { getThreadAffinity(key); lease.release(); process.exit(0); } +if (action.startsWith("late-")) { + bindThreadAffinity(key, "a", Date.now()); + const record = readCodexAccountRecord("a")!; + rememberConversationStateIssuer(key, "a", undefined, { generation: record.generation, chatgptAccountId: record.credential!.chatgptAccountId }); + lease.release(); + if (action === "late-bind") bindThreadAffinity(key, "b", Date.now()); + if (action === "late-delete") deleteThreadAffinity(key); + if (action === "late-issuer") rememberConversationStateIssuer(key, "b"); + const next = acquireSpendLedgerOwner(); + const freshKey = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; + const pin = getThreadAffinity(freshKey)?.accountId; + const issuer = peekConversationStateIssuer(freshKey); + next.release(); + console.log(JSON.stringify({ pin, issuer })); + process.exit(0); +} if (action === "shared-write") { bindThreadAffinity(key, "a", Date.now(), "shared"); lease.release(); diff --git a/tests/responses/native-compaction-gateway-credentials.test.ts b/tests/responses/native-compaction-gateway-credentials.test.ts index fb53f6d4ea8..95baea53637 100644 --- a/tests/responses/native-compaction-gateway-credentials.test.ts +++ b/tests/responses/native-compaction-gateway-credentials.test.ts @@ -6,6 +6,8 @@ import { join } from "node:path"; import { createHash } from "node:crypto"; import { clearAccountNeedsReauth, clearAccountQuota } from "../../src/codex/auth-api"; import { clearCodexUpstreamHealth, clearThreadAccountMap } from "../../src/codex/routing"; +import { codexPoolAffinityKey } from "../../src/codex/auth-context"; +import { rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; import { handleResponsesCompact } from "../../src/server/responses"; import { clearCompactHandoffRoutesForTests } from "../../src/server/responses/compact"; import { clearResponseStateForTests, clearResponseStateMemoryForTests } from "../../src/responses/state"; @@ -107,6 +109,26 @@ afterEach(() => { }); describe("native compact gateway credential boundary", () => { + test("gateway turn state is removed after an account-change scrub", async () => { + const realMode = registry.providerCodexAccountMode; + const modeSpy = spyOn(registry, "providerCodexAccountMode").mockImplementation((id, provider) => + id === "gw" ? "pool" : realMode(id, provider)); + const request = compactRequest(); + const key = codexPoolAffinityKey(request.headers)!; + rememberConversationStateIssuer(key, "other"); + const config = gatewayConfig(); + config.providers.gw!.headers!["x-codex-turn-state"] = "configured"; + const states: Array = []; + globalThis.fetch = (async (_input, init) => { + states.push(new Headers(init?.headers).get("x-codex-turn-state")); + return Response.json({ object: "response.compaction", output: [] }); + }) as typeof fetch; + try { + const response = await handleResponsesCompact(request, config, { model: "", provider: "" }); + expect(response.status).toBe(200); + expect(states).toEqual([null]); + } finally { modeSpy.mockRestore(); } + }); test.each([401, 429])("gateway %i never receives a stored Codex credential", async status => { // The router pins the only pool-mode row to the canonical URL today. Model a relaxation of that // pin: the native gateway send must hold the boundary on its own. diff --git a/tests/responses/native-compaction-gateway.test.ts b/tests/responses/native-compaction-gateway.test.ts index e7e3509913a..aa3744069f1 100644 --- a/tests/responses/native-compaction-gateway.test.ts +++ b/tests/responses/native-compaction-gateway.test.ts @@ -117,6 +117,19 @@ describe("native gateway transport", () => { expect((await calls[0]!.json()).model).toBe("some-model"); }); + test("configured gateway turn state survives when no account changed", async () => { + let sent: Request | undefined; + globalThis.fetch = (async (url, init) => { + sent = new Request(url, init); + return Response.json({ object: "response.compaction", output: [compactItem] }); + }) as typeof fetch; + const provider = { ...gateway, supportsNativeCompactEndpoint: true, headers: { "x-codex-turn-state": "configured" } }; + const config = { ...getDefaultConfig(), defaultProvider: "gw", providers: { gw: provider } }; + const response = await handleResponsesCompact(compactionRequest(baseCompactionBody()), config, { model: "", provider: "" }); + expect(response.status).toBe(200); + expect(sent?.headers.get("x-codex-turn-state")).toBe("configured"); + }); + test.each([ { replay: undefined, blob: nativeBlob }, { replay: true, blob: encodeCompactionSummary("Portable retained task fixture.") }, From 34b616e87c52f54889af38ec54a52477c2d28e5c Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:50:54 +0900 Subject: [PATCH 10/21] fix(codex): bounded flush under the size cap, scope-aware adoption, eager startup prune --- src/codex/routing/thread-affinity-store.ts | 64 +++++++++++++--- src/codex/routing/thread-affinity.ts | 10 +-- src/lib/spend-ledger-owner.ts | 12 ++- structure/providers/codex-pool-affinity.md | 2 +- .../codex-affinity-store.test.ts | 73 ++++++++++++++++++- tests/helpers/codex-affinity-matrix-worker.ts | 3 +- tests/lib/spend-ledger-owner.test.ts | 10 +++ 7 files changed, 152 insertions(+), 22 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index af094b08a79..4cb3fa5a15f 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -16,6 +16,7 @@ import { assertStorageOwned, currentSpendLedgerOwnerGeneration, mintSpendLedgerStorage, + onSpendLedgerOwnerAcquired, onSpendLedgerOwnerReleased, onSpendLedgerOwnerReleasing, spendLedgerStoragePath, @@ -236,7 +237,10 @@ function open(s: Scope): void { s.db = db; s.secret = secret; s.pruneTimer = setInterval(() => { - if (current === s && !isPassiveRead()) flush(s, false); + if (current === s) { + if (s.poisoned || isPassiveRead()) pruneOnly(s); + else flush(s, false); + } }, 60 * 60_000); s.pruneTimer.unref(); } catch { @@ -254,9 +258,9 @@ function open(s: Scope): void { warning(); } } -function scope(): Scope | undefined { - if (isPassiveRead()) return undefined; - let home = resolve(getConfigDir()); +function scope(acquiredHome?: string): Scope | undefined { + if (!acquiredHome && isPassiveRead()) return undefined; + let home = resolve(acquiredHome ?? getConfigDir()); try { home = realpathSync.native(home); } catch { @@ -416,15 +420,31 @@ function prune(db: Database, now: number): void { db.query("DELETE FROM threads WHERE created_at > ?").run(now); db.exec("DELETE FROM threads WHERE rowid IN (SELECT rowid FROM threads ORDER BY (scope LIKE 'model-detour:%') ASC, last_used_at DESC, row_key, scope LIMIT -1 OFFSET 2048); DELETE FROM issuers WHERE rowid IN (SELECT rowid FROM issuers ORDER BY last_used_at DESC, row_key LIMIT -1 OFFSET 2048)"); } +function pruneOnly(s: Scope): boolean { + if (!s.db || s.disabled) return false; + try { + assertStorageOwned(s.token); + s.db.exec("BEGIN IMMEDIATE"); + prune(s.db, Date.now()); + s.db.exec("COMMIT"); + return true; + } catch { + try { s.db.exec("ROLLBACK"); } catch { /* no transaction */ } + warning(); + return false; + } +} function flush(s: Scope, clean: boolean): boolean { if (!s.db || s.disabled || s.poisoned || isPassiveRead()) return false; const db = s.db; - try { + for (let attempt = 0; attempt < 2; attempt++) try { assertStorageOwned(s.token); db.exec("BEGIN IMMEDIATE"); - for (const op of s.log) { + for (let i = 0; i < s.log.length; i++) { + const op = s.log[i]!; if (op.args.length) db.query(op.sql).run(...op.args); else db.exec(op.sql); + if ((i + 1) % 256 === 0) prune(db, Date.now()); } // Read touches at flush time, after callers have updated the live entries. for (const t of s.threads.values()) { @@ -455,17 +475,20 @@ function flush(s: Scope, clean: boolean): boolean { s.issuers.clear(); s.failures = 0; return true; - } catch { + } catch (error) { try { db.exec("ROLLBACK"); } catch { /* preserve failure */ } + if (attempt === 0 && error instanceof Error && "code" in error && error.code === "SQLITE_FULL" && pruneOnly(s)) continue; if (++s.failures >= 3) poison(s); else warning(); return false; } + return false; } +onSpendLedgerOwnerAcquired((home) => { scope(home); }); // Commit clean before closing; a later close failure cannot undo that commit. onSpendLedgerOwnerReleasing(() => { const s = current; @@ -503,17 +526,20 @@ function validScope(value: string): boolean { export function adoptStoredThreadRows( key: string, now: number, + scopeName?: string, ): StoredThreadRow[] { const s = usable(); if (!s) return []; const rk = rowKey(s, key, "conversation"); - if (!rk || pending(s, rk, false) || s.consulted.has(`t:${rk}`)) return []; + if (!rk || pending(s, rk, false) || (scopeName !== undefined && s.consulted.has(`t:${rk}:${scopeName}`))) return []; try { const rows = s .db!.query( - "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? ORDER BY last_used_at DESC, scope LIMIT 64", + scopeName === undefined + ? "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? ORDER BY last_used_at DESC, scope LIMIT 64" + : "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? AND scope=? LIMIT 1", ) - .all(rk) as { + .all(...(scopeName === undefined ? [rk] : [rk, scopeName])) as { scope: string; account_id: string; fence: string; @@ -528,7 +554,8 @@ export function adoptStoredThreadRows( op.kind === "deleteScope" && op.key === rk && op.scope === r.scope, ) || s.log.some( - (op) => op.kind === "deleteAccount" && op.args[0] === r.account_id, + (op) => (op.kind === "deleteAccount" && op.args[0] === r.account_id) || + (op.kind === "deleteKeyAccount" && op.key === rk && op.args[1] === r.account_id), ) ) continue; @@ -561,7 +588,8 @@ export function adoptStoredThreadRows( lastUsedAt: r.last_used_at, }); } - consulted(s, `t:${rk}`); + for (const row of rows) consulted(s, `t:${rk}:${row.scope}`); + if (scopeName !== undefined) consulted(s, `t:${rk}:${scopeName}`); return result; } catch { warning(); @@ -676,6 +704,18 @@ export function recordThreadDelete(key: string, scopeName?: string): void { }, ); } +export function recordThreadAccountDeleted(key: string, id: string): void { + const s = usable(); + if (!s) return; + const rk = rowKey(s, key, "conversation"); + if (!rk) return; + const known = s.threadKeys.get(key); + if (known) { + for (const [scopeName, account] of known.scopes) if (account === id) known.scopes.delete(scopeName); + if (!known.scopes.size) s.threadKeys.delete(key); + } + enqueue(s, { kind: "deleteKeyAccount", key: rk, sql: "DELETE FROM threads WHERE row_key=? AND account_id=?", args: [rk, id] }); +} export function recordAccountPinsCleared(id: string): void { const s = usable(); if (s) { diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index 92342f2d7e3..80448917454 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -6,7 +6,7 @@ import type { CodexThreadLineage } from "../lineage"; import type { CodexQuotaScope } from "./health-store"; import type { TransientProbeLease } from "../../routing/probe-lease"; import { clearPoolRecoveryState } from "../../routing/probe-lease"; -import { identityFence, adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; +import { identityFence, adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordThreadAccountDeleted, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; export { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "./thread-affinity-store"; export type ThreadAffinityEntry = { @@ -393,12 +393,12 @@ function getThreadAffinityForScope( ): ThreadAffinityEntry | undefined { if (!admissibleAffinityComponent(threadId)) return undefined; recordThreadTouch(threadId, scope); - if (!threadAccountMap.get(threadId)?.get(scope)) adoptThreadAffinities(threadId, Date.now()); + if (!threadAccountMap.get(threadId)?.get(scope)) adoptThreadAffinities(threadId, Date.now(), scope); return threadAccountMap.get(threadId)?.get(scope); } -function adoptThreadAffinities(threadId: string, now: number): void { - for (const row of adoptStoredThreadRows(threadId, now)) { +function adoptThreadAffinities(threadId: string, now: number, scopeName?: ThreadAffinityScope): void { + for (const row of adoptStoredThreadRows(threadId, now, scopeName)) { const record = readCodexAccountRecord(row.accountId); if (!record?.credential || record.deletedAt != null || identityFence(record.quotaHistoryIdentity) !== row.fence) continue; const scope = row.scope as ThreadAffinityScope; @@ -451,7 +451,7 @@ export function deleteModelDetourAffinity( /** Remove only the matching failed account's affinities for one thread. */ export function deleteThreadAffinitiesForAccount(threadId: string, accountId: string): void { if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; - adoptThreadAffinities(threadId, Date.now()); + recordThreadAccountDeleted(threadId, accountId); const affinities = threadAccountMap.get(threadId); if (!affinities) return; for (const [scope, entry] of affinities) { diff --git a/src/lib/spend-ledger-owner.ts b/src/lib/spend-ledger-owner.ts index 00c6f9a0187..46833f4de63 100644 --- a/src/lib/spend-ledger-owner.ts +++ b/src/lib/spend-ledger-owner.ts @@ -47,6 +47,12 @@ let boundLedgerHome: string | null = null; */ let ownerGeneration = 0; const releaseHooks: Array<() => void> = []; +const acquiredHooks: Array<(home: string) => void> = []; + +/** Run once after each new ownership generation is acquired. */ +export function onSpendLedgerOwnerAcquired(hook: (home: string) => void): void { + acquiredHooks.push(hook); +} const releasingHooks: Array<() => void> = []; let releasing = false; @@ -209,7 +215,11 @@ export function acquireSpendLedgerOwner(configDir = getConfigDir()): SpendLedger activeOwner = { home: prepared.home, database, references: 1 }; ownerGeneration += 1; - return leaseFor(activeOwner); + const owner = activeOwner; + for (const hook of acquiredHooks) { + try { hook(owner.home); } catch { /* one hook must not prevent acquisition */ } + } + return leaseFor(owner); } function leaseFor(owner: ActiveOwner): SpendLedgerOwnerLease { diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index e7ed5d31f0c..0ee1b3dc138 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -9,7 +9,7 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse; idle rows are deleted within about an hour while the proxy runs and at the next start after an offline period. SQLite secure_delete overwrites freed pages; its transient rollback journal is removed after each commit. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse; expired rows are deleted at lease acquisition and at most hourly while the proxy runs, including after write failures. A store that failed to open is disabled and cannot prune. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index c77369e70e0..8c28e550b91 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -31,11 +31,12 @@ beforeEach(() => { }); afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); registerThreadAffinityLiveState(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); -test("inactive and passive operations do not create a database", async () => { +test("inactive operations do not open and passive operations do not change the database", async () => { key(); expect(existsSync(path())).toBe(false); lease = acquireSpendLedgerOwner(); + const before = readFileSync(path()); await runPassiveRead(async () => { key(); recordAllCleared(); }); - expect(existsSync(path())).toBe(false); + expect(readFileSync(path()).equals(before)).toBe(true); }); test("bind survives release and touch reads the live timestamp at flush", () => { lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); const newer = Date.now(); @@ -206,3 +207,71 @@ test("adopted rows retain delete keys after anchor eviction", () => { lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); }); + +test("large worst-case batch flushes and restores newest rows", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const now = Date.now(); + for (let i = 0; i < 7000; i++) { + const k = codexConversationKeyFor(`batch-${i}`, `batch-${i}`); + const scope = `model-detour:shared:${String(i).padStart(4, "0")}${"é".repeat(254)}`; + recordThreadBind(k, scope, { scope, accountId: "account-a", fence, createdAt: now, lastUsedAt: now - 7000 + i }); + } + flushCodexAffinityStoreForTests(); lease.release(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(codexConversationKeyFor("batch-6999", "batch-6999"), Date.now())).toHaveLength(1); + const db = new Database(path()); expect(db.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 2048 }); db.close(); +}); + +test("scoped lookup and account delete reach rows beyond the adoption limit", async () => { + const affinity = await import("../../src/codex/routing/thread-affinity"); + registerThreadAffinityLiveState(); lease = acquireSpendLedgerOwner(); const fence = seed(); const k = key(); const now = Date.now(); + recordThreadBind(k, "shared", { scope: "shared", accountId: "account-a", fence, createdAt: now - 100, lastUsedAt: now - 100 }); + for (let i = 0; i < 70; i++) { const scope = `model-detour:shared:model-${i}`; recordThreadBind(k, scope, { scope, accountId: "account-a", fence, createdAt: now, lastUsedAt: now }); } + lease.release(); lease = acquireSpendLedgerOwner(); const restored = key(); + affinity.getThreadAffinityScopes(restored); + expect(affinity.getThreadAffinity(restored, "shared")?.accountId).toBe("account-a"); + affinity.deleteThreadAffinitiesForAccount(restored, "account-a"); lease.release(); lease = acquireSpendLedgerOwner(); key(); + const db = new Database(path()); expect(db.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 0 }); db.close(); +}); + +test("lease acquisition prunes expired rows without affinity access", () => { + lease = acquireSpendLedgerOwner(); bind(seed()); lease.release(); + const db = new Database(path()); db.query("UPDATE threads SET last_used_at=?").run(Date.now() - 25 * 60 * 60_000); db.close(); + lease = acquireSpendLedgerOwner(); + const check = new Database(path()); expect(check.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 0 }); check.close(); +}); + +test("poisoned store prunes on its hourly timer", () => { + const originalInterval = globalThis.setInterval; let tick: (() => void) | undefined; + globalThis.setInterval = ((fn: () => void, ms: number) => { if (ms === 3600000) tick = fn; return originalInterval(fn, ms); }) as typeof setInterval; + try { + lease = acquireSpendLedgerOwner(); bind(seed()); flushCodexAffinityStoreForTests(); + const warn = console.warn; console.warn = () => {}; + try { for (let i = 0; i < 16385; i++) recordAccountPinsCleared("other-account"); } finally { console.warn = warn; } + expect(tick).toBeDefined(); const realNow = Date.now; Date.now = () => realNow() + 25 * 60 * 60_000; + try { tick!(); } finally { Date.now = realNow; } + const check = new Database(path()); expect(check.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 0 }); expect(check.query("SELECT value FROM meta WHERE name='clean'").get()).toEqual({ value: 0 }); check.close(); + } finally { globalThis.setInterval = originalInterval; } +}); + +test("account deletion removes unadopted scopes and blocks pending adoption", async () => { + const affinity = await import("../../src/codex/routing/thread-affinity"); + registerThreadAffinityLiveState(); lease = acquireSpendLedgerOwner(); const fence = seed(); const k = key(); const now = Date.now(); + for (let i = 0; i < 70; i++) { const scope = `model-detour:shared:delete-${i}`; recordThreadBind(k, scope, { scope, accountId: "account-a", fence, createdAt: now, lastUsedAt: now }); } + lease.release(); lease = acquireSpendLedgerOwner(); const restored = key(); + affinity.deleteThreadAffinitiesForAccount(restored, "account-a"); + expect(affinity.getModelDetourAffinity(restored, "delete-69", "shared")).toBeUndefined(); + lease.release(); lease = acquireSpendLedgerOwner(); + const db = new Database(path()); expect(db.query("SELECT count(*) AS count FROM threads").get()).toEqual({ count: 0 }); db.close(); +}); + +test("SQLITE_FULL triggers one prune-only transaction before retrying the flush", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); + const original = Database.prototype.exec; let begins = 0; + Database.prototype.exec = function (sql: string) { + if (sql === "BEGIN IMMEDIATE" && ++begins === 1) throw Object.assign(new Error("injected full"), { code: "SQLITE_FULL" }); + return original.call(this, sql); + }; + try { flushCodexAffinityStoreForTests(); } finally { Database.prototype.exec = original; } + expect(begins).toBe(3); + lease.release(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); expect(k).toBeDefined(); +}); diff --git a/tests/helpers/codex-affinity-matrix-worker.ts b/tests/helpers/codex-affinity-matrix-worker.ts index 2feae2ff07d..4870a0ac057 100644 --- a/tests/helpers/codex-affinity-matrix-worker.ts +++ b/tests/helpers/codex-affinity-matrix-worker.ts @@ -31,8 +31,9 @@ if (action === "passive") { peekConversationStateIssuer(passiveKey); clearThreadAccountMap(); }); + const unchanged = before.equals(readFileSync(file)); lease.release(); - console.log(JSON.stringify({ unchanged: before.equals(readFileSync(file)) })); + console.log(JSON.stringify({ unchanged })); process.exit(0); } const key = codexPoolAffinityKey(new Headers({ "session-id": "restart-session", "thread-id": "restart-thread" }))!; diff --git a/tests/lib/spend-ledger-owner.test.ts b/tests/lib/spend-ledger-owner.test.ts index 49690a34ec0..e0091e7e839 100644 --- a/tests/lib/spend-ledger-owner.test.ts +++ b/tests/lib/spend-ledger-owner.test.ts @@ -8,6 +8,7 @@ import { SPEND_LEDGER_OWNER_FILENAME, mintSpendLedgerStorage, onSpendLedgerOwnerReleasing, + onSpendLedgerOwnerAcquired, assertStorageOwned, SPEND_LEDGER_RESTART_PARENT_ENV, SpendLedgerOwnerError, @@ -241,6 +242,15 @@ describe("in-process references and privacy", () => { for (const lease of leases.splice(0).reverse()) lease.release(); }); + test("acquired hooks run once per new owner and isolate failures", () => { + let enabled = true; let calls = 0; let later = 0; + onSpendLedgerOwnerAcquired(() => { if (enabled) { calls++; throw new Error("failure"); } }); + onSpendLedgerOwnerAcquired(() => { if (enabled) later++; }); + const first = acquireSpendLedgerOwner(); const second = acquireSpendLedgerOwner(); + expect([calls, later]).toEqual([1, 1]); first.release(); second.release(); + const third = acquireSpendLedgerOwner(); expect([calls, later]).toEqual([2, 2]); third.release(); enabled = false; + }); + test("final release runs isolated hooks with existing storage but refuses new ownership", () => { const first = acquireSpendLedgerOwner(); const second = acquireSpendLedgerOwner(); From a8963c3b6045013059e80ba7459b8d3b8fba2df4 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:54:13 +0900 Subject: [PATCH 11/21] fix(codex): touches before chunk prunes, full scope enumeration, prune after failed flush, passive-safe lease hooks --- src/codex/routing/thread-affinity-store.ts | 66 ++++++++++++------- .../codex-affinity-restart-matrix.test.ts | 6 +- .../codex-affinity-store.test.ts | 34 ++++++++++ 3 files changed, 79 insertions(+), 27 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index 4cb3fa5a15f..f8c7ea107dd 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -425,6 +425,8 @@ function pruneOnly(s: Scope): boolean { try { assertStorageOwned(s.token); s.db.exec("BEGIN IMMEDIATE"); + // Pending touches first, so a pin used since its last persisted timestamp is not evicted as idle. + if (!s.poisoned) applyTouches(s, s.db); prune(s.db, Date.now()); s.db.exec("COMMIT"); return true; @@ -434,39 +436,50 @@ function pruneOnly(s: Scope): boolean { return false; } } +/** Write the live last-used time of every touched row; UPDATE-only, so it never creates or revives a row. */ +function applyTouches(s: Scope, db: Database): void { + for (const t of s.threads.values()) { + const row = reader?.thread(t.key, t.scope); + if (row?.identityFence) + db.query( + "UPDATE threads SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?", + ).run( + row.lastUsedAt, + t.rowKey, + t.scope, + row.accountId, + row.identityFence, + ); + } + for (const t of s.issuers.values()) { + const row = reader?.issuer(t.key); + if (row) + db.query( + "UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?", + ).run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); + } +} +/** + * Commit the ordered log and touches. The final release flush (clean=true) commits work that was + * logged outside any passive read, so it runs even if the release happens inside one. + */ function flush(s: Scope, clean: boolean): boolean { - if (!s.db || s.disabled || s.poisoned || isPassiveRead()) return false; + if (!s.db || s.disabled || s.poisoned || (!clean && isPassiveRead())) return false; const db = s.db; for (let attempt = 0; attempt < 2; attempt++) try { assertStorageOwned(s.token); db.exec("BEGIN IMMEDIATE"); + // Touches go before the log as well as after it, so the pruning between chunks sees the live + // last-used times and cannot evict a pin that is in use but persisted with an old timestamp. + applyTouches(s, db); for (let i = 0; i < s.log.length; i++) { const op = s.log[i]!; if (op.args.length) db.query(op.sql).run(...op.args); else db.exec(op.sql); if ((i + 1) % 256 === 0) prune(db, Date.now()); } - // Read touches at flush time, after callers have updated the live entries. - for (const t of s.threads.values()) { - const row = reader?.thread(t.key, t.scope); - if (row?.identityFence) - db.query( - "UPDATE threads SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND scope=? AND account_id=? AND fence=?", - ).run( - row.lastUsedAt, - t.rowKey, - t.scope, - row.accountId, - row.identityFence, - ); - } - for (const t of s.issuers.values()) { - const row = reader?.issuer(t.key); - if (row) - db.query( - "UPDATE issuers SET last_used_at=max(last_used_at, ?) WHERE row_key=? AND account_id=? AND fence IS ?", - ).run(row.lastUsedAt, t.rowKey, row.accountId, row.fence ?? null); - } + // Again after the log: rows the log just inserted take their live last-used time. + applyTouches(s, db); prune(db, Date.now()); if (clean) db.exec("UPDATE meta SET value=1 WHERE name='clean'"); db.exec("COMMIT"); @@ -484,11 +497,14 @@ function flush(s: Scope, clean: boolean): boolean { if (attempt === 0 && error instanceof Error && "code" in error && error.code === "SQLITE_FULL" && pruneOnly(s)) continue; if (++s.failures >= 3) poison(s); else warning(); + // Deleting expired rows does not depend on the failed batch; keep the retention bound. + pruneOnly(s); return false; } return false; } -onSpendLedgerOwnerAcquired((home) => { scope(home); }); +// Open eagerly so the startup prune runs without traffic; a passive read never opens the store. +onSpendLedgerOwnerAcquired((home) => { if (!isPassiveRead()) scope(home); }); // Commit clean before closing; a later close failure cannot undo that commit. onSpendLedgerOwnerReleasing(() => { const s = current; @@ -533,10 +549,12 @@ export function adoptStoredThreadRows( const rk = rowKey(s, key, "conversation"); if (!rk || pending(s, rk, false) || (scopeName !== undefined && s.consulted.has(`t:${rk}:${scopeName}`))) return []; try { + // Enumerating every scope matters: routing decides account-wide release from this view. The + // table itself is trimmed to 2048 rows at open and on every flush, which bounds the read. const rows = s .db!.query( scopeName === undefined - ? "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? ORDER BY last_used_at DESC, scope LIMIT 64" + ? "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? ORDER BY last_used_at DESC, scope LIMIT 2048" : "SELECT scope, account_id, fence, created_at, last_used_at FROM threads WHERE row_key=? AND scope=? LIMIT 1", ) .all(...(scopeName === undefined ? [rk] : [rk, scopeName])) as { diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts index 45b2901f4cb..01b6ded2120 100644 --- a/tests/codex-integration/codex-affinity-restart-matrix.test.ts +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -132,7 +132,7 @@ test("hourly timer prunes an idle open store", () => inHome(home => { expect(result.issuers.n).toBe(0); })); -test("adoption loads at most 64 newest scopes for a row key", () => inHome(home => { +test("adoption enumerates every stored scope for a row key", () => inHome(home => { seed(home); const db = new Database(join(home, names[0]!)); const row = db.query("SELECT row_key, account_id, fence, created_at, last_used_at FROM threads LIMIT 1").get() as { row_key: string; account_id: string; fence: string; created_at: number; last_used_at: number }; @@ -140,9 +140,9 @@ test("adoption loads at most 64 newest scopes for a row key", () => inHome(home for (let i = 0; i < 80; i++) insert.run(row.row_key, `model-detour:shared:model-${i}`, row.account_id, row.fence, row.created_at, row.last_used_at - i - 1); db.close(); const scopes = matrix(home, "scopes").scopes; - expect(scopes).toHaveLength(64); + expect(scopes).toHaveLength(81); expect(scopes).toContain("model-detour:shared:model-0"); - expect(scopes).not.toContain("model-detour:shared:model-79"); + expect(scopes).toContain("model-detour:shared:model-79"); })); test("restoration accepts a 512-byte model component and rejects 513 bytes", () => inHome(home => { diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 8c28e550b91..bd7d5750f46 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -275,3 +275,37 @@ test("SQLITE_FULL triggers one prune-only transaction before retrying the flush" lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); expect(k).toBeDefined(); }); + +test("a pin touched since its last flush survives a chunk prune at the row limit", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); flushCodexAffinityStoreForTests(); + const now = Date.now(); + const db = new Database(path()); const insert = db.query("INSERT INTO threads VALUES (?, ?, ?, ?, ?, ?)"); + db.exec("BEGIN"); for (let i = 0; i < 2047; i++) insert.run(`filler-${i}`, "legacy", "account-a", fence, now - 500, now - 500); db.exec("COMMIT"); db.close(); + recordThreadTouch(k, "legacy"); live!.lastUsedAt = now; + for (let i = 0; i < 256; i++) { const fresh = codexConversationKeyFor(`fresh-${i}`, `fresh-${i}`); recordThreadBind(fresh, "shared", { scope: "shared", accountId: "account-a", fence, createdAt: now, lastUsedAt: now }); } + flushCodexAffinityStoreForTests(); lease.release(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); +test("a failed flush still deletes expired rows", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); flushCodexAffinityStoreForTests(); + const db = new Database(path()); + db.query("INSERT INTO threads VALUES (?, ?, ?, ?, ?, ?)").run("expired-row", "legacy", "account-a", fence, Date.now() - 25 * 60 * 60_000, Date.now() - 25 * 60 * 60_000); + db.exec("CREATE TRIGGER fail_bind BEFORE INSERT ON threads WHEN NEW.account_id = 'boom' BEGIN SELECT RAISE(ABORT, 'injected'); END"); + db.close(); + recordThreadBind(codexConversationKeyFor("boom-family", "boom-family"), "legacy", { scope: "legacy", accountId: "boom", fence, createdAt: Date.now(), lastUsedAt: Date.now() }); + flushCodexAffinityStoreForTests(); + const check = new Database(path()); + expect((check.query("SELECT count(*) AS n FROM threads WHERE row_key = 'expired-row'").get() as { n: number }).n).toBe(0); + check.exec("DROP TRIGGER fail_bind"); check.close(); +}); +test("a lease acquired inside a passive read does not open the store", async () => { + await runPassiveRead(async () => { lease = acquireSpendLedgerOwner(); }); + expect(existsSync(path())).toBe(false); +}); +test("a release inside a passive read still commits the final flush", async () => { + lease = acquireSpendLedgerOwner(); bind(seed()); + await runPassiveRead(async () => { lease!.release(); }); + lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); + From 5ab255a0bbc1c2d39814ccd2ecc14bf0e9b12cb0 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:57:36 +0900 Subject: [PATCH 12/21] docs(codex): describe affinity pruning as attempted, with its exceptions --- structure/providers/codex-pool-affinity.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 0ee1b3dc138..30976de4b8d 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -9,7 +9,7 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse; expired rows are deleted at lease acquisition and at most hourly while the proxy runs, including after write failures. A store that failed to open is disabled and cannot prune. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. From 0af877babfb750a4f2437c016eb7042a4790296b Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 17:58:54 +0900 Subject: [PATCH 13/21] fix(codex): apply touches before every chunk prune --- src/codex/routing/thread-affinity-store.ts | 6 +++++- tests/codex-integration/codex-affinity-store.test.ts | 12 ++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index f8c7ea107dd..81cf7182256 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -476,7 +476,11 @@ function flush(s: Scope, clean: boolean): boolean { const op = s.log[i]!; if (op.args.length) db.query(op.sql).run(...op.args); else db.exec(op.sql); - if ((i + 1) % 256 === 0) prune(db, Date.now()); + // Touches before every chunk prune, so rows this chunk inserted carry their live last-used time. + if ((i + 1) % 256 === 0) { + applyTouches(s, db); + prune(db, Date.now()); + } } // Again after the log: rows the log just inserted take their live last-used time. applyTouches(s, db); diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index bd7d5750f46..e68f231cdfc 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -309,3 +309,15 @@ test("a release inside a passive read still commits the final flush", async () = expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); }); + +test("a pin bound and touched in the same pending batch survives a chunk prune", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); flushCodexAffinityStoreForTests(); + const now = Date.now(); + const db = new Database(path()); const insert = db.query("INSERT INTO threads VALUES (?, ?, ?, ?, ?, ?)"); + db.exec("BEGIN"); for (let i = 0; i < 2048; i++) insert.run(`filler-${i}`, "legacy", "account-a", fence, now - 500, now - 500); db.exec("COMMIT"); db.close(); + const k = bind(fence); recordThreadTouch(k, "legacy"); live!.lastUsedAt = now; + for (let i = 0; i < 255; i++) { const fresh = codexConversationKeyFor(`batch-${i}`, `batch-${i}`); recordThreadBind(fresh, "shared", { scope: "shared", accountId: "account-a", fence, createdAt: now, lastUsedAt: now }); } + flushCodexAffinityStoreForTests(); lease.release(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); +}); + From 7326977572e48d4c958da46128468ff29e320e64 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 18:00:15 +0900 Subject: [PATCH 14/21] chore: trim trailing blank lines --- devlog/_plan/261102_codex_affinity_persist/010_plan.md | 1 - devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md | 1 - devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md | 1 - devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md | 1 - devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md | 1 - devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md | 1 - devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md | 1 - .../261102_codex_affinity_persist/080_implementation_spec.md | 1 - structure/providers/codex-pool-affinity.md | 1 - tests/codex-integration/codex-affinity-store.test.ts | 1 - 10 files changed, 10 deletions(-) diff --git a/devlog/_plan/261102_codex_affinity_persist/010_plan.md b/devlog/_plan/261102_codex_affinity_persist/010_plan.md index af1b9bdf773..3eef8172862 100644 --- a/devlog/_plan/261102_codex_affinity_persist/010_plan.md +++ b/devlog/_plan/261102_codex_affinity_persist/010_plan.md @@ -143,4 +143,3 @@ Issue: https://github.com/lidge-jun/opencodex/issues/8091 (same protection class as `admin-api-token`), and the snapshot holds only digests of keys derived from it. - Two processes sharing one config home: exclusive create means the loser keeps a process-local key; acceptable and documented. - Write amplification: at most one coalesced write per second while traffic flows, same as the Claude store. - diff --git a/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md index d63a2316b8a..cfb69bc1020 100644 --- a/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/020_plan_revision.md @@ -149,4 +149,3 @@ reservation map). Locale copies found per locale (fr, ko, zh-cn, and any others) Focused new + affected tests, `bun run typecheck`, `bun run structure:check`, `bun run privacy:scan`, docs-site build, `bun scripts/test.ts --changed=dev --parallel=1`, red-on-old recorded. Before the PR: a dedicated Terra security review (secret file permissions/ACL, key scope, cross-restart linkability, snapshot contents, deletion on pool/config reset); its verdict goes in the PR description. - diff --git a/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md index 0a163b5c671..6be70fbd03d 100644 --- a/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/030_plan_revision.md @@ -96,4 +96,3 @@ snapshot, an already scheduled write, passive getters and observer calls, and ex Dedicated Terra security review before the PR; per MAINTAINERS.md security-sensitive changes also need explicit maintainer security review — the PR says so and requests it. - diff --git a/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md index c7491bcfc2a..0c24d47275b 100644 --- a/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/040_plan_revision.md @@ -92,4 +92,3 @@ record's current generation AND `credential.chatgptAccountId` both match (as `ac structure/providers/openai-tiers.md "Durable Codex pool affinity" describes the SQLite store (file, schema, secret, fences, queue, touch semantics, bounds, ownership, contract limits). Rest as in 010/030. - diff --git a/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md index c762f6e4f4d..52e8b476845 100644 --- a/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/050_plan_revision.md @@ -77,4 +77,3 @@ Windows test: arm, use, reset, `removeTreeWithRetry` succeeds. `tests/ci-workflows/test-home-guard.test.ts` (the new writer refuses before any filesystem mutation when pointed at the real home), `tests/lab/core-lab-boundary.test.ts` (new imports do not reach Lab; startServer window scan), config-ownership tests (INITIAL list), file-size ratchet, test-layout guards, plus the 020/030/040 test list adjusted: cross-process cases become single-writer cases above. - diff --git a/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md index 747af52310d..2febd334108 100644 --- a/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/060_plan_revision.md @@ -66,4 +66,3 @@ generation and `assertStorageOwned(token)` refuses it after any release/reacquir - Windows: start, use, stop (release hook closes), `removeTreeWithRetry` succeeds. - Everything else from 020/030/040 (fences, unknown issuer via responses/compact/retry, home switch, passive, bounds, privacy). - Guards: test-home guard (real home refused before ownership init), core-lab boundary, config-ownership tests, file-size ratchet, test layout. - diff --git a/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md b/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md index a41b1448550..cb9316439c9 100644 --- a/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md +++ b/devlog/_plan/261102_codex_affinity_persist/070_plan_revision.md @@ -42,4 +42,3 @@ cannot pass `assertStorageOwned`. Change (minimal, in spend-ledger-owner.ts): and linked from it. - Unsupported version/shape tests assert the application wrote nothing (no schema/meta change, no row change); they do not claim byte-for-byte identity where SQLite's own hot-journal recovery may touch the file. - diff --git a/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md b/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md index afc55c18519..44f5913513a 100644 --- a/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md +++ b/devlog/_plan/261102_codex_affinity_persist/080_implementation_spec.md @@ -185,4 +185,3 @@ red-on-old recorded. PR gets `ci:full` (subprocess tests, docs, on-disk contract `src/codex/routing/selection.ts` (Lane C), `src/codex/auth-context.ts` and the affinity log label (Lane D), `src/server/claude-*` and `src/server/caller-session-identity.ts` (Lane A), `src/codex/routing.ts` and `src/server/index.ts` (size caps), the ownership API fix (#8093). - diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 30976de4b8d..18466bc20cb 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -12,4 +12,3 @@ The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Wind The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. - diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index e68f231cdfc..5d896bfee4d 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -320,4 +320,3 @@ test("a pin bound and touched in the same pending batch survives a chunk prune", flushCodexAffinityStoreForTests(); lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); }); - From 267f40706616bfa15f244a721fe2626c719d204d Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 18:46:44 +0900 Subject: [PATCH 15/21] =?UTF-8?q?fix(codex):=20address=20CodeRabbit=20revi?= =?UTF-8?q?ew=20=E2=80=94=20unscoped=20adoption=20marker,=20relative=20tes?= =?UTF-8?q?t=20timestamps,=20acquired-hook=20and=20restart-eligibility=20d?= =?UTF-8?q?ocs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../content/docs/fr/reference/cli/providers-accounts.md | 2 +- .../content/docs/fr/reference/configuration/providers.md | 4 ++-- .../content/docs/ja/reference/cli/providers-accounts.md | 2 +- .../content/docs/ja/reference/configuration/providers.md | 4 ++-- .../content/docs/ko/reference/cli/providers-accounts.md | 2 +- .../content/docs/ko/reference/configuration/providers.md | 4 ++-- .../src/content/docs/reference/cli/providers-accounts.md | 2 +- .../content/docs/reference/configuration/providers.md | 4 ++-- .../content/docs/ru/reference/cli/providers-accounts.md | 2 +- .../content/docs/ru/reference/configuration/providers.md | 4 ++-- .../content/docs/tr/reference/cli/providers-accounts.md | 2 +- .../content/docs/tr/reference/configuration/providers.md | 4 ++-- .../docs/zh-cn/reference/cli/providers-accounts.md | 2 +- .../docs/zh-cn/reference/configuration/providers.md | 4 ++-- .../docs/zh-tw/reference/cli/providers-accounts.md | 2 +- .../docs/zh-tw/reference/configuration/providers.md | 4 ++-- src/codex/routing/thread-affinity-store.ts | 9 ++++++++- structure/transports/responses-spend.md | 4 ++++ tests/codex-integration/codex-affinity-restart.test.ts | 8 ++++++-- 19 files changed, 42 insertions(+), 27 deletions(-) diff --git a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md index 185723f8efb..6652fc2283b 100644 --- a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md @@ -230,7 +230,7 @@ cet état et quitte toujours 0. `--json` renvoie : Sélectionne un compte Codex, un compte OAuth ou une clé API existant. Pour `openai`, `main` sélectionne la connexion Codex App. Une sélection en mode Codex Pool efface l'affinité et s'applique à la requête suivante, -dont un provenant d'une tâche visible existante . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; +dont un provenant d'une tâche visible existante . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; Le mode direct continue d’utiliser les informations d’identification principales de l’appelant. Commutation proactive basée sur l'utilisation, 401/403 réauthentification, 429/retry-after temps de recharge, exclusion et pré-sortie 429/402 échec la récupération peut sélectionner ultérieurement un autre compte Pool éligible. Ces chemins de récupération restent actifs lorsque diff --git a/docs-site/src/content/docs/fr/reference/configuration/providers.md b/docs-site/src/content/docs/fr/reference/configuration/providers.md index 7ef4a918b3b..9897783dcea 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/fr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Après une inscription ou une connexion OAuth dans l’interface, une boîte de | `activeCodexAccountPinned?` | `string` | — | Identifiant du compte du dernier opérateur sélectionné manuellement. Lorsqu'il est défini, un niveau `codexAccountPriorities` supérieur ne peut pas le préempter jusqu'à ce que la broche soit libérée par drainage, exclusion, suppression ou un failover/promotion explicite. Un mouvement circulaire ordinaire à l’intérieur du niveau plafonné ne le libère pas. L'écriture d'une entrée `codexAccountPriorities` libère également le pin, donc un pin créé avant qu'un ordre n'existe ne peut pas surpasser un ensemble par la suite. `GET /api/codex-auth/active` indique à la fois si le compte effectif est épinglé (`pinned`) et le compte portant le plafond (`pinnedAccountId`). | | `autoSwitchThreshold?` | `number` | `80` | Seuil d'utilisation pour la commutation proactive. `quota` peut réévaluer les requêtes non liées lors de leur prochaine requête. Les tâches liées conservent leur compte au-delà du seuil par défaut (`pool.cacheAffinity`) jusqu'à ce que ce compte soit épuisé ou ne puisse plus servir, et ne basculent alors que vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour réévaluer aussi les tâches liées à ce seuil. `fill-first` ne l'utilise que comme seuil d'évacuation pour l'affectation des requêtes non liées ; `round-robin` ne l'utilise que pour les nouvelles affectations non liées : tant qu'un compte admissible reste sous son seuil effectif, il ignore ceux qui l'atteignent ou le dépassent, sans jamais déplacer une tâche liée pour cette raison. Le score retient la plus élevée des fenêtres de quota connues sur 5 heures, une semaine ou 30 jours. `0` désactive uniquement la commutation proactive fondée sur l'utilisation, pas l'affectation des requêtes non liées ni la récupération après incident. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Seuils par compte remplaçant `autoSwitchThreshold` : identifiant → entier de `0` à `100`. Une entrée absente hérite du seuil global ; `0` désactive uniquement les déplacements fondés sur l’utilisation depuis ce compte. Le compte principal `__main__` est pris en charge. La carte du compte dans Codex Auth gère cette valeur. Activer la valeur spécifique au compte copie le seuil global actuel dans une valeur fixe. Cette valeur, y compris `0`, reste prioritaire après toute modification du seuil global. La désactiver envoie `threshold: null`, supprime l'entrée et rétablit l'héritage du seuil global actuel et de ses modifications futures. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | | `pool.cacheAffinity?` | `boolean` | `true` | Ordre d'affinité de cache pour les threads Codex liés, indépendant de `pool.kernel`. Activé par défaut ; omettre la clé ou la définir à `true` conserve la liaison, et une valeur autre que `false` est lue comme activée. Une liaison active prime sur la marge de quota : `quota` ne déplace pas le thread simplement parce que l'utilisation a franchi `autoSwitchThreshold`, car déplacer une conversation liée jette le cache d'invites isolé par compte. Le thread quitte encore le compte s'il ne peut plus servir — suspendu, inutilisable, exclu du plan, identifiants invalides, génération remplacée, TTL expiré, refus de quota 429/402, ou réellement épuisé (utilisation connue à 100 %) — et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Un compte dont l'utilisation est inconnue n'est jamais choisi comme destination d'une tâche liée. Si tous les comptes dépassent le seuil, la tâche liée reste, car aucune destination n'est meilleure. Définissez `false` pour rétablir la réaffectation au seuil, toujours sous la même contrainte de destination. L'affinité est un réordonnancement, pas un verrouillage. | | `accountPoolStickyLimit?` | `number` | `1` | Nombre d'affectations de tâches nouvelles ou non liées conservées sur une même sélection tournante avant de passer à la suivante ; le compteur avance lorsqu'une tâche est liée, et non après une réponse réussie en amont. Plage : 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Nombre d'échecs transitoires consécutifs avant le basculement des futures nouvelles sessions. Réglez `0` pour désactiver ce mécanisme. Pour les requêtes Responses ordinaires et les envois compacts natifs, les échecs avérés d'accessibilité DNS/TCP avant connexion sont suivis au niveau du couple fournisseur-hôte : ils n'affectent jamais l'état ni la temporisation du compte, l'affinité de tâche ou de session, la sélection du compte actif ou le routage du pool, et ne sont jamais comptabilisés dans ce seuil. | @@ -235,7 +235,7 @@ Utilisez **Codex Auth** dans le tableau de bord pour ajouter des comptes au grou métadonnées non secrètes ; les jetons d'accès et d'actualisation utilisent le magasin d'identifiants renforcé. Le routage du pool distingue l'affectation des requêtes nouvelles ou non liées, la commutation proactive fondée sur l'utilisation et la récupération après incident. Une tâche liée conserve normalement son affinité. Par défaut (`pool.cacheAffinity`), cette réaffectation attend que le compte lié soit épuisé ou ne puisse plus servir, et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour laisser `quota` la relier dès la requête suivante une fois le seuil franchi, toujours sous la même contrainte de destination. Un refus de quota 429/402, une pause, une invalidation des identifiants ou l'expiration du TTL libèrent encore la liaison immédiatement ; une série d'échecs transitoires (5xx et autres échecs hors quota) sert la requête sur un autre compte sans supprimer la liaison active. -Une requête non liée ne possède aucune liaison active à un compte . Les liaisons admissibles survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un +Une requête non liée ne possède aucune liaison active à un compte . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un autre compte admissible au sein de la même requête, même lorsque la commutation proactive fondée sur l'utilisation est désactivée. Les changements de compte préservent et relisent le contexte de la conversation, mais la réutilisation du cache d'invites du fournisseur entre plusieurs comptes n'est pas garantie et le cache peut devoir être réchauffé. diff --git a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md index 3d36b06cc96..4e6058915c7 100644 --- a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md @@ -180,7 +180,7 @@ Codex クライアント連携を無効にしても、既存のネイティブ `auto` は手動の選択を解除し、プールが自身の戦略で再び配置するようにします — ただし id が `auto` の Codex アカウントが存在する場合は完全一致の id が優先され、`ocx account clear ` が常に自動選択を復元します。Codex アカウントは id の代わりに `ocx account alias` で付けたエイリアスでも指定でき、`priority`、`pause`、`resume`、`clear-cooldown`、`remove`、`alias` でも同様です。Codex アカウントでは `auto`、`main`、`__main__` は大文字・小文字を区別せず予約語として扱われるため、エイリアスとして設定できません。OAuth アカウントと API キーの表示名には従来のルールが適用されます。 既存の Codex アカウント、OAuth アカウント、または API key を選びます。`openai` で `main` は Codex App ログインを -選択します。Codex Pool の選択は affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 +選択します。Codex Pool の選択は affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 不明なプロバイダーや id は終了コード 1 です。`--json` は次を返します。 **401/403** では、そのアカウントへのプロセスローカルな affinity を解除し、再認証を要求します。 **429** では `Retry-After` を尊重してアカウントの cooldown を開始し、affinity を解除したうえで、 diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index cf71d84dc52..19637d6ee29 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI で登録または OAuth ログインが完了すると、Models ページ | `codexAccountPriorities?` | `Record` | — | Codex pool のアカウント別選択順。アカウント ID → `-100` から `100` の整数で、**大きいほど先に使われ**、未設定は `0` です。これは eligibility ではなく順序の境界です。選択は適格なアカウントを、まだ quota に余裕がある最上位 tier に絞り込み、その tier の中を `accountPoolStrategy` が選びます。tier が飛ばされるのは、そのメンバー全員が自身の 0 以外の実効しきい値（アカウント別上書き、未設定ならグローバル値）以上、cooldown 中、soft-avoid、一時停止、または再認証待ちのときだけで、usage 不明が tier を drain させることはありません。順序付けが不適格なアカウントを選択可能にすることはなく、すでにアカウントが結び付いた thread を再 bind することもありません。メインの `__main__` も同じ条件で参加するため、Codex Desktop ログインを最後に使わせられます。エントリが 1 つもなければ挙動は従来どおりです。map が不正な場合は警告を出して順序付けを無効にします（config の修復処理は走りません）。`ocx account priority` と Codex Auth ページで管理します。 | | `autoSwitchThreshold?` | `number` | `80` | 使用量ベースのプロアクティブ切り替えしきい値。`quota` は未紐付けタスクの次のリクエストを再評価できます。紐付け済みタスクは既定（`pool.cacheAffinity`）ではしきい値を超えても同じアカウントを維持し、アカウントが使い切られるか処理できなくなったときだけ離れ、その場合も実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`pool.cacheAffinity: false` にするとしきい値で紐付け済みタスクも再評価します。`fill-first` は未紐付け割り当ての使い切り基準としてのみ使用し、`round-robin` は新規/未紐付けの割り当て時にのみ使用し、しきい値未満の適格アカウントがある間は実効しきい値以上のアカウントを除外し、この理由で紐付け済みタスクを移動しません。既知の 5 時間、週次、30 日 quota window の最大スコアを使います。`0` は使用量ベースの切り替えだけを無効にし、未紐付け割り当てや障害回復は無効にしません。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | `autoSwitchThreshold` のアカウント別上書き（アカウント ID → `0`〜`100` の整数）。未設定はグローバル値を継承し、`0` はそのアカウントからの使用量ベース切り替えだけを無効にします。メインの `__main__` も指定できます。Codex Auth の各アカウントカードで管理します。 上書きを有効にすると、現在のグローバルしきい値が固定のアカウント別値としてコピーされます。`0` を含む上書き値は、その後グローバル値が変更されても優先されます。無効にすると `threshold: null` を送信してエントリを削除し、現在のグローバルしきい値と今後の変更を継承する状態に戻ります。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けです。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けです。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | | `pool.cacheAffinity?` | `boolean` | `true` | 紐付け済み Codex スレッド向けの cache-affinity 順序。`pool.kernel` とは独立で、既定はオンです。不正な値はオンとして読みます。live な紐付けが quota 余裕より優先され、`quota` は使用量が `autoSwitchThreshold` を超えたという理由だけではスレッドを移しません。一時停止、使用不可、または実際に使い切られたアカウント（既知 usage 100%）では離れますが、実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`false` にするとしきい値での再紐付けに戻ります。affinity は固定ではなく並べ替えです。 | | `accountPoolStickyLimit?` | `number` | `1` | 1 回の round-robin 選択で次へ進む前に保持する新規/未紐付けタスク割り当て数。カウンターは上流の成功後ではなくタスクの紐付け時に増えます。範囲 1–100。`accountPoolStrategy` が `round-robin` のときのみ。 | | `upstreamFailoverThreshold?` | `number` | `3` |今後の新しいセッションがフェイルオーバーする前に一時的なエラーが連続して発生する。 `0` を無効に設定します。通常のResponses送信とネイティブcompact送信では、実証済みの接続前DNS/TCP到達不能障害はprovider-host単位で記録され、アカウントの健全性、アカウントのクールダウン、スレッド/セッションの親和性、アクティブアカウントの選択、Poolルーティングには影響せず、この閾値にもカウントされません。 | @@ -220,7 +220,7 @@ pool アカウントの追加と quota 更新はダッシュボードの **Codex 使い切られるか処理できなくなるまで再紐付けを延期し、離れるときは実際に quota 余裕があり usage が より低いアカウントへだけ移ります。フラグをオフにすると、`quota` はしきい値超過後の次のリクエストで 再紐付けできます。 -pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには有効なアカウントの紐付けがありません。対象の紐付けはプロキシの正常な再起動後も最長24時間の非活動期間まで保持されます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。出力前の **429/402** は使用量ベースの +pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには有効なアカウントの紐付けがありません。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。出力前の **429/402** は使用量ベースの 切り替えがオフでも同じリクエストで適格な代替アカウントへ 1 回再試行できます。アカウント変更後も会話 コンテキストは保持・再生されますが、アカウント間の provider prompt cache 再利用は保証されません。 一時停止したアカウントと quota metadata は表示されたままですが、自動切り替え、再試行/failover 選択、cooldown 復旧プローブ、手動有効化の対象外です。 diff --git a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md index dfdb335f3cd..2cf22c568da 100644 --- a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md @@ -298,7 +298,7 @@ Codex JSON 행에는 등록된 `id`와 같은 `slotId`, 그리고 `native-main` `auto`는 수동 선택을 지워 풀이 다시 자체 전략으로 작업을 배치하게 합니다 — 단 id가 `auto`인 Codex 계정이 있으면 정확한 id 일치가 우선되며 `ocx account clear `는 항상 자동 선택을 복원합니다. Codex 계정은 id 대신 `ocx account alias`로 지정한 별칭으로도 가리킬 수 있으며, `priority`, `pause`, `resume`, `clear-cooldown`, `remove`, `alias`에서도 마찬가지입니다. Codex 계정에서 `auto`, `main`, `__main__`은 대소문자 구분 없이 예약어이므로 별칭으로 지정할 수 없습니다. OAuth 계정과 API 키의 표시 이름에는 기존 규칙이 그대로 적용됩니다. 기존 Codex 계정, OAuth 계정 또는 API key를 선택합니다. `openai`에서 `main`은 Codex App 로그인을 -선택합니다. Codex Pool 선택은 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. +선택합니다. Codex Pool 선택은 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. 알 수 없는 프로바이더나 id는 종료 코드 1입니다. `--json`은 다음을 반환합니다. **401/403**이 발생하면 해당 계정의 프로세스 로컬 affinity를 해제하고 재인증을 요구합니다. **429**에서는 `Retry-After`를 준수해 계정 cooldown을 시작하고 affinity를 해제한 뒤, diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index d94d07da409..9d8c3bced16 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI에서 등록이나 OAuth 로그인을 마치면 Models 페이지로 이동 | `codexAccountPriorities?` | `Record` | — | Codex pool의 계정별 선택 순서. 계정 ID → `-100`부터 `100`까지의 정수이며 **값이 클수록 먼저** 쓰이고, 항목이 없으면 `0`입니다. 이는 eligibility 경계가 아니라 순서 경계입니다. 선택은 이미 적격한 계정들을 quota 여유가 남은 최상위 tier로 좁히고, 그 tier 안에서 `accountPoolStrategy`가 계정을 고릅니다. tier를 건너뛰는 경우는 그 구성원 전부가 자신의 0이 아닌 유효 임계값(계정별 재정의 또는 전역 기본값) 이상, cooldown, soft-avoid, 일시 중지 또는 재인증 대기일 때뿐이며, usage를 알 수 없다고 해서 tier가 소진되지는 않습니다. 순서는 부적격 계정을 선택 가능하게 만들지 않고, 이미 계정에 묶인 thread를 다시 bind하지도 않습니다. 메인 `__main__` 계정도 동일한 조건으로 참여하므로 Codex Desktop 로그인을 마지막에 쓰도록 둘 수 있습니다. 항목이 하나도 없으면 동작은 이전과 같습니다. map이 잘못된 경우 경고를 출력하고 순서 지정을 끕니다(config 복구는 하지 않습니다). `ocx account priority`와 Codex Auth 페이지에서 관리합니다. | | `autoSwitchThreshold?` | `number` | `80` | 사용량 기반 선제 전환 임계값입니다. `quota`는 바인딩 없는 작업의 다음 요청을 재평가할 수 있습니다. 바인딩된 작업은 기본값(`pool.cacheAffinity`)에서 이 임계값을 넘어도 계정을 유지하며, 해당 계정이 소진되었거나 더 이상 처리할 수 없을 때에만 떠나고, 그때도 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `pool.cacheAffinity: false`로 두면 임계값에서 바인딩된 작업도 재평가합니다. `fill-first`는 바인딩 없는 작업 배정의 소진 기준으로만 사용하며, `round-robin`은 새/바인딩 없는 작업 배정에만 사용하며 임계값 미만인 적격 계정이 남아 있는 동안 유효 임계값 이상인 계정을 건너뛰고 이 때문에 바인딩된 작업을 옮기지 않습니다. 알려진 5시간, 주간, 30일 quota window 중 가장 높은 점수를 씁니다. `0`은 사용량 기반 전환만 끄며 바인딩 없는 작업 배정이나 실패 복구는 끄지 않습니다. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 계정별 `autoSwitchThreshold` 재정의입니다(계정 ID → `0`~`100` 정수). 항목이 없으면 전역 값을 상속하고, `0`은 해당 계정에서 시작하는 사용량 기반 전환만 끕니다. 메인 `__main__` 계정도 지원합니다. Codex Auth의 계정 카드에서 관리합니다. 재정의를 켜면 현재 전역 임계값을 고정된 계정별 값으로 복사합니다. `0`을 포함한 재정의 값은 이후 전역 값이 바뀌어도 우선합니다. 끄면 `threshold: null`을 보내 항목을 삭제하고 현재 전역 임계값과 향후 변경 사항을 상속하는 상태로 돌아갑니다. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청입니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청입니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | | `pool.cacheAffinity?` | `boolean` | `true` | 바인딩된 Codex 스레드의 cache-affinity 순서입니다. `pool.kernel`과는 별개이며 기본값은 켜짐입니다. 잘못된 값은 켜진 것으로 읽습니다. live 바인딩이 quota 여유보다 우선하므로 `quota`는 사용량이 `autoSwitchThreshold`를 넘었다는 이유만으로 스레드를 옮기지 않습니다. 해당 계정이 일시 중지되었거나 사용할 수 없거나 실제로 소진된 경우(알려진 usage 100%)에는 떠나되, 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `false`로 두면 임계값 재바인딩이 복원됩니다. affinity는 고정이 아니라 재정렬입니다. | | `accountPoolStickyLimit?` | `number` | `1` | 한 round-robin 선택이 다음으로 넘어가기 전에 유지하는 새 작업/바인딩 없는 작업 배정 수입니다. 카운터는 업스트림 성공 뒤가 아니라 작업을 바인딩할 때 증가합니다. 범위 1–100이며 `accountPoolStrategy`가 `round-robin`일 때만 적용됩니다. | | `upstreamFailoverThreshold?` | `number` | `3` | 연속된 일시적 실패가 이 횟수에 도달하면 이후 새 세션은 failover됩니다. `0`으로 두면 비활성화됩니다. 일반 Responses와 네이티브 compact 전송에서 입증된 연결 전 DNS/TCP 도달 불가 실패는 provider-host 범위로 기록되며 계정 상태, 계정 쿨다운, 스레드/세션 선호도, 활성 계정 선택 또는 Pool 라우팅에 영향을 주지 않고 이 임계값에도 집계되지 않습니다. | @@ -221,7 +221,7 @@ pool 계정 추가와 quota 갱신은 대시보드의 **Codex Auth** 페이지 구분됩니다. 바인딩된 작업은 보통 affinity를 유지합니다. 기본값(`pool.cacheAffinity`)에서는 바인딩된 계정이 소진되었거나 더 이상 처리할 수 없을 때까지 재바인딩을 미루고, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 `quota`가 사용량 임계값을 넘은 뒤 다음 요청에서 재바인딩할 수 있습니다. 일시 중지, cooldown, 재인증, 실패 처리도 -독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청에는 live 계정 바인딩이 없습니다. 대상 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 +독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청에는 live 계정 바인딩이 없습니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 전환이 꺼져 있어도 같은 요청에서 적격 대체 계정으로 한 번 재시도할 수 있습니다. 계정이 바뀌어도 대화 문맥은 보존·재생되지만 계정 간 프로바이더 측 prompt cache 재사용은 보장되지 않아 다시 예열해야 할 수 있습니다. diff --git a/docs-site/src/content/docs/reference/cli/providers-accounts.md b/docs-site/src/content/docs/reference/cli/providers-accounts.md index 5401508adc8..e36abdad0d8 100644 --- a/docs-site/src/content/docs/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/reference/cli/providers-accounts.md @@ -846,7 +846,7 @@ that state and still exits 0. `--json` returns: Selects an existing Codex account, OAuth account, or API key. For `openai`, `main` selects the Codex App login. A Codex Pool selection clears affinity and applies to the next request, -including one from an existing visible task. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound, while in-flight requests keep their captured account. This controls Pool routing only; +including one from an existing visible task. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound, while in-flight requests keep their captured account. This controls Pool routing only; Direct mode keeps using the caller-owned/native main credential. Usage-based proactive switching, 401/403 reauthentication, 429/retry-after cooldowns, exclusion, and pre-output 429/402 failure recovery may later select another eligible Pool account. Those recovery paths remain active when diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 7a43a0013b7..33f3061d6cb 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -125,7 +125,7 @@ separate. Full request URLs such as `/api/v1/responses` are not provider base UR | `codexAccountPriorityFailback?` | `boolean` | `false` | Opt-in return of ongoing Codex Pool tasks to a usable higher-priority account under the `quota` strategy. Requests trigger a background quota refresh at most once every five minutes, including failed refresh attempts. When recent quota observations show headroom, the next request may rebind even if its current account is below the threshold. This explicit preference can discard a warm prompt cache; it does not change default cache affinity. No requests means no polling. Manual pins, model access, cooldowns, pauses and reauthentication exclusions still apply; in-flight requests keep their captured account. Requires the current source account's effective threshold to be positive (an override of 0 disables it; a positive override works with global 0); exact account routes and Direct mode are unchanged. | | `autoSwitchThreshold?` | `number` | `80` | Usage threshold for placing new/unbound work. `quota` can re-evaluate unbound tasks on their next request once usage crosses this threshold. Bound tasks keep their account past the threshold by default (`pool.cacheAffinity`); they leave only when that account is exhausted or otherwise cannot serve, and then only for an account with genuine quota headroom and strictly lower usage. Set `pool.cacheAffinity: false` to re-evaluate bound tasks at this threshold, still only onto such a destination. `fill-first` uses it only as the drain point for unbound assignment; `round-robin` uses it only when placing new/unbound work: it skips accounts at or over their effective threshold while any eligible account is still under it, and never moves a bound task for it. The score uses the hottest known 5h, weekly, or 30d quota window. `0` disables usage-based proactive switching only, not unbound assignment or failure recovery. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Per-account overrides for `autoSwitchThreshold`: account id → integer from `0` to `100`. An absent entry inherits the global value; `0` disables usage-driven switching only when that account is the source. `quota` uses the current source account's effective threshold for bound-task re-evaluation and unbound switching. Fill-first drain and selection-order tier headroom are evaluated against each account's own effective threshold; main-account pins and subagent fallback also use the relevant account's effective threshold. Supports the main `__main__` account. A malformed map is ignored with a warning. Managed by each account card on the Codex Auth page; disabling an override removes its entry. Enabling the override copies the current global threshold into a fixed account value. The override, including `0`, takes precedence over later global changes. Disabling it sends `threshold: null`, removes the entry, and restores inheritance of the current global threshold and future global changes. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | | `pool.cacheAffinity?` | `boolean` | `true` | Cache-affinity ordering for bound Codex threads, independent of `pool.kernel`. On by default; omitting the key or setting `true` keeps a bound task on its account until that account genuinely cannot serve. Only an explicit `false` restores threshold-based rebinding of bound tasks. A live binding outranks quota headroom: `quota` does not move the thread merely because usage crossed `autoSwitchThreshold`. The thread still leaves if that account cannot serve — paused, unusable, or genuinely exhausted (known usage at 100%) — and then only to an account with genuine quota headroom and strictly lower usage. Under either setting, an account with unknown usage is never chosen as a destination for a bound task, so when every account sits above the threshold the task stays put. Affinity is a reordering, not a pin. | | `pool.credentialGroups?` | `Array<{ id: string; credentials: string[]; note?: string }>` | `[]` | Accepted and validated, but not yet consumed by routing: declaring a group changes no routing decision until a consuming layer lands. Operator-declared quota domains: groups of credentials that demonstrably share one upstream usage limit. Members of one group count once toward available capacity, and a quota refusal inside a group is not answered by rotating to another member — the limit is the same, so the move would pay a cold prefix for zero new capacity. Declared groups speak only to quota; sharing a limit says nothing about prompt-cache compatibility, which is classified separately. Each member is written provider-qualified as `":"`, because a credential id means something only inside its provider; the provider segment accepts the usual aliases (`chatgpt:` and `codex:` both mean OpenAI). Group ids must be unique, `credentials` must be non-empty, and a credential may appear in at most one group — an ambiguous declaration is rejected on write and dropped with a warning on load rather than resolved by whichever group is listed first, since that would merge two unrelated quota domains. A malformed list costs only the grouping: `pool.kernel` and `pool.cacheAffinity` are preserved. Absent or empty means no declared grouping, so an unconfigured install behaves exactly as before. | | `accountPoolStickyLimit?` | `number` | `1` | New/unbound task assignments retained on one round-robin selection before advancing; the counter advances when a task is bound, not after an upstream success. Range 1–100. | @@ -1022,7 +1022,7 @@ Set `pool.cacheAffinity: false` to restore threshold rebinding of bound tasks, s a destination. Pause, cooldown, reauthentication, and quota refusals still release a live binding immediately. A transient-failure streak serves the request from another account while keeping the binding, and the task returns once that account serves again; after 10 minutes the binding is -released normally. An unbound request has no live account binding. Eligible bindings survive a graceful proxy restart for up to 24 hours idle; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. A pre-stream 429 or 402, or a 5xx response whose bounded +released normally. An unbound request has no live account binding. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. A pre-stream 429 or 402, or a 5xx response whose bounded body explicitly reports quota exhaustion, retries once on an eligible alternate account in the same request, even when usage-based proactive switching is off. The ordinary transient-5xx policy runs first, so a wrapped quota response may make up to three sends on the exhausted account before pool diff --git a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md index abf6cf8b6ff..d7b251a0c54 100644 --- a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md @@ -211,7 +211,7 @@ credential'а, это состояние тоже печатается, но к `auto` снимает ручной выбор, и пул снова распределяет работу по своей стратегии — если только аккаунт Codex буквально не имеет id `auto`: точное совпадение id выигрывает, а `ocx account clear ` всегда восстанавливает автоматический выбор. Аккаунт Codex можно указать по псевдониму, заданному через `ocx account alias`, вместо id; это относится и к `priority`, `pause`, `resume`, `clear-cooldown`, `remove` и `alias`. Для аккаунтов Codex значения `auto`, `main` и `__main__` зарезервированы независимо от регистра и не могут назначаться как псевдонимы. Для отображаемых имён аккаунтов OAuth и API-ключей действуют прежние правила. Выбирает существующий аккаунт Codex, OAuth-аккаунт или API-ключ. Для `openai` значение `main` -выбирает вход Codex App. Выбор Codex Pool очищает affinity и применяется к следующему запросу, включая запрос существующей видимой задачи. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры +выбирает вход Codex App. Выбор Codex Pool очищает affinity и применяется к следующему запросу, включая запрос существующей видимой задачи. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры или id завершаются с кодом 1. `--json` возвращает: При **401/403** локальная для процесса привязка к аккаунту сбрасывается и требуется повторная аутентификация. При **429** учитывается `Retry-After`, для аккаунта запускается cooldown, привязка сбрасывается, diff --git a/docs-site/src/content/docs/ru/reference/configuration/providers.md b/docs-site/src/content/docs/ru/reference/configuration/providers.md index 91c2d47f0e5..7d596518c85 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ru/reference/configuration/providers.md @@ -56,7 +56,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Порядок выбора для каждого аккаунта пула Codex: id аккаунта → целое число от `-100` до `100`, **больше — используется раньше**, отсутствие означает `0`. Это граница порядка, а не пригодности: выбор сужает уже подходящие аккаунты до самого высокого уровня, у которого ещё есть запас квоты, а внутри этого уровня аккаунт выбирает `accountPoolStrategy`. Уровень пропускается, только когда все его аккаунты достигли собственного ненулевого эффективного порога (переопределение аккаунта или глобальное значение), находятся в cooldown, под soft-avoid, на паузе или требуют повторной аутентификации; неизвестный usage никогда не исчерпывает уровень. Порядок не делает выбираемым непригодный аккаунт и не перепривязывает поток, у которого аккаунт уже есть. Основной аккаунт `__main__` участвует на равных — именно так логин Codex Desktop можно оставить на самый конец. Без записей поведение остаётся прежним. Некорректная map игнорируется с предупреждением в консоли (порядок отключается, восстановление config не запускается). Управляется через `ocx account priority` и страницу Codex Auth. | | `autoSwitchThreshold?` | `number` | `80` | Порог проактивного переключения по использованию. `quota` может повторно оценить следующий непривязанный запрос. Привязанная задача по умолчанию (`pool.cacheAffinity`) сохраняет аккаунт после порога, пока он не исчерпан и ещё может обслуживать запрос, и переносится только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы повторно оценивать привязанные задачи на этом пороге. `fill-first` использует его только как точку исчерпания для непривязанных назначений; `round-robin` применяет его только к новым назначениям: пока есть подходящий аккаунт ниже своего порога, аккаунты на пороге или выше пропускаются; привязанная задача из-за этого не переносится. Оценка берёт самое горячее из окон 5 часов, недели и 30 дней. `0` отключает только переключение по использованию, но не назначение непривязанных задач и не восстановление после сбоев. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Переопределения `autoSwitchThreshold` по аккаунтам: id → целое число `0`–`100`. Без записи наследуется глобальное значение; `0` отключает только переключения по использованию с этого аккаунта. Поддерживается основной аккаунт `__main__`. Управляется в карточке аккаунта на странице Codex Auth. Включение переопределения копирует текущий глобальный порог в фиксированное значение аккаунта. Переопределение, включая `0`, имеет приоритет и после изменения глобального порога. Отключение отправляет `threshold: null`, удаляет запись и восстанавливает наследование текущего глобального порога и его будущих изменений. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | | `pool.cacheAffinity?` | `boolean` | `true` | Порядок cache-affinity для привязанных потоков Codex, независимый от `pool.kernel`. По умолчанию включён; пропуск ключа или `true` сохраняет привязку, а любое значение кроме `false` читается как включённое. Живая привязка важнее запаса квоты: `quota` не переносит поток только потому, что usage пересёк `autoSwitchThreshold` — перенос живого разговора сбрасывает изолированный по аккаунту prompt cache. Поток всё равно уходит, если аккаунт не может обслуживать запрос — на паузе, непригоден, исключён планом, с недействительными credential'ами, со сменённой generation, с истёкшим TTL, при отказе квоты 429/402 или реально исчерпан (известный usage 100%) — и только на аккаунт со строго меньшим usage и реальным запасом квоты. Аккаунт с неизвестным usage никогда не выбирается как назначение для привязанной задачи. Если все аккаунты выше порога, привязанная задача остаётся: лучшего назначения нет. `false` возвращает перепривязку по порогу, всё равно только при той же проверке назначения. Affinity меняет порядок, а не закрепляет учётные данные. | | `accountPoolStickyLimit?` | `number` | `1` | Число назначений новых/непривязанных задач на одном выборе round-robin перед переходом дальше. Счётчик растёт при привязке задачи, а не после успеха upstream. Диапазон 1–100; только при `accountPoolStrategy` = `round-robin`. | | `upstreamFailoverThreshold?` | `number` | `3` | Сколько подряд transient failure допустить, прежде чем новые сессии начнут делать failover. `0` отключает эту логику. Для обычных Responses-запросов и нативных compact-отправок доказанные ошибки доступности DNS/TCP до соединения учитываются на уровне пары «провайдер, хост» и не влияют на здоровье аккаунта, кулдауны аккаунта, привязку потока/сессии, выбор активного аккаунта или маршрутизацию пула, а также не учитываются в этом пороге. | @@ -249,7 +249,7 @@ redirect'ов для обычных provider-request'ов реализована защищённом хранилище учётных данных аккаунтов Codex. Pool routing разделяет назначение новых/непривязанных задач, проактивное переключение по использованию и восстановление после сбоев. Привязанная задача обычно сохраняет affinity. По умолчанию (`pool.cacheAffinity`) эта перепривязка ждёт, пока -привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Подходящие привязки сохраняются после штатного перезапуска прокси при простое до 24 часов; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. Отказ **429/402** до вывода допускает одну попытку +привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. Отказ **429/402** до вывода допускает одну попытку на подходящем альтернативном аккаунте даже при выключенном переключении по использованию. Контекст разговора сохраняется и воспроизводится, но prompt cache провайдера между аккаунтами может не переиспользоваться и потребовать прогрева. diff --git a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md index 1e8a5307792..8513002ef58 100644 --- a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md @@ -244,7 +244,7 @@ yine de 0 ile çıkar. `--json` şunu döndürür: Mevcut bir Codex hesabını, OAuth hesabını veya API anahtarını seçer. `openai` için `main` Codex App girişini seçer. Bir Codex Havuzu seçimi bağlılığı temizler ve mevcut görünür bir görevden gelen istekler de dahil olmak -üzere bir sonraki isteğe uygulanır. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilirken devam eden istekler +üzere bir sonraki isteğe uygulanır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilirken devam eden istekler yakaladıkları hesabı tutar. Bu yalnızca Havuz yönlendirmesini denetler; Direct modu arayana ait/yerel ana kimlik bilgisini kullanmaya devam eder. Kullanıma dayalı proaktif geçiş, 401/403 yeniden kimlik doğrulaması, 429/retry-after diff --git a/docs-site/src/content/docs/tr/reference/configuration/providers.md b/docs-site/src/content/docs/tr/reference/configuration/providers.md index a9040c5f9da..4aba2f8c083 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/tr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Arayüzde kayıt veya OAuth girişi tamamlanınca Models sayfasını açan bir b | `activeCodexAccountPinned?` | `string` | — | Operatörün en son elle seçtiği hesap kimliği. Ayarlandığı sürece, pin tükenme, hariç tutma, silme veya açık bir yük devretme/yükseltme ile serbest bırakılana kadar daha yüksek bir `codexAccountPriorities` katmanı onu öncelikleyemez. Sınırlı katman içindeki sıradan round-robin hareketi onu serbest bırakmaz. Herhangi bir `codexAccountPriorities` girdisi yazmak da pini serbest bırakır, böylece bir sıra var olmadan önce yapılan bir pin daha sonra ayarlanan bir pinin önüne geçemez. `GET /api/codex-auth/active`, hem geçerli hesabın sabitlenip sabitlenmediğini (`pinned`) hem de tavanı taşıyan hesabı (`pinnedAccountId`) bildirir. | | `autoSwitchThreshold?` | `number` | `80` | Proaktif geçiş için kullanım eşiği. `quota`, bağımsız görevlerin bir sonraki isteğini yeniden değerlendirebilir. Bağlı görevler varsayılan olarak (`pool.cacheAffinity`) eşiğin ötesinde hesabını korur; hesap tükenene veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile bağlı görevler de bu eşikte yeniden değerlendirilir. `fill-first` bunu yalnızca bağımsız atama için tükenme noktası olarak kullanır; `round-robin` bunu yalnızca yeni/bağımsız atamalarda kullanır: etkin eşiğinin altında uygun bir hesap kaldıkça eşiğine ulaşmış veya aşmış hesapları atlar ve bağlı bir görevi bu nedenle taşımaz. Puan, bilinen en sıcak 5 saatlik, haftalık veya 30 günlük kota penceresini kullanır. `0`, yalnızca kullanıma dayalı proaktif geçişi devre dışı bırakır, bağımsız atamayı veya arıza kurtarmayı devre dışı bırakmaz. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Hesap başına `autoSwitchThreshold` geçersiz kılmaları: hesap kimliği → `0`–`100` tam sayısı. Kayıt yoksa genel değer devralınır; `0` yalnızca bu hesaptan kullanıma dayalı geçişi kapatır. Ana `__main__` hesabı desteklenir. Codex Auth hesap kartından yönetilir. Hesaba özel değeri etkinleştirmek, mevcut genel eşiği sabit bir hesap değerine kopyalar. `0` dahil bu değer, genel eşik daha sonra değişse de önceliğini korur. Devre dışı bırakmak `threshold: null` gönderir, kaydı siler ve mevcut genel eşikle gelecekteki değişikliklerini devralmayı yeniden etkinleştirir. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | | `pool.cacheAffinity?` | `boolean` | `true` | Bağlı Codex iş parçacıkları için önbellek bağlılığı sıralaması; `pool.kernel`'dan bağımsızdır. Varsayılan olarak açıktır; anahtarı atlamak veya `true` vermek bağlamayı korur ve `false` dışındaki bir değer açık okunur. Canlı bağlama kota payından öndedir: `quota`, kullanımın `autoSwitchThreshold`'u geçmesi nedeniyle iş parçacığını taşımaz, çünkü canlı bir konuşmayı taşımak hesaba özel istem önbelleğini atar. Hesap duraklatılmış, kullanılamaz, plandan dışlanmış, kimlik bilgisi geçersiz, generation değişmiş, TTL dolmuş, 429/402 kota reddi almış veya gerçekten tükenmişse (bilinen kullanım %100) iş parçacığı yine ayrılır ve yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Kullanımı bilinmeyen bir hesap bağlı bir görev için hedef olarak asla seçilmez. Tüm hesaplar eşiğin üzerindeyse bağlı görev yerinde kalır; daha iyi bir hedef yoktur. `false` eşiğe göre yeniden bağlamayı geri getirir; hedef kısıtı yine geçerlidir. Bağlılık bir sabitleme değil yeniden sıralamadır. | | `accountPoolStickyLimit?` | `number` | `1` | İlerlemeden önce bir round-robin seçiminde tutulan yeni/bağımsız görev atamaları; sayaç yukarı akış başarısından sonra değil, bir görev bağlandığında ilerler. Aralık 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Gelecekteki yeni oturumların yük devretmesinden önceki ardışık geçici arızalar. Devre dışı bırakmak için `0` ayarlayın. Düzenli Responses ve yerel sıkıştırma gönderimleri için kanıtlanmış bağlantı öncesi DNS/TCP erişilebilirlik arızaları sağlayıcı-ana bilgisayar düzeyinde izlenir: hesap sağlığını, hesap soğuma sürelerini, iş parçacığı/oturum bağlılığını, aktif hesap seçimini veya Havuz yönlendirmesini asla etkilemez ve bu eşiğe asla sayılmaz. | @@ -253,7 +253,7 @@ yönlendirmesi yeni/bağımsız atamayı, kullanıma dayalı proaktif geçişi v kurtarmayı ayırır. Bağlı bir görev normalde bağlılığı korur. Varsayılan olarak (`pool.cacheAffinity`) bu yeniden bağlama, bağlı hesap tükenene veya hizmet veremez hale gelene kadar bekler ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile `quota`, kullanım eşiği aşıldıktan sonraki isteğinde onu yeniden bağlayabilir; hedef kısıtı yine geçerlidir. 429/402 kota reddi, duraklatma, kimlik bilgisi geçersizliği ve TTL dolması bağlamayı hemen serbest bırakır; geçici arıza serisi (5xx ve diğer kota dışı arızalar) isteği başka bir hesapta sunar ama canlı bağlamayı silmez. Bağımsız bir -isteğin canlı hesap bağlaması yoktur. Uygun bağlar, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. Akış öncesi +isteğin canlı hesap bağlaması yoktur. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. Akış öncesi bir 429 veya 402, kullanıma dayalı proaktif geçiş kapalı olsa bile aynı istekte uygun alternatif bir hesapta bir kez yeniden dener. Hesap değişiklikleri görüşme bağlamını korur ve yeniden oynatır, ancak hesaplar arasında sağlayıcı tarafı diff --git a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md index 5de702e9901..11020b09199 100644 --- a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md @@ -200,7 +200,7 @@ OAuth 账号会显示为 `Account N`，而 plan/label 列会在 plan、屏蔽后 `auto` 会清除手动选择，让 Pool 重新按自身策略分配工作 — 但如果某个 Codex 账号的 id 恰为 `auto`，则精确 id 匹配优先；`ocx account clear ` 始终恢复自动选择。Codex 账号可以用 `ocx account alias` 设置的别名代替 id 来指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 和 `alias` 同样如此。对于 Codex 账号，`auto`、`main` 和 `__main__` 为保留字（不区分大小写），不能设为别名。OAuth 账号和 API 密钥的显示名称仍遵循原有规则。 选择已有的 Codex 账号、OAuth 账号或 API key。对 `openai` 而言，`main` 选择 Codex App 登录。 -Codex Pool 选择会清除 affinity，并从下一次请求开始生效，包括已有可见任务的请求；符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： +Codex Pool 选择会清除 affinity，并从下一次请求开始生效，包括已有可见任务的请求；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： 遇到 **401/403** 时，App 登录会清除该账户的进程内 affinity 并要求重新认证。 遇到 **429** 时，它会遵循 `Retry-After`、启动账户 cooldown、清除 affinity， 并可将请求切换到另一个符合条件的 Pool 账户。即使 `autoSwitchThreshold: 0`， diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index 43a48aa4935..adc0d3dfe5b 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -55,7 +55,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Codex pool 各账号的选择顺序：账号 ID → `-100` 到 `100` 的整数，**数值越大越先使用**，未设置即为 `0`。这是顺序边界而非资格边界：选择会把已经合格的账号收窄到仍有 quota 余量的最高 tier，再由 `accountPoolStrategy` 在该 tier 内挑选。只有当某个 tier 的所有成员都达到各自非零的有效阈值（账号覆盖值，未设置则使用全局值）、处于 cooldown、被 soft-avoid、已暂停或需要重新认证时，该 tier 才会被跳过；usage 未知不会让 tier 耗尽。顺序不会让不合格的账号变得可选，也不会重新绑定已经绑定账号的 thread。主账号 `__main__` 同样参与排序，因此可以让 Codex Desktop 登录账号最后才被用到。没有任何条目时，行为与以往完全一致。映射格式非法时会打印警告并关闭排序（不会触发 config 修复）。可通过 `ocx account priority` 和 Codex Auth 页面管理。 | | `autoSwitchThreshold?` | `number` | `80` | 基于用量的主动切换阈值。`quota` 可在下一次请求中重新评估未绑定任务。已绑定任务默认（`pool.cacheAffinity`）在越过阈值后仍保留账号，直到该账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号。将 `pool.cacheAffinity` 设为 `false` 才会在该阈值重新评估已绑定任务。`fill-first` 仅把它用作未绑定分配的耗尽点；`round-robin` 仅在新建/未绑定任务分配时使用它：只要仍有低于自身有效阈值的合格账号，就跳过达到或超过阈值的账号，且不会因此移动已绑定任务。分数取已知 5 小时、周或 30 天 quota window 的最高值。`0` 只关闭基于用量的主动切换，不关闭未绑定任务分配或故障恢复。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各账号对 `autoSwitchThreshold` 的覆盖：账号 ID → `0`–`100` 的整数。没有条目时继承全局值；`0` 只关闭从该账号发起的基于用量的切换。支持主账号 `__main__`。可在 Codex Auth 的账号卡片中管理。 启用覆盖时，会将当前全局阈值复制为固定的账号值。覆盖值（包括 `0`）在之后修改全局阈值时仍优先。禁用时发送 `threshold: null`，删除条目，并恢复继承当前全局阈值及其未来的更改。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | | `pool.cacheAffinity?` | `boolean` | `true` | 已绑定 Codex 线程的 cache-affinity 排序，独立于 `pool.kernel`。默认开启；省略该键或设为 `true` 即为开启，非法值视为开启。live 绑定优先于 quota 余量：`quota` 不会仅因用量越过 `autoSwitchThreshold` 就移动线程。账号暂停、不可用或真正耗尽（已知 usage 为 100%）时仍会离开，且只改绑到确有额度余量且 usage 严格更低的账号。用量未知的账号不会作为已绑定任务的改绑目标。设为 `false` 可恢复按阈值改绑。affinity 是重排而非钉死。 | | `accountPoolStickyLimit?` | `number` | `1` | 一次 round-robin 选择在推进前保留的新建/未绑定任务分配数。计数在任务绑定时增加，而不是在上游成功后增加。范围 1–100；仅当 `accountPoolStrategy` 为 `round-robin` 时生效。 | | `upstreamFailoverThreshold?` | `number` | `3` | 连续发生多少次瞬态故障后，后续新会话会切换到备用上游。设为 `0` 可禁用。对于常规 Responses 和原生 compact 发送，已证明的连接前 DNS/TCP 不可达故障按 provider-host 粒度记录，不影响账户健康、账户冷却、线程/会话亲和性、活动账户选择或 Pool 路由，也不会计入此阈值。 | @@ -249,7 +249,7 @@ OpenCodex 不会依据网关名称或 URL 猜测可信度。如果同一网关 请在仪表盘 **Codex Auth** 页面添加 pool account 并刷新 quota。配置只保存非 secret account metadata；access/refresh token 存放在加固的 Codex account credential store 中。Pool routing 分为新建/未绑定任务分配、基于用量的主动切换和故障恢复。已绑定任务通常保持 affinity。默认（`pool.cacheAffinity`）下，该重新绑定会等到绑定账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号；所有账号都高于阈值时，已绑定任务留在原账号。关闭该标志后，`quota` 可在超过阈值后的下一次请求中重新绑定，但仍只改绑到确有额度余量且 usage 严格更低的账号。暂停、cooldown、重新认证和故障处理也能独立清除或改变 -routing。未绑定请求没有 live 账号绑定。符合条件的绑定在代理正常重启后可保留，空闲时间最长为 24 小时；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。输出前的 +routing。未绑定请求没有 live 账号绑定。带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。输出前的 **429/402** 即使在关闭基于用量的主动切换时，也可在同一请求中对合格替代账号重试一次。 账号变化后会保留并重放对话上下文，但账号间的 provider prompt cache 不保证复用，可能需要重新预热。 暂停后仍会显示账号及其 quota metadata，但不会参与自动切换、重试/failover 选择、cooldown 恢复探测或手动激活。 diff --git a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md index 9bcc719a758..836be478529 100644 --- a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md @@ -158,7 +158,7 @@ Codex 池選擇套用於清除既有親和性後的下一個請求；進行中 `auto` 會清除手動選擇，讓池重新依自身策略分配工作 — 但若 Codex 帳號的 id 恰為 `auto`，則精確 id 比對優先；`ocx account clear ` 一律還原自動選擇。Codex 帳號可以用 `ocx account alias` 設定的別名代替 id 來指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 與 `alias` 亦然。對於 Codex 帳號，`auto`、`main` 和 `__main__` 為保留字（不區分大小寫），不能設為別名。OAuth 帳號與 API 金鑰的顯示名稱仍遵循原有規則。 -選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除親和性並套用於下一個請求，包含來自既有可見任務的請求；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 +選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除親和性並套用於下一個請求，包含來自既有可見任務的請求；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 `--json` 回傳： diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index 33bc739cc02..c2adfed1320 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -53,7 +53,7 @@ ocx models provider openrouter on | `activeCodexAccountId?` | `string` | — | 為下一個請求手動選擇的池帳號。選擇清除執行緒親和性；進行中的請求保留擷取的憑證。 | | `autoSwitchThreshold?` | `number` | `80` | 主動切換的用量閾值。`quota` 可在下一個請求時重新評估未綁定任務。綁定任務預設（`pool.cacheAffinity`）在越過閾值後仍會保留帳號，直到該帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號。將 `pool.cacheAffinity` 設為 `false` 才會在此閾值重新評估綁定任務。`fill-first` 僅將其用作未綁定指派的排空點；`round-robin` 僅在新／未綁定工作指派時使用它：只要仍有低於自身有效閾值的合格帳號，就略過達到或超過閾值的帳號，且不會因此移動已綁定任務。分數使用最熱的已知 5h、週或 30d 配額視窗。`0` 僅停用基於用量的主動切換，而非未綁定指派或失敗復原。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各帳號對 `autoSwitchThreshold` 的覆寫：帳號 ID → `0`–`100` 的整數。沒有項目時繼承全域值；`0` 只停用從該帳號發起的使用量主動切換。支援主帳號 `__main__`。可在 Codex Auth 的帳號卡片中管理。 啟用覆寫時，會將目前全域閾值複製為固定的帳號值。覆寫值（包括 `0`）在之後修改全域閾值時仍優先。停用時傳送 `threshold: null`，刪除項目，並恢復繼承目前全域閾值及其未來的變更。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | | `pool.cacheAffinity?` | `boolean` | `true` | 綁定 Codex 執行緒的 cache-affinity 排序，獨立於 `pool.kernel`。預設開啟；省略該鍵或設為 `true` 即為開啟，格式錯誤視為開啟。即時綁定優先於配額餘裕：`quota` 不會只因用量越過 `autoSwitchThreshold` 就移動執行緒。帳號暫停、無法使用或真正耗盡（已知用量 100%）時仍會離開，且只改綁到確有額度餘裕且用量嚴格更低的帳號。用量未知的帳號不會作為綁定任務的改綁目標。設為 `false` 可恢復依閾值重新綁定。親和性是重排而非釘死。 | | `accountPoolStickyLimit?` | `number` | `1` | 在前進一個 round-robin 選擇前保留的新／未綁定任務指派；計數器在任務綁定時前進，而非在上游成功後。範圍 1–100。 | | `upstreamFailoverThreshold?` | `number` | `3` | 未來新 session 容錯移轉前的連續暫時性失敗。設 `0` 停用。 | @@ -186,7 +186,7 @@ API-key 供應商可持有字面值金鑰或環境參考。OAuth 供應商使用 ## Codex 帳號池 -在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；符合條件的綁定在代理正常重啟後可保留，閒置時間最長為 24 小時；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 +在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index 81cf7182256..9f6b682b3a4 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -551,7 +551,13 @@ export function adoptStoredThreadRows( const s = usable(); if (!s) return []; const rk = rowKey(s, key, "conversation"); - if (!rk || pending(s, rk, false) || (scopeName !== undefined && s.consulted.has(`t:${rk}:${scopeName}`))) return []; + // A completed unscoped read (marker "*") already adopted every stored scope for this key. + if ( + !rk + || pending(s, rk, false) + || s.consulted.has(`t:${rk}:*`) + || (scopeName !== undefined && s.consulted.has(`t:${rk}:${scopeName}`)) + ) return []; try { // Enumerating every scope matters: routing decides account-wide release from this view. The // table itself is trimmed to 2048 rows at open and on every flush, which bounds the read. @@ -612,6 +618,7 @@ export function adoptStoredThreadRows( } for (const row of rows) consulted(s, `t:${rk}:${row.scope}`); if (scopeName !== undefined) consulted(s, `t:${rk}:${scopeName}`); + else consulted(s, `t:${rk}:*`); return result; } catch { warning(); diff --git a/structure/transports/responses-spend.md b/structure/transports/responses-spend.md index 9545a45c199..1cc505212a6 100644 --- a/structure/transports/responses-spend.md +++ b/structure/transports/responses-spend.md @@ -372,6 +372,10 @@ journal instead of inheriting figures. Final-release hooks run before that discard while existing storage tokens remain valid; each hook is isolated from errors in the others. Acquiring another lease or minting storage during these hooks is refused. Releasing a non-final reference or releasing the same lease twice runs no hooks. +Acquired hooks run once per new ownership generation, after the owner is recorded and before +`acquireSpendLedgerOwner` returns; a hook error does not block acquisition, and a nested lease on an +existing owner does not run them again. The durable Codex pool affinity store uses one to open and +prune at startup ([Durable Codex Pool Affinity](../providers/codex-pool-affinity.md)). The ledger records the ownership it was built under and proves that exact identity on every accounting read and change, so a handle kept across a release and a reacquire of the same directory is refused rather than resuming over writes another owner diff --git a/tests/codex-integration/codex-affinity-restart.test.ts b/tests/codex-integration/codex-affinity-restart.test.ts index 40d2450e8ab..babd27cac24 100644 --- a/tests/codex-integration/codex-affinity-restart.test.ts +++ b/tests/codex-integration/codex-affinity-restart.test.ts @@ -141,8 +141,11 @@ test("a legacy account without an identity falls back and removes its stored row expect(result.rows).toBe(0); })); -for (const [label, timestamp] of [["expired", Date.now() - 25 * 60 * 60_000], ["future", Date.now() + 60_000]] as const) { +// Offsets, not timestamps: each case computes its time when it runs, because the subprocess +// workers can take long enough on a loaded host for a load-time "future" value to become past. +for (const [label, offsetMs] of [["expired", -25 * 60 * 60_000], ["future", 60 * 60_000]] as const) { test(`${label} restored pin falls back and is deleted`, () => withSeededHome(home => { + const timestamp = Date.now() + offsetMs; const db = new Database(join(home, "codex-affinity.sqlite")); db.query("UPDATE threads SET last_used_at=?").run(timestamp); db.close(); @@ -218,8 +221,9 @@ test("resolver reuse persists its updated last-used time on release", () => with expect(result.row.timestamp).toBe(result.timestamp); })); -for (const [label, timestamp] of [["expired", Date.now() - 25 * 60 * 60_000], ["future", Date.now() + 60_000]] as const) { +for (const [label, offsetMs] of [["expired", -25 * 60 * 60_000], ["future", 60 * 60_000]] as const) { test(`${label} issuer is removed when read`, () => withSeededHome(home => { + const timestamp = Date.now() + offsetMs; const db = new Database(join(home, "codex-affinity.sqlite")); db.query("UPDATE issuers SET last_used_at=?").run(timestamp); db.close(); From 19d80965fe563dc8c0d746d4077cb82ee249c264 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 18:52:42 +0900 Subject: [PATCH 16/21] fix(compact): rescrub turn state on 401 replay and on caller-owned main alternates --- src/server/responses/compact.ts | 16 ++- ...ive-compaction-gateway-credentials.test.ts | 102 +++++++++++++++++- 2 files changed, 113 insertions(+), 5 deletions(-) diff --git a/src/server/responses/compact.ts b/src/server/responses/compact.ts index 212e3325fb6..07e772c0575 100644 --- a/src/server/responses/compact.ts +++ b/src/server/responses/compact.ts @@ -1,5 +1,6 @@ import { conversationStateAccountId, releaseRequestOwnedMainOnOutcome, isKeyedRequestOwnedMain } from "../../codex/request-owned-affinity"; import { CodexInstallationIdUnavailableError } from "../../codex/installation-id"; +import { MAIN_CODEX_ACCOUNT_ID } from "../../codex/account-id"; import { isolateCodexPoolClientMetadata } from "../../codex/forward-transport-headers"; import { captureAccountAutoPauseAttempt, wasAccountAutoPaused, wasPermanentAccountRefusal } from "../../oauth/account-auto-pause"; import { spendLedgerStorageErrorResponse } from "./spend-storage-error"; @@ -1395,6 +1396,8 @@ export async function handleResponsesCompact( outcomeCtx = replay.authCtx; compactProvider = replay.provider; headers = replay.headers; + const replayBinding = conversationStateBindingFromAuth(replay.authCtx, codexPoolAffinityKey(req.headers)); + if (replayBinding) applyAccountChangeConversationStateScrub({ body: compactBody, headers, bindingKey: replayBinding.bindingKey, servingAccountId: replayBinding.accountId, logCtx }); logCtx.accountLogLabel = codexAuthContextLogLabel(replay.authCtx, config); try { upstream = await sendCompactAttempt(compactProvider, headers, "single", authCtx); @@ -1439,10 +1442,15 @@ export async function handleResponsesCompact( return formatErrorResponse(502, "upstream_error", "Failed to connect to compact upstream"); }; const scrubCompactStateForAlternate = (alternateAuthCtx: CodexAuthContext, alternateHeaders: Headers): void => { - const binding = conversationStateBindingFromAuth( - alternateAuthCtx, - compactAlternateAffinityKey(authCtx, () => codexPoolAffinityKey(req.headers)), - ); + const affinityKey = compactAlternateAffinityKey(authCtx, () => codexPoolAffinityKey(req.headers)) + ?? codexPoolAffinityKey(req.headers); + const priorAccountId = conversationStateAccountId(authCtx); + // An unkeyed caller-owned main alternate has no binding of its own, but moving off the issuing + // account still makes the carried state foreign. Scrub for this request only; nothing persists. + const binding = conversationStateBindingFromAuth(alternateAuthCtx, affinityKey) + ?? (alternateAuthCtx.kind === "main" && priorAccountId && affinityKey + ? { bindingKey: affinityKey, accountId: priorAccountId === MAIN_CODEX_ACCOUNT_ID ? "__caller_main__" : MAIN_CODEX_ACCOUNT_ID } + : null); if (!binding) return; for (const body of [raw, compactBody]) { applyAccountChangeConversationStateScrub({ diff --git a/tests/responses/native-compaction-gateway-credentials.test.ts b/tests/responses/native-compaction-gateway-credentials.test.ts index 95baea53637..035a6fe1bee 100644 --- a/tests/responses/native-compaction-gateway-credentials.test.ts +++ b/tests/responses/native-compaction-gateway-credentials.test.ts @@ -1,13 +1,17 @@ import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test"; import * as registry from "../../src/providers/registry"; +import * as authContext from "../../src/codex/auth-context"; import { mkdtempSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { createHash } from "node:crypto"; import { clearAccountNeedsReauth, clearAccountQuota } from "../../src/codex/auth-api"; import { clearCodexUpstreamHealth, clearThreadAccountMap } from "../../src/codex/routing"; +import { clearPoolRotationState } from "../../src/codex/pool-rotation"; +import { clearPoolRecoveryState } from "../../src/routing/probe-lease"; +import { setAccountQuotaFromParsed } from "../../src/codex/quota"; import { codexPoolAffinityKey } from "../../src/codex/auth-context"; -import { rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; +import { peekConversationStateIssuer, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; import { handleResponsesCompact } from "../../src/server/responses"; import { clearCompactHandoffRoutesForTests } from "../../src/server/responses/compact"; import { clearResponseStateForTests, clearResponseStateMemoryForTests } from "../../src/responses/state"; @@ -84,6 +88,9 @@ beforeEach(() => { clearAccountQuota(); clearCodexUpstreamHealth(); clearThreadAccountMap(); + clearPoolRotationState(); + clearPoolRecoveryState(); + setAccountQuotaFromParsed("work", { weeklyPercent: 10 }); clearResponseStateMemoryForTests(); writeFileSync(join(home, "codex-accounts.json"), JSON.stringify({ work: record("work-access", "work-refresh", "acc-work"), @@ -100,6 +107,8 @@ afterEach(() => { clearAccountQuota(); clearCodexUpstreamHealth(); clearThreadAccountMap(); + clearPoolRotationState(); + clearPoolRecoveryState(); clearResponseStateForTests(); if (previousOcxHome === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = previousOcxHome; @@ -146,3 +155,94 @@ describe("native compact gateway credential boundary", () => { } }); }); + +describe("native compact account-change replay", () => { + function request(key: string, file = false): Request { + return new Request("http://localhost/v1/responses/compact", { + method: "POST", + headers: { "content-type": "application/json", authorization: "Bearer caller-token", "chatgpt-account-id": "caller-account", "x-codex-parent-thread-id": key, "x-codex-turn-state": "old-turn" }, + body: JSON.stringify({ model: "gpt-5.5", previous_response_id: "old-response", conversation: "old-conversation", client_metadata: { "x-codex-turn-state": "old-turn" }, input: file ? [{ type: "message", role: "user", content: [{ type: "input_file", file_id: "file-fixture" }] }] : [{ role: "user", content: "history" }] }), + }); + } + function config(): OcxConfig { + return { ...gatewayConfig(), defaultProvider: "openai", providers: { openai: { + adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "pool", + } }, codexAccounts: [{ id: "work", label: "work" }] } as OcxConfig; + } + test.each(["other", "work"])("401 refresh preserves only same-account turn state, issuer=%s", async issuer => { + const req = request(`refresh-${issuer}`); + rememberConversationStateIssuer(codexPoolAffinityKey(req.headers)!, issuer); + const resolve = authContext.resolveCodexAuthContext; + const spy = spyOn(authContext, "resolveCodexAuthContext").mockImplementation((headers, cfg, mode, options) => { + const poolHeaders = new Headers(headers); poolHeaders.delete("authorization"); poolHeaders.delete("chatgpt-account-id"); + return resolve(poolHeaders, cfg, mode, { ...options, requestScopedMainCredential: false }); + }); + const states: Array = []; + globalThis.fetch = (async (input, init) => { + if (String(input).includes("auth.openai.com")) return Response.json({ access_token: "refreshed-access", refresh_token: "rotated-refresh", expires_in: 3600 }); + states.push(new Headers(init?.headers).get("x-codex-turn-state")); + return states.length === 1 ? Response.json({ error: "unauthorized" }, { status: 401 }) : Response.json({ object: "response.compaction", output: [] }); + }) as typeof fetch; + try { + const response = await handleResponsesCompact(req, config(), { model: "", provider: "" }); + expect(response.status).toBe(200); + expect(states).toEqual(issuer === "other" ? [null, null] : ["old-turn", "old-turn"]); + } finally { spy.mockRestore(); } + }); + + test.each([429, 503])("pool to caller main scrubs request on %i alternate", async status => { + const req = request(`alternate-${status}`); + const key = codexPoolAffinityKey(req.headers)!; + rememberConversationStateIssuer(key, "work"); + const resolve = authContext.resolveCodexAuthContext; + const spy = spyOn(authContext, "resolveCodexAuthContext").mockImplementation((headers, cfg, mode, options) => { + if (options?.excludeAccountId) return Promise.resolve({ kind: "main", accountId: null }); + const poolHeaders = new Headers(headers); poolHeaders.delete("authorization"); poolHeaders.delete("chatgpt-account-id"); + return resolve(poolHeaders, cfg, mode, { ...options, requestScopedMainCredential: false }); + }); + const calls: Array<{ headers: Headers; body: Record }> = []; + globalThis.fetch = (async (_input, init) => { + calls.push({ headers: new Headers(init?.headers), body: JSON.parse(String(init?.body)) }); + return calls.length === 1 ? Response.json({ error: { message: "unavailable" } }, { status, headers: { "retry-after": "600" } }) : Response.json({ object: "response.compaction", output: [] }); + }) as typeof fetch; + try { + const response = await handleResponsesCompact(req, config(), { model: "", provider: "" }); + expect(response.status).toBe(200); + expect(calls).toHaveLength(2); + expect(calls[0]!.headers.get("authorization")).toBe("Bearer work-access"); + expect(calls[1]!.headers.get("authorization")).toBe("Bearer caller-token"); + expect(calls[0]!.headers.get("x-codex-turn-state")).toBe("old-turn"); + expect(calls[0]!.body.previous_response_id).toBe("old-response"); + expect(calls[1]!.headers.get("x-codex-turn-state")).toBeNull(); + expect(calls[1]!.body.previous_response_id).toBeUndefined(); + expect(calls[1]!.body.conversation).toBeUndefined(); + expect(calls[1]!.body.client_metadata?.["x-codex-turn-state"]).toBeUndefined(); + expect(peekConversationStateIssuer(key)).toBe("work"); + } finally { spy.mockRestore(); } + }); + + test.each([429, 503])("file references prevent a caller-main alternate on %i", async status => { + const req = request(`file-${status}`, true); + const key = codexPoolAffinityKey(req.headers)!; + rememberConversationStateIssuer(key, "work"); + const resolve = authContext.resolveCodexAuthContext; + let alternateResolutions = 0; + const spy = spyOn(authContext, "resolveCodexAuthContext").mockImplementation((headers, cfg, mode, options) => { + if (options?.excludeAccountId) { alternateResolutions++; return Promise.resolve({ kind: "main", accountId: null }); } + const poolHeaders = new Headers(headers); poolHeaders.delete("authorization"); poolHeaders.delete("chatgpt-account-id"); + return resolve(poolHeaders, cfg, mode, { ...options, requestScopedMainCredential: false }); + }); + let sends = 0; + globalThis.fetch = (async () => { + sends++; + return Response.json({ error: { message: "unavailable" } }, { status, headers: { "retry-after": "600" } }); + }) as typeof fetch; + try { + const response = await handleResponsesCompact(req, config(), { model: "", provider: "" }); + expect(response.status).toBe(status); + expect(sends).toBe(1); + expect(alternateResolutions).toBe(0); + expect(peekConversationStateIssuer(key)).toBe("work"); + } finally { spy.mockRestore(); } + }); +}); From 31c2324d32c85f3b24f7ed7408b86c5d12848a77 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 19:16:24 +0900 Subject: [PATCH 17/21] fix(codex): keep request-owned keys out of the durable store; align with manual-switch issuer retention; memory wins after a lease gap; persist in unowned homes --- scripts/test-layout/layout.json | 2 +- src/codex/request-owned-affinity-key.ts | 11 ++ src/codex/request-owned-affinity.ts | 9 +- src/codex/routing/thread-affinity-store.ts | 103 +++++++++++++++--- src/codex/routing/thread-affinity.ts | 11 +- structure/providers/codex-pool-affinity.md | 6 +- .../providers/codex-request-owned-affinity.md | 4 +- .../codex-affinity-request-owned.test.ts | 86 +++++++++++++++ .../codex-affinity-restart-matrix.test.ts | 25 +++-- .../codex-affinity-store.test.ts | 3 +- tests/fixtures/test-layout-expected.json | 2 +- 11 files changed, 224 insertions(+), 38 deletions(-) create mode 100644 src/codex/request-owned-affinity-key.ts create mode 100644 tests/codex-integration/codex-affinity-request-owned.test.ts diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index fa8a4f43414..f4380feb870 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -706,7 +706,7 @@ "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", diff --git a/src/codex/request-owned-affinity-key.ts b/src/codex/request-owned-affinity-key.ts new file mode 100644 index 00000000000..72afaf4a4a4 --- /dev/null +++ b/src/codex/request-owned-affinity-key.ts @@ -0,0 +1,11 @@ +/** + * The namespace of caller-owned affinity keys, kept in a module with no imports so that stores + * which must refuse these keys (the durable Codex pool affinity store) can test for them without + * importing the routing modules that request-owned affinity itself depends on. + */ +export const REQUEST_OWNED_AFFINITY_KEY_PREFIX = "request-owned:v1:"; + +/** Identify namespace keys that must remain process-local. */ +export function isRequestOwnedAffinityKey(value: string): boolean { + return value.startsWith(REQUEST_OWNED_AFFINITY_KEY_PREFIX); +} diff --git a/src/codex/request-owned-affinity.ts b/src/codex/request-owned-affinity.ts index 514dc300c0b..a4adeb3190e 100644 --- a/src/codex/request-owned-affinity.ts +++ b/src/codex/request-owned-affinity.ts @@ -10,9 +10,11 @@ import { getThreadAffinity, getModelDetourAffinity, deleteThreadAffinity, delete import type { CodexQuotaScope } from "./routing/health-store"; import { classifyCodexUpstreamOutcome } from "./routing/cooldown-math"; import type { CodexAuthContext } from "./auth-context"; +import { isRequestOwnedAffinityKey, REQUEST_OWNED_AFFINITY_KEY_PREFIX } from "./request-owned-affinity-key"; +export { isRequestOwnedAffinityKey } from "./request-owned-affinity-key"; const key = randomBytes(32); -const prefix = "request-owned:v1:"; +const prefix = REQUEST_OWNED_AFFINITY_KEY_PREFIX; /** Derive the opaque key for this caller and conversation, if both exist. */ export function requestOwnedAffinityKey(headers: Headers): string | undefined { @@ -24,11 +26,6 @@ export function requestOwnedAffinityKey(headers: Headers): string | undefined { .digest("base64url"); } -/** Identify namespace keys that must remain process-local. */ -export function isRequestOwnedAffinityKey(value: string): boolean { - return value.startsWith(prefix); -} - /** Report a bind only when the selection-time entry names the selected account. */ export function honestAffinityDecision( decision: CodexAffinityDecision | undefined, diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index 9f6b682b3a4..b2ead0dfcc8 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -10,7 +10,7 @@ import { } from "node:fs"; import { resolve } from "node:path"; import { getConfigDir } from "../../config/paths"; -import { recordOwnedConfigPath } from "../../lib/config-ownership"; +import { initializeConfigOwnership, recordOwnedConfigPath } from "../../lib/config-ownership"; import { isPassiveRead } from "../../lib/passive-read"; import { assertStorageOwned, @@ -25,6 +25,7 @@ import { import { assertNotRealHomeUnderTest } from "../../lib/test-home-guard"; import { hardenSecretPath } from "../../lib/windows-secret-acl"; import { readCodexAccountRecord } from "../account-store"; +import { isRequestOwnedAffinityKey } from "../request-owned-affinity-key"; import { setCodexConversationAnchorObserver } from "../lineage"; import type { CodexQuotaScope } from "./health-store"; @@ -52,6 +53,11 @@ type LiveReader = { issuer( key: string, ): { accountId: string; fence?: string; lastUsedAt: number } | undefined; + /** Every live pin and issuer, for re-persisting memory into a newly opened database. */ + forEach( + thread: (key: string, scope: string, entry: { accountId: string; identityFence?: string; createdAt: number; lastUsedAt: number }) => void, + issuer: (key: string, entry: { accountId: string; fence?: string; lastUsedAt: number }) => void, + ): void; clearLive(): void; }; type Op = { @@ -95,7 +101,12 @@ const BASE_SCOPES_EXHAUSTIVE: Exclude< void BASE_SCOPES_EXHAUSTIVE; let current: Scope | undefined; let observedHome: string | undefined; -let hadScope = false; +/** Whether an earlier scope in this process actually opened (and so may have persisted) the database. */ +let hadPersistentScope = false; +/** Pins or issuers changed while no lease was held; the next scope re-persists memory. */ +let changedWithoutLease = false; +/** Anchors of the last retired scope, handed to the next scope of the same home. */ +let carried: { home: string; anchors: Scope["anchors"] } | undefined; let reader: LiveReader | undefined; export function registerCodexAffinityLiveState(value: LiveReader): void { @@ -110,6 +121,9 @@ function retire(): void { const s = current; if (!s) return; current = undefined; + // Anchors are process-local facts about this home's conversations; the next scope of the same + // home needs them to re-derive row keys for pins that are still live in memory. + if (s.db) carried = { home: s.home, anchors: s.anchors }; if (s.timer) clearTimeout(s.timer); if (s.pruneTimer) clearInterval(s.pruneTimer); try { @@ -134,8 +148,12 @@ function open(s: Scope): void { let db: Database | undefined; try { assertNotRealHomeUnderTest(s.home); + // An owned home must list these files before they exist, so uninstall removes them. A home with + // no ownership manifest (one that predates ownership tracking) cannot record anything, and + // uninstall refuses to remove anything from it; like the spend-ledger owner database, the store + // still works there rather than silently losing persistence. for (const suffix of ["", "-journal", "-wal", "-shm"]) - if (!recordOwnedConfigPath(s.home, path + suffix)) + if (!recordOwnedConfigPath(s.home, path + suffix) && initializeConfigOwnership(s.home)) throw new Error("ownership"); try { closeSync(openSync(path, "wx", 0o600)); @@ -270,6 +288,7 @@ function scope(acquiredHome?: string): Scope | undefined { // A lease ends a store scope; only a change of home invalidates live maps. if (observedHome !== undefined && observedHome !== home) { retire(); + carried = undefined; reader?.clearLive(); } observedHome = home; @@ -292,8 +311,6 @@ function scope(acquiredHome?: string): Scope | undefined { } catch { return undefined; } - if (hadScope) reader?.clearLive(); - hadScope = true; const s: Scope = { home, generation, @@ -304,21 +321,58 @@ function scope(acquiredHome?: string): Scope | undefined { log: [], threads: new Map(), issuers: new Map(), - anchors: new Map(), + anchors: carried?.home === home ? carried.anchors : new Map(), threadKeys: new Map(), issuerKeys: new Map(), consulted: new Set(), scheduled: false, }; + carried = undefined; current = s; open(s); + if (s.db) { + if (hadPersistentScope && changedWithoutLease) persistLiveState(s); + changedWithoutLease = false; + hadPersistentScope = true; + } + return s; +} +/** + * The scope for a change to which pins or issuers exist. With no scope open after an earlier one + * persisted, the change happened while no lease was held, so the next scope must re-persist memory. + */ +function structuralScope(): Scope | undefined { + const s = usable(); + if (!s && !current && hadPersistentScope && !isPassiveRead()) changedWithoutLease = true; return s; } +/** + * A later scope in the same process makes the database equal to memory when memory changed while + * no lease was held (a late bind, delete or issuer write after a + * release). Memory is what this process routes with, so it wins: every stored row is dropped and + * every live pin and issuer with a known anchor is written again. A pin deleted during the gap is + * therefore not revived by a later restart, and a pin rebound during the gap is saved as rebound. + * Live entries without a known anchor stay memory-only, like every other unanchored key. + */ +function persistLiveState(s: Scope): void { + s.threadKeys.clear(); + s.issuerKeys.clear(); + enqueue(s, { kind: "clearAll", sql: "DELETE FROM threads; DELETE FROM issuers", args: [] }); + reader?.forEach( + (key, scopeName, entry) => { + if (!entry.identityFence || entry.accountId === "__main__") return; + recordThreadBind(key, scopeName, { scope: scopeName, accountId: entry.accountId, fence: entry.identityFence, createdAt: entry.createdAt, lastUsedAt: entry.lastUsedAt }); + }, + (key, entry) => recordIssuerSet(key, entry.accountId, entry.fence ?? null, entry.lastUsedAt), + ); +} function usable(): Scope | undefined { const s = scope(); return s && !s.disabled && !s.poisoned && s.db ? s : undefined; } function observeAnchor(key: string, family: string, thread: string): void { + // Caller-owned keys are process-local by contract (structure/providers/codex-request-owned-affinity.md). + if (isRequestOwnedAffinityKey(key)) return; const s = usable(); if (!s) return; const now = Date.now(); @@ -337,6 +391,9 @@ function rowKey( key: string, kind: "conversation" | "issuer", ): string | undefined { + // The single choke point for every bind, touch, delete, adoption and issuer path: a request-owned + // key never has a durable row key, so nothing for it is ever written or restored. + if (isRequestOwnedAffinityKey(key)) return undefined; const a = s.anchors.get(key); if (!a || Date.now() - a.at > TTL) { s.anchors.delete(key); @@ -525,7 +582,7 @@ function pending(s: Scope, row: string, issuer: boolean): boolean { (issuer ? op.kind === "clearIssuers" || (op.kind === "issuerDelete" && op.key === row) - : op.kind === "deleteKey" && op.key === row), + : op.kind === "clearThreads" || (op.kind === "deleteKey" && op.key === row)), ); } function consulted(s: Scope, key: string): boolean { @@ -678,7 +735,7 @@ export function recordThreadBind( scopeName: string, row: StoredThreadRow, ): void { - const s = usable(); + const s = structuralScope(); if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -708,7 +765,7 @@ export function recordThreadTouch(key: string, scopeName: string): void { scheduleTouch(s); } export function recordThreadDelete(key: string, scopeName?: string): void { - const s = usable(); + const s = structuralScope(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -734,7 +791,7 @@ export function recordThreadDelete(key: string, scopeName?: string): void { ); } export function recordThreadAccountDeleted(key: string, id: string): void { - const s = usable(); + const s = structuralScope(); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -746,7 +803,7 @@ export function recordThreadAccountDeleted(key: string, id: string): void { enqueue(s, { kind: "deleteKeyAccount", key: rk, sql: "DELETE FROM threads WHERE row_key=? AND account_id=?", args: [rk, id] }); } export function recordAccountPinsCleared(id: string): void { - const s = usable(); + const s = structuralScope(); if (s) { for (const [key, entry] of s.threadKeys) { for (const [scopeName, account] of entry.scopes) if (account === id) entry.scopes.delete(scopeName); @@ -760,7 +817,7 @@ export function recordAccountPinsCleared(id: string): void { } } export function recordAllCleared(): void { - const s = usable(); + const s = structuralScope(); if (s) { s.threadKeys.clear(); s.issuerKeys.clear(); enqueue(s, { @@ -770,13 +827,25 @@ export function recordAllCleared(): void { }); } } +/** A manual account switch clears pins but keeps the issuers of carried state. */ +export function recordThreadsCleared(): void { + const s = structuralScope(); + if (s) { + s.threadKeys.clear(); + enqueue(s, { kind: "clearThreads", sql: "DELETE FROM threads", args: [] }); + } +} +/** Test seam: feed the anchor observer directly, as a future derivation path might. */ +export function noteCodexAffinityAnchorForTests(key: string, family: string, thread: string): void { + observeAnchor(key, family, thread); +} export function recordIssuerSet( key: string, id: string, fence: string | null, lastUsedAt: number, ): void { - const s = usable(); + const s = structuralScope(); if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; @@ -798,7 +867,7 @@ export function recordIssuerTouch(key: string): void { scheduleTouch(s); } export function recordIssuerDelete(key: string): void { - const s = usable(); + const s = structuralScope(); if (!s) return; const rk = rowKey(s, key, "issuer"); if (rk) { @@ -812,7 +881,7 @@ export function recordIssuerDelete(key: string): void { } } export function recordIssuersCleared(): void { - const s = usable(); + const s = structuralScope(); if (s) { s.issuerKeys.clear(); enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); @@ -826,4 +895,8 @@ export function resetCodexAffinityStoreForTests(): void { flush(current, false); retire(); } + // Model a fresh process: no earlier persisted scope, no change pending from a lease gap. + hadPersistentScope = false; + changedWithoutLease = false; + carried = undefined; } diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index 80448917454..d0b5e42a037 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -6,7 +6,7 @@ import type { CodexThreadLineage } from "../lineage"; import type { CodexQuotaScope } from "./health-store"; import type { TransientProbeLease } from "../../routing/probe-lease"; import { clearPoolRecoveryState } from "../../routing/probe-lease"; -import { identityFence, adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordThreadAccountDeleted, recordAccountPinsCleared, recordAllCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; +import { identityFence, adoptStoredThreadRows, adoptStoredIssuer, CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER, registerCodexAffinityLiveState, recordThreadBind, recordThreadTouch, recordThreadDelete, recordThreadAccountDeleted, recordAccountPinsCleared, recordAllCleared, recordThreadsCleared, recordIssuerSet, recordIssuerTouch, recordIssuerDelete, recordIssuersCleared } from "./thread-affinity-store"; export { CODEX_UNKNOWN_CONVERSATION_STATE_ISSUER } from "./thread-affinity-store"; export type ThreadAffinityEntry = { @@ -221,13 +221,20 @@ registerCodexAffinityLiveState({ const entry = conversationStateIssuerMap.get(key); return entry && { accountId: entry.storedAccountId ?? entry.accountId, fence: entry.storedAccountId ? entry.storedFence ?? undefined : entry.fence, lastUsedAt: entry.lastUsedAt }; }, + forEach: (thread, issuer) => { + for (const [key, affinities] of threadAccountMap) for (const [scope, entry] of affinities) thread(key, scope, entry); + for (const [key, entry] of conversationStateIssuerMap) { + issuer(key, { accountId: entry.storedAccountId ?? entry.accountId, fence: entry.storedAccountId ? entry.storedFence ?? undefined : entry.fence, lastUsedAt: entry.lastUsedAt }); + } + }, clearLive: () => { threadAccountMap.clear(); threadAffinityEntryTotal = 0; conversationStateIssuerMap.clear(); }, }); } registerThreadAffinityLiveState(); export function clearThreadAccountMap(options: { preserveConversationStateIssuers?: boolean } = {}): void { - recordAllCleared(); + if (options.preserveConversationStateIssuers) recordThreadsCleared(); + else recordAllCleared(); threadAccountMap.clear(); threadAffinityEntryTotal = 0; // A refresh cooldown is per-account runtime state learned alongside these bindings. Leaving it diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 18466bc20cb..65325bc83fd 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -7,8 +7,8 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp `src/codex/routing/thread-affinity-store.ts` restores eligible pool bindings after a graceful proxy restart for up to 24 hours idle. Session-bearing requests and session-less requests anchored on their immediate parent have derivable anchors. Deeper session-less chains, keys derived under another config home, legacy raw-parent keys and evicted anchors remain memory-only. Removed, re-logged, expired or identity-less accounts are not restored; normal selection and usability rules still apply. -The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. +The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. A new lease scope clears live affinity before restoring from the database. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. When pins or issuers changed while no lease was held, the next lease rewrites the database from memory, so a restart restores what the process was routing with and never a row it had already replaced or deleted. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. -`src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). `tests/codex-integration/codex-affinity-store.test.ts` and `tests/codex-integration/codex-affinity-restart.test.ts` cover store and restart behavior. +`src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). A home without an ownership manifest, created before ownership tracking, still uses the store; uninstall removes nothing from such a home, as for every other file in it. `tests/codex-integration/codex-affinity-store.test.ts`, `tests/codex-integration/codex-affinity-restart.test.ts` and `tests/codex-integration/codex-affinity-request-owned.test.ts` cover store, restart and request-owned exclusion behavior. diff --git a/structure/providers/codex-request-owned-affinity.md b/structure/providers/codex-request-owned-affinity.md index d9945fff821..9e0d73efecd 100644 --- a/structure/providers/codex-request-owned-affinity.md +++ b/structure/providers/codex-request-owned-affinity.md @@ -66,7 +66,9 @@ is not caller-scoped by design and is unchanged by this feature. Persistence of thread affinity must, for every key where `isRequestOwnedAffinityKey` is true: never register an anchor for it, never serialize its thread rows (any scope, including model detours) or issuer rows, and never adopt a restored row into it. The conversation-state key carries the same prefix, so the predicate -covers issuer records too. +covers issuer records too. The durable store enforces this in `src/codex/routing/thread-affinity-store.ts` +([Durable Codex Pool Affinity](codex-pool-affinity.md)); `tests/codex-integration/codex-affinity-request-owned.test.ts` +proves it across a flush and restart. ## Tests diff --git a/tests/codex-integration/codex-affinity-request-owned.test.ts b/tests/codex-integration/codex-affinity-request-owned.test.ts new file mode 100644 index 00000000000..29a97723086 --- /dev/null +++ b/tests/codex-integration/codex-affinity-request-owned.test.ts @@ -0,0 +1,86 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { Database } from "bun:sqlite"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { readCodexAccountRecord, saveCodexAccountCredential } from "../../src/codex/account-store"; +import { codexPoolAffinityKey } from "../../src/codex/auth-context"; +import { isRequestOwnedAffinityKey, requestOwnedAffinityKey } from "../../src/codex/request-owned-affinity"; +import { bindThreadAffinity, clearThreadAccountMap, getThreadAffinity, peekConversationStateIssuer, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; +import { adoptStoredIssuer, adoptStoredThreadRows, flushCodexAffinityStoreForTests, noteCodexAffinityAnchorForTests, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; +import { acquireSpendLedgerOwner, type SpendLedgerOwnerLease } from "../../src/lib/spend-ledger-owner"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +let home: string; +let prior: string | undefined; +let lease: SpendLedgerOwnerLease | undefined; +const headers = () => new Headers({ authorization: "Bearer caller-owned-token", "session-id": "request-owned-session", "thread-id": "request-owned-thread" }); + +beforeEach(() => { + prior = process.env.OPENCODEX_HOME; home = mkdtempSync(join(tmpdir(), "ocx-affinity-request-owned-")); process.env.OPENCODEX_HOME = home; + clearThreadAccountMap(); resetCodexAffinityStoreForTests(); +}); +afterEach(() => { + lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); clearThreadAccountMap(); + if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; + removeTreeWithRetry(home); +}); + +function rows(): { threads: number; issuers: number } { + const db = new Database(join(home, "codex-affinity.sqlite")); + try { + return { + threads: (db.query("SELECT count(*) AS n FROM threads").get() as { n: number }).n, + issuers: (db.query("SELECT count(*) AS n FROM issuers").get() as { n: number }).n, + }; + } finally { db.close(); } +} + +test("request-owned bindings and issuers are never saved or restored, even with an anchor", () => { + lease = acquireSpendLedgerOwner(); + saveCodexAccountCredential("account-a", { accessToken: "access", refreshToken: "refresh", expiresAt: Date.now() + 60_000, chatgptAccountId: "chat-account" }); + const record = readCodexAccountRecord("account-a")!; + const credential = { generation: record.generation, chatgptAccountId: "chat-account" }; + const conversationKey = codexPoolAffinityKey(headers())!; + const requestOwnedKey = requestOwnedAffinityKey(headers())!; + expect(isRequestOwnedAffinityKey(requestOwnedKey)).toBe(true); + // No production path anchors a request-owned key today; feed one in as a future derivation might. + // A distinct anchor matters: sharing the conversation's anchor would collapse both onto one row. + noteCodexAffinityAnchorForTests(requestOwnedKey, "future-anchor", "future-anchor"); + const now = Date.now(); + bindThreadAffinity(conversationKey, "account-a", now); + rememberConversationStateIssuer(conversationKey, "account-a", now, credential); + bindThreadAffinity(requestOwnedKey, "account-a", now); + rememberConversationStateIssuer(requestOwnedKey, "account-a", now, credential); + flushCodexAffinityStoreForTests(); + // Only the ordinary conversation reached disk. + expect(rows()).toEqual({ threads: 1, issuers: 1 }); + + // Restart: a new lease scope reads the database. Live memory survives an in-process lease change, + // so the restored state is read through the store's adoption, which sees only stored rows. + lease.release(); lease = acquireSpendLedgerOwner(); + noteCodexAffinityAnchorForTests(requestOwnedKey, "future-anchor", "future-anchor"); + const restoredConversation = codexPoolAffinityKey(headers())!; + expect(adoptStoredThreadRows(restoredConversation, Date.now()).map(row => row.accountId)).toEqual(["account-a"]); + expect(adoptStoredIssuer(restoredConversation, Date.now())?.issuer).toBe("account-a"); + expect(adoptStoredThreadRows(requestOwnedKey, Date.now())).toEqual([]); + expect(adoptStoredIssuer(requestOwnedKey, Date.now())).toBeUndefined(); +}); + +test("a manual switch keeps saved issuers while clearing saved pins", () => { + lease = acquireSpendLedgerOwner(); + saveCodexAccountCredential("account-a", { accessToken: "access", refreshToken: "refresh", expiresAt: Date.now() + 60_000, chatgptAccountId: "chat-account" }); + const record = readCodexAccountRecord("account-a")!; + const conversationKey = codexPoolAffinityKey(headers())!; + const now = Date.now(); + bindThreadAffinity(conversationKey, "account-a", now); + rememberConversationStateIssuer(conversationKey, "account-a", now, { generation: record.generation, chatgptAccountId: "chat-account" }); + flushCodexAffinityStoreForTests(); + clearThreadAccountMap({ preserveConversationStateIssuers: true }); + flushCodexAffinityStoreForTests(); + expect(rows()).toEqual({ threads: 0, issuers: 1 }); + lease.release(); lease = acquireSpendLedgerOwner(); + const restored = codexPoolAffinityKey(headers())!; + expect(getThreadAffinity(restored)).toBeUndefined(); + expect(peekConversationStateIssuer(restored)).toBe("account-a"); +}); diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts index 01b6ded2120..775da1b7812 100644 --- a/tests/codex-integration/codex-affinity-restart-matrix.test.ts +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -55,10 +55,12 @@ for (const kind of ["symlink", "directory"] as const) { })); } -test("an unowned nonempty home cannot create an affinity database", () => inHome(home => { +test("a home without an ownership manifest still persists affinity and gains no manifest", () => inHome(home => { writeFileSync(join(home, "unowned"), "keep"); - matrix(home, "probe"); - expect(existsSync(join(home, names[0]!))).toBe(false); + expect(seed(home).account).toBe("a"); + expect(existsSync(join(home, names[0]!))).toBe(true); + expect(worker(home, "codex-affinity-restart-worker.ts", "read").account).toBe("a"); + expect(existsSync(join(home, ".opencodex-uninstall.json"))).toBe(false); })); for (const kind of ["extra-table", "wrong-column"] as const) { @@ -178,13 +180,20 @@ test("home switching drops pending writes and does not carry recorded anchors ac expect(JSON.parse(result.stdout.toString())).toEqual({ first: "a", rowsB: 1, restoredB: "a" }); }))); -for (const mutation of ["bind", "delete", "issuer"]) { - test(`late ${mutation} after release is discarded on reacquisition`, () => inHome(home => { +for (const mutation of ["bind", "delete", "issuer"] as const) { + test(`late ${mutation} after release is what a reacquired lease persists`, () => inHome(home => { seed(home); - expect(matrix(home, `late-${mutation}`)).toEqual({ pin: "a", issuer: "a" }); + // Memory wins when a later lease opens the database: the change made while no lease was held is + // what this process routes with, and a restart must restore exactly that, never the older row. + const expected = { + bind: { pin: "b", issuer: "a", restoredPin: "b", restoredIssuer: "a" }, + delete: { pin: undefined, issuer: "a", restoredPin: undefined, restoredIssuer: "a" }, + issuer: { pin: "a", issuer: "b", restoredPin: "a", restoredIssuer: "__unknown_codex_conversation_state_issuer__" }, + }[mutation]; + expect(matrix(home, `late-${mutation}`)).toEqual({ pin: expected.pin, issuer: expected.issuer }); const restored = matrix(home, "read"); - expect(restored.before).toBe("a"); - expect(restored.issuer).toBe("a"); + expect(restored.before).toBe(expected.restoredPin); + expect(restored.issuer).toBe(expected.restoredIssuer); })); } diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 5d896bfee4d..8c03f15c2a2 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -27,13 +27,14 @@ function bind(fence: string): string { } beforeEach(() => { prior = process.env.OPENCODEX_HOME; home = mkdtempSync(join(tmpdir(), "ocx-affinity-")); process.env.OPENCODEX_HOME = home; live = undefined; - registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, clearLive: () => { live = undefined; } }); + registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, forEach: (thread) => { if (live) thread(key(), "legacy", live); }, clearLive: () => { live = undefined; } }); }); afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); registerThreadAffinityLiveState(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); test("inactive operations do not open and passive operations do not change the database", async () => { key(); expect(existsSync(path())).toBe(false); lease = acquireSpendLedgerOwner(); + flushCodexAffinityStoreForTests(); const before = readFileSync(path()); await runPassiveRead(async () => { key(); recordAllCleared(); }); expect(readFileSync(path()).equals(before)).toBe(true); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index c1a4d9c35cd..cab52deebab 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -698,7 +698,7 @@ "codex-restart-contract-parity.test.ts": "codex-integration", "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", "codex-service-manager-probe.test.ts": "codex-integration", "codex-systemd-env-probe.test.ts": "codex-integration", From 7d72ec78b870e4c94316384fd1b60e31d981451d Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 19:24:57 +0900 Subject: [PATCH 18/21] fix(codex): replay lease-gap changes instead of rewriting the store; conditional restart wording; French grammar --- .../fr/reference/cli/providers-accounts.md | 2 +- .../fr/reference/configuration/providers.md | 4 +- .../ja/reference/cli/providers-accounts.md | 2 +- .../ja/reference/configuration/providers.md | 4 +- .../ko/reference/cli/providers-accounts.md | 2 +- .../ko/reference/configuration/providers.md | 4 +- .../docs/reference/cli/providers-accounts.md | 2 +- .../docs/reference/configuration/providers.md | 4 +- .../ru/reference/cli/providers-accounts.md | 2 +- .../ru/reference/configuration/providers.md | 4 +- .../tr/reference/cli/providers-accounts.md | 2 +- .../tr/reference/configuration/providers.md | 4 +- .../zh-cn/reference/cli/providers-accounts.md | 2 +- .../reference/configuration/providers.md | 4 +- .../zh-tw/reference/cli/providers-accounts.md | 2 +- .../reference/configuration/providers.md | 4 +- scripts/test-layout/layout.json | 2 +- src/codex/routing/thread-affinity-store.ts | 109 +++++++++--------- src/codex/routing/thread-affinity.ts | 6 - structure/providers/codex-pool-affinity.md | 2 +- .../codex-affinity-lease-gap.test.ts | 78 +++++++++++++ .../codex-affinity-store.test.ts | 2 +- tests/fixtures/test-layout-expected.json | 2 +- 23 files changed, 161 insertions(+), 88 deletions(-) create mode 100644 tests/codex-integration/codex-affinity-lease-gap.test.ts diff --git a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md index 6652fc2283b..b378545e45b 100644 --- a/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/fr/reference/cli/providers-accounts.md @@ -230,7 +230,7 @@ cet état et quitte toujours 0. `--json` renvoie : Sélectionne un compte Codex, un compte OAuth ou une clé API existant. Pour `openai`, `main` sélectionne la connexion Codex App. Une sélection en mode Codex Pool efface l'affinité et s'applique à la requête suivante, -dont un provenant d'une tâche visible existante . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; +y compris si elle provient d'une tâche visible existante. Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, peuvent survivre à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité, si l’enregistrement final de l’affinité à l’arrêt réussit. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée, tandis que les demandes en cours conservent leur compte capturé. Cela contrôle uniquement le routage du pool ; Le mode direct continue d’utiliser les informations d’identification principales de l’appelant. Commutation proactive basée sur l'utilisation, 401/403 réauthentification, 429/retry-after temps de recharge, exclusion et pré-sortie 429/402 échec la récupération peut sélectionner ultérieurement un autre compte Pool éligible. Ces chemins de récupération restent actifs lorsque diff --git a/docs-site/src/content/docs/fr/reference/configuration/providers.md b/docs-site/src/content/docs/fr/reference/configuration/providers.md index 9897783dcea..c4bd73f020c 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/fr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Après une inscription ou une connexion OAuth dans l’interface, une boîte de | `activeCodexAccountPinned?` | `string` | — | Identifiant du compte du dernier opérateur sélectionné manuellement. Lorsqu'il est défini, un niveau `codexAccountPriorities` supérieur ne peut pas le préempter jusqu'à ce que la broche soit libérée par drainage, exclusion, suppression ou un failover/promotion explicite. Un mouvement circulaire ordinaire à l’intérieur du niveau plafonné ne le libère pas. L'écriture d'une entrée `codexAccountPriorities` libère également le pin, donc un pin créé avant qu'un ordre n'existe ne peut pas surpasser un ensemble par la suite. `GET /api/codex-auth/active` indique à la fois si le compte effectif est épinglé (`pinned`) et le compte portant le plafond (`pinnedAccountId`). | | `autoSwitchThreshold?` | `number` | `80` | Seuil d'utilisation pour la commutation proactive. `quota` peut réévaluer les requêtes non liées lors de leur prochaine requête. Les tâches liées conservent leur compte au-delà du seuil par défaut (`pool.cacheAffinity`) jusqu'à ce que ce compte soit épuisé ou ne puisse plus servir, et ne basculent alors que vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour réévaluer aussi les tâches liées à ce seuil. `fill-first` ne l'utilise que comme seuil d'évacuation pour l'affectation des requêtes non liées ; `round-robin` ne l'utilise que pour les nouvelles affectations non liées : tant qu'un compte admissible reste sous son seuil effectif, il ignore ceux qui l'atteignent ou le dépassent, sans jamais déplacer une tâche liée pour cette raison. Le score retient la plus élevée des fenêtres de quota connues sur 5 heures, une semaine ou 30 jours. `0` désactive uniquement la commutation proactive fondée sur l'utilisation, pas l'affectation des requêtes non liées ni la récupération après incident. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Seuils par compte remplaçant `autoSwitchThreshold` : identifiant → entier de `0` à `100`. Une entrée absente hérite du seuil global ; `0` désactive uniquement les déplacements fondés sur l’utilisation depuis ce compte. Le compte principal `__main__` est pris en charge. La carte du compte dans Codex Auth gère cette valeur. Activer la valeur spécifique au compte copie le seuil global actuel dans une valeur fixe. Cette valeur, y compris `0`, reste prioritaire après toute modification du seuil global. La désactiver envoie `threshold: null`, supprime l'entrée et rétablit l'héritage du seuil global actuel et de ses modifications futures. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Stratégie d'affectation des requêtes Codex nouvelles ou non liées. Une requête est non liée lorsqu'elle ne possède aucune affinité active, définie par l'identifiant de la tâche parente et la portée du quota . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, peuvent survivre à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité, si l’enregistrement final de l’affinité à l’arrêt réussit. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. `quota` sélectionne le compte admissible le moins utilisé lorsqu'aucun compte actif n'existe, conserve un compte actif admissible sous `autoSwitchThreshold` et, une fois le seuil franchi, peut déplacer une requête non liée. Les tâches liées suivent `pool.cacheAffinity` (activé par défaut) : elles restent jusqu'à ce que le compte soit épuisé (utilisation connue à 100 %) ou ne puisse plus servir, et ne se relient alors qu'à un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez le drapeau à `false` pour relier de manière proactive une tâche liée à un compte admissible moins utilisé au seuil, toujours sous la même contrainte de destination. `round-robin` répartit équitablement les requêtes non liées entre les comptes admissibles sous leur `autoSwitchThreshold` effectif, ou entre tous les comptes admissibles s'ils ont tous atteint ou dépassé leur seuil ; `fill-first` continue de les attribuer au compte actif jusqu'à sa temporisation, son indisponibilité ou le seuil d'évacuation configuré. `reset-first`: Parmi les comptes sous le seuil, privilégier le prochain reset de 5 heures ou hebdomadaire. Les tâches liées suivent la politique d’affinité configurée. Les quotas de modèles indépendants suivent l’ordre de consommation. Les resets mensuels ne déterminent pas cet ordre. | | `pool.cacheAffinity?` | `boolean` | `true` | Ordre d'affinité de cache pour les threads Codex liés, indépendant de `pool.kernel`. Activé par défaut ; omettre la clé ou la définir à `true` conserve la liaison, et une valeur autre que `false` est lue comme activée. Une liaison active prime sur la marge de quota : `quota` ne déplace pas le thread simplement parce que l'utilisation a franchi `autoSwitchThreshold`, car déplacer une conversation liée jette le cache d'invites isolé par compte. Le thread quitte encore le compte s'il ne peut plus servir — suspendu, inutilisable, exclu du plan, identifiants invalides, génération remplacée, TTL expiré, refus de quota 429/402, ou réellement épuisé (utilisation connue à 100 %) — et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Un compte dont l'utilisation est inconnue n'est jamais choisi comme destination d'une tâche liée. Si tous les comptes dépassent le seuil, la tâche liée reste, car aucune destination n'est meilleure. Définissez `false` pour rétablir la réaffectation au seuil, toujours sous la même contrainte de destination. L'affinité est un réordonnancement, pas un verrouillage. | | `accountPoolStickyLimit?` | `number` | `1` | Nombre d'affectations de tâches nouvelles ou non liées conservées sur une même sélection tournante avant de passer à la suivante ; le compteur avance lorsqu'une tâche est liée, et non après une réponse réussie en amont. Plage : 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Nombre d'échecs transitoires consécutifs avant le basculement des futures nouvelles sessions. Réglez `0` pour désactiver ce mécanisme. Pour les requêtes Responses ordinaires et les envois compacts natifs, les échecs avérés d'accessibilité DNS/TCP avant connexion sont suivis au niveau du couple fournisseur-hôte : ils n'affectent jamais l'état ni la temporisation du compte, l'affinité de tâche ou de session, la sélection du compte actif ou le routage du pool, et ne sont jamais comptabilisés dans ce seuil. | @@ -235,7 +235,7 @@ Utilisez **Codex Auth** dans le tableau de bord pour ajouter des comptes au grou métadonnées non secrètes ; les jetons d'accès et d'actualisation utilisent le magasin d'identifiants renforcé. Le routage du pool distingue l'affectation des requêtes nouvelles ou non liées, la commutation proactive fondée sur l'utilisation et la récupération après incident. Une tâche liée conserve normalement son affinité. Par défaut (`pool.cacheAffinity`), cette réaffectation attend que le compte lié soit épuisé ou ne puisse plus servir, et seulement vers un compte dont l'utilisation est strictement inférieure et qui dispose d'une véritable marge de quota. Définissez `pool.cacheAffinity: false` pour laisser `quota` la relier dès la requête suivante une fois le seuil franchi, toujours sous la même contrainte de destination. Un refus de quota 429/402, une pause, une invalidation des identifiants ou l'expiration du TTL libèrent encore la liaison immédiatement ; une série d'échecs transitoires (5xx et autres échecs hors quota) sert la requête sur un autre compte sans supprimer la liaison active. -Une requête non liée ne possède aucune liaison active à un compte . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, survivent à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un +Une requête non liée ne possède aucune liaison active à un compte . Les liaisons des requêtes avec un identifiant de session (y compris les tours habituels de Codex CLI et App et les sous-agents), ou des requêtes sans session ancrées sur leur parent immédiat, peuvent survivre à un redémarrage normal du proxy jusqu’à 24 heures d’inactivité, si l’enregistrement final de l’affinité à l’arrêt réussit. Les chaînes sans session plus profondes et les liaisons à la connexion principale de Codex App restent en mémoire uniquement et sont réattribuées après le redémarrage ; une expulsion, la suppression du compte, une reconnexion, un plantage ou une réinitialisation de l’affinité peuvent laisser une tâche non liée. Un 429 ou un 402 reçu avant le début de la diffusion déclenche une nouvelle tentative unique sur un autre compte admissible au sein de la même requête, même lorsque la commutation proactive fondée sur l'utilisation est désactivée. Les changements de compte préservent et relisent le contexte de la conversation, mais la réutilisation du cache d'invites du fournisseur entre plusieurs comptes n'est pas garantie et le cache peut devoir être réchauffé. diff --git a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md index 4e6058915c7..0a939338274 100644 --- a/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ja/reference/cli/providers-accounts.md @@ -180,7 +180,7 @@ Codex クライアント連携を無効にしても、既存のネイティブ `auto` は手動の選択を解除し、プールが自身の戦略で再び配置するようにします — ただし id が `auto` の Codex アカウントが存在する場合は完全一致の id が優先され、`ocx account clear ` が常に自動選択を復元します。Codex アカウントは id の代わりに `ocx account alias` で付けたエイリアスでも指定でき、`priority`、`pause`、`resume`、`clear-cooldown`、`remove`、`alias` でも同様です。Codex アカウントでは `auto`、`main`、`__main__` は大文字・小文字を区別せず予約語として扱われるため、エイリアスとして設定できません。OAuth アカウントと API キーの表示名には従来のルールが適用されます。 既存の Codex アカウント、OAuth アカウント、または API key を選びます。`openai` で `main` は Codex App ログインを -選択します。Codex Pool の選択は affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 +選択します。Codex Pool の選択は affinity を消去し、既存の表示タスクを含む次のリクエストから適用されます。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、シャットダウン時の最終的な affinity 保存が成功した場合、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されることがあります。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ますが、処理中のリクエストは取得済みアカウントを維持します。この選択は Pool routing のみを制御し、Direct mode は caller-owned/native main credential を使い続けます。使用量ベースのプロアクティブ切り替え、401/403 再認証、429/retry-after cooldown、除外、出力前 429/402 の障害回復により、後で別の適格 Pool アカウントが選ばれる場合があります。これらの回復経路は使用量ベース切り替えが off でも有効です。アカウント変更後も OpenCodex は会話コンテキストを再生しますが、provider prompt cache は再ウォームアップが必要な場合があります。 不明なプロバイダーや id は終了コード 1 です。`--json` は次を返します。 **401/403** では、そのアカウントへのプロセスローカルな affinity を解除し、再認証を要求します。 **429** では `Retry-After` を尊重してアカウントの cooldown を開始し、affinity を解除したうえで、 diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index 19637d6ee29..3459c6b2caf 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI で登録または OAuth ログインが完了すると、Models ページ | `codexAccountPriorities?` | `Record` | — | Codex pool のアカウント別選択順。アカウント ID → `-100` から `100` の整数で、**大きいほど先に使われ**、未設定は `0` です。これは eligibility ではなく順序の境界です。選択は適格なアカウントを、まだ quota に余裕がある最上位 tier に絞り込み、その tier の中を `accountPoolStrategy` が選びます。tier が飛ばされるのは、そのメンバー全員が自身の 0 以外の実効しきい値（アカウント別上書き、未設定ならグローバル値）以上、cooldown 中、soft-avoid、一時停止、または再認証待ちのときだけで、usage 不明が tier を drain させることはありません。順序付けが不適格なアカウントを選択可能にすることはなく、すでにアカウントが結び付いた thread を再 bind することもありません。メインの `__main__` も同じ条件で参加するため、Codex Desktop ログインを最後に使わせられます。エントリが 1 つもなければ挙動は従来どおりです。map が不正な場合は警告を出して順序付けを無効にします（config の修復処理は走りません）。`ocx account priority` と Codex Auth ページで管理します。 | | `autoSwitchThreshold?` | `number` | `80` | 使用量ベースのプロアクティブ切り替えしきい値。`quota` は未紐付けタスクの次のリクエストを再評価できます。紐付け済みタスクは既定（`pool.cacheAffinity`）ではしきい値を超えても同じアカウントを維持し、アカウントが使い切られるか処理できなくなったときだけ離れ、その場合も実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`pool.cacheAffinity: false` にするとしきい値で紐付け済みタスクも再評価します。`fill-first` は未紐付け割り当ての使い切り基準としてのみ使用し、`round-robin` は新規/未紐付けの割り当て時にのみ使用し、しきい値未満の適格アカウントがある間は実効しきい値以上のアカウントを除外し、この理由で紐付け済みタスクを移動しません。既知の 5 時間、週次、30 日 quota window の最大スコアを使います。`0` は使用量ベースの切り替えだけを無効にし、未紐付け割り当てや障害回復は無効にしません。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | `autoSwitchThreshold` のアカウント別上書き（アカウント ID → `0`〜`100` の整数）。未設定はグローバル値を継承し、`0` はそのアカウントからの使用量ベース切り替えだけを無効にします。メインの `__main__` も指定できます。Codex Auth の各アカウントカードで管理します。 上書きを有効にすると、現在のグローバルしきい値が固定のアカウント別値としてコピーされます。`0` を含む上書き値は、その後グローバル値が変更されても優先されます。無効にすると `threshold: null` を送信してエントリを削除し、現在のグローバルしきい値と今後の変更を継承する状態に戻ります。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けです。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新規/未紐付け Codex リクエストの割り当て戦略。live な `(parent thread id, quota scope)` affinity がなければ未紐付けです。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、シャットダウン時の最終的な affinity 保存が成功した場合、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されることがあります。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。`quota` はアクティブアカウントがなければ既知 usage 最小の適格アカウントを選び、適格なアクティブアカウントが `autoSwitchThreshold` 未満なら維持します。しきい値到達後は未紐付けリクエストを移せます。紐付け済みタスクは既定ではアカウントが使い切られるか（既知 usage 100%）処理できなくなるまで維持され、離れるときは実際に quota 余裕があり usage がより低いアカウントへだけ移ります。フラグをオフにすると、しきい値で紐付け済みタスクの次のリクエストも実際に quota 余裕があり usage がより低い適格アカウントへ移せます。`round-robin` は実効 `autoSwitchThreshold` 未満の適格アカウントに未紐付けリクエストを均等分散し、全員がしきい値以上なら全適格アカウントに分散し、`fill-first` は cooldown、使用不可、または drain threshold までアクティブアカウントへ割り当てます。 `reset-first`: 使用率のしきい値未満から、次の5時間枠または週次枠のリセットが最も近いアカウントを選びます。紐付け済みタスクは設定されたアフィニティ方針に従います。独立したモデル枠は使用率順です。 月次リセットはこの順序に使用しません。 | | `pool.cacheAffinity?` | `boolean` | `true` | 紐付け済み Codex スレッド向けの cache-affinity 順序。`pool.kernel` とは独立で、既定はオンです。不正な値はオンとして読みます。live な紐付けが quota 余裕より優先され、`quota` は使用量が `autoSwitchThreshold` を超えたという理由だけではスレッドを移しません。一時停止、使用不可、または実際に使い切られたアカウント（既知 usage 100%）では離れますが、実際に quota 余裕があり usage がより低いアカウントへだけ移ります。`false` にするとしきい値での再紐付けに戻ります。affinity は固定ではなく並べ替えです。 | | `accountPoolStickyLimit?` | `number` | `1` | 1 回の round-robin 選択で次へ進む前に保持する新規/未紐付けタスク割り当て数。カウンターは上流の成功後ではなくタスクの紐付け時に増えます。範囲 1–100。`accountPoolStrategy` が `round-robin` のときのみ。 | | `upstreamFailoverThreshold?` | `number` | `3` |今後の新しいセッションがフェイルオーバーする前に一時的なエラーが連続して発生する。 `0` を無効に設定します。通常のResponses送信とネイティブcompact送信では、実証済みの接続前DNS/TCP到達不能障害はprovider-host単位で記録され、アカウントの健全性、アカウントのクールダウン、スレッド/セッションの親和性、アクティブアカウントの選択、Poolルーティングには影響せず、この閾値にもカウントされません。 | @@ -220,7 +220,7 @@ pool アカウントの追加と quota 更新はダッシュボードの **Codex 使い切られるか処理できなくなるまで再紐付けを延期し、離れるときは実際に quota 余裕があり usage が より低いアカウントへだけ移ります。フラグをオフにすると、`quota` はしきい値超過後の次のリクエストで 再紐付けできます。 -pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには有効なアカウントの紐付けがありません。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されます。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。出力前の **429/402** は使用量ベースの +pause、cooldown、再認証、障害処理も独立して routing を消去または変更できます。未紐付けリクエストには有効なアカウントの紐付けがありません。セッションIDを持つリクエスト（通常の Codex CLI・App のターンやサブエージェントを含む）と、直近の親に紐付くセッションなしリクエストの紐付けは、シャットダウン時の最終的な affinity 保存が成功した場合、プロキシの正常な再起動後も最大24時間の非活動期間まで保持されることがあります。より深いセッションなしの連鎖と Codex App のメインログインへの紐付けはメモリ内にのみ残り、再起動後に再割り当てされます。eviction、アカウント削除、再ログイン、クラッシュ、affinity リセットではタスクが未紐付けになり得ます。出力前の **429/402** は使用量ベースの 切り替えがオフでも同じリクエストで適格な代替アカウントへ 1 回再試行できます。アカウント変更後も会話 コンテキストは保持・再生されますが、アカウント間の provider prompt cache 再利用は保証されません。 一時停止したアカウントと quota metadata は表示されたままですが、自動切り替え、再試行/failover 選択、cooldown 復旧プローブ、手動有効化の対象外です。 diff --git a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md index 2cf22c568da..8e77626c88f 100644 --- a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md @@ -298,7 +298,7 @@ Codex JSON 행에는 등록된 `id`와 같은 `slotId`, 그리고 `native-main` `auto`는 수동 선택을 지워 풀이 다시 자체 전략으로 작업을 배치하게 합니다 — 단 id가 `auto`인 Codex 계정이 있으면 정확한 id 일치가 우선되며 `ocx account clear `는 항상 자동 선택을 복원합니다. Codex 계정은 id 대신 `ocx account alias`로 지정한 별칭으로도 가리킬 수 있으며, `priority`, `pause`, `resume`, `clear-cooldown`, `remove`, `alias`에서도 마찬가지입니다. Codex 계정에서 `auto`, `main`, `__main__`은 대소문자 구분 없이 예약어이므로 별칭으로 지정할 수 없습니다. OAuth 계정과 API 키의 표시 이름에는 기존 규칙이 그대로 적용됩니다. 기존 Codex 계정, OAuth 계정 또는 API key를 선택합니다. `openai`에서 `main`은 Codex App 로그인을 -선택합니다. Codex Pool 선택은 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. +선택합니다. Codex Pool 선택은 affinity를 지우고 기존에 보이던 작업을 포함한 다음 요청부터 적용됩니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 종료 시 마지막 affinity 저장이 성공하면 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지될 수 있습니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 진행 중인 요청은 이미 확보한 계정을 유지합니다. 이 선택은 Pool 라우팅만 제어하며 Direct mode는 호출자 소유/native main credential을 계속 사용합니다. 사용량 기반 선제 전환, 401/403 재인증, 429/retry-after cooldown, 제외, 출력 전 429/402 실패 복구는 나중에 다른 적격 Pool 계정을 선택할 수 있습니다. 이러한 복구 경로는 사용량 기반 전환이 꺼져 있어도 동작합니다. 계정이 바뀌어도 OpenCodex는 대화 문맥을 재생하지만 프로바이더 측 prompt cache는 다시 예열해야 할 수 있습니다. 알 수 없는 프로바이더나 id는 종료 코드 1입니다. `--json`은 다음을 반환합니다. **401/403**이 발생하면 해당 계정의 프로세스 로컬 affinity를 해제하고 재인증을 요구합니다. **429**에서는 `Retry-After`를 준수해 계정 cooldown을 시작하고 affinity를 해제한 뒤, diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index 9d8c3bced16..d59609c3db8 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -55,7 +55,7 @@ GUI에서 등록이나 OAuth 로그인을 마치면 Models 페이지로 이동 | `codexAccountPriorities?` | `Record` | — | Codex pool의 계정별 선택 순서. 계정 ID → `-100`부터 `100`까지의 정수이며 **값이 클수록 먼저** 쓰이고, 항목이 없으면 `0`입니다. 이는 eligibility 경계가 아니라 순서 경계입니다. 선택은 이미 적격한 계정들을 quota 여유가 남은 최상위 tier로 좁히고, 그 tier 안에서 `accountPoolStrategy`가 계정을 고릅니다. tier를 건너뛰는 경우는 그 구성원 전부가 자신의 0이 아닌 유효 임계값(계정별 재정의 또는 전역 기본값) 이상, cooldown, soft-avoid, 일시 중지 또는 재인증 대기일 때뿐이며, usage를 알 수 없다고 해서 tier가 소진되지는 않습니다. 순서는 부적격 계정을 선택 가능하게 만들지 않고, 이미 계정에 묶인 thread를 다시 bind하지도 않습니다. 메인 `__main__` 계정도 동일한 조건으로 참여하므로 Codex Desktop 로그인을 마지막에 쓰도록 둘 수 있습니다. 항목이 하나도 없으면 동작은 이전과 같습니다. map이 잘못된 경우 경고를 출력하고 순서 지정을 끕니다(config 복구는 하지 않습니다). `ocx account priority`와 Codex Auth 페이지에서 관리합니다. | | `autoSwitchThreshold?` | `number` | `80` | 사용량 기반 선제 전환 임계값입니다. `quota`는 바인딩 없는 작업의 다음 요청을 재평가할 수 있습니다. 바인딩된 작업은 기본값(`pool.cacheAffinity`)에서 이 임계값을 넘어도 계정을 유지하며, 해당 계정이 소진되었거나 더 이상 처리할 수 없을 때에만 떠나고, 그때도 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `pool.cacheAffinity: false`로 두면 임계값에서 바인딩된 작업도 재평가합니다. `fill-first`는 바인딩 없는 작업 배정의 소진 기준으로만 사용하며, `round-robin`은 새/바인딩 없는 작업 배정에만 사용하며 임계값 미만인 적격 계정이 남아 있는 동안 유효 임계값 이상인 계정을 건너뛰고 이 때문에 바인딩된 작업을 옮기지 않습니다. 알려진 5시간, 주간, 30일 quota window 중 가장 높은 점수를 씁니다. `0`은 사용량 기반 전환만 끄며 바인딩 없는 작업 배정이나 실패 복구는 끄지 않습니다. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 계정별 `autoSwitchThreshold` 재정의입니다(계정 ID → `0`~`100` 정수). 항목이 없으면 전역 값을 상속하고, `0`은 해당 계정에서 시작하는 사용량 기반 전환만 끕니다. 메인 `__main__` 계정도 지원합니다. Codex Auth의 계정 카드에서 관리합니다. 재정의를 켜면 현재 전역 임계값을 고정된 계정별 값으로 복사합니다. `0`을 포함한 재정의 값은 이후 전역 값이 바뀌어도 우선합니다. 끄면 `threshold: null`을 보내 항목을 삭제하고 현재 전역 임계값과 향후 변경 사항을 상속하는 상태로 돌아갑니다. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청입니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 새 작업/바인딩 없는 Codex 요청의 계정 배정 전략입니다. `(parent thread id, quota scope)`의 live affinity가 없으면 바인딩 없는 요청입니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 종료 시 마지막 affinity 저장이 성공하면 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지될 수 있습니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. `quota`는 활성 계정이 없을 때 알려진 usage가 가장 낮은 적격 계정을 선택하고, 적격 활성 계정이 `autoSwitchThreshold` 미만이면 유지합니다. 임계값 도달 뒤에는 바인딩 없는 요청을 옮길 수 있습니다. 바인딩된 작업은 기본값에서 계정이 소진되었거나(알려진 usage 100%) 더 이상 처리할 수 없을 때까지 유지되며, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 임계값에서 바인딩된 작업의 다음 요청도 실제 quota 여유가 있고 usage가 더 낮은 적격 계정으로 옮길 수 있습니다. `round-robin`은 유효 `autoSwitchThreshold` 미만인 적격 계정에 바인딩 없는 요청을 균등 분배하고 모두 임계값 이상이면 모든 적격 계정에 분배하며, `fill-first`는 cooldown, 사용 불가 또는 drain threshold까지 활성 계정에 배정합니다. `reset-first`: 사용량 임계값 미만인 계정 중 다음 5시간·주간 초기화가 가장 가까운 계정을 고릅니다. 연결된 작업은 설정된 어피니티 정책을 따릅니다. 독립 모델 한도에는 사용량 순서를 적용합니다. 월간 초기화는 이 순서에 사용하지 않습니다. | | `pool.cacheAffinity?` | `boolean` | `true` | 바인딩된 Codex 스레드의 cache-affinity 순서입니다. `pool.kernel`과는 별개이며 기본값은 켜짐입니다. 잘못된 값은 켜진 것으로 읽습니다. live 바인딩이 quota 여유보다 우선하므로 `quota`는 사용량이 `autoSwitchThreshold`를 넘었다는 이유만으로 스레드를 옮기지 않습니다. 해당 계정이 일시 중지되었거나 사용할 수 없거나 실제로 소진된 경우(알려진 usage 100%)에는 떠나되, 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. `false`로 두면 임계값 재바인딩이 복원됩니다. affinity는 고정이 아니라 재정렬입니다. | | `accountPoolStickyLimit?` | `number` | `1` | 한 round-robin 선택이 다음으로 넘어가기 전에 유지하는 새 작업/바인딩 없는 작업 배정 수입니다. 카운터는 업스트림 성공 뒤가 아니라 작업을 바인딩할 때 증가합니다. 범위 1–100이며 `accountPoolStrategy`가 `round-robin`일 때만 적용됩니다. | | `upstreamFailoverThreshold?` | `number` | `3` | 연속된 일시적 실패가 이 횟수에 도달하면 이후 새 세션은 failover됩니다. `0`으로 두면 비활성화됩니다. 일반 Responses와 네이티브 compact 전송에서 입증된 연결 전 DNS/TCP 도달 불가 실패는 provider-host 범위로 기록되며 계정 상태, 계정 쿨다운, 스레드/세션 선호도, 활성 계정 선택 또는 Pool 라우팅에 영향을 주지 않고 이 임계값에도 집계되지 않습니다. | @@ -221,7 +221,7 @@ pool 계정 추가와 quota 갱신은 대시보드의 **Codex Auth** 페이지 구분됩니다. 바인딩된 작업은 보통 affinity를 유지합니다. 기본값(`pool.cacheAffinity`)에서는 바인딩된 계정이 소진되었거나 더 이상 처리할 수 없을 때까지 재바인딩을 미루고, 떠날 때는 실제 quota 여유가 있고 usage가 더 낮은 계정으로만 옮깁니다. 플래그를 끄면 `quota`가 사용량 임계값을 넘은 뒤 다음 요청에서 재바인딩할 수 있습니다. 일시 중지, cooldown, 재인증, 실패 처리도 -독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청에는 live 계정 바인딩이 없습니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지됩니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 +독립적으로 라우팅을 지우거나 바꿀 수 있습니다. 바인딩 없는 요청에는 live 계정 바인딩이 없습니다. 세션 ID가 있는 요청(일반적인 Codex CLI 및 App 턴과 서브에이전트 포함)이나 바로 위 부모에 연결된 세션 없는 요청의 바인딩은 종료 시 마지막 affinity 저장이 성공하면 프록시를 정상적으로 재시작해도 최대 24시간의 유휴 기간 동안 유지될 수 있습니다. 더 깊은 세션 없는 체인과 Codex App의 주 로그인에 대한 바인딩은 메모리에만 남으며 재시작 후 다시 배정됩니다. eviction, 계정 삭제, 재로그인, 충돌 또는 affinity 초기화로 작업이 바인딩 없는 상태가 될 수 있습니다. 출력 전 **429/402**는 사용량 기반 선제 전환이 꺼져 있어도 같은 요청에서 적격 대체 계정으로 한 번 재시도할 수 있습니다. 계정이 바뀌어도 대화 문맥은 보존·재생되지만 계정 간 프로바이더 측 prompt cache 재사용은 보장되지 않아 다시 예열해야 할 수 있습니다. diff --git a/docs-site/src/content/docs/reference/cli/providers-accounts.md b/docs-site/src/content/docs/reference/cli/providers-accounts.md index e36abdad0d8..092133e51de 100644 --- a/docs-site/src/content/docs/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/reference/cli/providers-accounts.md @@ -846,7 +846,7 @@ that state and still exits 0. `--json` returns: Selects an existing Codex account, OAuth account, or API key. For `openai`, `main` selects the Codex App login. A Codex Pool selection clears affinity and applies to the next request, -including one from an existing visible task. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound, while in-flight requests keep their captured account. This controls Pool routing only; +including one from an existing visible task. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, can survive a graceful proxy restart, for up to 24 hours idle, when the final affinity save at shutdown succeeds. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound, while in-flight requests keep their captured account. This controls Pool routing only; Direct mode keeps using the caller-owned/native main credential. Usage-based proactive switching, 401/403 reauthentication, 429/retry-after cooldowns, exclusion, and pre-output 429/402 failure recovery may later select another eligible Pool account. Those recovery paths remain active when diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 33f3061d6cb..7b739f5f860 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -125,7 +125,7 @@ separate. Full request URLs such as `/api/v1/responses` are not provider base UR | `codexAccountPriorityFailback?` | `boolean` | `false` | Opt-in return of ongoing Codex Pool tasks to a usable higher-priority account under the `quota` strategy. Requests trigger a background quota refresh at most once every five minutes, including failed refresh attempts. When recent quota observations show headroom, the next request may rebind even if its current account is below the threshold. This explicit preference can discard a warm prompt cache; it does not change default cache affinity. No requests means no polling. Manual pins, model access, cooldowns, pauses and reauthentication exclusions still apply; in-flight requests keep their captured account. Requires the current source account's effective threshold to be positive (an override of 0 disables it; a positive override works with global 0); exact account routes and Direct mode are unchanged. | | `autoSwitchThreshold?` | `number` | `80` | Usage threshold for placing new/unbound work. `quota` can re-evaluate unbound tasks on their next request once usage crosses this threshold. Bound tasks keep their account past the threshold by default (`pool.cacheAffinity`); they leave only when that account is exhausted or otherwise cannot serve, and then only for an account with genuine quota headroom and strictly lower usage. Set `pool.cacheAffinity: false` to re-evaluate bound tasks at this threshold, still only onto such a destination. `fill-first` uses it only as the drain point for unbound assignment; `round-robin` uses it only when placing new/unbound work: it skips accounts at or over their effective threshold while any eligible account is still under it, and never moves a bound task for it. The score uses the hottest known 5h, weekly, or 30d quota window. `0` disables usage-based proactive switching only, not unbound assignment or failure recovery. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Per-account overrides for `autoSwitchThreshold`: account id → integer from `0` to `100`. An absent entry inherits the global value; `0` disables usage-driven switching only when that account is the source. `quota` uses the current source account's effective threshold for bound-task re-evaluation and unbound switching. Fill-first drain and selection-order tier headroom are evaluated against each account's own effective threshold; main-account pins and subagent fallback also use the relevant account's effective threshold. Supports the main `__main__` account. A malformed map is ignored with a warning. Managed by each account card on the Codex Auth page; disabling an override removes its entry. Enabling the override copies the current global threshold into a fixed account value. The override, including `0`, takes precedence over later global changes. Disabling it sends `threshold: null`, removes the entry, and restores inheritance of the current global threshold and future global changes. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Assignment strategy for new/unbound Codex requests. A request is unbound when it has no live (parent thread id, quota scope) affinity. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, can survive a graceful proxy restart, for up to 24 hours idle, when the final affinity save at shutdown succeeds. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. `quota` picks the lowest-usage eligible account when no active account exists, keeps an eligible active account below `autoSwitchThreshold`, and after the threshold may move an unbound request. Bound tasks follow `pool.cacheAffinity` (on by default): they stay until the account is exhausted (known usage at 100%) or otherwise cannot serve, and then may rebind only to an account with genuine quota headroom and strictly lower usage. Set the flag `false` to proactively rebind a bound task at the threshold, still only onto such a destination. `round-robin` distributes unbound requests evenly across eligible accounts still under their effective `autoSwitchThreshold`, and across all eligible accounts when every one is at or over it; `fill-first` keeps assigning unbound requests to the active account until cooldown, unavailability, or the configured drain threshold. `reset-first`: Prefer the nearest future 5-hour or weekly reset among accounts below the usage threshold. Bound tasks follow the configured affinity policy. Independent model quotas use quota ordering. Monthly resets do not determine this ordering. Under every strategy, a Claude Code turn that reaches a Codex account route through `/v1/messages` is bound by its Claude session: a caller `session_id`/`session-id` alias if present, else the metadata session, else `x-claude-code-session-id`; the binding is keyed to the admitted credential (or to local loopback use) with a server-held secret, so callers cannot share, guess or reuse another credential's binding. The binding is used only to choose the account; the headers sent upstream are unchanged. Spawn turns and turns that already carry `x-codex-parent-thread-id`, `thread-id` or a complete session/thread pair keep their own identity. A bound session normally stays on one account; the affinity and failure-recovery rules above can still move it. | | `pool.cacheAffinity?` | `boolean` | `true` | Cache-affinity ordering for bound Codex threads, independent of `pool.kernel`. On by default; omitting the key or setting `true` keeps a bound task on its account until that account genuinely cannot serve. Only an explicit `false` restores threshold-based rebinding of bound tasks. A live binding outranks quota headroom: `quota` does not move the thread merely because usage crossed `autoSwitchThreshold`. The thread still leaves if that account cannot serve — paused, unusable, or genuinely exhausted (known usage at 100%) — and then only to an account with genuine quota headroom and strictly lower usage. Under either setting, an account with unknown usage is never chosen as a destination for a bound task, so when every account sits above the threshold the task stays put. Affinity is a reordering, not a pin. | | `pool.credentialGroups?` | `Array<{ id: string; credentials: string[]; note?: string }>` | `[]` | Accepted and validated, but not yet consumed by routing: declaring a group changes no routing decision until a consuming layer lands. Operator-declared quota domains: groups of credentials that demonstrably share one upstream usage limit. Members of one group count once toward available capacity, and a quota refusal inside a group is not answered by rotating to another member — the limit is the same, so the move would pay a cold prefix for zero new capacity. Declared groups speak only to quota; sharing a limit says nothing about prompt-cache compatibility, which is classified separately. Each member is written provider-qualified as `":"`, because a credential id means something only inside its provider; the provider segment accepts the usual aliases (`chatgpt:` and `codex:` both mean OpenAI). Group ids must be unique, `credentials` must be non-empty, and a credential may appear in at most one group — an ambiguous declaration is rejected on write and dropped with a warning on load rather than resolved by whichever group is listed first, since that would merge two unrelated quota domains. A malformed list costs only the grouping: `pool.kernel` and `pool.cacheAffinity` are preserved. Absent or empty means no declared grouping, so an unconfigured install behaves exactly as before. | | `accountPoolStickyLimit?` | `number` | `1` | New/unbound task assignments retained on one round-robin selection before advancing; the counter advances when a task is bound, not after an upstream success. Range 1–100. | @@ -1022,7 +1022,7 @@ Set `pool.cacheAffinity: false` to restore threshold rebinding of bound tasks, s a destination. Pause, cooldown, reauthentication, and quota refusals still release a live binding immediately. A transient-failure streak serves the request from another account while keeping the binding, and the task returns once that account serves again; after 10 minutes the binding is -released normally. An unbound request has no live account binding. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, survive a graceful proxy restart for up to 24 hours idle. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. A pre-stream 429 or 402, or a 5xx response whose bounded +released normally. An unbound request has no live account binding. Bindings for requests with a session ID (including normal Codex CLI and App turns and subagents), or session-less requests anchored on their immediate parent, can survive a graceful proxy restart, for up to 24 hours idle, when the final affinity save at shutdown succeeds. Deeper session-less chains and bindings to the Codex App main login stay in memory only and are assigned again after restart; eviction, account removal, re-login, a crash, or an affinity reset can leave a task unbound. A pre-stream 429 or 402, or a 5xx response whose bounded body explicitly reports quota exhaustion, retries once on an eligible alternate account in the same request, even when usage-based proactive switching is off. The ordinary transient-5xx policy runs first, so a wrapped quota response may make up to three sends on the exhausted account before pool diff --git a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md index d7b251a0c54..8adca4b9a4e 100644 --- a/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ru/reference/cli/providers-accounts.md @@ -211,7 +211,7 @@ credential'а, это состояние тоже печатается, но к `auto` снимает ручной выбор, и пул снова распределяет работу по своей стратегии — если только аккаунт Codex буквально не имеет id `auto`: точное совпадение id выигрывает, а `ocx account clear ` всегда восстанавливает автоматический выбор. Аккаунт Codex можно указать по псевдониму, заданному через `ocx account alias`, вместо id; это относится и к `priority`, `pause`, `resume`, `clear-cooldown`, `remove` и `alias`. Для аккаунтов Codex значения `auto`, `main` и `__main__` зарезервированы независимо от регистра и не могут назначаться как псевдонимы. Для отображаемых имён аккаунтов OAuth и API-ключей действуют прежние правила. Выбирает существующий аккаунт Codex, OAuth-аккаунт или API-ключ. Для `openai` значение `main` -выбирает вход Codex App. Выбор Codex Pool очищает affinity и применяется к следующему запросу, включая запрос существующей видимой задачи. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры +выбирает вход Codex App. Выбор Codex Pool очищает affinity и применяется к следующему запросу, включая запрос существующей видимой задачи. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, могут сохраниться после штатного перезапуска прокси при простое до 24 часов, если финальное сохранение affinity при завершении работы прошло успешно. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки, а выполняющиеся запросы сохраняют захваченный аккаунт. Это управляет только Pool routing; Direct mode продолжает использовать caller-owned/native main credential. Проактивное переключение по использованию, повторная аутентификация 401/403, cooldown 429/retry-after, исключение и восстановление после отказа 429/402 до вывода могут позже выбрать другой подходящий Pool-аккаунт. Эти пути восстановления остаются активными, когда переключение по использованию выключено. После смены аккаунта OpenCodex воспроизводит контекст разговора, но prompt cache провайдера может потребовать прогрева. Неизвестные провайдеры или id завершаются с кодом 1. `--json` возвращает: При **401/403** локальная для процесса привязка к аккаунту сбрасывается и требуется повторная аутентификация. При **429** учитывается `Retry-After`, для аккаунта запускается cooldown, привязка сбрасывается, diff --git a/docs-site/src/content/docs/ru/reference/configuration/providers.md b/docs-site/src/content/docs/ru/reference/configuration/providers.md index 7d596518c85..4ffac816f4d 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ru/reference/configuration/providers.md @@ -56,7 +56,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Порядок выбора для каждого аккаунта пула Codex: id аккаунта → целое число от `-100` до `100`, **больше — используется раньше**, отсутствие означает `0`. Это граница порядка, а не пригодности: выбор сужает уже подходящие аккаунты до самого высокого уровня, у которого ещё есть запас квоты, а внутри этого уровня аккаунт выбирает `accountPoolStrategy`. Уровень пропускается, только когда все его аккаунты достигли собственного ненулевого эффективного порога (переопределение аккаунта или глобальное значение), находятся в cooldown, под soft-avoid, на паузе или требуют повторной аутентификации; неизвестный usage никогда не исчерпывает уровень. Порядок не делает выбираемым непригодный аккаунт и не перепривязывает поток, у которого аккаунт уже есть. Основной аккаунт `__main__` участвует на равных — именно так логин Codex Desktop можно оставить на самый конец. Без записей поведение остаётся прежним. Некорректная map игнорируется с предупреждением в консоли (порядок отключается, восстановление config не запускается). Управляется через `ocx account priority` и страницу Codex Auth. | | `autoSwitchThreshold?` | `number` | `80` | Порог проактивного переключения по использованию. `quota` может повторно оценить следующий непривязанный запрос. Привязанная задача по умолчанию (`pool.cacheAffinity`) сохраняет аккаунт после порога, пока он не исчерпан и ещё может обслуживать запрос, и переносится только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы повторно оценивать привязанные задачи на этом пороге. `fill-first` использует его только как точку исчерпания для непривязанных назначений; `round-robin` применяет его только к новым назначениям: пока есть подходящий аккаунт ниже своего порога, аккаунты на пороге или выше пропускаются; привязанная задача из-за этого не переносится. Оценка берёт самое горячее из окон 5 часов, недели и 30 дней. `0` отключает только переключение по использованию, но не назначение непривязанных задач и не восстановление после сбоев. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Переопределения `autoSwitchThreshold` по аккаунтам: id → целое число `0`–`100`. Без записи наследуется глобальное значение; `0` отключает только переключения по использованию с этого аккаунта. Поддерживается основной аккаунт `__main__`. Управляется в карточке аккаунта на странице Codex Auth. Включение переопределения копирует текущий глобальный порог в фиксированное значение аккаунта. Переопределение, включая `0`, имеет приоритет и после изменения глобального порога. Отключение отправляет `threshold: null`, удаляет запись и восстанавливает наследование текущего глобального порога и его будущих изменений. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Стратегия назначения для новых/непривязанных запросов Codex. Запрос непривязан, если у него нет live affinity `(parent thread id, quota scope)`. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, могут сохраниться после штатного перезапуска прокси при простое до 24 часов, если финальное сохранение affinity при завершении работы прошло успешно. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. `quota` выбирает подходящий аккаунт с наименьшим известным usage, когда активного аккаунта нет, сохраняет подходящий активный аккаунт ниже `autoSwitchThreshold`, а после порога может перевести непривязанный запрос. Привязанные задачи следуют `pool.cacheAffinity` (по умолчанию включён): они остаются, пока аккаунт не исчерпан (известный usage 100%) или не может обслуживать запрос, и тогда переносятся только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите флаг в `false`, чтобы на пороге проактивно перепривязать задачу к подходящему аккаунту с меньшим usage, всё равно только при той же проверке назначения. `round-robin` распределяет непривязанные запросы равномерно между подходящими аккаунтами ниже их эффективного `autoSwitchThreshold`, а если все достигли порога — между всеми подходящими; `fill-first` назначает их активному аккаунту до cooldown, недоступности или порога исчерпания. `reset-first`: Среди аккаунтов ниже порога выбирается ближайший сброс 5-часовой или недельной квоты. Привязанные задачи следуют настроенной политике привязки. Независимые квоты моделей упорядочиваются по использованию. Месячный сброс не определяет этот порядок. | | `pool.cacheAffinity?` | `boolean` | `true` | Порядок cache-affinity для привязанных потоков Codex, независимый от `pool.kernel`. По умолчанию включён; пропуск ключа или `true` сохраняет привязку, а любое значение кроме `false` читается как включённое. Живая привязка важнее запаса квоты: `quota` не переносит поток только потому, что usage пересёк `autoSwitchThreshold` — перенос живого разговора сбрасывает изолированный по аккаунту prompt cache. Поток всё равно уходит, если аккаунт не может обслуживать запрос — на паузе, непригоден, исключён планом, с недействительными credential'ами, со сменённой generation, с истёкшим TTL, при отказе квоты 429/402 или реально исчерпан (известный usage 100%) — и только на аккаунт со строго меньшим usage и реальным запасом квоты. Аккаунт с неизвестным usage никогда не выбирается как назначение для привязанной задачи. Если все аккаунты выше порога, привязанная задача остаётся: лучшего назначения нет. `false` возвращает перепривязку по порогу, всё равно только при той же проверке назначения. Affinity меняет порядок, а не закрепляет учётные данные. | | `accountPoolStickyLimit?` | `number` | `1` | Число назначений новых/непривязанных задач на одном выборе round-robin перед переходом дальше. Счётчик растёт при привязке задачи, а не после успеха upstream. Диапазон 1–100; только при `accountPoolStrategy` = `round-robin`. | | `upstreamFailoverThreshold?` | `number` | `3` | Сколько подряд transient failure допустить, прежде чем новые сессии начнут делать failover. `0` отключает эту логику. Для обычных Responses-запросов и нативных compact-отправок доказанные ошибки доступности DNS/TCP до соединения учитываются на уровне пары «провайдер, хост» и не влияют на здоровье аккаунта, кулдауны аккаунта, привязку потока/сессии, выбор активного аккаунта или маршрутизацию пула, а также не учитываются в этом пороге. | @@ -249,7 +249,7 @@ redirect'ов для обычных provider-request'ов реализована защищённом хранилище учётных данных аккаунтов Codex. Pool routing разделяет назначение новых/непривязанных задач, проактивное переключение по использованию и восстановление после сбоев. Привязанная задача обычно сохраняет affinity. По умолчанию (`pool.cacheAffinity`) эта перепривязка ждёт, пока -привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, сохраняются после штатного перезапуска прокси при простое до 24 часов. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. Отказ **429/402** до вывода допускает одну попытку +привязанный аккаунт не будет исчерпан или не сможет обслуживать запрос, и тогда только на аккаунт со строго меньшим usage и реальным запасом квоты. Установите `pool.cacheAffinity: false`, чтобы `quota` перепривязывала задачу при следующем запросе после превышения порога, всё равно только при той же проверке назначения. Отказ квоты 429/402, пауза, инвалидация credential'ов и истечение TTL по-прежнему снимают привязку сразу; серия временных сбоев (5xx и другие не-квотные ошибки) обслуживает запрос на другом аккаунте, не удаляя живую привязку. Привязки запросов с идентификатором сессии (включая обычные ходы Codex CLI и App и субагентов) или запросов без сессии, опирающихся на непосредственного родителя, могут сохраниться после штатного перезапуска прокси при простое до 24 часов, если финальное сохранение affinity при завершении работы прошло успешно. Более глубокие цепочки без сессии и привязки к основному входу Codex App остаются только в памяти и назначаются заново после перезапуска; вытеснение, удаление аккаунта, повторный вход, сбой или сброс affinity могут оставить задачу без привязки. Отказ **429/402** до вывода допускает одну попытку на подходящем альтернативном аккаунте даже при выключенном переключении по использованию. Контекст разговора сохраняется и воспроизводится, но prompt cache провайдера между аккаунтами может не переиспользоваться и потребовать прогрева. diff --git a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md index 8513002ef58..f0cec9d17ac 100644 --- a/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/tr/reference/cli/providers-accounts.md @@ -244,7 +244,7 @@ yine de 0 ile çıkar. `--json` şunu döndürür: Mevcut bir Codex hesabını, OAuth hesabını veya API anahtarını seçer. `openai` için `main` Codex App girişini seçer. Bir Codex Havuzu seçimi bağlılığı temizler ve mevcut görünür bir görevden gelen istekler de dahil olmak -üzere bir sonraki isteğe uygulanır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilirken devam eden istekler +üzere bir sonraki isteğe uygulanır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, kapatma sırasında son bağlılık kaydı başarılı olursa proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunabilir. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilirken devam eden istekler yakaladıkları hesabı tutar. Bu yalnızca Havuz yönlendirmesini denetler; Direct modu arayana ait/yerel ana kimlik bilgisini kullanmaya devam eder. Kullanıma dayalı proaktif geçiş, 401/403 yeniden kimlik doğrulaması, 429/retry-after diff --git a/docs-site/src/content/docs/tr/reference/configuration/providers.md b/docs-site/src/content/docs/tr/reference/configuration/providers.md index 4aba2f8c083..5c9a70f54a1 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/tr/reference/configuration/providers.md @@ -57,7 +57,7 @@ Arayüzde kayıt veya OAuth girişi tamamlanınca Models sayfasını açan bir b | `activeCodexAccountPinned?` | `string` | — | Operatörün en son elle seçtiği hesap kimliği. Ayarlandığı sürece, pin tükenme, hariç tutma, silme veya açık bir yük devretme/yükseltme ile serbest bırakılana kadar daha yüksek bir `codexAccountPriorities` katmanı onu öncelikleyemez. Sınırlı katman içindeki sıradan round-robin hareketi onu serbest bırakmaz. Herhangi bir `codexAccountPriorities` girdisi yazmak da pini serbest bırakır, böylece bir sıra var olmadan önce yapılan bir pin daha sonra ayarlanan bir pinin önüne geçemez. `GET /api/codex-auth/active`, hem geçerli hesabın sabitlenip sabitlenmediğini (`pinned`) hem de tavanı taşıyan hesabı (`pinnedAccountId`) bildirir. | | `autoSwitchThreshold?` | `number` | `80` | Proaktif geçiş için kullanım eşiği. `quota`, bağımsız görevlerin bir sonraki isteğini yeniden değerlendirebilir. Bağlı görevler varsayılan olarak (`pool.cacheAffinity`) eşiğin ötesinde hesabını korur; hesap tükenene veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile bağlı görevler de bu eşikte yeniden değerlendirilir. `fill-first` bunu yalnızca bağımsız atama için tükenme noktası olarak kullanır; `round-robin` bunu yalnızca yeni/bağımsız atamalarda kullanır: etkin eşiğinin altında uygun bir hesap kaldıkça eşiğine ulaşmış veya aşmış hesapları atlar ve bağlı bir görevi bu nedenle taşımaz. Puan, bilinen en sıcak 5 saatlik, haftalık veya 30 günlük kota penceresini kullanır. `0`, yalnızca kullanıma dayalı proaktif geçişi devre dışı bırakır, bağımsız atamayı veya arıza kurtarmayı devre dışı bırakmaz. | | `codexAccountAutoSwitchThresholds?` | `Record` | — | Hesap başına `autoSwitchThreshold` geçersiz kılmaları: hesap kimliği → `0`–`100` tam sayısı. Kayıt yoksa genel değer devralınır; `0` yalnızca bu hesaptan kullanıma dayalı geçişi kapatır. Ana `__main__` hesabı desteklenir. Codex Auth hesap kartından yönetilir. Hesaba özel değeri etkinleştirmek, mevcut genel eşiği sabit bir hesap değerine kopyalar. `0` dahil bu değer, genel eşik daha sonra değişse de önceliğini korur. Devre dışı bırakmak `threshold: null` gönderir, kaydı siler ve mevcut genel eşikle gelecekteki değişikliklerini devralmayı yeniden etkinleştirir. | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | Yeni/bağımsız Codex istekleri için atama stratejisi. Bir istek, canlı (üst iş parçacığı kimliği, kota kapsamı) bağlılığı olmadığında bağımsızdır. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, kapatma sırasında son bağlılık kaydı başarılı olursa proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunabilir. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. `quota`, aktif bir hesap olmadığında en düşük kullanımlı uygun hesabı seçer, `autoSwitchThreshold` altında uygun bir aktif hesabı tutar ve eşikten sonra bağımsız bir isteği taşıyabilir. Bağlı görevler `pool.cacheAffinity`'yi izler (varsayılan açık): hesap tükenene (bilinen kullanım %100) veya hizmet veremez hale gelene kadar kalır ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Bayrağı `false` yapınca bağlı bir görev eşiğinde proaktif olarak daha düşük kullanımlı uygun bir hesaba yeniden bağlanabilir; hedef kısıtı yine geçerlidir. `round-robin`, bağımsız istekleri etkin `autoSwitchThreshold` altında kalan uygun hesaplara eşit dağıtır; hepsi eşiğe ulaşmış veya aşmışsa tüm uygun hesaplara dağıtır; `fill-first`, soğuma, kullanılamama veya yapılandırılmış tükenme eşiğine kadar bağımsız istekleri aktif hesaba atamaya devam eder. `reset-first`: Eşiğin altındaki hesaplar arasından sonraki 5 saatlik veya haftalık sıfırlaması en yakın olanı seçer. Bağlı görevler yapılandırılmış bağlılık politikasını izler. Bağımsız model kotaları kullanıma göre sıralanır. Aylık sıfırlamalar bu sıralamayı belirlemez. | | `pool.cacheAffinity?` | `boolean` | `true` | Bağlı Codex iş parçacıkları için önbellek bağlılığı sıralaması; `pool.kernel`'dan bağımsızdır. Varsayılan olarak açıktır; anahtarı atlamak veya `true` vermek bağlamayı korur ve `false` dışındaki bir değer açık okunur. Canlı bağlama kota payından öndedir: `quota`, kullanımın `autoSwitchThreshold`'u geçmesi nedeniyle iş parçacığını taşımaz, çünkü canlı bir konuşmayı taşımak hesaba özel istem önbelleğini atar. Hesap duraklatılmış, kullanılamaz, plandan dışlanmış, kimlik bilgisi geçersiz, generation değişmiş, TTL dolmuş, 429/402 kota reddi almış veya gerçekten tükenmişse (bilinen kullanım %100) iş parçacığı yine ayrılır ve yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. Kullanımı bilinmeyen bir hesap bağlı bir görev için hedef olarak asla seçilmez. Tüm hesaplar eşiğin üzerindeyse bağlı görev yerinde kalır; daha iyi bir hedef yoktur. `false` eşiğe göre yeniden bağlamayı geri getirir; hedef kısıtı yine geçerlidir. Bağlılık bir sabitleme değil yeniden sıralamadır. | | `accountPoolStickyLimit?` | `number` | `1` | İlerlemeden önce bir round-robin seçiminde tutulan yeni/bağımsız görev atamaları; sayaç yukarı akış başarısından sonra değil, bir görev bağlandığında ilerler. Aralık 1–100. | | `upstreamFailoverThreshold?` | `number` | `3` | Gelecekteki yeni oturumların yük devretmesinden önceki ardışık geçici arızalar. Devre dışı bırakmak için `0` ayarlayın. Düzenli Responses ve yerel sıkıştırma gönderimleri için kanıtlanmış bağlantı öncesi DNS/TCP erişilebilirlik arızaları sağlayıcı-ana bilgisayar düzeyinde izlenir: hesap sağlığını, hesap soğuma sürelerini, iş parçacığı/oturum bağlılığını, aktif hesap seçimini veya Havuz yönlendirmesini asla etkilemez ve bu eşiğe asla sayılmaz. | @@ -253,7 +253,7 @@ yönlendirmesi yeni/bağımsız atamayı, kullanıma dayalı proaktif geçişi v kurtarmayı ayırır. Bağlı bir görev normalde bağlılığı korur. Varsayılan olarak (`pool.cacheAffinity`) bu yeniden bağlama, bağlı hesap tükenene veya hizmet veremez hale gelene kadar bekler ve o zaman yalnızca kullanımı kesin olarak daha düşük ve gerçek kota payı olan bir hesaba geçer. `pool.cacheAffinity: false` ile `quota`, kullanım eşiği aşıldıktan sonraki isteğinde onu yeniden bağlayabilir; hedef kısıtı yine geçerlidir. 429/402 kota reddi, duraklatma, kimlik bilgisi geçersizliği ve TTL dolması bağlamayı hemen serbest bırakır; geçici arıza serisi (5xx ve diğer kota dışı arızalar) isteği başka bir hesapta sunar ama canlı bağlamayı silmez. Bağımsız bir -isteğin canlı hesap bağlaması yoktur. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunur. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. Akış öncesi +isteğin canlı hesap bağlaması yoktur. Oturum kimliği taşıyan isteklerin (normal Codex CLI ve App turları ile alt ajanlar dahil) veya doğrudan üst öğesine bağlanan oturumsuz isteklerin bağları, kapatma sırasında son bağlılık kaydı başarılı olursa proxy düzgün yeniden başlatıldığında 24 saate kadar boşta kalma süresiyle korunabilir. Daha derin oturumsuz zincirler ve Codex App ana girişine ait bağlar yalnızca bellekte kalır ve yeniden başlatmadan sonra yeniden atanır; çıkarma, hesap silme, yeniden giriş, çökme veya bağlılık sıfırlaması bir görevi bağımsız bırakabilir. Akış öncesi bir 429 veya 402, kullanıma dayalı proaktif geçiş kapalı olsa bile aynı istekte uygun alternatif bir hesapta bir kez yeniden dener. Hesap değişiklikleri görüşme bağlamını korur ve yeniden oynatır, ancak hesaplar arasında sağlayıcı tarafı diff --git a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md index 11020b09199..1c3f0893a21 100644 --- a/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-cn/reference/cli/providers-accounts.md @@ -200,7 +200,7 @@ OAuth 账号会显示为 `Account N`，而 plan/label 列会在 plan、屏蔽后 `auto` 会清除手动选择，让 Pool 重新按自身策略分配工作 — 但如果某个 Codex 账号的 id 恰为 `auto`，则精确 id 匹配优先；`ocx account clear ` 始终恢复自动选择。Codex 账号可以用 `ocx account alias` 设置的别名代替 id 来指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 和 `alias` 同样如此。对于 Codex 账号，`auto`、`main` 和 `__main__` 为保留字（不区分大小写），不能设为别名。OAuth 账号和 API 密钥的显示名称仍遵循原有规则。 选择已有的 Codex 账号、OAuth 账号或 API key。对 `openai` 而言，`main` 选择 Codex App 登录。 -Codex Pool 选择会清除 affinity，并从下一次请求开始生效，包括已有可见任务的请求；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： +Codex Pool 选择会清除 affinity，并从下一次请求开始生效，包括已有可见任务的请求；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在关闭时最后一次 affinity 保存成功的情况下，代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定，但进行中的请求保留已捕获账号。此选择只控制 Pool routing；Direct mode 继续使用 caller-owned/native main credential。基于用量的主动切换、401/403 重新认证、429/retry-after cooldown、排除，以及输出前 429/402 故障恢复之后仍可能选择其他合格 Pool 账号。这些恢复路径在关闭基于用量的切换时仍然有效。账号变化后 OpenCodex 会重放对话上下文，但 provider prompt cache 可能需要重新预热。未知 provider 或 id 返回退出码 1。`--json` 返回： 遇到 **401/403** 时，App 登录会清除该账户的进程内 affinity 并要求重新认证。 遇到 **429** 时，它会遵循 `Retry-After`、启动账户 cooldown、清除 affinity， 并可将请求切换到另一个符合条件的 Pool 账户。即使 `autoSwitchThreshold: 0`， diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index adc0d3dfe5b..41890df5aa3 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -55,7 +55,7 @@ ocx models provider openrouter on | `codexAccountPriorities?` | `Record` | — | Codex pool 各账号的选择顺序：账号 ID → `-100` 到 `100` 的整数，**数值越大越先使用**，未设置即为 `0`。这是顺序边界而非资格边界：选择会把已经合格的账号收窄到仍有 quota 余量的最高 tier，再由 `accountPoolStrategy` 在该 tier 内挑选。只有当某个 tier 的所有成员都达到各自非零的有效阈值（账号覆盖值，未设置则使用全局值）、处于 cooldown、被 soft-avoid、已暂停或需要重新认证时，该 tier 才会被跳过；usage 未知不会让 tier 耗尽。顺序不会让不合格的账号变得可选，也不会重新绑定已经绑定账号的 thread。主账号 `__main__` 同样参与排序，因此可以让 Codex Desktop 登录账号最后才被用到。没有任何条目时，行为与以往完全一致。映射格式非法时会打印警告并关闭排序（不会触发 config 修复）。可通过 `ocx account priority` 和 Codex Auth 页面管理。 | | `autoSwitchThreshold?` | `number` | `80` | 基于用量的主动切换阈值。`quota` 可在下一次请求中重新评估未绑定任务。已绑定任务默认（`pool.cacheAffinity`）在越过阈值后仍保留账号，直到该账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号。将 `pool.cacheAffinity` 设为 `false` 才会在该阈值重新评估已绑定任务。`fill-first` 仅把它用作未绑定分配的耗尽点；`round-robin` 仅在新建/未绑定任务分配时使用它：只要仍有低于自身有效阈值的合格账号，就跳过达到或超过阈值的账号，且不会因此移动已绑定任务。分数取已知 5 小时、周或 30 天 quota window 的最高值。`0` 只关闭基于用量的主动切换，不关闭未绑定任务分配或故障恢复。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各账号对 `autoSwitchThreshold` 的覆盖：账号 ID → `0`–`100` 的整数。没有条目时继承全局值；`0` 只关闭从该账号发起的基于用量的切换。支持主账号 `__main__`。可在 Codex Auth 的账号卡片中管理。 启用覆盖时，会将当前全局阈值复制为固定的账号值。覆盖值（包括 `0`）在之后修改全局阈值时仍优先。禁用时发送 `threshold: null`，删除条目，并恢复继承当前全局阈值及其未来的更改。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新建/未绑定 Codex 请求的分配策略。没有 live `(parent thread id, quota scope)` affinity 的请求属于未绑定；带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在关闭时最后一次 affinity 保存成功的情况下，代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。`quota` 在没有活跃账号时选择已知 usage 最低的合格账号；活跃账号合格且低于 `autoSwitchThreshold` 时继续使用；达到阈值后，可把未绑定请求切换到 usage 更低的合格账号。已绑定任务默认会保留到账号耗尽（已知 usage 为 100%）或无法继续服务，改绑时只前往确有额度余量且 usage 严格更低的账号。关闭该标志后，也可在该阈值把已绑定任务的下一次请求改绑到确有额度余量且 usage 严格更低的账号。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格账号间均匀分配未绑定请求，若全部达到或超过阈值则在所有合格账号间分配；`fill-first` 在 cooldown、不可用或耗尽阈值前持续分配给活跃账号。 `reset-first`: 在低于用量阈值的账号中，优先选择下次5小时或周额度重置最早的账号。已绑定任务遵循配置的亲和策略。独立模型额度按用量排序。 此排序不使用月额度重置时间。 | | `pool.cacheAffinity?` | `boolean` | `true` | 已绑定 Codex 线程的 cache-affinity 排序，独立于 `pool.kernel`。默认开启；省略该键或设为 `true` 即为开启，非法值视为开启。live 绑定优先于 quota 余量：`quota` 不会仅因用量越过 `autoSwitchThreshold` 就移动线程。账号暂停、不可用或真正耗尽（已知 usage 为 100%）时仍会离开，且只改绑到确有额度余量且 usage 严格更低的账号。用量未知的账号不会作为已绑定任务的改绑目标。设为 `false` 可恢复按阈值改绑。affinity 是重排而非钉死。 | | `accountPoolStickyLimit?` | `number` | `1` | 一次 round-robin 选择在推进前保留的新建/未绑定任务分配数。计数在任务绑定时增加，而不是在上游成功后增加。范围 1–100；仅当 `accountPoolStrategy` 为 `round-robin` 时生效。 | | `upstreamFailoverThreshold?` | `number` | `3` | 连续发生多少次瞬态故障后，后续新会话会切换到备用上游。设为 `0` 可禁用。对于常规 Responses 和原生 compact 发送，已证明的连接前 DNS/TCP 不可达故障按 provider-host 粒度记录，不影响账户健康、账户冷却、线程/会话亲和性、活动账户选择或 Pool 路由，也不会计入此阈值。 | @@ -249,7 +249,7 @@ OpenCodex 不会依据网关名称或 URL 猜测可信度。如果同一网关 请在仪表盘 **Codex Auth** 页面添加 pool account 并刷新 quota。配置只保存非 secret account metadata；access/refresh token 存放在加固的 Codex account credential store 中。Pool routing 分为新建/未绑定任务分配、基于用量的主动切换和故障恢复。已绑定任务通常保持 affinity。默认（`pool.cacheAffinity`）下，该重新绑定会等到绑定账号耗尽或无法继续服务，并且只改绑到确有额度余量且 usage 严格更低的账号；所有账号都高于阈值时，已绑定任务留在原账号。关闭该标志后，`quota` 可在超过阈值后的下一次请求中重新绑定，但仍只改绑到确有额度余量且 usage 严格更低的账号。暂停、cooldown、重新认证和故障处理也能独立清除或改变 -routing。未绑定请求没有 live 账号绑定。带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。输出前的 +routing。未绑定请求没有 live 账号绑定。带会话 ID 的请求（包括常规 Codex CLI 和 App 轮次及子代理），或锚定到直接父任务的无会话请求，其绑定在关闭时最后一次 affinity 保存成功的情况下，代理正常重启后可保留，空闲时间最长为 24 小时。更深层的无会话链和 Codex App 主登录的绑定仅保存在内存中，重启后会重新分配；驱逐、账号删除、重新登录、崩溃或 affinity 重置仍可能使任务变为未绑定。输出前的 **429/402** 即使在关闭基于用量的主动切换时，也可在同一请求中对合格替代账号重试一次。 账号变化后会保留并重放对话上下文，但账号间的 provider prompt cache 不保证复用，可能需要重新预热。 暂停后仍会显示账号及其 quota metadata，但不会参与自动切换、重试/failover 选择、cooldown 恢复探测或手动激活。 diff --git a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md index 836be478529..c43aaade0e3 100644 --- a/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/zh-tw/reference/cli/providers-accounts.md @@ -158,7 +158,7 @@ Codex 池選擇套用於清除既有親和性後的下一個請求；進行中 `auto` 會清除手動選擇，讓池重新依自身策略分配工作 — 但若 Codex 帳號的 id 恰為 `auto`，則精確 id 比對優先；`ocx account clear ` 一律還原自動選擇。Codex 帳號可以用 `ocx account alias` 設定的別名代替 id 來指定；`priority`、`pause`、`resume`、`clear-cooldown`、`remove` 與 `alias` 亦然。對於 Codex 帳號，`auto`、`main` 和 `__main__` 為保留字（不區分大小寫），不能設為別名。OAuth 帳號與 API 金鑰的顯示名稱仍遵循原有規則。 -選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除親和性並套用於下一個請求，包含來自既有可見任務的請求；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 +選擇既有的 Codex 帳號、OAuth 帳號或 API 金鑰。對於 `openai`，`main` 選擇 Codex App 登入。Codex 池選擇清除親和性並套用於下一個請求，包含來自既有可見任務的請求；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在關閉時最後一次親和性儲存成功的情況下，代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定，而進行中的請求保留其擷取的帳號。這僅控制池路由；Direct 模式繼續使用呼叫者擁有／原生的 main 憑證。基於用量的主動切換、401/403 重新認證、429/retry-after 冷卻、排除，以及 pre-output 429/402 失敗復原稍後可能選擇另一個合格的池帳號。當基於用量的切換關閉時，這些復原路徑仍然活躍。OpenCodex 在帳號變更後重播對話，但供應商端的 prompt cache 可能是冷的。未知的供應商或 id 離開 1。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 `--json` 回傳： diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index c2adfed1320..80b14de4608 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -53,7 +53,7 @@ ocx models provider openrouter on | `activeCodexAccountId?` | `string` | — | 為下一個請求手動選擇的池帳號。選擇清除執行緒親和性；進行中的請求保留擷取的憑證。 | | `autoSwitchThreshold?` | `number` | `80` | 主動切換的用量閾值。`quota` 可在下一個請求時重新評估未綁定任務。綁定任務預設（`pool.cacheAffinity`）在越過閾值後仍會保留帳號，直到該帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號。將 `pool.cacheAffinity` 設為 `false` 才會在此閾值重新評估綁定任務。`fill-first` 僅將其用作未綁定指派的排空點；`round-robin` 僅在新／未綁定工作指派時使用它：只要仍有低於自身有效閾值的合格帳號，就略過達到或超過閾值的帳號，且不會因此移動已綁定任務。分數使用最熱的已知 5h、週或 30d 配額視窗。`0` 僅停用基於用量的主動切換，而非未綁定指派或失敗復原。 | | `codexAccountAutoSwitchThresholds?` | `Record` | — | 各帳號對 `autoSwitchThreshold` 的覆寫：帳號 ID → `0`–`100` 的整數。沒有項目時繼承全域值；`0` 只停用從該帳號發起的使用量主動切換。支援主帳號 `__main__`。可在 Codex Auth 的帳號卡片中管理。 啟用覆寫時，會將目前全域閾值複製為固定的帳號值。覆寫值（包括 `0`）在之後修改全域閾值時仍優先。停用時傳送 `threshold: null`，刪除項目，並恢復繼承目前全域閾值及其未來的變更。 | -| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | +| `accountPoolStrategy?` | `"quota" \| "round-robin" \| "fill-first" \| "reset-first"` | `"quota"` | 新／未綁定 Codex 請求的指派策略。當請求沒有即時（父執行緒 id、配額 scope）親和性時即為未綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在關閉時最後一次親和性儲存成功的情況下，代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。`quota` 在無現用帳號時選擇最低用量的合格帳號，將合格現用帳號保持在 `autoSwitchThreshold` 以下，且在閾值後可將未綁定請求移至較低用量的合格帳號。綁定任務預設會保留到帳號耗盡（已知用量 100%）或無法繼續服務，改綁時只前往確有額度餘裕且用量嚴格更低的帳號。關閉該設定後，也可在閾值將綁定任務的下一個請求改綁到確有額度餘裕且用量嚴格更低的帳號。`round-robin` 在有效 `autoSwitchThreshold` 以下的合格帳號間均勻分配未綁定請求，若全部達到或超過閾值則在所有合格帳號間分配；`fill-first` 持續將未綁定請求指派到現用帳號直到冷卻、不可用或設定的排空閾值。 `reset-first`: 在低於用量門檻的帳號中，優先選擇下次5小時或週額度重設最早的帳號。已綁定任務遵循設定的親和策略。獨立模型額度按用量排序。 此排序不使用月額度重設時間。 | | `pool.cacheAffinity?` | `boolean` | `true` | 綁定 Codex 執行緒的 cache-affinity 排序，獨立於 `pool.kernel`。預設開啟；省略該鍵或設為 `true` 即為開啟，格式錯誤視為開啟。即時綁定優先於配額餘裕：`quota` 不會只因用量越過 `autoSwitchThreshold` 就移動執行緒。帳號暫停、無法使用或真正耗盡（已知用量 100%）時仍會離開，且只改綁到確有額度餘裕且用量嚴格更低的帳號。用量未知的帳號不會作為綁定任務的改綁目標。設為 `false` 可恢復依閾值重新綁定。親和性是重排而非釘死。 | | `accountPoolStickyLimit?` | `number` | `1` | 在前進一個 round-robin 選擇前保留的新／未綁定任務指派；計數器在任務綁定時前進，而非在上游成功後。範圍 1–100。 | | `upstreamFailoverThreshold?` | `number` | `3` | 未來新 session 容錯移轉前的連續暫時性失敗。設 `0` 停用。 | @@ -186,7 +186,7 @@ API-key 供應商可持有字面值金鑰或環境參考。OAuth 供應商使用 ## Codex 帳號池 -在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 +在儀表板中使用 **Codex Auth** 新增池帳號並重新整理配額。`config.json` 儲存非秘密中繼資料；access 與 refresh token 使用強化的憑證存放。池路由將新／未綁定指派、基於用量的主動切換與失敗復原分開。綁定任務通常保留親和性。預設（`pool.cacheAffinity`）下，該重新綁定會等到綁定帳號耗盡或無法繼續服務，並且只改綁到確有額度餘裕且用量嚴格更低的帳號；所有帳號都高於閾值時，綁定任務留在原帳號。關閉該設定後，`quota` 可在超過用量閾值後的下一個請求時重新綁定它，但仍只改綁到確有額度餘裕且用量嚴格更低的帳號。暫停、冷卻、重新認證與失敗處理可獨立清除或移動路由。未綁定請求沒有即時帳號綁定；帶有工作階段 ID 的請求（包括一般 Codex CLI 和 App 回合及子代理），或錨定到直接父任務的無工作階段請求，其綁定在關閉時最後一次親和性儲存成功的情況下，代理正常重啟後可保留，閒置時間最長為 24 小時。更深層的無工作階段鏈和 Codex App 主要登入的綁定僅保留在記憶體中，重啟後會重新指派；驅逐、帳號刪除、重新登入、崩潰或親和性重置仍可能使任務未綁定。Pre-stream 的 429 或 402 在同一個請求中於一個合格的備用帳號上重試一次，即使基於用量的主動切換關閉。帳號變更保留並重播對話 context，但跨帳號的供應商端 prompt-cache 重用不保證，cache 可能需要重新暖機。 在 **401/403** 時，App 登入清除該帳號的行程本地親和性並要求重新認證。 在 **429** 時，opencodex 遵循 `Retry-After`、啟動帳號冷卻、清除親和性，並可能將請求輪換到另一個合格的池帳號。這些失敗轉換在 `autoSwitchThreshold: 0` 時仍然活躍；該設定僅停用基於用量的主動切換。 diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index f4380feb870..3285320c8f1 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -706,7 +706,7 @@ "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-affinity-lease-gap.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index b2ead0dfcc8..e1749db4013 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -53,11 +53,6 @@ type LiveReader = { issuer( key: string, ): { accountId: string; fence?: string; lastUsedAt: number } | undefined; - /** Every live pin and issuer, for re-persisting memory into a newly opened database. */ - forEach( - thread: (key: string, scope: string, entry: { accountId: string; identityFence?: string; createdAt: number; lastUsedAt: number }) => void, - issuer: (key: string, entry: { accountId: string; fence?: string; lastUsedAt: number }) => void, - ): void; clearLive(): void; }; type Op = { @@ -101,10 +96,14 @@ const BASE_SCOPES_EXHAUSTIVE: Exclude< void BASE_SCOPES_EXHAUSTIVE; let current: Scope | undefined; let observedHome: string | undefined; -/** Whether an earlier scope in this process actually opened (and so may have persisted) the database. */ -let hadPersistentScope = false; -/** Pins or issuers changed while no lease was held; the next scope re-persists memory. */ -let changedWithoutLease = false; +/** + * Structural changes made while no lease was held, after a scope of the same home had persisted. + * The next scope of that home replays them against its database in order. Bounded; an overflow + * makes the next scope clear the database instead, which loses pins but never revives one. + */ +let gapOps: Array<() => void> = []; +let gapOverflow = false; +const MAX_GAP_OPS = 4096; /** Anchors of the last retired scope, handed to the next scope of the same home. */ let carried: { home: string; anchors: Scope["anchors"] } | undefined; let reader: LiveReader | undefined; @@ -276,19 +275,24 @@ function open(s: Scope): void { warning(); } } -function scope(acquiredHome?: string): Scope | undefined { - if (!acquiredHome && isPassiveRead()) return undefined; +function resolvedHome(acquiredHome?: string): string { let home = resolve(acquiredHome ?? getConfigDir()); try { home = realpathSync.native(home); } catch { /* ownership check below refuses it */ } - if (process.platform === "win32") home = home.toLowerCase(); + return process.platform === "win32" ? home.toLowerCase() : home; +} +function scope(acquiredHome?: string): Scope | undefined { + if (!acquiredHome && isPassiveRead()) return undefined; + const home = resolvedHome(acquiredHome); // A lease ends a store scope; only a change of home invalidates live maps. if (observedHome !== undefined && observedHome !== home) { retire(); carried = undefined; + gapOps = []; + gapOverflow = false; reader?.clearLive(); } observedHome = home; @@ -330,41 +334,33 @@ function scope(acquiredHome?: string): Scope | undefined { carried = undefined; current = s; open(s); - if (s.db) { - if (hadPersistentScope && changedWithoutLease) persistLiveState(s); - changedWithoutLease = false; - hadPersistentScope = true; - } + replayGapOps(s); return s; } /** - * The scope for a change to which pins or issuers exist. With no scope open after an earlier one - * persisted, the change happened while no lease was held, so the next scope must re-persist memory. + * The scope for a change to which pins or issuers exist. With no lease held after a scope of this + * home persisted, the change is queued for the next scope instead of being lost: live memory keeps + * routing with it, so the database must learn it too, without disturbing rows nobody touched. */ -function structuralScope(): Scope | undefined { +function gapOrScope(replay: () => void): Scope | undefined { const s = usable(); - if (!s && !current && hadPersistentScope && !isPassiveRead()) changedWithoutLease = true; - return s; -} -/** - * A later scope in the same process makes the database equal to memory when memory changed while - * no lease was held (a late bind, delete or issuer write after a - * release). Memory is what this process routes with, so it wins: every stored row is dropped and - * every live pin and issuer with a known anchor is written again. A pin deleted during the gap is - * therefore not revived by a later restart, and a pin rebound during the gap is saved as rebound. - * Live entries without a known anchor stay memory-only, like every other unanchored key. - */ -function persistLiveState(s: Scope): void { - s.threadKeys.clear(); - s.issuerKeys.clear(); - enqueue(s, { kind: "clearAll", sql: "DELETE FROM threads; DELETE FROM issuers", args: [] }); - reader?.forEach( - (key, scopeName, entry) => { - if (!entry.identityFence || entry.accountId === "__main__") return; - recordThreadBind(key, scopeName, { scope: scopeName, accountId: entry.accountId, fence: entry.identityFence, createdAt: entry.createdAt, lastUsedAt: entry.lastUsedAt }); - }, - (key, entry) => recordIssuerSet(key, entry.accountId, entry.fence ?? null, entry.lastUsedAt), - ); + if (s || current || !carried || isPassiveRead() || carried.home !== resolvedHome()) return s; + if (gapOps.length >= MAX_GAP_OPS) gapOverflow = true; + else gapOps.push(replay); + return undefined; +} +/** Apply the changes queued while no lease was held, in order, to the newly opened database. */ +function replayGapOps(s: Scope): void { + const ops = gapOps; + const overflow = gapOverflow; + gapOps = []; + gapOverflow = false; + if (!s.db) return; + if (overflow) { + recordAllCleared(); + return; + } + for (const op of ops) op(); } function usable(): Scope | undefined { const s = scope(); @@ -735,7 +731,8 @@ export function recordThreadBind( scopeName: string, row: StoredThreadRow, ): void { - const s = structuralScope(); + if (isRequestOwnedAffinityKey(key)) return; + const s = gapOrScope(() => recordThreadBind(key, scopeName, row)); if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -765,7 +762,8 @@ export function recordThreadTouch(key: string, scopeName: string): void { scheduleTouch(s); } export function recordThreadDelete(key: string, scopeName?: string): void { - const s = structuralScope(); + if (isRequestOwnedAffinityKey(key)) return; + const s = gapOrScope(() => recordThreadDelete(key, scopeName)); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -791,7 +789,8 @@ export function recordThreadDelete(key: string, scopeName?: string): void { ); } export function recordThreadAccountDeleted(key: string, id: string): void { - const s = structuralScope(); + if (isRequestOwnedAffinityKey(key)) return; + const s = gapOrScope(() => recordThreadAccountDeleted(key, id)); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -803,7 +802,7 @@ export function recordThreadAccountDeleted(key: string, id: string): void { enqueue(s, { kind: "deleteKeyAccount", key: rk, sql: "DELETE FROM threads WHERE row_key=? AND account_id=?", args: [rk, id] }); } export function recordAccountPinsCleared(id: string): void { - const s = structuralScope(); + const s = gapOrScope(() => recordAccountPinsCleared(id)); if (s) { for (const [key, entry] of s.threadKeys) { for (const [scopeName, account] of entry.scopes) if (account === id) entry.scopes.delete(scopeName); @@ -817,7 +816,7 @@ export function recordAccountPinsCleared(id: string): void { } } export function recordAllCleared(): void { - const s = structuralScope(); + const s = gapOrScope(() => recordAllCleared()); if (s) { s.threadKeys.clear(); s.issuerKeys.clear(); enqueue(s, { @@ -829,7 +828,7 @@ export function recordAllCleared(): void { } /** A manual account switch clears pins but keeps the issuers of carried state. */ export function recordThreadsCleared(): void { - const s = structuralScope(); + const s = gapOrScope(() => recordThreadsCleared()); if (s) { s.threadKeys.clear(); enqueue(s, { kind: "clearThreads", sql: "DELETE FROM threads", args: [] }); @@ -845,7 +844,8 @@ export function recordIssuerSet( fence: string | null, lastUsedAt: number, ): void { - const s = structuralScope(); + if (isRequestOwnedAffinityKey(key)) return; + const s = gapOrScope(() => recordIssuerSet(key, id, fence, lastUsedAt)); if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; @@ -867,7 +867,8 @@ export function recordIssuerTouch(key: string): void { scheduleTouch(s); } export function recordIssuerDelete(key: string): void { - const s = structuralScope(); + if (isRequestOwnedAffinityKey(key)) return; + const s = gapOrScope(() => recordIssuerDelete(key)); if (!s) return; const rk = rowKey(s, key, "issuer"); if (rk) { @@ -881,7 +882,7 @@ export function recordIssuerDelete(key: string): void { } } export function recordIssuersCleared(): void { - const s = structuralScope(); + const s = gapOrScope(() => recordIssuersCleared()); if (s) { s.issuerKeys.clear(); enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); @@ -895,8 +896,8 @@ export function resetCodexAffinityStoreForTests(): void { flush(current, false); retire(); } - // Model a fresh process: no earlier persisted scope, no change pending from a lease gap. - hadPersistentScope = false; - changedWithoutLease = false; + // Model a fresh process: nothing carried from an earlier scope, no change pending from a lease gap. carried = undefined; + gapOps = []; + gapOverflow = false; } diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts index d0b5e42a037..4ada4044eaa 100644 --- a/src/codex/routing/thread-affinity.ts +++ b/src/codex/routing/thread-affinity.ts @@ -221,12 +221,6 @@ registerCodexAffinityLiveState({ const entry = conversationStateIssuerMap.get(key); return entry && { accountId: entry.storedAccountId ?? entry.accountId, fence: entry.storedAccountId ? entry.storedFence ?? undefined : entry.fence, lastUsedAt: entry.lastUsedAt }; }, - forEach: (thread, issuer) => { - for (const [key, affinities] of threadAccountMap) for (const [scope, entry] of affinities) thread(key, scope, entry); - for (const [key, entry] of conversationStateIssuerMap) { - issuer(key, { accountId: entry.storedAccountId ?? entry.accountId, fence: entry.storedAccountId ? entry.storedFence ?? undefined : entry.fence, lastUsedAt: entry.lastUsedAt }); - } - }, clearLive: () => { threadAccountMap.clear(); threadAffinityEntryTotal = 0; conversationStateIssuerMap.clear(); }, }); } diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 65325bc83fd..ce89aef8d86 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -9,6 +9,6 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. When pins or issuers changed while no lease was held, the next lease rewrites the database from memory, so a restart restores what the process was routing with and never a row it had already replaced or deleted. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. Structural changes made while no lease is held (binds, deletes, clears, issuer writes for durable keys) are queued and replayed in order against the database when the next lease of the same home opens it, so a restart never revives a pin the process replaced or deleted, and rows for conversations the process never loaded stay intact. A change of config home discards the queue; if the queue overflows its 4,096-entry bound, the next lease clears the database instead. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). A home without an ownership manifest, created before ownership tracking, still uses the store; uninstall removes nothing from such a home, as for every other file in it. `tests/codex-integration/codex-affinity-store.test.ts`, `tests/codex-integration/codex-affinity-restart.test.ts` and `tests/codex-integration/codex-affinity-request-owned.test.ts` cover store, restart and request-owned exclusion behavior. diff --git a/tests/codex-integration/codex-affinity-lease-gap.test.ts b/tests/codex-integration/codex-affinity-lease-gap.test.ts new file mode 100644 index 00000000000..e4147812f26 --- /dev/null +++ b/tests/codex-integration/codex-affinity-lease-gap.test.ts @@ -0,0 +1,78 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { readCodexAccountRecord, saveCodexAccountCredential } from "../../src/codex/account-store"; +import { codexPoolAffinityKey } from "../../src/codex/auth-context"; +import { requestOwnedAffinityKey } from "../../src/codex/request-owned-affinity"; +import { bindThreadAffinity, clearThreadAccountMap, getThreadAffinity, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; +import { adoptStoredIssuer, adoptStoredThreadRows, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; +import { acquireSpendLedgerOwner, type SpendLedgerOwnerLease } from "../../src/lib/spend-ledger-owner"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +let homes: string[] = []; +let prior: string | undefined; +let lease: SpendLedgerOwnerLease | undefined; +const conversation = (id: string) => new Headers({ authorization: "Bearer caller", "session-id": `gap-${id}`, "thread-id": `gap-${id}` }); + +function useHome(home: string): void { process.env.OPENCODEX_HOME = home; } +function newHome(): string { const home = mkdtempSync(join(tmpdir(), "ocx-affinity-gap-")); homes.push(home); return home; } +function seedAccounts(): void { + for (const id of ["a", "b"]) saveCodexAccountCredential(id, { accessToken: `access-${id}`, refreshToken: `refresh-${id}`, expiresAt: Date.now() + 600_000, chatgptAccountId: `chat-${id}` }); +} +function credential(id: string) { return { generation: readCodexAccountRecord(id)!.generation, chatgptAccountId: `chat-${id}` }; } +/** A fresh process: no live affinity, nothing carried in the store. */ +function freshProcess(): void { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); clearThreadAccountMap(); resetCodexAffinityStoreForTests(); } +function storedPin(id: string): string | undefined { return adoptStoredThreadRows(codexPoolAffinityKey(conversation(id))!, Date.now())[0]?.accountId; } +function storedIssuer(id: string): string | undefined { return adoptStoredIssuer(codexPoolAffinityKey(conversation(id))!, Date.now())?.issuer; } + +beforeEach(() => { prior = process.env.OPENCODEX_HOME; freshProcess(); }); +afterEach(() => { + freshProcess(); + if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; + for (const home of homes) removeTreeWithRetry(home); + homes = []; +}); + +test("a change made between leases keeps rows for conversations this process never loaded", () => { + useHome(newHome()); + lease = acquireSpendLedgerOwner(); seedAccounts(); + for (const id of ["one", "two"]) { + const key = codexPoolAffinityKey(conversation(id))!; + bindThreadAffinity(key, "a", Date.now()); + rememberConversationStateIssuer(key, "a", Date.now(), credential("a")); + } + freshProcess(); + // A later process loads only conversation "one", releases, and rebinds it with no lease held. + lease = acquireSpendLedgerOwner(); + const one = codexPoolAffinityKey(conversation("one"))!; + expect(getThreadAffinity(one)?.accountId).toBe("a"); + lease.release(); lease = undefined; + bindThreadAffinity(one, "b", Date.now()); + // A request-owned key changed in the same gap must not reach disk either. + bindThreadAffinity(requestOwnedAffinityKey(conversation("one"))!, "b", Date.now()); + lease = acquireSpendLedgerOwner(); + freshProcess(); lease = acquireSpendLedgerOwner(); + expect(storedPin("one")).toBe("b"); + expect(storedPin("two")).toBe("a"); + expect(storedIssuer("two")).toBe("a"); +}); + +test("a change made between leases in one home never touches another home's database", () => { + const homeB = newHome(); + useHome(homeB); + lease = acquireSpendLedgerOwner(); seedAccounts(); + bindThreadAffinity(codexPoolAffinityKey(conversation("b-only"))!, "a", Date.now()); + freshProcess(); + const homeA = newHome(); + useHome(homeA); + lease = acquireSpendLedgerOwner(); seedAccounts(); + const inA = codexPoolAffinityKey(conversation("a-only"))!; + bindThreadAffinity(inA, "a", Date.now()); + lease.release(); lease = undefined; + bindThreadAffinity(inA, "b", Date.now()); + useHome(homeB); + lease = acquireSpendLedgerOwner(); + freshProcess(); useHome(homeB); lease = acquireSpendLedgerOwner(); + expect(storedPin("b-only")).toBe("a"); +}); diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 8c03f15c2a2..1b471b80dad 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -27,7 +27,7 @@ function bind(fence: string): string { } beforeEach(() => { prior = process.env.OPENCODEX_HOME; home = mkdtempSync(join(tmpdir(), "ocx-affinity-")); process.env.OPENCODEX_HOME = home; live = undefined; - registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, forEach: (thread) => { if (live) thread(key(), "legacy", live); }, clearLive: () => { live = undefined; } }); + registerCodexAffinityLiveState({ thread: () => live, issuer: () => undefined, clearLive: () => { live = undefined; } }); }); afterEach(() => { lease?.release(); lease = undefined; resetCodexAffinityStoreForTests(); registerThreadAffinityLiveState(); if (prior === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = prior; removeTreeWithRetry(home); }); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index cab52deebab..91ca46d40b0 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -698,7 +698,7 @@ "codex-restart-contract-parity.test.ts": "codex-integration", "codex-restart-route.test.ts": "codex-integration", "codex-restore-app-rewrite.test.ts": "codex-integration", "codex-retained-root-serialization.test.ts": "codex-integration", - "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", + "codex-affinity-restart.test.ts": "codex-integration", "codex-affinity-restart-matrix.test.ts": "codex-integration", "codex-affinity-request-owned.test.ts": "codex-integration", "codex-affinity-lease-gap.test.ts": "codex-integration", "codex-routing-cache-affinity-detour.test.ts": "codex-integration", "codex-routing-elapsed-short-reset.test.ts": "codex-integration", "codex-routing.test.ts": "codex-integration", "codex-runtime.test.ts": "codex-integration", "codex-runtime-macos-desktop.test.ts": "codex-integration", "codex-runtime-wsl-desktop.test.ts": "codex-integration", "codex-clamp-notice.test.ts": "codex-integration", "codex-service-manager-probe-hardening.test.ts": "codex-integration", "codex-service-manager-probe.test.ts": "codex-integration", "codex-systemd-env-probe.test.ts": "codex-integration", From bb491809b61648501bfee35b3192f353c75e4756 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 19:31:27 +0900 Subject: [PATCH 19/21] fix(codex): fail closed after a lease gap instead of replaying; carry row keys across lease changes --- src/codex/routing/thread-affinity-store.ts | 85 +++++++++---------- structure/providers/codex-pool-affinity.md | 2 +- .../codex-affinity-lease-gap.test.ts | 33 +++++-- .../codex-affinity-restart-matrix.test.ts | 10 +-- .../codex-affinity-store.test.ts | 14 +++ 5 files changed, 86 insertions(+), 58 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index e1749db4013..25982a2e751 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -96,16 +96,19 @@ const BASE_SCOPES_EXHAUSTIVE: Exclude< void BASE_SCOPES_EXHAUSTIVE; let current: Scope | undefined; let observedHome: string | undefined; +/** The home whose database the most recent scope in this process actually opened. */ +let persistedHome: string | undefined; /** - * Structural changes made while no lease was held, after a scope of the same home had persisted. - * The next scope of that home replays them against its database in order. Bounded; an overflow - * makes the next scope clear the database instead, which loses pins but never revives one. + * A persistable change to pins or issuers happened while that home's database was not writable + * (between leases, or while a scope could not open it). The database may still hold the row the + * change replaced or deleted, so the next scope that opens it clears it first. Live memory keeps + * routing as before; a later restart then restores nothing rather than something stale. Only a + * process that releases and reacquires its lease reaches this: a serving proxy holds one lease for + * its whole life. */ -let gapOps: Array<() => void> = []; -let gapOverflow = false; -const MAX_GAP_OPS = 4096; +let needsClear = false; /** Anchors of the last retired scope, handed to the next scope of the same home. */ -let carried: { home: string; anchors: Scope["anchors"] } | undefined; +let carried: { home: string; anchors: Scope["anchors"]; threadKeys: Scope["threadKeys"]; issuerKeys: Scope["issuerKeys"] } | undefined; let reader: LiveReader | undefined; export function registerCodexAffinityLiveState(value: LiveReader): void { @@ -122,7 +125,8 @@ function retire(): void { current = undefined; // Anchors are process-local facts about this home's conversations; the next scope of the same // home needs them to re-derive row keys for pins that are still live in memory. - if (s.db) carried = { home: s.home, anchors: s.anchors }; + // Row keys too: after an anchor is evicted they are the only way a delete can still reach its row. + if (s.db && !s.poisoned) carried = { home: s.home, anchors: s.anchors, threadKeys: s.threadKeys, issuerKeys: s.issuerKeys }; if (s.timer) clearTimeout(s.timer); if (s.pruneTimer) clearInterval(s.pruneTimer); try { @@ -291,8 +295,8 @@ function scope(acquiredHome?: string): Scope | undefined { if (observedHome !== undefined && observedHome !== home) { retire(); carried = undefined; - gapOps = []; - gapOverflow = false; + persistedHome = undefined; + needsClear = false; reader?.clearLive(); } observedHome = home; @@ -326,41 +330,30 @@ function scope(acquiredHome?: string): Scope | undefined { threads: new Map(), issuers: new Map(), anchors: carried?.home === home ? carried.anchors : new Map(), - threadKeys: new Map(), - issuerKeys: new Map(), + threadKeys: carried?.home === home ? carried.threadKeys : new Map(), + issuerKeys: carried?.home === home ? carried.issuerKeys : new Map(), consulted: new Set(), scheduled: false, }; carried = undefined; current = s; open(s); - replayGapOps(s); + if (s.db) { + if (needsClear && persistedHome === home) recordAllCleared(); + needsClear = false; + persistedHome = home; + } return s; } /** - * The scope for a change to which pins or issuers exist. With no lease held after a scope of this - * home persisted, the change is queued for the next scope instead of being lost: live memory keeps - * routing with it, so the database must learn it too, without disturbing rows nobody touched. + * The scope for a change to which pins or issuers exist. When the change could have been persisted + * but this home's database is not writable right now, remember that the database is stale. */ -function gapOrScope(replay: () => void): Scope | undefined { +function writableScope(persistable: boolean): Scope | undefined { const s = usable(); - if (s || current || !carried || isPassiveRead() || carried.home !== resolvedHome()) return s; - if (gapOps.length >= MAX_GAP_OPS) gapOverflow = true; - else gapOps.push(replay); - return undefined; -} -/** Apply the changes queued while no lease was held, in order, to the newly opened database. */ -function replayGapOps(s: Scope): void { - const ops = gapOps; - const overflow = gapOverflow; - gapOps = []; - gapOverflow = false; - if (!s.db) return; - if (overflow) { - recordAllCleared(); - return; - } - for (const op of ops) op(); + if (!s && persistable && persistedHome !== undefined && !isPassiveRead() && persistedHome === resolvedHome()) + needsClear = true; + return s; } function usable(): Scope | undefined { const s = scope(); @@ -732,7 +725,7 @@ export function recordThreadBind( row: StoredThreadRow, ): void { if (isRequestOwnedAffinityKey(key)) return; - const s = gapOrScope(() => recordThreadBind(key, scopeName, row)); + const s = writableScope(row.accountId !== "__main__" && !!row.fence); if (!s || row.accountId === "__main__" || !row.fence) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -763,7 +756,7 @@ export function recordThreadTouch(key: string, scopeName: string): void { } export function recordThreadDelete(key: string, scopeName?: string): void { if (isRequestOwnedAffinityKey(key)) return; - const s = gapOrScope(() => recordThreadDelete(key, scopeName)); + const s = writableScope(true); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -790,7 +783,7 @@ export function recordThreadDelete(key: string, scopeName?: string): void { } export function recordThreadAccountDeleted(key: string, id: string): void { if (isRequestOwnedAffinityKey(key)) return; - const s = gapOrScope(() => recordThreadAccountDeleted(key, id)); + const s = writableScope(true); if (!s) return; const rk = rowKey(s, key, "conversation"); if (!rk) return; @@ -802,7 +795,7 @@ export function recordThreadAccountDeleted(key: string, id: string): void { enqueue(s, { kind: "deleteKeyAccount", key: rk, sql: "DELETE FROM threads WHERE row_key=? AND account_id=?", args: [rk, id] }); } export function recordAccountPinsCleared(id: string): void { - const s = gapOrScope(() => recordAccountPinsCleared(id)); + const s = writableScope(true); if (s) { for (const [key, entry] of s.threadKeys) { for (const [scopeName, account] of entry.scopes) if (account === id) entry.scopes.delete(scopeName); @@ -816,7 +809,7 @@ export function recordAccountPinsCleared(id: string): void { } } export function recordAllCleared(): void { - const s = gapOrScope(() => recordAllCleared()); + const s = writableScope(true); if (s) { s.threadKeys.clear(); s.issuerKeys.clear(); enqueue(s, { @@ -828,7 +821,7 @@ export function recordAllCleared(): void { } /** A manual account switch clears pins but keeps the issuers of carried state. */ export function recordThreadsCleared(): void { - const s = gapOrScope(() => recordThreadsCleared()); + const s = writableScope(true); if (s) { s.threadKeys.clear(); enqueue(s, { kind: "clearThreads", sql: "DELETE FROM threads", args: [] }); @@ -845,7 +838,7 @@ export function recordIssuerSet( lastUsedAt: number, ): void { if (isRequestOwnedAffinityKey(key)) return; - const s = gapOrScope(() => recordIssuerSet(key, id, fence, lastUsedAt)); + const s = writableScope(id !== "__main__"); if (!s || id === "__main__") return; const rk = rowKey(s, key, "issuer"); if (!rk) return; @@ -868,7 +861,7 @@ export function recordIssuerTouch(key: string): void { } export function recordIssuerDelete(key: string): void { if (isRequestOwnedAffinityKey(key)) return; - const s = gapOrScope(() => recordIssuerDelete(key)); + const s = writableScope(true); if (!s) return; const rk = rowKey(s, key, "issuer"); if (rk) { @@ -882,7 +875,7 @@ export function recordIssuerDelete(key: string): void { } } export function recordIssuersCleared(): void { - const s = gapOrScope(() => recordIssuersCleared()); + const s = writableScope(true); if (s) { s.issuerKeys.clear(); enqueue(s, { kind: "clearIssuers", sql: "DELETE FROM issuers", args: [] }); @@ -896,8 +889,8 @@ export function resetCodexAffinityStoreForTests(): void { flush(current, false); retire(); } - // Model a fresh process: nothing carried from an earlier scope, no change pending from a lease gap. + // Model a fresh process: nothing carried from an earlier scope, nothing known to be stale. carried = undefined; - gapOps = []; - gapOverflow = false; + persistedHome = undefined; + needsClear = false; } diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index ce89aef8d86..c7dc74a6ab4 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -9,6 +9,6 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. Structural changes made while no lease is held (binds, deletes, clears, issuer writes for durable keys) are queued and replayed in order against the database when the next lease of the same home opens it, so a restart never revives a pin the process replaced or deleted, and rows for conversations the process never loaded stay intact. A change of config home discards the queue; if the queue overflows its 4,096-entry bound, the next lease clears the database instead. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. A persistable change to pins or issuers made while the home's database is not writable (between two leases of one process, or while a scope could not open it) marks that database stale; the next scope that opens it clears it before use, so a restart restores nothing rather than a row the process had replaced or deleted. This also drops rows for conversations the process never loaded, which loses pins but never revives one; a serving proxy holds one lease for its whole life and does not reach this path. Row keys and anchors carry across a lease change in the same home, so deletes still find their rows after an anchor is evicted. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). A home without an ownership manifest, created before ownership tracking, still uses the store; uninstall removes nothing from such a home, as for every other file in it. `tests/codex-integration/codex-affinity-store.test.ts`, `tests/codex-integration/codex-affinity-restart.test.ts` and `tests/codex-integration/codex-affinity-request-owned.test.ts` cover store, restart and request-owned exclusion behavior. diff --git a/tests/codex-integration/codex-affinity-lease-gap.test.ts b/tests/codex-integration/codex-affinity-lease-gap.test.ts index e4147812f26..4e1cea8f4cb 100644 --- a/tests/codex-integration/codex-affinity-lease-gap.test.ts +++ b/tests/codex-integration/codex-affinity-lease-gap.test.ts @@ -1,11 +1,11 @@ import { afterEach, beforeEach, expect, test } from "bun:test"; -import { mkdtempSync } from "node:fs"; +import { mkdirSync, mkdtempSync, renameSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { readCodexAccountRecord, saveCodexAccountCredential } from "../../src/codex/account-store"; import { codexPoolAffinityKey } from "../../src/codex/auth-context"; import { requestOwnedAffinityKey } from "../../src/codex/request-owned-affinity"; -import { bindThreadAffinity, clearThreadAccountMap, getThreadAffinity, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; +import { bindThreadAffinity, clearThreadAccountMap, deleteThreadAffinity, getThreadAffinity, rememberConversationStateIssuer } from "../../src/codex/routing/thread-affinity"; import { adoptStoredIssuer, adoptStoredThreadRows, resetCodexAffinityStoreForTests } from "../../src/codex/routing/thread-affinity-store"; import { acquireSpendLedgerOwner, type SpendLedgerOwnerLease } from "../../src/lib/spend-ledger-owner"; import { removeTreeWithRetry } from "../helpers/remove-tree"; @@ -34,7 +34,7 @@ afterEach(() => { homes = []; }); -test("a change made between leases keeps rows for conversations this process never loaded", () => { +test("a change made between leases makes the next lease clear stale rows instead of reviving them", () => { useHome(newHome()); lease = acquireSpendLedgerOwner(); seedAccounts(); for (const id of ["one", "two"]) { @@ -53,9 +53,11 @@ test("a change made between leases keeps rows for conversations this process nev bindThreadAffinity(requestOwnedAffinityKey(conversation("one"))!, "b", Date.now()); lease = acquireSpendLedgerOwner(); freshProcess(); lease = acquireSpendLedgerOwner(); - expect(storedPin("one")).toBe("b"); - expect(storedPin("two")).toBe("a"); - expect(storedIssuer("two")).toBe("a"); + // The replaced pin must not come back as "a". Rows the process never loaded are cleared too: + // the database could not record the change, so it is treated as stale as a whole (fail closed). + expect(storedPin("one")).toBeUndefined(); + expect(storedPin("two")).toBeUndefined(); + expect(storedIssuer("two")).toBeUndefined(); }); test("a change made between leases in one home never touches another home's database", () => { @@ -76,3 +78,22 @@ test("a change made between leases in one home never touches another home's data freshProcess(); useHome(homeB); lease = acquireSpendLedgerOwner(); expect(storedPin("b-only")).toBe("a"); }); + +test("a change made while the store could not open is still honoured by the next successful open", () => { + const home = newHome(); + useHome(home); + lease = acquireSpendLedgerOwner(); seedAccounts(); + const key = codexPoolAffinityKey(conversation("failed-open"))!; + bindThreadAffinity(key, "a", Date.now()); + lease.release(); lease = undefined; + // The next lease cannot open the database (a directory sits at its path); delete during it. + const file = join(home, "codex-affinity.sqlite"); + renameSync(file, `${file}.aside`); mkdirSync(file); + lease = acquireSpendLedgerOwner(); + deleteThreadAffinity(key); + lease.release(); lease = undefined; + removeTreeWithRetry(file); renameSync(`${file}.aside`, file); + lease = acquireSpendLedgerOwner(); + freshProcess(); lease = acquireSpendLedgerOwner(); + expect(storedPin("failed-open")).toBeUndefined(); +}); diff --git a/tests/codex-integration/codex-affinity-restart-matrix.test.ts b/tests/codex-integration/codex-affinity-restart-matrix.test.ts index 775da1b7812..470f0bb221f 100644 --- a/tests/codex-integration/codex-affinity-restart-matrix.test.ts +++ b/tests/codex-integration/codex-affinity-restart-matrix.test.ts @@ -183,12 +183,12 @@ test("home switching drops pending writes and does not carry recorded anchors ac for (const mutation of ["bind", "delete", "issuer"] as const) { test(`late ${mutation} after release is what a reacquired lease persists`, () => inHome(home => { seed(home); - // Memory wins when a later lease opens the database: the change made while no lease was held is - // what this process routes with, and a restart must restore exactly that, never the older row. + // Live memory keeps the change made while no lease was held. The database could not record it, + // so the next lease clears it: a restart restores nothing rather than the replaced row. const expected = { - bind: { pin: "b", issuer: "a", restoredPin: "b", restoredIssuer: "a" }, - delete: { pin: undefined, issuer: "a", restoredPin: undefined, restoredIssuer: "a" }, - issuer: { pin: "a", issuer: "b", restoredPin: "a", restoredIssuer: "__unknown_codex_conversation_state_issuer__" }, + bind: { pin: "b", issuer: "a", restoredPin: undefined, restoredIssuer: undefined }, + delete: { pin: undefined, issuer: "a", restoredPin: undefined, restoredIssuer: undefined }, + issuer: { pin: "a", issuer: "b", restoredPin: undefined, restoredIssuer: undefined }, }[mutation]; expect(matrix(home, `late-${mutation}`)).toEqual({ pin: expected.pin, issuer: expected.issuer }); const restored = matrix(home, "read"); diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 1b471b80dad..5c746fef083 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -321,3 +321,17 @@ test("a pin bound and touched in the same pending batch survives a chunk prune", flushCodexAffinityStoreForTests(); lease.release(); lease = acquireSpendLedgerOwner(); expect(adoptStoredThreadRows(key(), Date.now())).toHaveLength(1); }); + +test("row keys carried across a lease change still let a delete reach its row after anchor eviction", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); const k = bind(fence); + recordIssuerSet(k, "account-a", fence, Date.now()); lease.release(); + lease = acquireSpendLedgerOwner(); const adopted = key(); + expect(adoptStoredThreadRows(adopted, Date.now())).toHaveLength(1); + expect(adoptStoredIssuer(adopted, Date.now())?.issuer).toBe("account-a"); + lease.release(); lease = acquireSpendLedgerOwner(); + for (let i = 0; i < 4096; i++) codexConversationKeyFor(`carry-family-${i}`, `carry-thread-${i}`); + recordThreadDelete(adopted, "legacy"); recordIssuerDelete(adopted); lease.release(); + resetCodexAffinityStoreForTests(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); + expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); +}); From fcbde646796f2fc07adf02421ae39ad568227963 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 19:36:13 +0900 Subject: [PATCH 20/21] fix(codex): keep carried row keys past a failed reopen; document the post-release limit --- src/codex/routing/thread-affinity-store.ts | 3 +++ structure/providers/codex-pool-affinity.md | 2 +- .../codex-affinity-store.test.ts | 18 +++++++++++++++++- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/src/codex/routing/thread-affinity-store.ts b/src/codex/routing/thread-affinity-store.ts index 25982a2e751..5fcf67f2292 100644 --- a/src/codex/routing/thread-affinity-store.ts +++ b/src/codex/routing/thread-affinity-store.ts @@ -335,9 +335,12 @@ function scope(acquiredHome?: string): Scope | undefined { consulted: new Set(), scheduled: false, }; + const prior = carried; carried = undefined; current = s; open(s); + // A scope that could not open never touched the carried maps; keep them for the next one. + if (!s.db && prior?.home === home) carried = prior; if (s.db) { if (needsClear && persistedHome === home) recordAllCleared(); needsClear = false; diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index c7dc74a6ab4..16e12418722 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -9,6 +9,6 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. -The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. A persistable change to pins or issuers made while the home's database is not writable (between two leases of one process, or while a scope could not open it) marks that database stale; the next scope that opens it clears it before use, so a restart restores nothing rather than a row the process had replaced or deleted. This also drops rows for conversations the process never loaded, which loses pins but never revives one; a serving proxy holds one lease for its whole life and does not reach this path. Row keys and anchors carry across a lease change in the same home, so deletes still find their rows after an anchor is evicted. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. +The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. A persistable change to pins or issuers made while the home's database is not writable (between two leases of one process, or while a scope could not open it) marks that database stale; the next scope that opens it clears it before use, so a restart restores nothing rather than a row the process had replaced or deleted. This also drops rows for conversations the process never loaded, which loses pins but never revives one; a serving proxy holds one lease for its whole life and does not reach this path. Row keys and anchors carry across a lease change in the same home, including past a scope that failed to open, so deletes still find their rows after an anchor is evicted. A change made after the final release, before the process exits without taking the lease again, cannot be saved: the lease is gone and another process may already own the database, so the next process can restore the row that change replaced. Such a row still has to pass the identity fence and the resolver's usability checks, and a serving proxy releases its lease only after its listeners have stopped. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both. `src/lib/config-ownership.ts` includes the database and sidecars in `INITIAL_OWNED_PATHS`; existing homes use `recordOwnedConfigPath`, whose ownership race is tracked in [#8093](https://github.com/lidge-jun/opencodex/issues/8093). A home without an ownership manifest, created before ownership tracking, still uses the store; uninstall removes nothing from such a home, as for every other file in it. `tests/codex-integration/codex-affinity-store.test.ts`, `tests/codex-integration/codex-affinity-restart.test.ts` and `tests/codex-integration/codex-affinity-request-owned.test.ts` cover store, restart and request-owned exclusion behavior. diff --git a/tests/codex-integration/codex-affinity-store.test.ts b/tests/codex-integration/codex-affinity-store.test.ts index 5c746fef083..5e496adc5a2 100644 --- a/tests/codex-integration/codex-affinity-store.test.ts +++ b/tests/codex-integration/codex-affinity-store.test.ts @@ -1,7 +1,7 @@ import { registerThreadAffinityLiveState } from "../../src/codex/routing/thread-affinity"; import { afterEach, beforeEach, expect, test } from "bun:test"; import { Database } from "bun:sqlite"; -import { existsSync, mkdtempSync, readFileSync, statSync } from "node:fs"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, renameSync, statSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { saveCodexAccountCredential } from "../../src/codex/account-store"; @@ -335,3 +335,19 @@ test("row keys carried across a lease change still let a delete reach its row af expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); expect(adoptStoredIssuer(key(), Date.now())).toBeUndefined(); }); + +test("row keys survive a failed reopen, so a later delete still reaches its row after anchor eviction", () => { + lease = acquireSpendLedgerOwner(); const fence = seed(); bind(fence); lease.release(); + lease = acquireSpendLedgerOwner(); const adopted = key(); + expect(adoptStoredThreadRows(adopted, Date.now())).toHaveLength(1); + lease.release(); + // One reacquisition cannot open the database (a directory sits at its path). + renameSync(path(), `${path()}.aside`); mkdirSync(path()); + lease = acquireSpendLedgerOwner(); lease.release(); + removeTreeWithRetry(path()); renameSync(`${path()}.aside`, path()); + lease = acquireSpendLedgerOwner(); + for (let i = 0; i < 4096; i++) codexConversationKeyFor(`reopen-family-${i}`, `reopen-thread-${i}`); + recordThreadDelete(adopted, "legacy"); lease.release(); + resetCodexAffinityStoreForTests(); lease = acquireSpendLedgerOwner(); + expect(adoptStoredThreadRows(key(), Date.now())).toEqual([]); +}); From d219e304d90eb5cbff2912491ecb140fd5f16f07 Mon Sep 17 00:00:00 2001 From: JUN Date: Mon, 2 Nov 2026 20:01:35 +0900 Subject: [PATCH 21/21] docs(structure): state that affinity rows store the local pool account id --- structure/providers/codex-pool-affinity.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/structure/providers/codex-pool-affinity.md b/structure/providers/codex-pool-affinity.md index 16e12418722..756d667e472 100644 --- a/structure/providers/codex-pool-affinity.md +++ b/structure/providers/codex-pool-affinity.md @@ -7,7 +7,7 @@ the single-writer lease is owned by [Responses spend](../transports/responses-sp `src/codex/routing/thread-affinity-store.ts` restores eligible pool bindings after a graceful proxy restart for up to 24 hours idle. Session-bearing requests and session-less requests anchored on their immediate parent have derivable anchors. Deeper session-less chains, keys derived under another config home, legacy raw-parent keys and evicted anchors remain memory-only. Removed, re-logged, expired or identity-less accounts are not restored; normal selection and usability rules still apply. -The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret; raw identifiers and process-local conversation keys are not stored. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. +The config home contains `codex-affinity.sqlite`, mode 0600 with a hardened Windows ACL. Its v1 schema has `meta` (store secret and clean flag), `threads` (scoped bindings), and `issuers` (conversation state issuers). Row keys are domain-separated HMACs of the cohort anchor under the store's own secret. Each row stores the local pool account id (the label that config.json and the credential store in this directory already key the account by, as the Claude affinity store does), an identity fence digest and timestamps. Session ids, conversation keys (raw or process-local), emails, ChatGPT account ids and tokens are not stored; a keyed digest of the pool id would hide nothing from a reader of this directory, which holds both the store secret and the account list. The identity fence derives from `quotaHistoryIdentity`, captured at bind and at issuance. Issuance requires both generation and `chatgptAccountId` to match the current record. `__main__` pins and issuers are never persisted. Request-owned keys (`isRequestOwnedAffinityKey`, from `src/codex/request-owned-affinity-key.ts`) are refused at the store's row-key derivation and anchor observer, so they are never anchored, written or restored, as the [request-owned persistence contract](codex-request-owned-affinity.md#persistence-contract) requires. An unknown restored issuer scrubs continuation state; uploaded files are refused when the stored account differs from the serving account, as for known account changes and prior-account moves. The spend-ledger lease is the single-writer authority. The final releasing hook flushes pending work and marks the store clean before ownership is released; it does so even when the release happens inside a passive read, because it commits only work recorded outside one. A crash, failed final flush or poisoned store leaves the clean flag unset, so the next owner discards rows. Structural changes use an ordered write log; UPDATE-only touches read live timestamps at flush, avoiding stale timestamps and resurrection. The 24-hour idle limit bounds reuse. Expired rows are removed by the next prune that commits: one is attempted when the store opens at lease acquisition, on an hourly timer while it stays open, and after every failed flush, including in a poisoned store. A prune can still be delayed or fail: a lease acquired inside a passive read does not open the store, a store that failed to open is disabled and cannot prune, a prune transaction can roll back, and timers can run late. SQLite secure_delete overwrites freed pages in the database file, but not filesystem snapshots, backups or storage remapping; its transient rollback journal is removed after each commit. The store secret pseudonymizes row keys and does not protect integrity from a local process that can already read the config directory. Pruning also removes future-dated rows and bounds each table to 2,048 entries, evicting model detours first. Other passive reads do not open, adopt or mutate the store. Live affinity survives a lease change in one process. A persistable change to pins or issuers made while the home's database is not writable (between two leases of one process, or while a scope could not open it) marks that database stale; the next scope that opens it clears it before use, so a restart restores nothing rather than a row the process had replaced or deleted. This also drops rows for conversations the process never loaded, which loses pins but never revives one; a serving proxy holds one lease for its whole life and does not reach this path. Row keys and anchors carry across a lease change in the same home, including past a scope that failed to open, so deletes still find their rows after an anchor is evicted. A change made after the final release, before the process exits without taking the lease again, cannot be saved: the lease is gone and another process may already own the database, so the next process can restore the row that change replaced. Such a row still has to pass the identity fence and the resolver's usability checks, and a serving proxy releases its lease only after its listeners have stopped. A manual account switch deletes saved pins and keeps saved issuers, matching the live maps; a full reset deletes both.