Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/codex/ocx-compaction-history.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
unlinkSync,
writeFileSync,
} from "node:fs";
import { basename, isAbsolute, join, relative, resolve } from "node:path";
import { basename, isAbsolute, join, relative, resolve, toNamespacedPath } from "node:path";

import { Database } from "bun:sqlite";

Expand Down Expand Up @@ -80,7 +80,8 @@ function writePrivateFile(path: string, content: string): void {
const fd = openSync(path, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL, 0o600);
try {
if (process.platform !== "win32") chmodSync(path, 0o600);
else hardenSecretPath(path, { required: true, timeoutMemoKey: path });
// icacls needs the extended Windows spelling once backup/temp names exceed MAX_PATH.
else hardenSecretPath(toNamespacedPath(path), { required: true, timeoutMemoKey: path });
writeFileSync(fd, content, "utf8");
fsyncSync(fd);
} finally {
Expand Down
2 changes: 1 addition & 1 deletion structure/transports/responses-failover.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ build both honor for canonical ChatGPT and explicitly native-capable custom dest
The portable Responses summarizer in `src/adapters/openai-responses/passthrough.ts` uses `src/adapters/openai-responses/compaction-search-history.ts` to render remaining top-level `web_search_call` items, after bridge restoration, as assistant reference notes labeled untrusted historical metadata (not instructions or fetched page content). The allowlist keeps string status and action fields (type, query/queries, URL, pattern, source URL/title/type); opaque state, IDs, unknown fields and malformed values are omitted. Strings are cut at 2048 code units, lists at 20 entries, and one request carries at most 64 KiB of notes; later hosted cells collapse into a single omission note reserved inside that budget. Tool declarations stay absent, bridge-restored call/result pairs and existing messages/citations stay intact, and this portable path changes only compact summary input, never stored history or native compaction. Ordinary Codex continuation policy is separate: see [hosted-search-continuation](../providers-and-adapters.md#hosted-search-continuation-binding). A failed summary publishes no replacement history. Lite requests retain `parallel_tool_calls=false`, which the upstream requires even without tool declarations.

`tests/responses/responses-compaction-override.test.ts` covers source filtering, trigger selection, config
validation, native and routed handlers, credential retention, portable summaries and replay, and combo failover. Only explicit `reasoningRetention.enabled: true` (default off, including legacy limit-only blocks) makes routed v1 `/responses/compact` and direct routed v2 `compaction_trigger` use `src/responses/reasoning-retention.ts` to withhold readable reasoning from its summarizer; exact text including its historical-context frame within the smaller configured percentage/token cap is restored as framed historical context after successful compaction, including routed v1 Combo failover to an opaque native result. Repeated compact removes that frame locally again; ordinary requests replay it as a user context message, not a fresh reasoning item. Overflow is provisionally archived once per routed turn and kept only when successful replacement history names its path, in the readable summary or a separate historical notice alongside an opaque native v1 result; failed, cancelled, or invalid outcomes remove that attempt's file without deleting earlier successful archives. The percentage/token cap bounds retained context only; each overflow file is request-admission bounded under the resolved OpenCodex home’s `reasoning-archive/`, and also shares a compiled-in aggregate ceiling under that directory: at most 512 MiB and 256 files, counted across every committed, remnant, or foreign file. The ceiling is a conservative policy bound, not a retention-duration or free-space guarantee. `src/responses/reasoning-retention.ts` enforces it inside a dedicated sibling SQLite write transaction (`reasoning-archive.lock`, `BEGIN IMMEDIATE`, `busy_timeout = 0`) that serializes each non-recursive scan, exclusive create, write, and close across cooperating writers; older builds and external OS actors stay outside that guarantee. An archive directory or lock path that is not a plain regular object, a non-regular or uninspectable directory entry, and an at- or over-capacity directory all refuse the new archive closed without deleting or mutating anything already there, surfacing distinct typed capacity, busy, and I/O errors that keep the original input intact for a client retry with no upstream send. Successful archives require manual deletion; switching off or resetting does not delete them. Crash or filesystem cleanup failure can leave provisional files, which then count toward the ceiling. Failure yields no replacement history. Opaque compaction preserves its unchanged item and ciphertext. Routed v1 failover appends any locally withheld retained messages or an archive notice; direct native compact keeps existing behavior. Direct routed v2 remains a single compaction output item: `src/responses/retained-compaction.ts` encodes summary and framed reasoning separately as versioned `ocx2:` JSON/Base64 (encoding, not encryption). Replay expands them into separate historical messages; repeated v1/v2 compaction withholds the reasoning again. Existing `ocx1:` summaries remain readable. `src/codex/ocx-compaction-history.ts` also lowers `ocx2:` into separate summary and reasoning messages for explicit native-history recovery, so later proxy compaction still withholds reasoning. `src/server/responses/compaction-retention.ts` owns provisional archives and request-budget retention; failure or body cancellation releases the attempt. Emergency recovery preserves the structured envelope for JSON and SSE instead of flattening reasoning into the summary. `tests/responses/reasoning-retention-v2.test.ts` and `tests/responses/responses-compaction-recovery.test.ts` cover v2 replay and recovery; `tests/responses/reasoning-retention-routing.test.ts` covers these public contracts; `tests/responses/reasoning-retention.test.ts` covers envelopes, limits and archive failures; `tests/responses/reasoning-retention-quota.test.ts` covers the aggregate ceiling and the archive lock. Archive accounting uses a streaming one-entry directory buffer, stops actual reads when refusal is certain, and closes on every outcome; vanished entries consume the inspection allowance and excessive churn refuses closed. At the final v1/v2 HTTP boundary in `src/server/index/serve-options.ts`, after internal recovery unwinds, `src/server/responses/reasoning-archive-error.ts` maps only typed archive failures to local-storage HTTP 503: `reasoning_archive_quota_exceeded` requests manual review/removal of older archives, `reasoning_archive_busy` requests retry after the other writer finishes, and `reasoning_archive_unavailable` requests permission/storage checks. Responses expose no raw filesystem message, path, cause or guessed Retry-After. `tests/responses/reasoning-archive-http.test.ts` exercises the production router's quota, lock, unsafe-directory and partial-write refusals with zero sends, including enabled recovery and streaming. The WebSocket transport's pre-response catch in `src/server/index/websocket-handler.ts` applies the same mapping as a 503 error frame with the same stable code; `tests/responses/reasoning-archive-ws.test.ts` covers its quota, lock and unsafe-directory refusals with zero sends.
validation, native and routed handlers, credential retention, portable summaries and replay, and combo failover. Only explicit `reasoningRetention.enabled: true` (default off, including legacy limit-only blocks) makes routed v1 `/responses/compact` and direct routed v2 `compaction_trigger` use `src/responses/reasoning-retention.ts` to withhold readable reasoning from its summarizer; exact text including its historical-context frame within the smaller configured percentage/token cap is restored as framed historical context after successful compaction, including routed v1 Combo failover to an opaque native result. Repeated compact removes that frame locally again; ordinary requests replay it as a user context message, not a fresh reasoning item. Overflow is provisionally archived once per routed turn and kept only when successful replacement history names its path, in the readable summary or a separate historical notice alongside an opaque native v1 result; failed, cancelled, or invalid outcomes remove that attempt's file without deleting earlier successful archives. The percentage/token cap bounds retained context only; each overflow file is request-admission bounded under the resolved OpenCodex home’s `reasoning-archive/`, and also shares a compiled-in aggregate ceiling under that directory: at most 512 MiB and 256 files, counted across every committed, remnant, or foreign file. The ceiling is a conservative policy bound, not a retention-duration or free-space guarantee. `src/responses/reasoning-retention.ts` enforces it inside a dedicated sibling SQLite write transaction (`reasoning-archive.lock`, `BEGIN IMMEDIATE`, `busy_timeout = 0`) that serializes each non-recursive scan, exclusive create, write, and close across cooperating writers; older builds and external OS actors stay outside that guarantee. An archive directory or lock path that is not a plain regular object, a non-regular or uninspectable directory entry, and an at- or over-capacity directory all refuse the new archive closed without deleting or mutating anything already there, surfacing distinct typed capacity, busy, and I/O errors that keep the original input intact for a client retry with no upstream send. Successful archives require manual deletion; switching off or resetting does not delete them. Crash or filesystem cleanup failure can leave provisional files, which then count toward the ceiling. Failure yields no replacement history. Opaque compaction preserves its unchanged item and ciphertext. Routed v1 failover appends any locally withheld retained messages or an archive notice; direct native compact keeps existing behavior. Direct routed v2 remains a single compaction output item: `src/responses/retained-compaction.ts` encodes summary and framed reasoning separately as versioned `ocx2:` JSON/Base64 (encoding, not encryption). Replay expands them into separate historical messages; repeated v1/v2 compaction withholds the reasoning again. Existing `ocx1:` summaries remain readable. `src/codex/ocx-compaction-history.ts` also lowers `ocx2:` into separate summary and reasoning messages for explicit native-history recovery, so later proxy compaction still withholds reasoning. Recovery uses extended-length Windows ACL paths for backups and atomic-repair files, preserving required owner-only ACLs on long paths (`tests/codex-integration/history-ocx-compaction-recovery.test.ts`). `src/server/responses/compaction-retention.ts` owns provisional archives and request-budget retention; failure or body cancellation releases the attempt. Emergency recovery preserves the structured envelope for JSON and SSE instead of flattening reasoning into the summary. `tests/responses/reasoning-retention-v2.test.ts` and `tests/responses/responses-compaction-recovery.test.ts` cover v2 replay and recovery; `tests/responses/reasoning-retention-routing.test.ts` covers these public contracts; `tests/responses/reasoning-retention.test.ts` covers envelopes, limits and archive failures; `tests/responses/reasoning-retention-quota.test.ts` covers the aggregate ceiling and the archive lock. Archive accounting uses a streaming one-entry directory buffer, stops actual reads when refusal is certain, and closes on every outcome; vanished entries consume the inspection allowance and excessive churn refuses closed. At the final v1/v2 HTTP boundary in `src/server/index/serve-options.ts`, after internal recovery unwinds, `src/server/responses/reasoning-archive-error.ts` maps only typed archive failures to local-storage HTTP 503: `reasoning_archive_quota_exceeded` requests manual review/removal of older archives, `reasoning_archive_busy` requests retry after the other writer finishes, and `reasoning_archive_unavailable` requests permission/storage checks. Responses expose no raw filesystem message, path, cause or guessed Retry-After. `tests/responses/reasoning-archive-http.test.ts` exercises the production router's quota, lock, unsafe-directory and partial-write refusals with zero sends, including enabled recovery and streaming. The WebSocket transport's pre-response catch in `src/server/index/websocket-handler.ts` applies the same mapping as a 503 error frame with the same stable code; `tests/responses/reasoning-archive-ws.test.ts` covers its quota, lock and unsafe-directory refusals with zero sends.
## Ambiguous connection-reset replay boundary

Three failures look alike from the outside — the turn may have executed and we cannot
Expand Down
2 changes: 1 addition & 1 deletion tests/cli/cli-help.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,7 @@ describe("CLI subcommand help", () => {
);

expectSpawnFinished(result, "ocx recover-history --ocx-compaction");
expect(result.status).toBe(0);
expect(result.status, result.stderr).toBe(0);
expect(result.stdout).toContain("Recovered 1 ocx1 compaction item(s)");
expect(readFileSync(rollout, "utf8")).toContain("portable summary");
expect(readFileSync(rollout, "utf8")).not.toContain("ocx1:");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,17 @@ describe("OpenCodeX compaction history recovery", () => {
expect(rewriteOcxCompactionsForNativeReplay(source)).toEqual({ content: source, replaced: 0 });
});

test("backs up and atomically repairs one database-selected rollout", () => {
test.each([false, true])("backs up and atomically repairs one database-selected rollout (long paths: %s)", longPaths => {
const root = mkdtempSync(join(tmpdir(), "ocx-compaction-recovery-"));
try {
const codexHome = join(root, "codex");
const rolloutDir = join(codexHome, "sessions", "2026", "09", "07");
const backupRoot = join(root, "backups");
const segment = longPaths ? "long-path-".repeat(18) : "07";
const rolloutDir = join(codexHome, "sessions", "2026", "09", segment);
const backupRoot = join(root, "backups", segment);
mkdirSync(rolloutDir, { recursive: true });
const threadId = "01a018e6-242f-7801-81b8-ffc0a5c6d589";
const rolloutPath = join(rolloutDir, `rollout-fixture-${threadId}.jsonl`);
if (longPaths) expect(rolloutPath.length).toBeGreaterThan(260);
const original = `${JSON.stringify({
type: "compacted",
payload: {
Expand Down
Loading