Skip to content

feat(codex): ingest compressed and shared-history rollouts - #54

Merged
jamesyong-42 merged 6 commits into
mainfrom
feat/codex-rollout-storage
Nov 13, 2026
Merged

jamesyong-42 merged 6 commits into
mainfrom
feat/codex-rollout-storage

Conversation

@jamesyong-42

@jamesyong-42 jamesyong-42 commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Older Codex sessions could appear as empty metadata because Rosetta did not discover .jsonl.zst rollouts or archived transcripts. Resuming a session in Codex restored its plain file and made the history appear again. Import both representations directly, following the writer protocol pinned to Codex 0.167.2.

  • Preserve logical source identity and decoded-byte checkpoints across compression, restoration and archive moves. Verify the complete committed prefix before continuing through a representation change.
  • Follow the authoritative SQLite rollout selector and resolve nested fixed history ranges, including reverts and attached threads. Capture a self-contained raw view for offline rebuilding, with inherited usage and artifact ownership preserved.
  • Keep any pre-existing independent branch cursors on that same selected view, including later reverts, so upgrading an older Rosetta database cannot resurrect abandoned messages. Fresh imports create one logical transcript per thread.
  • Hold incomplete or corrupt replacements without discarding the last valid transcript. Cover missing ancestors, cycles, invalid range boundaries, migration rewrites, missed watcher events and later recovery. Quiet polls compare file metadata without decompressing transcripts.
  • Keep native files read-only and bound decoding, headers, lineage depth and cancellation. Extend the raw metadata journal with source readiness obligations; preserve older journals and checkpoints without a canonical database migration.

Validation includes 25 focused storage regressions, cross-adapter replay/live/snapshot tests, 165 pipeline unit tests, 253 daemon tests, 68 ground-truth tooling tests, full-workspace strict Clippy, and checks of all five shared-history range references found in the local census. An isolated import of two copied native sessions produced 92 and 199 canonical messages, respectively, with zero hard errors and a clean SQLite integrity check. No private transcripts or fixtures are committed.

A release-mode paired synthetic run produced identical 8,000-message projections: 485.63 ms for plain input and 493.43 ms for zstd. Twenty quiet polls started zero capture passes in either representation. This is a small warm-filesystem measurement, not a full-archive performance estimate. The reproducible benchmark and storage contract are recorded in docs/ARCHITECTURE.md.

@jamesyong-42
jamesyong-42 force-pushed the feat/codex-rollout-storage branch from 530ac19 to 0b65666 Compare November 13, 2026 00:33
@jamesyong-42
jamesyong-42 merged commit 3de2da2 into main Nov 13, 2026
8 checks passed
@jamesyong-42
jamesyong-42 deleted the feat/codex-rollout-storage branch November 13, 2026 00:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant