[hskl18]

What changed

• Replace new-table v1 writes with a checksummed 4096-byte paged v2 format and explicit corruption errors.
• Add incrementally mutated persistent per-field B+ trees with leaf and internal splits, sibling validation, stable untouched pages, and root replacement.
• Add a complete-image checksummed WAL with dirty-page application, explicit fsync ordering, deterministic crash injection, committed redo, stale-WAL rejection, and idempotent recovery.
• Add sql_migrate_v1 with read-only source access, dry-run inspection, byte-for-byte row verification, source revalidation, an exclusively owned temporary file, and atomic no-replace promotion.
• Set the generated CMake and CLI version to 2.0.0, add storage inspection, document the exact byte layout, and add a release checklist.

Feature verified

• Observed working: SQL create, insert, reopen, equality/range filtering, batch fixtures, v1 migration, storage inspection, incremental leaf/internal/root splits, and crash recovery.
• Release proof: fresh warning-as-error build, 9 of 9 CTest targets passed.
• Sanitizer proof: fresh combined AddressSanitizer and UndefinedBehaviorSanitizer warning-as-error build, 9 of 9 CTest targets passed with halt-on-error enabled.
• Crash proof: child processes terminate with _exit(86) at dynamically enumerated WAL, dirty-page, fsync, directory-fsync, and WAL-clear boundaries. Recovery exposes only the old or new complete state and remains idempotent.
• Migration proof: empty strings, exact 100-byte values, duplicate/range keys, and a 64-field 6,400-byte logical row remain equivalent, while source bytes remain unchanged.

Regression guardrails

• Release build with warnings as errors
• SQL and parser behavior tests
• Randomized incremental persistent-index tests
• Leaf, internal, and root split tests
• Page, superblock, child-pointer, partial-record, and WAL corruption tests
• Dynamic create, insert, and deep-split crash matrices
• V1 dry-run, migration, query-equivalence, and promotion-crash tests
• CLI version and non-mutating storage-inspection tests
• AddressSanitizer and UndefinedBehaviorSanitizer

How to reproduce

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

cmake -S . -B build-sanitized -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=ON -DSQL_ENABLE_SANITIZERS=ON -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build build-sanitized --parallel
UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1 ctest --test-dir build-sanitized --output-on-failure

Scope and evidence boundary

• The durability scope is one process, one writer, and one atomic statement at a time.
• This does not claim multi-statement ACID transactions, concurrency control, updates, deletes, or broader SQL compatibility.
• The crash harness proves abrupt process termination and recovery ordering with the kernel page cache intact. It does not emulate physical media failure.
• Linux, macOS, and Linux sanitizer GitHub Actions checks remain the hosted evidence gate.
• This PR does not create or merge a tag or GitHub Release.

Summary by CodeRabbit

• New Features

  • Introduced Storage Format v2 with checksummed pages, persistent indexes, and crash recovery.
  • Added reliable row insertion and indexed filtering for v2 tables.
  • Added CLI commands for version reporting and storage inspection.
  • Added copy-first migration from legacy tables, including dry-run validation.
  • Legacy tables remain available in read-only mode.

• Bug Fixes

  • Improved detection and rejection of corrupted or incomplete storage and recovery data.

• Documentation

  • Updated usage, persistence, migration, inspection, and release guidance for v2.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6fa53174-6044-4968-be81-2f8a33448e38

📥 Commits

Reviewing files that changed from the base of the PR and between b53cc14 and 68eec3c.

📒 Files selected for processing (22)

• CMakeLists.txt
• DESIGN.md
• README.md
• docs/release-checklist-v2.md
• docs/storage-v2.md
• includes/binary_files/file_record.cpp
• includes/binary_files/file_record.h
• includes/storage/page_buffer.cpp
• includes/storage/page_buffer.h
• includes/storage/storage_v2.cpp
• includes/storage/storage_v2.h
• includes/table/table.cpp
• includes/table/table.h
• includes/version.h.in
• run_sql.cpp
• tests/cli_tests.cpp
• tests/migration_tests.cpp
• tests/sql_tests.cpp
• tests/storage_crash_tests.cpp
• tests/storage_crash_worker.cpp
• tests/storage_v2_tests.cpp
• tools/sql_migrate_v1.cpp

---

📝 Walkthrough

Walkthrough

SQL in C++ 2.0 introduces POSIX-only paged storage v2 with checksummed pages, persistent B+ tree indexes, WAL recovery, v1 migration, CLI inspection/version commands, and expanded crash, corruption, migration, and integration tests.

Changes

Storage v2 format and buffering

Layer / File(s)	Summary
Storage format contracts and page buffering DESIGN.md, README.md, docs/storage-v2.md, includes/storage/storage_v2.h, includes/storage/page_buffer.*	Defines the v2 page, index, WAL, recovery, inspection, and migration formats, and adds public storage and page-buffer APIs.
Page codec and persistent indexes includes/storage/storage_v2.cpp	Implements checksummed page encoding, chained streams, image validation, persistent B+ trees, and incremental splitting.
WAL persistence and storage API includes/storage/storage_v2.cpp	Adds dirty-page commits, WAL validation, crash injection, recovery, table lifecycle operations, queries, and inspection output.

Runtime and migration integration

Layer / File(s)	Summary
Table backend and CLI integration includes/table/*, includes/binary_files/file_record.*, run_sql.cpp	Routes v2 tables through storage APIs, retains read-only v1 loading, and adds --version and --inspect-storage.
v1 migration flow tools/sql_migrate_v1.cpp, tests/migration_tests.cpp	Decodes v1 files, verifies equivalent v2 data, preserves sources, and promotes temporary destinations atomically.
Build wiring and storage validation CMakeLists.txt, includes/version.h.in, tests/*	Adds generated version headers, migration and storage targets, and tests for indexes, crashes, WALs, corruption, CLI inspection, and v2 SQL behavior.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant Table
  participant StorageV2
  participant WAL
  participant Database
  CLI->>Table: execute INSERT or SELECT
  Table->>StorageV2: insert_row or query_index
  StorageV2->>WAL: persist committed after-image
  StorageV2->>Database: write and fsync database pages
  StorageV2->>WAL: clear and fsync WAL
  Table-->>CLI: return query results or status

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/storage-engine-v2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}