Skip to content

fix(sqlite): publish complete stores and recover interrupted initialization - #4805

Merged
ScriptedAlchemy merged 75 commits into
masterfrom
fix/4513-reserve-sqlite-sidecars-20261108
Nov 12, 2026
Merged

ScriptedAlchemy merged 75 commits into
masterfrom
fix/4513-reserve-sqlite-sidecars-20261108

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

SQLite initialization could expose an incomplete canonical store or reinterpret a failed creation as an existing database. Initialization now completes in a private sibling directory, joins its SQLite workers, and publishes the completed main file last through capability-relative, no-replace moves.

The publisher retains the original main and sidecar handles, validates their file identities and the main-file digest, and refuses uncheckpointed WAL or journal data. A durable preparation receipt supports interrupted publication and handled schema-failure recovery when the recorded family remains provable. Ambiguous families preserve their bytes and refuse admission for offline reconciliation. Read-only admission accepts an empty completed staging directory and refuses every unproven leftover entry without creating a writer or coordination lock.

Existing-store admission preserves live SQLite writer locks, validates initialized schema, and uses a bounded live-WAL inspection when an immutable main-only view is inconsistent during a checkpoint. Reserved sidecars are authenticated before and after writer startup; substituted entries refuse startup without overwriting the replacement bytes.

Validation:

  • Native macOS Bazel at 0f4a3da377: all 473 rusqlite-runtime, 554 runtime-core, and 37 private-filesystem unit tests passed. The read-only staging fixture failed at the preceding head and passed after the fixture was aligned with the marker-refusal contract. Affected formatting checks passed.
  • Parent verification at 214d89fcee: all 28 attachment tests passed, including real cross-process sidecar substitution/retry, failed-start preservation, independent writer exclusion, and shutdown. All 18 initialization tests passed with strict runtime-core Clippy and formatting.
  • Native macOS at 8f8d82a550: all 502 global-db and 667 sessions unit tests passed, alongside their strict Clippy and formatting checks. The previous hosted Linux global-db timeout remains unexplained; the native result does not establish Linux timing.

Refs #4513. Hosted checks for the new head are pending. The broader publication-to-reattach and existing-store sidecar-provenance boundary, sustained production observation, and remaining platform qualification stay tracked in #4513; this PR does not establish whole-issue closure.

@changeset-bot

changeset-bot Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 07de719

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Copy Markdown
Owner Author

Review/validation hold: This PR is marked ready solely so the existing PR CI can execute on its exact head. Please do not merge until nonzero hosted Bazel tests, native macOS/Windows SQLite sidecar and locking behavior, and the preserved-failure/retry tradeoff are reviewed. The default PR workflow skips native macOS; a skipped check is not platform acceptance. The cross-process replacement and initial sidecar refusal tests are intended to be exercised by //crates/tracedecay-rusqlite-runtime:unit_test on the current commit. The broader #4513 acceptance remains open.

Copy link
Copy Markdown
Owner Author

Refs #4513 — follow-up fix published on this same issue-specific PR at 1738c8e, fast-forwarded from exact prior head 92d53ff3f0de without force, three owned files only.

New data-loss path on retry: create_new creates the empty main DB before exclusively reserving -wal/-shm/-journal. If any sidecar is occupied, it refuses without deleting foreign files, but leaves the empty sessions.db. A later existing-mode attachment can open that empty file, start SQLite in WAL mode, and erase the exact foreign sidecar that caused the earlier refusal. An independent disposable Linux/native SQLite 3.46.1 schedule reproduced sentinel loss for all 3 suffixes on retry; these are standalone reproductions, not Bazel/Rust passes.

Patch: both attach and attach_read_only now call OpenedDatabaseFile::pin_initialized before starting SQLite workers. It verifies the completed SQLite 16-byte file magic through the already pinned descriptor, avoiding an extra Unix close that would drop unrelated POSIX locks. Missing/invalid headers fail closed, rather than allowing SQLite to bootstrap an incomplete basename. Added falsifiable Rust regression for each occupied suffix's failed-create → existing-retry sequence, plus invalid-header refusal and valid initialized DB acceptance.

Independent native SQLite external read-only subprocess / concurrent writer probes also passed 32 nonzero Linux assertions, 8 each in rollback and WAL modes (reader result and quick_check). Neither those probes nor this review count as repository Bazel, hosted CI, native macOS or Windows qualification. New-head hosted CI must run nonzero //crates/tracedecay-rusqlite-runtime:unit_test and all relevant integration gates; macOS still needs a native workflow-dispatched run.

Not complete acceptance: an initialization that got far enough to write a valid database header can still leave an incomplete family, and post-admission POSIX pathname replacement lacks a stronger private ownership/VFS fence. Daemon-level external sqlite3 endurance, restart recovery and platform verification remain open. Keep #4513 OPEN and this PR on review/verification hold. Ordinary merging belongs to the existing PR-maintenance lane; do not merge held draft #4776 (Kev/decision model).

Refs #4513

Remove the stale discard_created call left after unsafe pathname cleanup
was deleted. Abort now preserves the database and all sidecars, releases
shared descriptors only after normal worker teardown, and transfers a
lost-file initialization pin to the existing deferred worker cleanup.
Reject initialization publication once drain or close has begun.

Add real runtime regressions for abort/read-back/stale handles, separate-
process replacement of the database and WAL/SHM/journal after worker
close, idempotent abort, and a lost writer held across initialization abort.
These Rust tests still require nonzero Bazel and native macOS execution;
no Cargo or direct Rust invocation was used.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refs #4513 — code review of exact head 1738c8e1de983e062053fb2544c62f3a9c222300.

Three actionable safety findings are attached inline: (1) invalid-header rejection closes an independent raw database descriptor and can release another local SQLite connection's POSIX locks; (2) the SQLite magic does not establish completed initialization or authorize reusing a preserved sidecar family; (3) sidecar reservations do not fence a replacement between reservation and SQLite's first access.

I reproduced the mechanisms in disposable Linux fixtures with native SQLite 3.46.1: the rejected-header/raw-close schedule changed an independent subprocess writer from SQLITE_BUSY to admitted before the original transaction ended; a separate-process post-reservation replacement lost sentinel bytes for each of WAL/SHM/journal; and a header-bearing but otherwise uninitialized WAL database accepted a retry that overwrote/deleted each sentinel. These are standalone mechanism reproductions, NOT execution of this repository's Rust/Bazel tests, and not macOS/Windows qualification.

Keep the removal of unsafe pathname-based cleanup. The next change needs production-path regressions through attachment startup/failure/retry, rather than only calling repository_start_failure directly. Require nonzero Bazel results on the resulting exact head, plus native macOS locking/sidecar coverage and relevant Windows coverage. This review is not an approval or a claim of master completion. Keep #4513 open; ordinary merging remains with the PR-maintenance owner. Do not merge or modify held draft #4776 or release #4763.

Apply the exact rustfmt changes reported by hosted Bazel Repository
gates without touching SQLite behavior or other PR files.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — repaired the exact four formatting differences reported by the failed Repository gates job at the peer's head a3c0a165. Commit 2e9b650 is a nonforce expected-old-head fast-forward and changes only formatting in the three pre-existing SQLite files, preserving the peer's new abort-initialization tests and production changes. New exact-head CI is queued; no Bazel Rust pass is claimed yet. This does not resolve the three existing P1 review findings: rejected pin/header read lock lifetime, valid-header incomplete-initialization retry, and sidecar-only replacement after O_EXCL reservation. Native macOS/Windows tests and daemon/external sqlite3 endurance also remain outstanding. Keep #4513 open and this PR on review hold. Do not merge protected draft #4776 or touch release #4763.

Avoid opening a raw database descriptor when an existing attachment
cannot contain the SQLite header. Closing that descriptor could unlock
another live SQLite writer; cover the refusal with a cross-process test.

This does not fix header-bearing incomplete stores or sidecar swaps.
Refs #4513
SQLite's minimum initialized database occupies one 512-byte page.
Refuse header-bearing shorter files without opening a raw Unix handle;
otherwise they bypass magic-only admission and can release live locks.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — published narrow POSIX-lock preservation follow-ups on this PR: e6ea022 and 83d0221, via exact-old-head, non-force fast-forwards, preserving all prior commits.

pin_initialized previously opened a fresh raw Unix descriptor even when an existing database was too short to contain an initialized SQLite page; its error return closed that descriptor and could release another live SQLite connection's POSIX writer locks. Unix now refuses files shorter than SQLite's minimum 512-byte page by metadata alone, without opening/closing a raw descriptor. Added a real Rust subprocess lock regression for the zero-byte BEGIN IMMEDIATE case (blocked before and after refused admission, allowed after rollback), plus short-file cases bearing the correct 16-byte magic (lengths 16, 100, 511), which the original magic-only check incorrectly admitted. SQLite file-format minimum: https://www.sqlite.org/fileformat2.html.

Independent Linux system SQLite 3.46.1 probe confirmed the pre-fix raw-close lock loss and 5 nonzero metadata-only mitigation assertions; these are not repository Rust/Bazel passes. Hosted exact-head Bazel //crates/tracedecay-rusqlite-runtime:unit_test, broader applicable CI, and native macOS/Windows qualification remain unverified.

Scope is deliberately partial: this does not make invalid-header refusal safe for files >=512 bytes; a raw pin still closes on those errors. It also does not prove a header-bearing store completed initialization or protect the sidecar names against noncooperating POSIX replacement after reservation. Keep the earlier three P1 review threads open and PR on review hold. Require an ownership-safe lifecycle and daemon+external sqlite3 endurance/restart tests before #4513 can be closed on master. No merge/release/draft #4776 changes.

Existing-mode admission must not open sidecars for databases whose
file size cannot be a whole number of SQLite pages. Refuse those paths
using metadata before opening a descriptor, preserving foreign journal
entries and avoiding POSIX lock-loss on the refusal path.

Add production attachment regressions for truncated header-bearing
files and WAL, SHM, and journal sentinel preservation.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — published 761cc49 to this issue-only PR, fast-forwarding exact old head 83d0221 without force and preserving concurrent changes.

Additional defensive admission fix: an existing SQLite DB must consist of complete pages; every legal SQLite page size is a multiple of 512 bytes. The prior >=512-byte check admitted header-bearing files of size 513 or 1025. Those could still reach the SQLite worker and allow it to delete or change unrelated journal/SHM sidecars despite the database being incomplete. Admission now rejects non-512-aligned lengths using filesystem metadata before opening a raw descriptor (avoiding an additional POSIX close on this refusal path), on all supported platforms.

Rust regressions added: direct existing-header tests for 513/1023/1025 bytes, plus a test through real RepositoryPhysicalAttachmentFactory::attach asserting InvalidHeader and byte-exact preservation of sessions.db and foreign -wal/-shm/-journal sentinels for 513/1025-byte databases. No CI bypass or test weakening.

Independent Linux SQLite 3.46.1 mechanism probe (NOT Bazel/Rust tests): of 6 unprotected invalid-page-length cases (2 lengths x 3 suffixes), 2 journal sentinels were deleted, and 6 metadata-only refusal cases preserved all original sentinels. The probe did not demonstrate every suffix is destroyed in this specific invalid-header schedule; it did confirm a real journal-loss path.

Still outstanding: this is only a necessary structural guard, not proof of completed initialization or sidecar ownership. Page-aligned but invalid or partially initialized DBs, a header-read failure after a fresh raw pin, and post-reservation pathname replacement remain unresolved. Native macOS/Windows, full external sqlite3 daemon-reader endurance, and nonzero exact-head Bazel tests are not yet proven. CI has not yet reported a run for this commit. Keep this PR under review hold and issue #4513 OPEN; no master completion/merge is claimed. Mixed draft #4776 and release #4763 remain untouched.

Match the pinned Bazel rustfmt_check_2024 output for the refused
empty-attachment writer-lock regression without changing its behavior.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — fixed the exact-head hosted Bazel Repository gates rustfmt failure on #4805 without changing behavior. Commit 9a6e93d changes one line-wrap in crates/tracedecay-rusqlite-runtime/src/connection/path_identity_tests.rs, matching the pinned rustfmt diff at line 177 from failed run 42452325096 (job 128643991644). This was committed as a child of verified old head 761cc49 and pushed with force=false and expected_sha lease; all peer work remains intact. New-head CI is queued, not a Bazel passing result.

Independent disposable Linux SQLite 3.46.1 checks this run reproduced the successful-init overwrite of pre-existing -wal/-shm/-journal bytes (3/3), demonstrated 3/3 exclusive-create refusals preserving sentinels, and verified separate-process WAL read of committed row 1 while an uncommitted row 2 existed, plus external writer exclusion/release in both WAL and DELETE modes. These probes do not run this Rust code, prove the new-head Bazel tests, or qualify native macOS/Windows.

The existing remaining holds are unchanged: full-page malformed/retry lock safety, complete initialization/sidecar ownership, post-reservation substitution, ordinary external sqlite3 + isolated-daemon endurance/restart, native macOS/Windows and proof on master. Keep #4513 open and PR-merging with the existing maintenance lane. Never merge held draft #4776 or touch release #4763.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published 45d0ed2 as an issue-focused regression-only update to this existing PR. The test now exercises RepositoryPhysicalAttachmentFactory::initialize, attach, and attach_read_only for each pre-existing -wal, -shm, -journal sentinel, verifying Reserve refusal, invalid-header retry refusal, unchanged zero-byte main file and byte-exact preservation of foreign sidecars. It covers the real factory entry points rather than only calling the pin helper. The commit is a one-file, 61-line addition and a verified nonforce expected-old-head fast-forward from 9a6e93d, preserving peer commits. Exact-head CI is pending; no repository Bazel or native macOS/Windows run is claimed. This does NOT fix post-reservation replacement, page-aligned incomplete initialization or full external-sqlite3 daemon endurance. Keep #4513 OPEN; ordinary merges remain with the maintenance owner. Draft #4776 and release #4763 remain untouched.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Refs #4513 — issue-specific follow-up published on #4805 (non-force, expected-old-head fast-forwards retaining prior and peer commits): b154e6b replaces the existing-store raw-file header read with a read-only immutable SQLite-VFS admission probe before acquiring the independent raw pin, and 02783ec adds a falsifiable Unix cross-process POSIX RESERVED-byte regression. 935d3d4 only normalizes test imports. Four SQLite-only files changed across these commits.

Specific fix: previously a page-aligned malformed database with a bad SQLite header could be opened via File::open for inspection, then refused and closed. On Unix, that extra close can release another live connection's process-owned record locks. The immutable SQLite probe is read-only, ignores WAL/SHM/journal and uses the SQLite VFS for its open/close lifecycle. The factory and direct tests now reject page-aligned false-header stores (512/4096 bytes) without altering foreign sidecars. A separate test preserves acceptance of a legitimate live WAL store whose schema changes remain uncheckpointed. The new subprocess test inspects SQLite's actual RESERVED advisory-lock byte before and after corrupt-header rejection, with a preheld writer and without vacuous zero-test acceptance.

Independent disposable Linux SQLite 3.46.1 mechanism checks passed: 6 malformed aligned-header/sidecar preservations and 10 external writer-lock/immutable-probe assertions, plus a separate direct POSIX-fcntl schedule demonstrating exclusion survives SQLite immutable refusal whereas an independent raw open/close releases it. These checks are NOT the Rust regressions, repository Bazel, or native macOS results.

Validation hold: exact-head CI run 42465671210 was queued when checked. Require nonzero bazel test //crates/tracedecay-rusqlite-runtime:unit_test --test_output=errors on this SHA, plus affected integration/format gates and a native macOS workflow run on the exact head. The full underlying issue is not solved: the immutable main-file probe does not prove completion of initialization or own the sidecars; post-reservation POSIX sidecar replacement, safe recovery from partial initializations, ordinary daemon + external sqlite3 endurance/restart, and native platform evidence remain open. Keep #4513 OPEN; do not merge from this comment. Ordinary PR merging stays with the maintenance lane. Held draft #4776 and release #4763 remain untouched.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published issue-specific formatting repair fcd316b6ca08 directly on this PR using an exact-head, nonforce fast-forward from 935d3d4d8733; the remote is now at fcd316b6ca08, one commit ahead / zero behind that prior head. Only connection/mod.rs and connection/path_identity_tests.rs changed, with the three formatting hunks requested by Bazel's pinned rustfmt.

The preceding exact-head CI run 42465671210 failed Repository gates at //crates/tracedecay-rusqlite-runtime:rustfmt_check_2024 on precisely those hunks (immutable-inspection let wrapping and two OpenOptions chain layouts). Other CI jobs were still pending/queued when checked, not passed. This commit is behavior-preserving and does not fix the known post-reservation replacement race or incomplete-initialization recovery. Do not infer Rust unit-test or macOS acceptance until actual nonzero results on this new SHA.

Keep #4513 open; ordinary PR merging is owned by the existing maintenance lane. Held draft #4776 and release #4763 are untouched.

Transfer creation pins into publication and close all raw main/sidecar
verification handles before clearing the admission receipt. Apply the
same ordering to prepared publication and published-family recovery.

Add deterministic Unix subprocess regressions that admit a real SQLite
connection at the completed-handoff event and verify rollback/WAL read
locks, independent WAL writer exclusion, release, and restart integrity.
Preserve refusal when cold recovery lacks durable allocation generations.

Standalone Linux SQLite 3.46.1 mechanism checks passed; repository Bazel
and native macOS qualification remain required on this exact commit.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — published a production handoff repair and four Unix regression tests in bc5cbc1fc5ad78e05748d1e73628f0e92296e74f.

Repair

The raw-SHM report led to a concrete ordering defect in the publisher: retire() cleared the preparation receipt while publish_prepared's verification descriptors, the outer publish creation descriptors, or recover_published's descriptors were still in scope. verify_read_only_handoff observes the receipt without acquiring the writer lease, so an admitted connection could acquire SQLite locks before a remaining raw main/SHM descriptor closed.

Fresh publication now transfers creation handles into the same owned collection as the verification handles. That entire collection is closed before the receipt is cleared. Prepared publication and already-published recovery apply the same order. Identity/content checks, no-replace publication, foreign-file preservation, durable receipt handling, and refusal on unproven allocation generations are unchanged. The completed-handoff tracing event now records the actual released boundary.

Regression coverage

The new initialization::tests::handoff module uses the canonical tracing-capture helper to admit a real SQLite connection exactly at that production boundary, without sleeps or test-only runtime hooks. A separate libtest subprocess must remain excluded after publisher teardown. It covers rollback read snapshots, WAL read snapshots, independent WAL writer exclusion, ordinary acquisition after release, committed data and restart integrity across fresh/prepared publication and published recovery. Cold recovery on filesystems without allocation generations explicitly exercises preservation/refusal; it does not pretend to qualify successful recovery there. The child process has a completion marker to reject a zero-test subprocess green.

Evidence and validation hold

  • 8 standalone Linux/native SQLite 3.46.1 tests passed. Negative controls reproduced lost rollback-reader/WAL-reader/WAL-writer exclusion when raw descriptors close after admission; the release-before-admission controls preserved exclusion and restart integrity. Additional cases preserved occupied WAL/SHM/journal names and foreign cross-process replacements. These are Python/native SQLite mechanism tests, not execution of the Rust regressions.
  • The original source was reconstructed byte-exactly and checked against blob 9933b1d56d586badfbfbaec1c46af82b889344f2 before editing; both published blobs match the locally reviewed bytes. Compare confirms exactly two files, one commit ahead/zero behind the former PR head. Publication used force=false and expected old head 77e765888ad1bd1f0f9ba4745d24e9ebb1097de7.
  • Exact-head CI is pending, not passed. No local Bazel/Rust execution is claimed; Cargo was not used.

Required exact-head checks:

bazel test //crates/tracedecay-runtime-core:unit_test --test_output=all --test_arg=initialization::tests::handoff
bazel test //crates/tracedecay-runtime-core:unit_test //crates/tracedecay-rusqlite-runtime:unit_test --test_output=errors
bazel build //crates/tracedecay-runtime-core:rustfmt_check_2024

Confirm nonzero execution of the four behavioral regressions, and run the same affected targets natively on macOS using the existing workflow. The available connector exposes job reads/reruns but no workflow-dispatch action; no workflow files or infrastructure were altered.

This repairs the publication/pin-lifetime window, not arbitrary same-user mutation of the active canonical SQLite namespace. That separate P1 limitation, current-head native qualification, daemon acceptance, and verification on master remain open. No approval, merge, or issue closure; ordinary merges remain with PR maintenance. #4776/model work and release #4763 are untouched.

Refs #4513. Apply the exact three rustfmt 2024 hunks reported by
hosted Bazel repository gates for bc5cbc1; no behavioral edits.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published a narrow hosted Bazel Rust 2024 formatting repair on the existing issue-specific PR #4805: 714474f9dc8e.

The previous exact-head CI run 42703851238 failed //crates/tracedecay-runtime-core:rustfmt_check_2024, not the Linux Rust suite. Its log specified exactly three hunks in src/shard_runtime/registry/initialization/tests/handoff.rs; this commit applies those three exact formatting changes with no behavioral, fixture, or assertion changes. The other repository-gate tests passed (8/8); Clippy, Dashboard, and the hosted Linux jobs succeeded on the previous head. The dependent Feature gates job failed because Repository gates failed; macOS was skipped, and Windows qualification was cancelled on that previous run.

One-parent, issue-focused commit; the branch advanced by expected-old-head (bc5cbc1fc5ad) nonforce fast-forward and remote re-read verified the exact published SHA and the one-file diff. Exact-head CI 42720083882 is queued, not passed.

A separate disposable Linux SQLite 3.46.1 subprocess baseline passed 57 nonzero assertions: read-only SQLite processes observed every completed WAL commit, restart/integrity checks passed, and O_EXCL correctly refused all three occupied WAL/SHM/journal names. These are standalone tests, not repository Bazel, an actual TraceDecay daemon, or native macOS results.

The existing P1 canonical sidecar namespace replacement review remains unresolved and requires a real ownership boundary, not another pathname recheck. PR #4805 is not master, and issue #4513 stays OPEN. No merge or change to draft #4776, release #4763, or the independent PR-maintenance workflow.

Refs #4513.

Use Arc::try_unwrap to fence late WeakReaderPool upgrades before
acknowledging initialization-reader or attachment shutdown. Restore the
same pool and retry within the existing deadline when another owner won.

Track worker owners extracted for out-of-lock joins so a concurrent lost
file callback cannot report closure and release main/sidecar pins while
SQLite connections are still alive. Preserve a failed join on retries.

Add deterministic production-path races and independent SQLite subprocess
checks for writer exclusion, committed data and integrity. Rust/Bazel and
native macOS execution remain pending; local standalone Linux SQLite
mechanism probes are not repository test results.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review repair published: shutdown ownership fences

Refs #4513. Published 89f84bd672c9 on this issue-specific branch. This is not an approval or a claim of default-branch completion.

Source defects repaired

  1. A quiescent Arc count did not establish exclusive shutdown ownership. Both initialization-reader close and ordinary attachment drain observed ReaderPool::is_quiescent, then dropped the pool as though that necessarily joined its SQLite workers. An exact-SQL weak handle can upgrade between the observation and retirement. The surviving strong reference would keep the workers alive after the caller acknowledged reader shutdown. ReaderPool::try_into_shutdown now uses Arc::try_unwrap to atomically claim the actual PoolInner and fence subsequent weak upgrades. A failed claim restores the same pool and retries within the existing deadline. Successful owners join outside the attachment mutex.

  2. Worker owners temporarily extracted for an out-of-lock join were invisible to lost-file observations. During that interval, the writer's lost-file callback can see empty normal slots. lost_file_closed could then treat empty normal/lost slots as proof of closure and release main/sidecar identity pins while SQLite connections were still live. The new workers_closing state covers that interval, prevents premature closure/pin release, and clears only after the joins return. close_and_join also requires completed drain. A writer join error is retained so a later drain cannot report success merely because its slots are empty.

The repair does not add any pathname deletion or relax existing WAL/SHM/journal preservation checks.

Regression coverage added

  • a_late_weak_upgrade_cannot_outlive_an_acknowledged_reader_shutdown
  • initialization_reader_close_waits_for_a_late_weak_upgrade
  • attachment_drain_waits_for_a_late_weak_upgrade
  • lost_notice_during_out_of_lock_join_preserves_pins_and_writer_exclusion (Unix)

The attachment regressions use real SQLite and deterministic one-shot hooks at the production ownership boundary, not a timing-only attempt to win the race. Fresh subprocesses check writer exclusion, committed-row visibility, and integrity_check. The snapshot helper requires successful exit, exactly one executed test, and a completion marker. The lost-notice test deliberately injects the real callback while the files remain in place so an independent process can probe the original locks; it does not claim to simulate a filesystem rename.

Verification and limits

Remote comparison confirms exactly one commit ahead, zero behind the inspected 714474f9dc8ed0362c7088d83cc36c143574795d, changing four SQLite runtime files. The branch was re-read immediately before an expected-old-head, nonforce fast-forward, and the resulting PR head was verified.

Fourteen nonzero standalone Linux SQLite 3.46.1 assertions passed. They demonstrate the raw-descriptor-close lock-loss mechanism for a rollback-mode main file and a WAL SHM file using independent subprocess writers, followed by committed-data and integrity checks. These are not execution of the new Rust regressions, repository Bazel results, a TraceDecay daemon journey, or native macOS qualification. Bazel is unavailable in this runtime; Cargo and direct rustc were not used.

Exact-head hosted CI for 89f84bd is queued. The new source still requires compilation, meaningful nonzero Bazel execution, and native macOS qualification. Previous-head platform results do not qualify this commit.

Keep #4513 open and the existing independent merge hold in place. This lifecycle repair does not resolve the previously documented canonical namespace/sidecar replacement P1 or prove full acceptance on master. No merge, issue closure, protected #4776 change, release #4763 change, CI weakening, or PR-maintenance automation modification was made.

Refs #4513

Apply the exact Rust 2024 rustfmt hunk from hosted repository gates
without changing shutdown ownership, SQLite behavior or test assertions.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published a narrow exact-head CI formatting repair to this existing issue-specific PR: d80c1b617abd.

The prior exact-head hosted Bazel repository-gates job failed //crates/tracedecay-rusqlite-runtime:rustfmt_check_2024. Its complete error output identified one exact rustfmt hunk in src/repository/attachment.rs:597, joining a multiline if quiescent && let Some(readers) = state.readers.take() condition onto one line. The commit applies only that mechanical adjustment (one file, +1/-3 lines), without changing reader shutdown ordering, SQLite policy, tests, workflows, or dependencies. All eight repository-gate tests in the previous run passed; the rustfmt build failure made the overall gate red.

The GitHub write used an expected-old-head (89f84bd672c9), nonforce fast-forward; remote re-read and compare confirmed exactly one commit and one modified file (1 ahead, 0 behind). Fresh exact-head CI #42748402187 is pending, not a Bazel pass. The preceding run's Linux fleet and Clippy remained queued at last inspection; macOS was skipped.

Separately, 17 nonzero local standalone Linux SQLite 3.46.1 cross-process assertions passed for WAL writer exclusion, external read-only visibility before/during/after commit, restart integrity, and occupied WAL/SHM/journal preservation. These do not execute TraceDecay Rust/Bazel or native macOS.

No acceptance claim or merge: issue #4513 stays OPEN. The documented canonical sidecar namespace replacement P1 and native macOS/current-head regression qualification remain outstanding. master is aa03f27ef074 and the issue-specific PR is unmerged. Protected Kev draft #4776, release #4763, and independent PR-maintenance ownership were untouched.

Refs #4513.

Require exact single-entry names before either native no-replace rename
or sibling replacement. A dirfd does not confine absolute paths, parent
traversal or intermediate symlinks. Reject normalized aliases and NUL
without opening any SQLite descriptor or mutating either namespace.

Add filesystem preservation regressions for the database and all three
sidecars, plus an independent-process intermediate-directory replacement.
Keep the separate full-path framed-log API and existing positive tests.

Native Linux renameat2 reproduces all three escapes with a committed
SQLite row. Repository Bazel and native macOS qualification are pending;
this does not resolve the remaining same-UID canonical leaf race.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published 73b01ef17b85, a production namespace-boundary repair plus regression tests. Updated this branch from exact old head d80c1b617abd13b2698a61f15e9d1758a79ef81d using expected_sha and force=false; remote verification confirms the new head. The comparison is exactly one new commit, zero behind its parent, and two changed files.

Defect and repair

capability_dir::rename_noreplace forwarded arbitrary source/destination paths to native rename operations. A held directory descriptor does not confine ../foreign/sessions.db, an absolute path, or bridge/sessions.db when another process replaces bridge with a symlink. Those paths can move a foreign file despite the advertised single-entry capability contract.

Both rename operands now require an exact single normal entry before any filesystem mutation. The shared validator also rejects empty/dot names, normalized aliases such as payload/., and NUL, and is reused by sibling replacement. It does not open another SQLite descriptor or introduce an inode-check/unlink race. The separate full-path framed-log API is unchanged. Current SQLite publication uses fixed or extracted leaf names; this repairs the underlying primitive's missing boundary, not a claim that a normal registered caller currently supplies traversal.

Regression coverage and verification limits

Added three behavioral Rust regressions plus a child-process helper in capability_dir/namespace_tests.rs: source/destination traversal and absolute-path refusal preserving the main/WAL/SHM/journal fixtures; malformed-name refusal for both rename variants; and an independent process replacing an intermediate directory after its parent is pinned. The subprocess test requires both a completion marker and a nonzero executed-test result. Existing successful-leaf and occupied-destination tests remain unchanged.

An independent disposable Linux/native renameat2 reproduction using real SQLite 3.46.1 databases reproduced all three escapes (parent traversal, absolute source, child-rebound intermediate symlink). In each case the foreign database's committed row was moved and remained readable/integrity-valid: 12 nonzero assertions. This is a standalone mechanism reproduction, not execution of the new Rust tests or proof of the patched repository behavior.

This runtime has no Bazel installation; Cargo was not used. Required repository verification remains bazel test //crates/tracedecay-private-fs:unit_test --test_output=errors and bazel build //crates/tracedecay-private-fs:rustfmt_check_2024, plus the relevant runtime/SQLite suites. Exact-head CI is pending. Native macOS/Windows qualification is not claimed.

Keep #4513 open and retain the merge hold. Rejecting path escapes does not make replacement of a valid canonical leaf by a noncooperating same-UID process safe. That separate namespace-ownership requirement and default-branch acceptance remain outstanding. No master merge, protected #4776 change, release #4763 change, infrastructure change, or maintenance-task modification was made.

Refs #4513.

Require HAS_MOVED attestation when verifying a live SQLite worker. An unchanged path cannot establish the identity of a VFS handle when file control is unsupported. Exercise the actual SQLite memdb VFS and confirm a normal disk worker still attaches to the unmodified store.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published 12f5b305b6b9 on this issue-specific PR.

Worker identity fail-closed repair: OpenedDatabaseFile::verify_connection previously accepted SQLITE_NOTFOUND from SQLITE_FCNTL_HAS_MOVED because its check rejected only Some(true). A stable canonical pathname authenticates the name, not an unverified SQLite VFS handle. Worker verification now refuses missing file-control proof with OpenedDatabaseFileError::Inspect; a positive Some(true) still rejects replacement and a supported Some(false) retains both pathname checks. The existing verified-reader and Darwin initial-pin paths already refused None. No sidecar deletion, extra raw SQLite descriptor, policy relaxation or CI change was introduced.

Real-VFS Rust regression added: worker_identity_refuses_a_vfs_without_move_attestation uses SQLite's built-in memdb VFS against the same on-disk pathname. It checks the actual SQLITE_NOTFOUND result, requires worker verification to refuse it, then opens the actual on-disk database through the ordinary VFS, verifies positive admission and reads the original committed row. This should fail before the repair, and tests the production method rather than a mocked file-control result.

Verification boundaries: 15 nonzero standalone Linux SQLite 3.46.1 C assertions passed: the ordinary file VFS returned SQLITE_OK for HAS_MOVED, memdb returned SQLITE_NOTFOUND, and the committed on-disk SQLite header remained intact. These are mechanism probes, not execution of the repository Rust test, Bazel, a daemon, or native macOS. This runtime has no Bazel runner; Cargo was not invoked. Require nonvacuous bazel test //crates/tracedecay-rusqlite-runtime:unit_test --test_output=errors plus the Rust 2024 formatting gate and current-head Linux/macOS qualification. Exact-head hosted CI #42764933700 was queued at last check.

The branch update used the exact previous head 73b01ef17b85, expected_sha, and force=false; post-write comparison confirms one commit (ahead 1, behind 0), exactly two SQLite-runtime files changed. Issue #4513 remains open: this closes a missing-VFS-attestation hole, not the unresolved same-UID canonical WAL/SHM/journal leaf-replacement boundary or full default-branch acceptance. master remained 2f9d659 at write time. Protected draft #4776, release #4763, and the separate PR maintenance workflow were not touched.

Refs #4513

Apply the four mechanical Rust 2024 formatting changes reported by
hosted Bazel run 42764933700. Preserve all assertions and behavior.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published an exact-head Bazel formatting repair on the existing issue-specific PR.

Commit: b1f093e8966e

The previous head's hosted Repository gates completed with eight Bazel test targets passing but failed two rustfmt_check_2024 build actions. The job log names exactly four formatting hunks: three in crates/tracedecay-private-fs/src/capability_dir/namespace_tests.rs and one in crates/tracedecay-rusqlite-runtime/src/connection/tests.rs. This commit applies those four mechanical changes, without altering production code, test assertions, dependencies, policies, or CI. GitHub comparison verifies one new commit and exactly those two files; the update used expected_sha=12f5b305b6b9 with force=false and the exact new branch head was read back.

New exact-head hosted run: #42773456440 is queued, not yet passed. Independently, 18 nonzero standalone Linux SQLite 3.46.1 assertions passed across three disposable databases, with subprocess read-only WAL snapshots during a pending write, commit visibility, sidecar presence, reopen and integrity_check. Those are not repository Rust/Bazel or native macOS results. No local Bazel runner or checked-out source tree is available in this environment; no Cargo used.

Hold: PR remains unmerged, issue #4513 remains OPEN, native macOS qualification of the latest source remains outstanding, and the canonical sidecar-namespace same-UID replacement boundary is unresolved. Existing draft #4776, release #4763 and ordinary PR-maintenance ownership remain untouched.

Refs #4513. Preserve all cancellation checkpoints and schema assertions,
but avoid repeatedly reopening the registered runtime just to inspect its
schema. The independent SQLite reader observes the published bytes directly.
Refs #4513. Bind the collected schema rows before returning so the
SQLite statement and iterator lifetimes are explicit.

Copy link
Copy Markdown
Owner Author

Refs #4513 — published a narrowly scoped CI/cancellation-regression repair on the existing issue-specific PR (not a master acceptance claim).

Exact-head hosted evidence. Run #42773456440, Linux fleet failed only //crates/tracedecay-global-db:unit_test. I downloaded and inspected its bazel-test-logs-Linux-fleet artifact (ID 13754645707): the test executable reports 501 tests, 500 ... ok completions, then Test timed out at 2026-11-10 09:56:55 UTC at Bazel's 300-second deadline, with no panic or failing assertion. The new tests::schema_install_cancellation::cancelled_daemon_attach_preserves_published_schema_and_reopens sweep has no completion line; its code repeatedly calls schema_objects, which itself republishes/reopens the registered daemon runtime for every simple schema observation. This is an evidence-backed suspect for the tail, not yet a proved root cause.

Published commits: 1e9f428b and 879ab9a4. They change only crates/tracedecay-global-db/src/tests/schema_install_cancellation.rs (test fixture): retain the registered-before-admission scan, every cancellation poll checkpoint and the existing schema assertions, but read after-cancellation/reopened schema through an independent ordinary read-only SQLite connection, without triggering additional registered runtime publications. The fixture now asserts the independent reader agrees with the original registered scan; the second commit makes statement/iterator destruction order explicit. Both pushes were nonforce expected-old-head fast-forwards with exact remote head confirmation. No production code, test limits, timeout budgets, CI settings, releases or protected PRs changed.

Verification limit: local repository Bazel and native macOS execution are unavailable here. Standalone Linux SQLite 3.46.1 checks passed 17 nonzero assertions for subprocess read-only WAL snapshot visibility, committed-row/integrity preservation and foreign sidecar exclusive-create refusal, but are not repository tests. The new exact-head CI run #42896310870 is pending; these commits are not yet qualified. The canonical same-account sidecar namespace-replacement threat remains independently unresolved; #4513 stays open pending proof on master. Mixed Kev/decision-model draft #4776 and ordinary PR merging remain under their existing owner holds.

Return the collected external SQLite schema rows directly, without the
redundant temporary that fails -D clippy::let_and_return. Preserve the
statement lifetime, reader independence, and regression assertions.

Refs #4513

Copy link
Copy Markdown
Owner Author

Refs #4513 — published 9f48ec0 to this existing issue-specific PR, as a one-file, one-parent nonforce expected-old-head fast-forward from 879ab9a4. The previous exact-head CI failed strict Clippy on schema_install_cancellation.rs:107 (clippy::let_and_return); the commit returns the collected independent SQLite read-only schema rows directly. No test assertions, worker lifecycle, or SQLite admission logic changed. New exact-head CI is queued, not a pass. Bazel and native macOS execution were unavailable locally.

The previous Linux Bazel fleet nevertheless passed //crates/tracedecay-rusqlite-runtime:unit_test, //crates/tracedecay-runtime-core:unit_test, and //crates/tracedecay-global-db:unit_test (226 s). Its two failed targets were //crates/tracedecay:hooks_lsp_suite (receipt backlog left 1 of 65 after the 10-second deadline) and //crates/tracedecay:runtime_acceptance_suite (LSP acknowledgement DeadlineExceeded). Neither failure is yet shown to originate from the SQLite change. Do not increase deadlines or weaken tests without root-cause evidence.

The acknowledged canonical sidecar namespace-substitution race remains on review hold. Issue #4513 stays OPEN until qualifying exact-master behavioral and native-platform proof exists. Protected draft #4776 and release #4763 remain untouched; ordinary merging remains owned by the separate PR-maintenance workflow.

@ScriptedAlchemy
ScriptedAlchemy merged commit 8228cf1 into master Nov 12, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant