Repository navigation
fix(storage): stream growth measurements and preserve registry errors - #4794
Conversation
|
Refs #4099. Aggregate each table's dbstat payload inside a correlated subquery instead of materializing all pages for GROUP BY. Preserve the existing reader snapshot, cancellation, deadline and typed-error path. Add nonzero filesystem-backed regressions for exact payload attribution, growth, logical deletion versus physical compaction, VM-work reduction, and interruption followed by a successful retry. No retention policy or measurement budget is relaxed; full 24-hour profile acceptance remains open.
Refs #4099. Apply the exact rustfmt delta reported by the hosted Repository gates; no production logic or acceptance criteria changed.
Refs #4099. Use a value assertion so the hosted Clippy assert_is_empty policy can display an unexpected store on failure.
Refs #4099. LIKE treats the underscore in sqlite_% as a wildcard, silently excluding valid user tables such as sqlitemessages. Match the literal reserved sqlite_ prefix instead without changing the streaming page census or its cancellation and snapshot boundaries. Add a file-backed regression that requires the omitted table's nonzero payload and subsequent growth to match an independent dbstat sum while still excluding sqlite_sequence. Preserve the concurrent CI repair.
|
Published a3654ac directly to this PR through GitHub ( Production bug fixed: the census excluded Regression added: the file-backed Rust test calls the production reader, requires a nonzero Executed SQL verification: using SQLite 3.46.1 on temporary files and SQL extracted from the candidate source, the old filter omitted 1,048,581 payload bytes; the new filter reports them and subsequently reports 2,097,162 bytes after another insert. The original fixture still has exact reference parity and 6,577 versus 19,043 VM steps. Interruption returns SQLITE_INTERRUPT, and retry returns the complete exact result. These are actual SQL checks, not a Rust/Bazel test pass. The uploaded source blob Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42442890231 (queued at verification). Local Bazel and Rust toolchain are unavailable. Require the real current-head Bazel/Clippy/format results before merging. This is a bounded correctness fix, not evidence that the original 153 GB incident or #4099's 24-hour master acceptance is resolved. |
Refs #4099. Align the new user-table growth assertion with the exact format expected by the Bazel rustfmt gate; no behavioral changes.
|
Published The Repository gates log showed only a formatting difference in the new On The master baseline is unchanged at |
Refs #4099. The offline registry snapshot followed symlinked WAL/SHM members and could copy arbitrarily large foreign files into scratch. Refuse nonregular members before copying, and keep physical family byte counts from charging symlink targets outside the inspected profile. Add real offline-report and family-copy regressions for symlinked source members, plus partial-coverage and size assertions for user sessions. No operator data is copied or deleted.
|
Published 1e3d62f (Refs #4099), direct to this issue-specific PR. One-file update in New root-cause finding: the offline Fix: fail closed on every nonregular/linked snapshot member via Behavioral regressions: a real on-disk SQLite registry with a symlinked WAL and SHM, each targeting a sparse 512 MiB foreign file, must reject copying before a destination is created; the end-to-end offline report must return a typed Database failure for a linked registry WAL; and an offline report with a regular 4096-byte Independent checks: manually reproduced the original This is partial correctness only; #4099 remains open until 24-hour fresh-profile, multi-store bounds and daemon-owned reclamation are proved on |
|
Published The hosted Repository gates on Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42462849057 (queued at check). The previous run's Repository gates failed only on this formatting hunk; the Linux/Clippy jobs were still queued there at review time and are not evidence of a pass. There is no local Bazel executable or supplied checkout, so I do not claim a local Rust test pass. Require passing real nonvacuous The |
Refs #4099. The SQLite snapshot enumerated sidecars before capturing its validation generation. A writer creating a WAL in between could commit new project registrations that the report then misclassified as unregistered. Capture the generation first, then inspect only regular members and validate after copying. Add a deterministic on-disk WAL race regression: stale generations must fail closed, while a fresh generation preserves the committed project. Also fix the pinned Clippy collapsible-if failure in family sizing.
Refs #4099. The registry snapshot preflight rejected symlinks, but fs::copy reopened each checked path and could follow a link swapped in after inspection, copying foreign bytes into report scratch. Open through a held parent capability with no-follow leaf policy, then copy from that file handle. A real filesystem regression swaps a checked WAL member to a foreign link and proves the copy refuses without creating a scratch member. Preserve existing CI repairs.
|
Published New independently identified correctness/safety issue: The offline SQLite family copier checked each source member with The Linux filesystem TOCTOU mechanism was independently reproduced using a regular-file probe followed by symlink replacement: ordinary copy followed the foreign target while |
Refs #4099. A no-follow open still accepts a regular-file replacement between preflight and copy, potentially copying a huge foreign WAL before post-copy generation validation rejects it. Compare the opened descriptor against checked identity and length before creating scratch; bound the copy to the inspected length plus one. Add a filesystem regression for same-sized and oversized regular-file swaps without reading the foreign bytes.
|
Published Independent bug found: The preceding no-follow fix rejects symlink replacement but still opens an ordinary file swapped into the WAL/SHM pathname after the first metadata check. Post-copy Production repair: Preserve checked metadata for each SQLite main/WAL/SHM member; compare the opened, held file descriptor's identity and inspected length before creating the scratch member (Unix device/inode; non-Unix conservative size/time checks with existing generation validation). Cap each copy at the previously measured length plus one byte and reject a concurrent size change, rather than chasing an unbounded WAL tail. No dependencies, CI policy, thresholds, retention ownership, or production data were changed. Nonvacuous Unix regression: Replaces a previously checked, 4 KiB regular WAL with a different regular inode of equal size and again with a 512 MiB sparse file. It calls the production copy helper, requires a typed Independent filesystem reproduction: On Linux, |
Refs #4099. A rollback-mode writer may spill uncommitted registration pages into global.db while the undo journal remains. Copying only the main/WAL/SHM family would turn those pages into false registry truth. Fail closed on the rollback journal before writing scratch. Add direct and end-to-end real-SQLite regressions for refusal during an open transaction and correct registration after rollback. Preserve all prior snapshot generation and no-follow protections.
|
Published focused Root cause: the offline registry reporter copies only Published repair: refuse a present rollback journal before writing snapshot scratch and propagate the existing typed report failure instead of classifying stores using an unsafe registry. Two real-SQLite Rust behavioral regressions test the lower-level snapshot refusal (including no output) and the end-to-end The exact-head non-force fast-forward preserved |
Apply eight exact Rust 2024 rustfmt differences from hosted Bazel Repository gates. No behavior, schema, or test assertion changes.
|
Refs #4099 — published follow-up to the no-follow footprint fix on this existing issue-specific PR.
Independent local POSIX reproduction verified why a child-path Verified exact-old-head non-force fast-forwards Still partial: no master-proven daemon reclamation completeness or 24-hour fresh-profile bounded-growth acceptance; #4099 remains open. Root-link checks close stable linked roots and do not claim to solve adversarial rename/symlink races between metadata inspection and later path access. |
Refs #4099. Preserve an exact empty entry set assertion with typed value-level diagnostics, rather than the Clippy-denied assert_is_empty form. No production semantics or coverage assertions change.
|
Refs #4099 — published commit Verified current-head CI repair: previous run #42772419833 failed strict Clippy ( New-head CI #42903848747 is queued; no Bazel/Rust pass is yet claimed. The prior Linux fleet separately failed two automation-runtime settlement tests and timed out in the global-db unit suite. Those targets are outside this storage change and remain for their owners, not modified or suppressed here. Local Bazel and an authenticated checkout are unavailable. Keep #4099 OPEN and #4794 unmerged: complete daemon-owned retention/reclamation and the 24-hour fresh-profile per-family bounded-growth acceptance on |
The stdio proxy reads its optional hint setting on a background connection. Keep the restart fixture listening when that settings RPC arrives before the host initialize request, returning method-not-found to unsupported RPCs instead of consuming the sentinel response. Preserve the restart deadline, authenticated preamble, response drain, and exact initialize sentinel assertion.
Retain measured bytes and configured bound when registered-root details or valid long evidence identifiers exceed Doctor's 512-byte text limit. Only compact optional details and hash the evidence reference if needed; full identities remain in the storage report. Add a filesystem-backed census-to-Doctor regression for both truncation cases. Refs #4099
Compare the bounded final statement against the same control-folded text Doctor publishes, so short checkout names with tabs or newlines do not lose their identity. Extend the real profile census regression to prove sanitized identity, intact measured overage, and existing long-root trimming. Refs #4099
Refs #4099 Read the maintenance-owned table-growth state through the existing no-follow, nonblocking bounded-file authority. Refuse sparse oversized state and foreign symlinks without importing their bytes; warn and re-baseline as for invalid state. Cap the corresponding writer output before publication, preserving the previous file on refusal. A real-filesystem regression proves valid persistence and readback, sparse oversize refusal without payload allocation, and that a valid foreign symlink is not accepted. No schema, budgets for live store measurement, daemon GC, or CI policies change.
Refs #4099 Do not replace an existing oversized, foreign-linked, or otherwise unreadable table-growth checkpoint after the new bounded reader refuses it. Surface the failure to persistence so the previous bytes remain untouched, while live sampling can establish a new in-memory baseline. Strengthen the real-file regression to assert refusal preserves sparse file length and foreign symlink/target through a persistence attempt.
Refs #4099 Match the actual I/O error variant without requiring the persisted TableGrowthStateV1 map to implement Debug. Preserve positive on-disk checkpoint and foreign-link identity assertions.
|
Refs #4616 — coordinated Doctor/graph census identity blocker. I confirmed this still exists on your current #4794 head |
Refs #4099 A failed dbstat timeout is a process-local wall-clock mark. If the wall clock moves backward, negative elapsed time currently passes the 24-hour suppression predicate and incorrectly blocks retries. Reject a future mark and cover the actual guarded telemetry port with a regression that starts from an established baseline and demands another live measured result.
|
Refs #4099 — published an issue-specific table-growth retry correction in Root cause: Fix: A failed-walk marker now suppresses a re-probe only if the current wall-clock reading is not earlier than its observation. The existing live-page/freelist threshold, 24-hour suppression for normal clocks, deadlines, and typed errors remain unchanged. This also avoids treating a regressed clock as permission to fabricate a size result. Behavioral regression: Extended the existing real-SQLite guarded telemetry-port test after establishing a baseline. It injects a mark one retry interval in the future, then demands an actual Publication/review: Compared GitHub's exact old head Merge hold: #4099 stays open pending completed exact-head Bazel CI, independent review and verified daemon-owned reclamation plus 24-hour per-store/profile bounded growth on |
|
Refs #4616 — independently reproduced a second cross-projection Doctor accounting defect, distinct from the previously reported generation-only sealed-head census. This is a source review / SQLite reproduction, not a code change or Bazel pass. On current AND replay.generation NOT IN (
SELECT generation FROM graph_publication_replay_dependencies_v1
)But the dependency table's durable FK binds I executed the current SELECT against a real in-memory SQLite database with the production column schema: two projections A/B, each with an old Replace the global generation-name exclusion with This source is actively owned by #4794. I have not edited its branch or changed #4819/other peers. No change to protected #4776, Kev/model, release #4763, CI, or schedules. |
Apply the pinned formatter layout reported by repository gates. Validation: owning Bazel formatter target passed; independent review completed.
Resolve the footprint Doctor-statement conflict by keeping both intents: the branch's bounded evidence-reference/long-path hashing and master's registered-project reclaim guidance with budget-aware root compaction. Both now share footprint_statement and the control-character-folded fit check so a sanitized short root is not mistaken for an overlong one.
`rename_preview_deadline_stops_the_worker_before_the_next_file` carries one deadline through three timers: the pre-dispatch gate, the admitted read, and the outer dispatch ceiling. The ceiling can settle first on a loaded host, and the graph-tool adapter reports that retryable route refusal as an unavailable pre-admission state under `tool_dispatch_deadline_exceeded` rather than the admitted read's `timed_out`. The test only accepted the other two terminals, so the ceiling's legal settlement failed the assertion. Pin that record exactly instead of accepting any deadline envelope.
This reverts commit 0ef6822. Accepting `unavailable`/`pre_admission` for a deadline that fires after admission would codify a likely terminality misclassification and is outside this PR's scope. The rename-preview race is pre-existing on master and should be fixed at its owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The projection-scoped census regression in `build_tests` calls
`census_sealed_store` but the module's `use super::{...}` list did not
import it, so the graph-db unit test binary failed to compile (E0425)
under Clippy and the Linux fleet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register the canonical global/session schema installer before publishing the isolated profile runtime. The projection-identity regression now executes independently instead of relying on test order; no production admission or assertion is relaxed.
|
Focused independent checks found an ordering defect in the Doctor projection-journal regression: it did not register the canonical global/session schema installer and refused before reaching its assertions when run alone. Fixed in the latest one-line test commit. Verified via Bazel + anti-vacuity: projection-scoped sealed census 1 passed, table-growth tests 8 passed, unreadable-registry footprint 1 passed, and isolated projection-journal liveness/retirement 1 passed after reproducing the fixture failure. Strict daemon-service unit Clippy and rustfmt passed. Production behavior/limits remain unchanged; current-head hosted CI still required. |
Seed the same 10,001 reconciliation rows in one canonical transaction. Make the Codex unsettled-clock regression independent of scheduler delay.
Resolve the storage report conflict against the Tokio runtime cutover and move the branch's remaining storage-growth tests to #[tokio::test]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scope
Refs #4099. This remains an issue-specific partial repair, not a claim that the deleted 153 GB profile has been reconstructed or that the 24-hour master acceptance has passed.
Measurement-path root cause and repair
The actual reader-worker
TableSizescommand joined onedbstatrow per database page tosqlite_schema, then usedGROUP BYto recover one total per table. That creates a temporary grouping sorter proportional to the number of pages being measured and consumes avoidable SQL VM work inside the existing measurement deadline. Large-store measurement was doing expensive page sorting in addition to the necessary page walk.The current reader initializes one total per user table from
sqlite_schema, then walksSELECT name, payload FROM dbstatonce on the same snapshot. It includes empty and rootless user tables, excludes index and internal pages from their totals, and returns an interruption or arithmetic error before publishing results. The earlier correlated per-tableSUMat8b403d1was an intermediate implementation.The existing registry fixes are preserved: the offline report distinguishes true
global.dbabsence from metadata/read errors and refuses registry symlinks rather than importing foreign registration identities into a profile whose census cannot account for those linked bytes.Behavioral regressions
Three new filesystem-backed tests call the production census implementation:
The earlier dangling/valid-external-registry behavioral tests remain in the PR.
Validation and review
9edee1472e796e6bde5bc1564b49a452c56e75d0before editing. Published blobs match the locally inspected source. Comparison against prior head43cef92shows only the production dispatch change and the new census/test module; the concurrent formatting fix is preserved.43cef92bc121f1bf9b451de1a3b92c8e40fd624f.8b403d1found only pinned-rustfmt differences in the new test module; the exact reported formatting corrections were published asd9559c8using a non-force fast-forward with expected old head8b403d1. The new-head CI run #42437543266 is queued; the old run's Rust jobs were cancelled by supersession, not passed. Required validation on the exact head:bazel test //crates/tracedecay-rusqlite-runtime:unit_test //crates/tracedecay-maintenance:unit_test --test_output=errors bazel build //crates/tracedecay-rusqlite-runtime:rustfmt_check_2024 //crates/tracedecay-maintenance:rustfmt_check_2024Still required before closing #4099
This removes a demonstrated source of measurement work amplification; it does not make the physical page walk constant-time or prove that every large store fits its deadline. Per-family bounds, daemon-owned retention/reclamation and the fresh-profile 24-hour normal-use acceptance must still be demonstrated on
master. Keep #4099 open; keep Kev/decision-model work and mixed #4776 draft and excluded from merging.Current CI repair (679f636)
On the prior head
d9559c8, pinned Repository gates passed. Clippy failed onstorage_report.rs:2492withclippy::assert_is_empty(-D warnings): the added test usedassert!(absent.stores.is_empty()). Commit679f636replaces that with a typedassert_eq!(absent.stores, Vec::<StoreSizeReportEntry>::new()), retaining the failure check and adding value-level diagnostic output. Published via exact-headexpected_sha=d9559c8,force=false, with no concurrent commits overwritten.The earlier fleet Linux job failed with
UNAVAILABLE: io exceptionfrom the external Bazel remote executor (not a repository test assertion). The earlier Clippy job additionally observed that transient executor failure. Fresh hosted CI is pending on679f636. No Bazel executable or Rust toolchain is available locally, so the current Rust test and Clippy results remain unverified. Do not claim passing gates, merge, or #4099 closure until actual results and master acceptance exist.Partial WAL findings
The long-key WAL review and its duplicate are addressed in
55ef65ea44. When Doctor hashes a footprint reference, its coverage statement now retains the storage family and the unreadable-entry count. Both the measured-byte floor and the configured size bound remain visible.Native macOS Bazel passed all 10 footprint tests and the repository formatting check. The WAL regression walks a real regular segment and an unreadable directory entry, then verifies both a proven overage and a partial finding below the bound. Linux exercises a key over 1,024 bytes. The native macOS path uses a control character to trigger the same reference fallback within the filesystem path limit. Exact-head Linux remains pending.
Nested graph sidecar accounting
At
4b542edc22, the project Doctor bound and the physical storage report use the same classifier for the container file and its recognized sidecars. Both count a spill's nested files, including an orphaned spill directory, against its named container. Unrelated suffixes and unrelated directories with a.grafeoname stay outside the container bound. A dedicated WAL retains its separate bound.On the preceding head, replacing the flat orphaned spill fixture with a real
.grafeo.spill/scratch/inflightpath made the Doctor finding disappear. The adjusted case now reports the exact nested bytes and a degraded container bound. Both whole macOS Bazel unit suites passed on the repaired head, with 176 maintenance and 431 rusqlite runtime cases, plus repository rustfmt and strict Clippy on the maintenance test target. Current-head hosted CI, independent review and the separate 24-hour master acceptance for #4099 remain open.