Skip to content

Share Cell publication and integrate verified bundle coverage - #67

Draft
forhappy wants to merge 50 commits into
mainfrom
codex/packed-write-publication
Draft

forhappy wants to merge 50 commits into
mainfrom
codex/packed-write-publication

Conversation

@forhappy

@forhappy forhappy commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Cellule's per-Cell publication work limits write throughput. This draft adds shared node-bundle selection with exact range proofs, signed append grants, bounded authenticated histories, actor receipt/read/retry integration, exact capture retirement and separately admitted asynchronous Cell roots. Authority fencing, origin verification, complete issued-range closure and joined shutdown remain required.

The latest fix observes authenticated selection readiness without owning the Cell publisher. A ready oldest prefix enters exact cleanup; older root debt can prepare while selection waits. The unchanged cleanup deadline begins after readiness. A real-actor regression fails with Fenced before the fix and passes afterward: delayed selection beyond ten seconds after valid Fleet ACKs, read/retry visibility, older-root progress, joined drain, cold state/results and released credits. The atomic gate-ID allocator also preserves checked uniqueness and Rust 1.97 compatibility without the API deprecated by current Clippy.

No acceptable throughput improvement or celld parity is established. Six fresh matched release cases compare 4a8f55c, candidate 6c909a6 and celld f2bf6486. They use 1,000 uniform Cells, 96-byte SQL INSERT+SELECT, a two-hour retry/result ledger, 128 clients/queue slots, 30-second warmup and 60-second window. The VM has 8 CPUs / 8 GiB shared memory; container ceilings exceed it. These overloaded completion counts are not sustainable capacities.

Mode Before writes/s Candidate celld Candidate successful scheduled p99 ms
Fleet, offered 15K/s 195.13 184.77 4,717.33 1,745.68
Bucket, offered 2K/s 247.55 248.68 1,321.83 4,512.72

Candidate Fleet returns 328,243 errors, drops 560,671 offers and fails 19,797 of 21,366 warm ACK checks; cold recovery and successful aggregate drain are unverified. Candidate Bucket has zero request errors and passes all 26,423 warm/cold ACK checks, but drops 104,823 offers. Celld Fleet is OOM-killed and fails its audit; celld Bucket passes all 124,185 warm/cold checks but drops 40,437 offers. All six points fail qualification. Fleet completion decreases 5.3%; Bucket's +0.46% difference does not establish a gain and bypasses the managed producer. The subsequent atomic compatibility fix was not part of these timed binaries; it is not presented as a performance improvement.

Fleet root density remains 11.12 commands/root; canonical PUT attempts are 0.5422/completed write and GET/range attempts 13.9287. Retention rises from 27.88 to 58.40 MiB of 64 MiB; oldest debt ends at 45,452 ms. Steady Bundle ACKs remain zero. The 215-command / conditional 0.05-PUT targets and bounded-debt qualification remain unmet. Sampled audit failures occur on POST retries after matched GETs; the pre-SQL backlog gate remains a concrete next regression target, not the proven cause of every failure.

Validation: the frozen selection-fix snapshot passes all eleven contributor routes, 1,962 passed / 0 failed / 38 ignored. All eleven contributor routes pass again after compatibility commit d852525, and Rust 1.99 Clippy passes for all targets/features with warnings denied. Final-head CI remains required. This compatibility change is outside the timed candidate binaries. Canonical reports and independent journal replay reconcile every offer/attempt/completion, error/drop and complete ACK cohort. Fixture, client/auditor, host and runner provenance match. All 9,530 indexed external evidence files rehash; raw journals, binaries and provider data stay outside Git.

Latest measurement and merge criteria, earlier repeat, delivery, runtime design.

Remaining before merge: restore pressure-time read/retry availability and successful joined drain; prove materializer progress and bounded debt; connect Bucket shared selection; reduce complete publication cost; qualify failed-owner Fleet suffix recovery and safe collection; finish sustainable capacity search, three paired five-minute repetitions and read-only/mixed guardrails with zero errors/drops and the original latency targets. The 2,000-Cell / 10K-write / 50K-read standard-node goal remains unqualified. Keep this PR in draft.

Keep per-Cell fenced root selection while uploading bounded application-scoped
capture cohorts. Issue signed append windows through fresh enrollment and
serialize native proof release with durable closure.

Document the remaining bundle-coverage protocol and preserve qualification
failures. Bundle-based bucket acknowledgments remain disabled.
@forhappy
forhappy force-pushed the codex/packed-write-publication branch from fc8147b to c590664 Compare October 7, 2026 09:04
@forhappy forhappy changed the title Share Cell publication and use signed follower append grants Share Cell publication and add verified bundle coverage APIs Oct 7, 2026
@forhappy
forhappy marked this pull request as ready for review October 8, 2026 01:11
@forhappy forhappy changed the title Share Cell publication and add verified bundle coverage APIs Share Cell publication and integrate verified bundle coverage Oct 8, 2026
@forhappy
forhappy marked this pull request as draft October 8, 2026 19:46
@forhappy
forhappy marked this pull request as ready for review October 8, 2026 23:18
@forhappy
forhappy marked this pull request as draft October 8, 2026 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant