[bradseiler]

Summary

Close admitted root and huddle-audio sockets when Redis misses a ban or membership change. Recheck durable state with bounded batches, concurrency, deadlines and a fail-closed database-error grace period. Timeouts retain the existing session policy.

Preserve verified owner delegation through scans, including NIP-FI token-owner admission when NIP-OA is disabled. Both real root and audio enrollment paths record that admission source. The new database regression admits both paths, scans them and then removes the owner without a Redis notification; it does not exercise JWT verification or transport.

Recovered corrections retry transient root AUTH policy-read failures, prevent stale-target scan starvation and report the captured owner safely. Common mesh and fixture fixes already merged on main are no longer duplicated in this layer.

Related issue

BUZZ-271. Stack: #8529 → #8241 → #8239 → #8235. Invitation revocation #8237 and workflow-owner bans #8242 are merged.

Testing

Current head d91fe8c485821dafe6b0296ba62691254352fb63 in native stack 8725, rebased by Brad through GitHub onto main 44b3c4823. Brad owns further ref updates.

The explicit AuthorizationUnavailable access-log label is authorization_unavailable; its close frame is unchanged. The selected outage-grace regression reads the result through production server_close_reason.

Independent review clears this correction and the restack. All 20 carried patches are unchanged in range-diff. The prior candidate passed relay clippy and 14 live-authorization/close-reason tests; a label mutation failed. That evidence carries over by unchanged patch review, and current-head hosted runtime CI is now terminal and green.

Historical private-relay tests are not current-head runtime acceptance. Huddle recovery during a continuing writer outage and human cross-pod verification remain separate follow-ups.

Normal pre-push hooks passed on integrated head 7ee3cf92f59a96349099d48fc115e5b8bb6a9716, including the branch-skew guard, unit tests, desktop checks/tests, Tauri checks, mobile checks/tests and file-size checks. These hooks validate the checked-out top tree; hosted CI remains the gate for each individual PR head.

Current-head hosted CI is terminal and green, including runtime unit and PostgreSQL jobs. McKeon independently cleared all four current heads: carried patches range-diff unchanged (8/20/21/6), with no overlap between the newer main delta and stack files. The separate main-only security-review authorization check remains failed for this stacked base; no bypass or retarget was performed.

Historical local validation at integrated head 7ee3cf92f5: buzz-db/buzz-relay clippy passed with all targets and warnings denied; package tests passed 2,035/2,035; desktop build, Tauri check/tests, web build and mobile tests passed. The full PostgreSQL lane finished 1,503/1,505, with two failures still unresolved in that run: cluster_global_sample_writer_fails_closed_when_activity_is_masked and hanging_redis_peer_times_out_before_the_health_listener_binds. The runner constructed a postgresql:// URI; the fence test requires literal postgres://, explaining its prefix panic. The Redis test exceeded its scheduling bound at 8.03 seconds; that local failure remains unexplained. No full local just ci pass is claimed. These receipts remain attributed to the historical head.

The separate security-review workflow only authorizes PRs targeting main. Do not retarget or retry this stacked PR to bypass that restriction.

Human review and verification remain required. Existing review decisions are preserved. No merge, deployment, production repair or human acceptance was performed.

Generated with Codex

[wpfleger96]

🤖 Automated review at exact head c0c22bbf808eaed1042d556867415ec813237c2a (base 425d89b4d412adce216e5f58bc985743b0da311b). Requesting changes for two blocking findings, both about how the scan behaves when the writer is slow.

IMPORTANT (correctness): a timed-out batch keeps its writer connection busy for up to two minutes. revalidate_live_authorization_sessions (state.rs:979) wraps each batch in a 1-second tokio::time::timeout, but live_authorization_state_batch (live_authorization.rs:71) runs on an ordinary pooled writer connection with no statement-level timeout. When the wrapper fires, the future is dropped and the PoolConnection goes back through sqlx-core 0.9's Floating::return_to_pool. That runs an untimed ping() (pool/connection.rs:314), which has to wait for the abandoned statement to finish first. The writer's session statement_timeout defaults to 2 minutes (buzz-db/src/runtime/mod.rs:750). So a slow writer turns each timed-out batch into a connection held for up to 120 s, and the next scan 5 s later starts four more. The documented "at most 4 queries at a time" bound covers the futures, not the database work. Under the conditions this timeout exists for, the scan can starve the serving writer pool. Fix: bound the statement on the server, e.g. run the batch in a transaction with SET LOCAL statement_timeout just under the wrapper deadline. Also make a checkout that is timed out or cancelled by the scan close its connection (with a bounded close) instead of returning it to the pool.

IMPORTANT (design): closing sessions on a failed lookup makes writer saturation worse. Once a target's last good check is more than 10 s old, the failure branch (state.rs:1141) closes every session it holds, and it does so for pool timeouts and query timeouts as well as hard outages. While the writer is down, no new ban or removal can commit. So in an outage, failing closed only protects against a revocation committed in the few seconds before the outage whose Redis fan-out was also missed. The cost is that every root and audio socket on every pod closes together. In the saturation case the loop compounds itself: scan timeouts close sockets without jitter, the clients reconnect together, and each admission now makes an extra writer read (the captured-owner membership read in final_admission_check). That puts more load on the writer that was already the problem. ARCHITECTURE.md acknowledges the reconnect herd but doesn't mitigate it. Please either keep sessions through lookup failures and apply the verdict on the next successful snapshot, or limit fail-closed to a much longer outage window and jitter the closes. Whichever you pick, state the security/availability tradeoff in ARCHITECTURE.md.

MINOR (non-blocking):

• relayAuthPolicy.ts:34 copies four relay error strings into SERVER_UNAVAILABLE_AUTH_REPLIES, with no shared constant or contract test. If a relay message is reworded, the desktop silently goes back to capped retries.
• The (None, true) invariant violation in final_admission_check (auth.rs:323) returns "internal error checking relay membership". The desktop now retries that message forever, even though here it signals a server bug rather than an outage.
• admission_db_error_class (auth.rs:236) is identical to live_authorization_db_error_class (state.rs:1218); keep one.
• The useKnownAgentPubkeys.test.mjs observer-race fix doesn't touch this feature and would be cleaner in its own PR.
• ARCHITECTURE.md:283 records workstation timings (70 ms, 129 ms) that will go stale. The batch, concurrency, and deadline bounds are the durable facts.

Checked and fine: users and community_bans are keyed by (community_id, pubkey), so batch rows are unique per target and the incomplete-result guard won't misfire. Scans can't overlap (MissedTickBehavior::Skip plus a sequential await). An admitted owner-derived session always has its captured owner equal to the stored owner, because owner_link_denial refuses admission when the link doesn't land. Every predicate is community-scoped. The test database URL edits are already on main. We ran the PR's live_revalidation E2E at this head against an isolated Postgres and Redis. Member and NIP-OA owner-linked root and audio sockets closed with 1008 access revoked about 4.5 s after database-only bans, and the clear bystander stayed usable. With the ban table made unavailable, the remaining sockets closed with authorization unavailable about 10 s later.

[wpfleger96]

🤖 Automated review at exact head fb8a7f50dae757241d45d10d18d4648bccbacc61 (base b7d7d61c7c5a19b28ba494e98d8f71301458a853). Requesting changes: the first blocker from the review at c0c22bbf is unchanged after the rebase, and the second is only partly addressed.

IMPORTANT (correctness), still present: a timed-out batch keeps its writer connection busy for up to two minutes. live_authorization_state_batch still checks out an ordinary pooled writer connection (live_authorization.rs:71) and runs the batch with no statement-level timeout (live_authorization.rs:120). revalidate_live_authorization_sessions still bounds it only with a 1-second tokio::time::timeout (state.rs:1082). That drops the future, but the statement keeps running on the server and the connection goes back through sqlx's return_to_pool ping, which waits for it. The writer session statement_timeout defaults to 2 minutes (buzz-db/src/runtime/mod.rs:750). So on a slow writer, every timed-out batch holds a writer connection for up to 120 s, and each 5-second scan adds up to four more. The "4 queries at a time" bound limits futures, not database work, and the scan can starve the serving writer pool under exactly the conditions the timeout exists for. Fix: bound the statement on the server, for example by running the batch in a transaction with SET LOCAL statement_timeout just under the 1-second wrapper. Also make a connection the scan times out or cancels close (with a bounded close) instead of returning to the pool.

IMPORTANT (design), partly addressed: closing sessions on a failed lookup still makes writer saturation worse. ARCHITECTURE.md:331-351 now states the security/availability tradeoff, which was one part of the earlier ask. Neither mitigation is in place, though. The failure branch (state.rs:1245) still closes every session on a target after the same 10-second grace, for pool and query timeouts as well as hard outages, and the closes still aren't jittered; ARCHITECTURE.md:349 says so. The desktop's jittered reconnect spreads that one client's re-admissions, but every root and audio socket on a pod still closes in the same sweep. Every other client reconnects on its own schedule, and each re-admission adds a writer read (the captured-owner read at auth.rs:420) against the writer that is already saturated. Please either keep sessions through lookup failures and apply the verdict on the next successful snapshot, or limit fail-closed to a much longer outage window and jitter the closes across it. Update the staleness budget and its compile-time assertion to match whichever you pick.

MINOR (non-blocking), all carried over from the earlier review and still present unless noted:

• relayAuthPolicy.ts:34 still copies four relay error strings into SERVER_UNAVAILABLE_AUTH_REPLIES with no shared constant or contract test. If a relay message is reworded, the desktop silently goes back to capped retries.
• The (None, true) invariant violation in final_admission_check (auth.rs:441) still returns "internal error checking relay membership". The desktop retries that message forever, but here it signals a server bug, not an outage.
• admission_db_error_class (auth.rs:353) is still identical to live_authorization_db_error_class (state.rs:1321); keep one.
• The useKnownAgentPubkeys.test.mjs observer-race fix still doesn't touch this feature and would be cleaner in its own PR.
• ARCHITECTURE.md:283 still records workstation timings (70 ms, 129 ms) that will go stale. The batch, concurrency, and deadline bounds are the durable facts.
• New: ARCHITECTURE.md:316 calls the admission captured-owner read "bounded", but is_relay_member_writer at auth.rs:420 has no deadline beyond pool acquisition and the 2-minute session statement_timeout. That's the first finding again, on the admission path. Give it the same server-side bound, or drop "bounded" from the doc.

Checked and fine: live_authorization_allowed (state.rs:1270) still scopes every predicate to the bound community and refuses a captured-owner mismatch before it considers membership. Ordering targets by their oldest last-successful check (state.rs:1054) gives a target that missed a deadline priority on the next scan without letting a newer sibling socket hide it. Token-owner admission is recorded on both the root (auth.rs:1075) and audio (audio/handler.rs:1168) enrollment paths, so NIP-FI owner-derived sessions keep the path they were admitted under when NIP-OA is disabled. The source review decided this verdict, so no E2E run was done at this head.