[TheSentinel454]

Stack (merge bottom-up): main ← #8325 ← #8326 (0087) ← #8323 (0088) ← #8211 (0089). Migrations are pre-numbered in stack order, so each PR's migration.rs count assumes the ones below it.

Important

Hold: do not merge until #8128 is deployed in every environment. Migration 0089 removes the database-side enforcement of rules that #8128 moved into the application. If a relay without #8128 runs against a database migrated past 0089, it loses the NIP-RS watermark, the superseded-row purge and the mention-liveness guard.

Rollout order: apply 0089 before the new binary serves traffic. This binary assumes 0089 has run. Against a migration-built database not yet at 0089, every NIP-RS hard delete hits 0011's BEFORE DELETE guard (check_violation). That affects replacement, NIP-09 and community deletion. It fails closed, so no data is lost. With autoMigrate=true (the Helm default), a failed migration stops startup, so this can't happen. Where BUZZ_AUTO_MIGRATE=false (the compose default, and the cutover runbook), run buzz-admin migrate before rolling out.

After rollout: run a one-off cleanup. A rolling update (maxSurge: 1, maxUnavailable: 0) applies 0089 while #8128 pods are still serving. Those pods classify a replacement by the incoming event, so replacing a conforming read-state or mesh-status head with a nonconforming event soft-deletes the old head, and with the purge triggers gone nothing removes it later. Only the coordinate's owner can cause this, and shipping clients publish the conforming shape, so expect at most a few hidden rows. Once every relay runs this binary, run this once per database. It is idempotent and only touches soft-deleted rows that match retention_free_event_predicate!:

Cleanup SQL

-- One-off: purge soft-deleted retention-free rows (NIP-RS read state, mesh status)
-- and their mentions. Predicate = retention_free_event_predicate! in store/event_deletion.rs.
BEGIN;
WITH purged AS (
  DELETE FROM events
  WHERE deleted_at IS NOT NULL
    AND kind IN (30078, 30003)
    AND (
      (kind = 30078
       AND d_tag ~ '^read-state:[0-9a-f]{32}$'
       AND (SELECT count(*) FROM jsonb_array_elements(CASE WHEN jsonb_typeof(tags) = 'array' THEN tags ELSE '[]'::jsonb END) tag
            WHERE jsonb_typeof(tag) = 'array' AND tag->0 = '"d"'::jsonb) = 1
       AND EXISTS (SELECT 1 FROM jsonb_array_elements(CASE WHEN jsonb_typeof(tags) = 'array' THEN tags ELSE '[]'::jsonb END) tag
            WHERE jsonb_typeof(tag) = 'array' AND jsonb_array_length(tag) >= 2
              AND jsonb_typeof(tag->1) = 'string' AND tag->>0 = 'd' AND tag->>1 = d_tag)
       AND (SELECT count(*) FROM jsonb_array_elements(CASE WHEN jsonb_typeof(tags) = 'array' THEN tags ELSE '[]'::jsonb END) tag
            WHERE tag = '["t", "read-state"]'::jsonb) = 1)
      OR
      (kind = 30003
       AND d_tag LIKE 'buzz-mesh-member-status:%'
       AND tags @> '[["k", "buzz-mesh-status"]]'::jsonb)
    )
  RETURNING community_id, id
), purged_mentions AS (
  DELETE FROM event_mentions m USING purged p
  WHERE m.community_id = p.community_id AND m.event_id = p.id
  RETURNING 1
)
SELECT (SELECT count(*) FROM purged) AS events_purged,
       (SELECT count(*) FROM purged_mentions) AS mentions_purged;
COMMIT;

Note

Migration numbering. 0089, on top of #8326 (0087) and #8323 (0088). The migrations are independent DROPs.

What

Migration 0089_drop_nip_rs_mesh_migration_triggers.sql drops five triggers and five functions. Migrations 0009, 0010, 0011 and 0019 installed them, but only on migration-built databases. schema/ never declared them.

Trigger	Function
trg_events_nip_rs_watermark	guard_nip_rs_watermark()
trg_events_guard_nip_rs_hard_delete	guard_nip_rs_hard_delete()
trg_events_purge_soft_deleted_nip_rs	purge_soft_deleted_nip_rs()
trg_events_purge_soft_deleted_buzz_mesh_status	purge_soft_deleted_buzz_mesh_status()
trg_event_mentions_require_live_event	guard_event_mention_live()

Since #8128, the application owns each of these rules inside the event transaction:

• It advances the watermark and rejects stale NIP-RS writes.
• It physically deletes superseded and NIP-09-deleted retention-free rows, along with their mentions.
• It indexes mentions in the same transaction as the event.

The buzz.nip_rs_hard_delete opt-in existed only to get past trg_events_guard_nip_rs_hard_delete. It is removed from community delete (deletion.rs), retention purge (event.rs) and the replace path (replaceable.rs).

One removal rule. The replace path used to decide "retention-free" with its own Rust code, which was stricter than the SQL predicates used by the delete and purge paths, and it classified the incoming event rather than the rows it removed. store/event_deletion.rs now defines the predicates once (event.rs re-exports them) (nip_rs_event_predicate! and mesh_status_event_predicate!, combined into retention_free_event_predicate!), and every removal path decides row by row from each row's stored shape:

• remove_coordinate_versions_in_tx() purges retention-free versions (with their mentions) and soft-deletes the rest. NIP-09 coordinate deletion and replacement both call it.
• The replace path only asks Postgres one question about the incoming event: is_nip_rs_event(), for the watermark.

Before this change, replacing a conforming read-state head with a nonconforming event left a soft-deleted row that only the dropped triggers would have purged. Replacing a nonconforming head with a conforming event hard-deleted history that should have been kept (this was already broken before the PR). Both now follow the row's own shape.

0089 sets SET LOCAL lock_timeout = '5s' before its DROPs, because DROP TRIGGER takes ACCESS EXCLUSIVE on events and its partitions.

Tests

• Removed: the opt-in and fence tests, and migration_schema_database_guard_covers_legacy_writer_and_nip09_deletion (about 310 lines). They tested only the dropped triggers.
• Renamed: mesh_status_replacement_keeps_one_physical_row now checks only the application behavior. Its trigger half is gone.
• NIP-09 classification table: still runs on both schemas. It now also asserts that no purge triggers exist, and runs every shape through replacement as well as both delete paths.
• mixed_shape_replacement_classifies_each_superseded_row: conforming then nonconforming purges the old head and its mention; nonconforming then conforming soft-deletes the old head and drops its mention. Main's fix(db): drop a superseded head's mention rows with it #8948 removes a superseded head's mention rows and CoordinateRemoval::Supersede does the same; only NIP-09 deletion keeps a soft-deleted row's mentions. Behavior change vs. this PR's earlier heads: a superseded nonconforming head no longer keeps its mention.
• NIP-RS replay test: deletes through soft_delete_by_coordinate instead of a raw UPDATE, then checks that neither the old head nor the deleted head can be replayed back.
• admin_schema_parity_between_desired_state_and_migrations: now runs every migration. It then requires the events and event_mentions row triggers to match schema/ exactly: name, function, enable mode and tgtype. Any trigger that exists only in migrations now fails this test.
• migration.rs: asserts len() == 89 and migrations[88].version == 89, keeps feat(buzz-db): retire the channel-TTL trigger #8210's 0065 checks, and checks all 10 DROP statements and that lock_timeout is set before the first one. The parity probe applies the remaining migrations through run_migrations.

What got simpler

The code is about 360 lines shorter (388 added, 746 removed, most of the additions are tests). Migration-built and desired-state databases now run the same event write path. The session GUC opt-in is gone. Replacement and NIP-09 coordinate deletion share one removal primitive, and the explicit mention DELETE and the two-field RetentionFreeClass are gone.

Verification

Head e9f3f8fcc, restacked onto #8323. CI and the Blox gate are running at this head; results replace this line when they land.

The previous evidence, at dddf67a7b on base e9269cbdf, is superseded by the rebase and kept here only for history: buzz-db PG 482/482 and buzz-relay PG 378/378, CI PG lane 913/913, and a migration-built E2E (49/49 live cases, the lock-timeout 55P03 probe and an incoming-shape mutation that failed 7 cases).

[wpfleger96]

🤖 Thanks, Luke. Nothing here blocks.

The replace path now removes old versions through remove_coordinate_versions_in_tx(), the same helper NIP-09 coordinate deletion uses, so each superseded row is classified by its own stored shape instead of by the incoming event. ID deletion uses the same SQL predicate, the purge deletes each removed row's mentions scoped by community, and the removal, insert, watermark and mention indexing all sit under the one savepoint, under the coordinate advisory lock. is_nip_rs_event() reuses nip_rs_event_predicate! for the watermark, so the Rust and SQL classifications can't drift apart again.

Verified at this head on an isolated, migration-built PostgreSQL stack:

• The real 0057 → 0058 upgrade removes all 37 trigger instances, partition clones included, and all five functions. With a conflicting lock held, 0058 fails at the 5s lock_timeout and leaves the ledger at 57 with every trigger in place; the retry succeeds.
• Signed events through a relay built from this tree: watermark advance, stale and exact replay (including a fresh stale event after a relay restart with Redis cleared), both mixed-shape replacement directions, and NIP-09 deletion by ID and by coordinate. SQL shows read-state and mesh-status rows physically gone with their mentions, and nonconforming look-alikes soft-deleted with mentions kept.
• Putting back classification by the incoming event turns mixed_shape_replacement_classifies_each_superseded_row red: (rows, live, mentions) = (1, 0, 1) against the required (0, 0, 0).
• The new and changed tests and admin_schema_parity_between_desired_state_and_migrations pass by name in the PostgreSQL job (913/913).

Optional

• The rollout note says relays with #8128 are safe against a database past 0058, but there's one case where they aren't. The chart rolls with maxSurge: 1 / maxUnavailable: 0 (deployment.yaml L13–L16), so the first new pod runs 0058 while #8128 pods are still serving. Those pods still classify by the incoming event (replaceable.rs L151–L167, L277–L282). If one of them replaces a conforming read-state head with a nonconforming event, it soft-deletes the old head, and with the purge trigger gone nothing removes it later, because the new purge only selects live rows (event.rs L1288). The old exact-two-element mesh check has the same gap for mesh-status tags with extra elements. Only the coordinate's owner can trigger this and the shipping clients all publish the conforming shape, so the realistic worst case is a hidden leftover row or two from a few minutes of rollout. Either drain the old pods before 0058 runs, or add a one-off cleanup to run after rollout (delete soft-deleted rows matching retention_free_event_predicate! plus their mentions), and say which in the rollout note.

[TheSentinel454]

Thanks, Will. Good catch: the rolling update does leave that window, and the new purge never revisits soft-deleted rows. We went with the post-rollout cleanup rather than draining, since draining means downtime or a Recreate rollout to avoid a handful of invisible rows. The hold in the description now has an "After rollout" step with an idempotent one-off SQL statement. It purges soft-deleted rows matching retention_free_event_predicate! and their community-scoped mentions, and it covers the mesh-status extra-element case too. Run it once per database after every relay is on this binary.

[github-actions]

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

• Exact PR diff: e2dc54e22fc113f298889d76c3acf674cb46db4b...b68b60f2bc061bcdb07acc5bbf01ae4010b1162e
• Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.

---

Review Summary

Overall Risk: MEDIUM

The updated PostgreSQL tests contain incorrect expectations that will fail CI. No concrete production security regression was identified.

Findings

[MEDIUM] Account for mention removal in replacement test expectations

• Category: Reliability
• Location: crates/buzz-db/src/store/replaceable.rs:2172 (source)
• Description: Adding Delete::Replace reuses expected(false), which expects one retained mention. However, replacement calls remove_coordinate_versions_in_tx with Supersede, whose superseded_mentions CTE deletes that mention. The nip-rs without t tag case therefore produces (1, 0, 0), while the assertion expects (1, 0, 1). Both schema variants fail. The new mixed_shape_replacement_classifies_each_superseded_row test contains the same incorrect expectation.
• Impact: These tests are automatically selected by the PostgreSQL CI lane and fail even when replacement behaves correctly.
• Recommendation: Make the expected mention count depend on the operation: zero for replacement and one for ordinary soft deletion. Also change the nonconforming mixed-shape replacement expectation to (1, 0, 0).

Notes

• Review used read-only source and history inspection. Builds and tests were not executed, as requested.

---

_{Generated by Codex Security Review |
Requested by: @TheSentinel454 |
Workflow run}

[wpfleger96]

🤖 Thanks, Luke. Still nothing blocking after the rebase.

Since the last review the only code change is the 0058 → 0066 renumber. A merge with current main (60de1db0b, which adds #8318 and #8322 on top of this base) is clean, and nothing in crates/ still sets buzz.nip_rs_hard_delete. Every production path that removes read-state or mesh-status rows still goes through remove_coordinate_versions_in_tx() or retention_free_event_predicate!, and 0059–0065 add no events or event_mentions trigger that the stricter parity check would trip on.

Verified at this head on an isolated, migration-built PostgreSQL stack:

• The real 0065 → 0066 upgrade removes all 37 trigger instances (32 of them partition clones) and all five functions. The ledger ends at 66 with every checksum matching this head.
• The cleanup SQL from the description, run verbatim, purged the two soft-deleted conforming rows (one read-state, one mesh status) and their mentions. Malformed look-alikes, live conforming rows and same-ID live copies in another community were left alone, and a second run returned 0 / 0. Its predicate matches both retention_free_event_predicate! expansions exactly. The kind IN (30078, 30003) prefilter is redundant but harmless.
• admin_schema_parity_between_desired_state_and_migrations, mixed_shape_replacement_classifies_each_superseded_row and both schema cases of nip09_deletion_purges_retention_free_rows_and_mentions pass by name in the PostgreSQL job.

Optional

• The cleanup DELETE fires the community write fence on events. That fence rejects anything other than READ COMMITTED (L17–L19), and it aborts the whole statement if any matching row belongs to a community that isn't active (L73–L75). Nothing gets partially purged, so it's safe, but the note could say it needs the default isolation level and should be rerun once an in-flight community deletion finishes.
• The Verification section still describes head 28e0aae87 with re-verification in progress.

[wesbillman]

🤖 Star Lord’s automated source review via Wes’s account.

No actionable findings in head 353b477d94b663443ec1a0d2157136220745fb66 against base/merge-base c0ab789499bde118a468178d23b726fd109a8e1c.

The shared removal path classifies each stored row, preserves ordinary history through soft deletion, and purges retention-free rows with community-scoped mentions. Replacement retains its coordinate lock and savepoint around removal, insertion, watermark updates and mention indexing. The mixed-shape, replay and dual-schema tests cover the intended contracts in source. The public description was also inspected; it contains no attached images.

The documented rollout hold remains: deploy #8128 everywhere, apply migration 0066 before this binary serves traffic, then perform the stated post-rollout cleanup. This review does not establish those prerequisites have been met. Existing optional documentation notes in the previous review are not repeated as new findings.

Validation: independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/retention/testing) source reviews plus Star Lord integration. No builds, tests, migrations, cleanup SQL or live deployment were run. The read-only check snapshot showed 57 successful and 34 skipped checks; GitHub currently reports merge conflicts. Runtime locking, performance and rollout behavior remain unverified by this review. This is a non-blocking COMMENT, not approval or merge/deploy authorization.

[loganj]

🤖 Larry and Auto: an independent adversarial review of b68b60f2bc. Source review only, plus local Postgres 17 runs and mutation tests. Approving. The code is right, but four tests at this head need updating before the Postgres lane can pass. They are the same four that fail in the PostgreSQL job (1483/1487), and all four pass on main e2dc54e22f. Each one broke when the branch was rebased onto newer main.

Must-fix (error): three tests still expect a superseded row to keep its mention.

• Failing: mixed_shape_replacement_classifies_each_superseded_row (replaceable.rs:1906), and both schema variants of nip09_deletion_purges_retention_free_rows_and_mentions (:2212, Replace mode).
• #8948 changed main so that a superseded head loses its mention rows (main's superseded_heads_drop_their_mention_rows pins this). The rebase routes replacement through CoordinateRemoval::Supersede, which does the same, so the code is right. These tests still expect (1, 0, 1) for a non-retention-free superseded row and get (1, 0, 0).
• Making Supersede keep mentions turns all three green and turns superseded_heads_drop_their_mention_rows red, which confirms the tests are stale.
• Fix (tests only): see the inline comment at :1844. Also, in the expected helper at :2163-2167, expect (1, 0, 0) for a non-purged row in Delete::Replace mode and keep (1, 0, 1) for Coordinate and Id.

Must-fix (error): media_edit_index_upgrade_builds_inline_or_verifies_prebuild now times out at 0087.

• runtime/migration.rs:4200: in the prebuilt case, the test holds LOCK TABLE public.events IN ROW EXCLUSIVE MODE and calls run_migrations, which now reaches 0087. Its DROP TRIGGER needs ACCESS EXCLUSIVE, so it fails with 55P03 at version 87. It fails the same way when run alone and in CI, so it is not load-related.
• Fix: run that step only through 86 (run_migrations_through(&pool, 86)) while the writer lock is held, or roll back the writer before applying the rest.

What we checked and found sound

• Watermark: the only writer that can store a conforming kind 30078 event is the parameterized replacement, which advances the watermark and rejects stale writes under the coordinate advisory lock. Ingest sends every kind 30000–39999 to replace_parameterized_event. We found no other production insert path for kind 30078; we searched INSERT INTO events and the insert_event* callers in buzz-db and buzz-relay.
• Purges: only three production soft-delete paths can reach kinds 30078 and 30003: parameterized supersede, NIP-09 by id and NIP-09 by coordinate. All three now purge in the app, classifying each row by its stored shape. Every other production UPDATE events SET deleted_at is limited to other kinds.
• Mention-live guard: mentions of a 30078 event are written in the event's own transaction.
• Supersede bound: the new created_at <= incoming bound removes the same rows as before, because the advisory lock and the dominance check already ensure every live version is at or before the incoming event.
• Rebase: compared with 353b477d94, the only differences come from main's superseded_with_mentions_sql!, and Supersede keeps that behaviour.
• Mutations: each one fails at least one test. Supersede skipping the purge fails 4 tests. Supersede keeping mentions fails superseded_heads_drop_their_mention_rows. Making is_nip_rs_event always false fails the NIP-RS replay test. Keeping the mention-live trigger or the watermark trigger in 0087 fails the new trigger-parity check.
• Rollout: the hold, the migrate-before-binary order, and the cleanup that runs after rollout are already discussed above, so we don't repeat them.

The other local failures are not from this PR. The event_follow_up_contract_* tests fail the same way on main in our local setup and pass in CI.

[loganj]

🤖 error: since #8948, a superseded head loses its mention rows, and CoordinateRemoval::Supersede follows that rule. So for "nonconforming then conforming", the old row is soft-deleted without its mention. This should be (1, 0, 0); at this head the test fails at :1906 with (1, 0, 0) vs (1, 0, 1). The doc comment's "keeps legacy retention" should also say that the mention goes. The expected helper at :2163-2167 needs the same change for Delete::Replace only. Coordinate and id deletion still keep mentions.

[loganj]

🤖 advice (optional): this adds one database round trip to every kind 30078 parameterized write. That covers all NIP-78 app data, and read state is high volume. For 30078/30003, supersede also now runs two statements (purge, then UPDATE) where it used to run one. Keeping one SQL definition is a good reason for this. The cost could come back by evaluating the predicate over $1/$2/$3 in the existing head or watermark SELECT, which is the same predicate in the same round trip.

[loganj]

🤖 convention (optional): the predicates and remove_coordinate_versions_in_tx live in store/event_deletion.rs; event.rs only re-exports them. The PR body is also stale: it still says 0066, "main ships through 0065", and that #8323/#8326 use 0061/0062. Its predicate location points at event.rs too. Separately, the cleanup SQL is one unbounded DELETE across every community on partitioned events. The docs/database.md checklist asks for repairs as a batched scripts/maintenance/ script that takes the community deletion lock.