Skip to content

feat(relay): keep plugin record lists current with NIP-AR sync - #1523

Merged
loganj merged 6 commits into
mainfrom
larry/artifact-sync
Oct 26, 2026
Merged

loganj merged 6 commits into
mainfrom
larry/artifact-sync

Conversation

@loganj

@loganj loganj commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

🤖

Summary

  • Plugins can now keep a list of shared records up to date without polling. A plugin asks for "the current records of this type in these channels", gets the complete list once, and then gets each change as it happens.
  • Before this, a plugin could only read once and read again later. Plugins had to poll, so changes showed up late, and a list over its read limit was silently incomplete.
  • Relays that cannot do this report "unsupported", so plugins keep polling there. This needs feat(relay): NIP-AR sync subscriptions buzz#8522 on the relay; until that ships, nothing changes for users.

Details

  • New host call: session.artifacts.sync(filter) returns a view with snapshot(), subscribe() and dispose(). The status is loading, live, stale (reconnecting; the last complete list is kept), restricted, error or unsupported.
  • The filter must name its channels with #h (at most 256). Paging keys such as limit are refused at the call site, because the relay refuses them too.
  • On the wire it is one relay subscription (REQ) with "artifact": "current" on the existing live socket, sent after sign-in (NIP-AR sync, block/buzz docs/nips/NIP-AR.md). Changes that arrive before the complete list ends (EOSE) are held, then the list is replaced at once. After that the view applies each new head, deletions, records that stop matching, and move removals.
  • When the relay ends the subscription to ask for a fresh copy (resync), or the socket drops, the view restarts with backoff and jitter (1 s to 60 s) and keeps the last list as stale. restricted, invalid, unsupported, blocked and auth-required end it.
  • Support comes from the relay's NIP-11 document (artifacts.sync.version 1), in both the native transport and the browser broker. The browser broker gets a matching stream route.
  • The filter checks live in their own module with no runtime imports, so the browser broker can load them.
  • The view types are exported to plugin authors next to RelayData.

Demo

I ran a container relay built from #8522 with three demo channels. In a browser build of this branch, a demo plugin synced three record types over those channels. When I edited a record from another client, the open plugin view showed the change about 30 ms after the write returned, with no refresh. The relay saw three sync subscriptions over the three channels.

@loganj
loganj force-pushed the larry/artifact-sync branch 2 times, most recently from ec94276 to ffac8ab Compare October 23, 2026 16:11
@loganj
loganj marked this pull request as ready for review October 23, 2026 16:49
@loganj
loganj requested a review from a team as a code owner October 23, 2026 16:49
wesbillman

This comment was marked as resolved.

loganj and others added 5 commits October 26, 2026 10:57
session.artifacts.sync(filter) keeps an exact set of current artifacts for
channels named with #h. It sends one REQ with "artifact": "current" on the
live relay socket, holds changes until EOSE, then replaces the set and applies
live heads, deletions and removals. It restarts with backoff after a resync or
a lost socket, and ends on restricted or invalid. Relays that do not advertise
artifacts.sync in NIP-11 report "unsupported", so plugins can keep polling.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
- When an open POST has no known outcome (timeout, network error, 5xx),
  the browser also closes that ID on the host. A 4xx refusal allocated
  nothing and sends no close.
- When a close has no known outcome, the browser resets the stream; the
  host ends every sync with its stream.
- The host remembers a close that arrives before its open, so the late
  open gets 409 and takes no slot.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
The host forgot its oldest close-before-open record past 64, so that sync's late open could take a slot again. At 64 unresolved closes the host now ends the stream. That ends every sync on it, and a late open finds no stream.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Access loss and Clear cache purged every other retained view but left
artifact sync sets readable. Each sync view now drops its committed and
in-flight sets, fences the old sync so its late EVENT or EOSE cannot
restore them, and starts a new sync under the relay's current access.
Notifications go through the session's revocation deferral.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
A rate-limited sync CLOSED went straight to the view, so it retried in
about a second and other queued dispatches also ran into the exhausted
quota. It now pauses the shared admission first, as channel and
publication refusals do.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj
loganj force-pushed the larry/artifact-sync branch from ffac8ab to 46f34cd Compare October 26, 2026 15:07
@loganj

This comment has been minimized.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating Rocket, Mantis and Groot: both prior P2 findings are resolved; no new required changes, source assessment 9/10. Access loss/cache clear now purge sync views and fence old frames; rate-limited syncs pause shared admission before retry. Reviewed head 46f34cd against base c40dfdf, including the fix tests, transport lifecycle and public PR material. Source-only: no tests or apps executed; the hosted-check snapshot still had browser/native checks running, and live/native acceptance remains unverified.

wesbillman

This comment was marked as resolved.

- Arm the retry timer before publishing a close, so a listener that
  restarts or disposes the view cancels it instead of leaking a second
  live sync.
- End with error, keeping the last set, on the relay's max_events cap.
- Retry auth-required and restricted: authorization denied, which end
  with the session, not with access.
- Validate the relay's predicate limits at the call site: 32 predicates,
  256 values and 64 KiB across all of them, and canonical channel UUIDs
  in #h.
- Share one open-sync capacity (32) between the session and the
  browser broker.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj

This comment has been minimized.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating Rocket, Mantis and Groot: the prior required fixes are addressed; no new changes requested, source assessment 9/10. Retry reentrancy, max-events termination, shared capacity and aggregate filter validation now match the reviewed contracts. Reviewed head 5b215a6 against base c40dfdf, including the follow-up tests and public PR material. Source-only: no tests or apps executed; the hosted-check snapshot still had browser, native and JavaScript jobs running, so runtime acceptance remains unverified.

@loganj
loganj merged commit 4acc21a into main Oct 26, 2026
37 checks passed
@loganj
loganj deleted the larry/artifact-sync branch October 26, 2026 19:54
baxen added a commit that referenced this pull request Oct 26, 2026
…views

* origin/main: (34 commits)
  chore(hooks): use stock lefthook 2.2.2 instead of patched build (#1644)
  perf(messages): encode video opening frames without blocking (#1658)
  perf(relay): cut per-notification work on the profile path (#1657)
  add feedback for chat attachment downloads (#1642)
  feat(buzz-mcp): guard memory overwrites, search memory, report core size (#1637)
  feat(relay): keep plugin record lists current with NIP-AR sync (#1523)
  feat(plugins): add pi catalog artwork (#1572)
  fix(avatars): pause packed avatars for live reduced motion and covered panes (#1652)
  fix(panels): bound the shell grid row during input (#1651)
  feat(attention): support agent-provided commands as sources (#1630)
  feat(agents): add self_handoff so an agent can hand off to another of its conversations (#1589)
  fix(pi): fail a mention queued behind a left plugin wake that fails (#1646)
  perf(threads): re-fold thread views only for their own changes (#1654)
  perf(threads): keep hidden thread tabs memoized and size thread budgets incrementally (#1619)
  feat(agents): add plugin Attention sources (#1553)
  feat(avatars): resize and reposition emoji avatars (#1582)
  fix(relay): replace a live socket that stops answering (#1629)
  Leave an archived or deleted Solo chat for its sidebar neighbor (#1648)
  perf(browser): page through first-seen history (#1276)
  Fix false date dividers after collapsed thread replies (#1454)
  ...

# Conflicts:
#	src/features/conversation/Conversation.tsx
dr0uleau added a commit that referenced this pull request Nov 4, 2026
## Why

Saved task records need updates after restart without a full download on
each sync.

## What

Keep each local collection current through the relay's incremental
reads. Resume from the last saved checkpoint with a 120-second overlap.

## How

One window owns each collection. It saves paged changes and current-ID
reads in order. Explicit removals preserve saved bodies and preferences.
Channel removal commits at most 100 saved records at a time. Failed
pages cannot advance progress. Access recovery revalidates saved IDs.

Incremental reads use `artifacts.incremental` and `artifact:
"incremental"`. The existing WebSocket service keeps `artifacts.sync`
for a complete set followed by live updates. The two capabilities work
independently. Long retries wait 60–90 seconds so owners do not all
retry at the same fixed interval.

## Risk

The overlap assumes bounded database delay. Incremental reads cover
permanent channels. Native end-to-end sync needs the compatible relay
and client together.

## Testing

At `09ee7d13d`, native macOS tests use the real Buzz relay. A host
collection saves bootstrap data, resumes from its checkpoint after app
restart, and reads a later update after a live hint. Channel removal
returns an explicit `remove`; the saved body and pinned preference
remain, with unavailable status.

The relay rejects an invalid initial checkpoint with HTTP 400. The
client makes one fresh pass. An injected failure of that fresh pass
preserves the prior checkpoint, body, and preference.

A real policy change returns HTTP 409 with `policy_changed`. Recovery
loads 101 records in pages of 100 and one, then rechecks saved IDs in
batches of 100 and one. The two delta reads take 156 ms and 50 ms. All
101 records remain saved, with a checkpoint and no refresh error. This
verifies the page boundary; large-scale performance remains unmeasured.

Human acceptance passes in the native test harness: Publish test update
reaches the saved collection and preserves the pinned preference. The
operator supplies the PASS screenshot. The harness uses the real host
service and signed reader. It does not establish adoption by a
production plugin.

Chromium and WebKit also pass the two ownership-transfer cases with real
IndexedDB and Web Locks. Command: `bin/pnpm exec playwright test
--config tests/browser/playwright.config.mjs
tests/browser/artifact-sync.spec.mjs --project chromium --project webkit
--no-deps`.

buzz-review-completed

## Bigger picture

The storage prerequisite, [PR
1501](#1501), is merged into
`main`. The relay prerequisite is [PR
8523](block/buzz#8523). The client preserves the
WebSocket service from [PR
1523](#1523).

Generated with Codex

---------

Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants