Skip to content

Tell the client when a command's change is on the screen - #2312

Open
marvin-bitterlich wants to merge 1 commit into
mainfrom
ccr-efcafb6c-sxkls9-command-reflected
Open

marvin-bitterlich wants to merge 1 commit into
mainfrom
ccr-efcafb6c-sxkls9-command-reflected

Conversation

@marvin-bitterlich

Copy link
Copy Markdown
Contributor

Why?

A command's reply reaches the client before the live queries it changed are updated. Nothing tells the client when those updates are done, or that none are coming, so a screen cannot say "saved and visible" and a test cannot wait for it without guessing. This is the mechanism half of marginalia-settle-command-pushes; the React hook, the settle() verb and the removal of withLiveAnswer follow in a second pull request.

How?

  1. The client gives each connection a random id and each command a counter. The pair travels in the x-shivaedev-sync header: <client> on $changes, <client>:<n> on a command.
  2. After the handler finishes, publishCommand in server hands the token to CommandPublication.publish(keys, sync) together with the command's remaining keys. A failed handler sends nothing.
  3. With PostgreSQL, sql sends the token through pg_notify on the change channel, in the same transaction as the keys and after them, or alone when the command published nothing. Inside an enclosing transaction it waits for that commit. Every host's listener reads it in channel order and calls synced.
  4. Without PostgreSQL the default publisher invalidates the keys and then hands the token to the SyncDelivery reference, which the host points at its live server.
  5. LiveServer.synced(token) queues the token in each subscription of the client it names, through the same hint queue as keys. A host that holds no stream of that client drops it.
  6. Each subscription finishes the work ahead of the token and then writes Synced{ids} to its own stream. A subscription that names its client also writes Syncing at the start of each full re-read, so the client can tell a host that confirms from one that never will.
  7. The client folds the frames into one signal per command: sent, accepted, reflected. untilReflected(command) completes at reflected, CommandSignal receives each state, client.sync waits the same way without writing, and client activity counts unreflectedCommands, which isClientSettled now waits for.

Per package:

  • changes: SyncToken, SyncDelivery, CommandPublication.publish(keys, sync?).
  • sql: token encoding in the NOTIFY payload, listener dispatch to PostgresChangeSync, the synced option of postgresChanges.
  • contract: the header, LiveServer.synced and register/stream taking the client name, the Syncing and Synced frames, the client's reflection scope, commandSignal.ts, LiveSync, MadeClient.sync, unreflectedCommands.
  • server: publishCommand sends the token, hostChanges names the client, appDatabase wires synced, liveSyncLayer serves the sync command.

Decisions

  • Per-stream frames, folded on the client. There is no per-client "done" frame. A command is reflected when every stream it waits on has confirmed, in whatever order the frames arrive, so correctness never depends on frame order across streams.
  • One client rule for every stream. At accept the command waits on each stream of its connection that is open and not known to be unsupported. A stream releases the wait with the command's own id or with any token issued after the reply, when it detaches, or when its first data frame arrives without a Syncing before it. A stream that was armed after the command's id was issued cannot have seen the token, so the client sends one catch-up sync for it; if that request fails the wait is abandoned rather than held.
  • Reconnect is a page reload. Reopened streams count as armed late and are covered by the catch-up sync, so a waiting command is reflected once the reopened query has its first result.
  • Paused live queries hold the wait. Sync frames are ignored while paused; resuming reopens the streams and the catch-up sync releases the command.
  • Listener recovery releases waits. A resync re-reads every subscription and begins with Syncing; the client's catch-up token rides behind it.
  • Older host, older client. A host that never sends Syncing is treated as not confirming, and its clients' commands are reflected at accept. A client that sends no header receives exactly the frames it received before. The token payload is a JSON object, which older listeners already ignore.
  • Guessable ids. A host acts only on tokens whose client it holds a stream for, and a stream confirms only its own client's tokens. A forged id can at worst end one client's wait early; other viewers receive nothing extra.
  • Token-only hints skip the session and policy recheck, because they read nothing.
  • The id rides on the command only over the default shared socket. A custom protocol uses the catch-up sync after the reply, since the host that took the command may not hold the streams.
  • postgresChanges requires synced rather than defaulting it to a no-op, so a host cannot silently drop tokens. This is the one breaking change a hand-wired host has to act on; appDatabase and appHost need no change.

Callouts

  • Not covered, and documented as such: work a command starts for later, such as a job. reflected covers what the handler published before it returned.
  • Rules without an end-to-end story: a real listener outage against PostgreSQL. The contract story takes the token away at the publication port and calls resync; sql's own stories prove that a recovered listener resyncs. Nothing here runs in a real browser, and the PostgreSQL kit uses two hosts.
  • Breaking entries are in the changeset: postgresChanges needs synced; isClientSettled waits for unreflectedCommands; exhaustive QueryUpdate matches gain two cases; CommandBoundary gets a third argument; clients declare platform.liveSync.request.
  • React specs that record raw socket deliveries now expect the Syncing frame; no React source changed. usePendingCommand still ends at the reply until the follow-up.
  • In the cloud sandbox packages/test-story/src/server/proxyForwarding.spec.ts fails at listen ::1 (no IPv6 loopback) and packages/server/src/databaseConfiguration.spec.ts skips without TEST_DATABASE_URL; neither touches this change, and CI covers both.

Generated by Claude Code

A command's reply reaches the client before the live queries it changed
are updated, and nothing told the client when the updates were done or
that none were coming.

The client now attaches an id to each command. Once the handler has
finished, the host sends a sync token with that id behind the command's
keys: through NOTIFY on the change channel with PostgreSQL, straight into
the live server without it. The host that holds the client's streams
queues the token in each of them through the hint path keys use, and each
stream confirms it in a frame of its own once it has finished the work
ahead of the token. The client folds the frames into one signal per
command with the states sent, accepted and reflected, and a standalone
sync waits the same way for writes the client did not make.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjCSiB3Dk7BRBmjic6wFM5
@tally-code-quality

Copy link
Copy Markdown

Coverage 92.0% (±0.0 pts) · Patch 97.8%

6 of 275 added lines are not covered, in 6 files.

Uncovered added lines

  • packages/changes/src/commandPublication.ts · 1 of 3 uncovered · L19
  • packages/changes/src/syncToken.ts · 1 of 3 uncovered · L10
  • packages/contract/src/clientOperations.ts · 1 of 15 uncovered · L104
  • packages/contract/src/reflection.ts · 1 of 54 uncovered · L71
  • packages/contract/src/rpcClient.ts · 1 of 8 uncovered · L81
  • packages/sql/src/postgresChanges.ts · 1 of 16 uncovered · L74

33 more changed files are fully covered.

Coverage map of 39 changed files
---
config:
  treemap:
    showValues: false
---
treemap-beta
"packages/contract/src"
    "reflection.ts 53/54": 54:::some
    "reflectCommand.ts 20/20": 20:::all
    "commandSignal.ts 17/17": 17:::all
    "reflectedStream.ts 17/17": 17:::all
    "adapters/openLiveHints.ts 15/15": 15:::all
    "clientOperations.ts 14/15": 15:::some
    "queryDelivery.ts 15/15": 15:::all
    "automaticLive.ts 13/13": 13:::all
    "rpcClient.ts 7/8": 8:::some
    "syncHeader.ts 8/8": 8:::all
    "automaticQuery.ts 5/5": 5:::all
    "connectionProtocol.ts 5/5": 5:::all
    "live-server/subscribe.ts 5/5": 5:::all
    "liveHintQueue.ts 5/5": 5:::all
    "live-server/synced.ts 4/4": 4:::all
    "live-server/syncFrames.ts 4/4": 4:::all
    "bind.ts 3/3": 3:::all
    "catchUp.ts 3/3": 3:::all
    "CommandGuard.ts 3/3": 3:::all
    "liveRetry.ts 3/3": 3:::all
    "LiveSync.ts 3/3": 3:::all
    "adapters/liveHintStream.ts 2/2": 2:::all
    "adapters/standaloneLiveServer.ts 2/2": 2:::all
    "live-server/initialize.ts 2/2": 2:::all
    "live-server/register.ts 2/2": 2:::all
    "live-server/stream.ts 2/2": 2:::all
    "liveSyncRequest.ts 2/2": 2:::all
    "queryUpdate.ts 2/2": 2:::all
    "clientActivities.ts 1/1": 1:::all
    "clientActivity.ts 1/1": 1:::all
"packages/sql/src"
    "postgresChanges.ts 15/16": 16:::some
    "notifyPayload.ts 4/4": 4:::all
    "listenForChanges.ts 3/3": 3:::all
"packages/changes/src"
    "commandPublication.ts 2/3": 3:::some
    "syncToken.ts 2/3": 3:::some
"packages/server/src"
    "publishCommand.ts 2/2": 2:::all
    "hostChanges.ts 1/1": 1:::all
    "hostRpc.ts 1/1": 1:::all
    "liveSyncLayer.ts 1/1": 1:::all
classDef all fill:#1a7f37,color:#ffffff,stroke:#ffffff,fill-opacity:1
classDef some fill:#9a6700,color:#ffffff,stroke:#ffffff,fill-opacity:1
classDef few fill:#cf222e,color:#ffffff,stroke:#ffffff,fill-opacity:1
Loading

Each box is a changed file, sized by its added lines: 🟩 all covered · 🟨 half or more · 🟥 under half.

Components · @shivaedev/changes −1.1 pts · @shivaedev/contract +0.1 pts · 1 more
Component Coverage vs main
@shivaedev/changes 97.0% −1.1 pts
@shivaedev/contract 98.1% +0.1 pts
@shivaedev/sql 93.7% −0.1 pts

b67d951 (merged onto 64d4061) vs main at 64d4061 · updated Nov 18, 16:28 UTC · Full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants