Repository navigation
fix: recover E2EE opens that outrun the ledger and cut cold dispatch reads - #2762
Merged
Merged
Conversation
…reads Streams can deliver a frame from a rotated epoch or a newly admitted device before the slower Convex ledger feed. Desktop and portable hosts now classify such frames (at most four epochs ahead, or an unknown author at the current epoch, for an active member device) as retryable key-pending instead of a terminal invalid-ledger; frames further ahead keep their original failure. The shared content adapter's existing refresh+retry then catches up, and its receptions are spaced to one per 3 s so unverifiable frames cannot drive a refresh per frame. Machine sharing: concurrent cold latest() reads of one binding share a single authenticated read (never across an invalidation); readHead may inline the active original, verified like readVersion and never persisted in the pin; desktop and CLI transports ask for it and fall back once per client if the backend refuses the field. Opening a workspace warms other members' machine-sharing grants in the background. Model: claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Model: claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zxch3n
marked this pull request as ready for review
November 14, 2026 06:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Problem / pressure
skew-convex-slow(P0): when the Convex ledger feed is slower than Streams (e.g. 900 ms vs 20 ms), a frame from a rotatedepoch + 1, or from a newly admitted device at the current epoch, reachesopenContentbefore the local verified ledger has that record. Core authentication fails withContentError('unauthorized'), which both hosts surfaced as terminale2ee_invalid-ledger(amissing-read-keyat a future epoch was also refused bykeyPending). The room then never recovers.prepareDispatch(P1): a cold machine-sharinglatest()does ledger refresh →readHead→readVersionin series (~1.2 s at 0 ms RTT, >10 s at 5 s RTT), and concurrent cold dispatches each repeat it. Warm (15-minute RAM grant) is 18–37 ms.Summary
Ledger lag on content opens
e2ee-runtime.mts) and portable (e2ee-clientsession/runtime)openContentnow also routeunauthorizedthroughkeyPending. For an active member device, a frame is retryablekey-pendingwhen its unverified epoch is at most 4 ahead of the verified epoch, or at the current epoch with an unknown author. Frames further ahead (forged999999), older unknown authors, malformed frames and access loss keep their original failure; no network is touched by the classification.desktop-e2ee-content.ts) already turnskey-pendinginto one sharedrefresh({ installKeys: true, keyMode: 'receive' })(ledger history, then this device's key) plus one retry, so the open returns plaintext transparently once the ledger catches up. Receptions are now spaced byKEY_PENDING_REFRESH_INTERVAL_MS = 3000per content instance: waiting reads join the next reception (still inside the 30 s wait), so a peer streaming unverifiable frames costs at most ~0.33 refresh/s per workspace. Refresh runs outsideopenContent, so no host lock or semaphore is held across it.@loro-dev/streams-crdt(external, 0.20.0) projects any provideropenfailure aspayload_protection_error,retryable: false, then retries the live read with a ≥1 s floor untilmaxAttemptsand disconnects. Resolving lag inside one adapteropenkeeps that path from being reached in the normal skew case; the adapter creates a fresh deadline per reception and never reuses an aborted signal.Cold dispatch reads (
e2ee-machine-sharing)latest(force = false)reads of one binding share one forked authenticated read; a read started before an invalidation is never joined after it, and forced/denial refreshes are unchanged.readHeadreplies may carry an optional inlinedversion(cloud-apirequestincludeVersion?: true, reply type and zod schema extended). It is used only for the exact head revision/digest, verified exactly like areadVersionreply, and stripped before the head is returned or pinned.includeVersionthroughmakeInlineVersionReadHead; if the backend's validator refuses the field (ArgumentValidationError … includeVersion), the call is repeated once without it and that client instance stops asking. Other transports (private Web) are untouched since core never adds the field itself. No duplicate refresh existed: desktop passes a no-opoptions.refreshLedgerand refreshes via the remote; CLI refreshes via options only.open).Docs: binding rules added to
packages/e2ee-machine-sharing/AGENTS.mdandapps/electron/src/main/services/AGENTS.md. The latter's ceiling is raised 1240 → 1290: the lag rule spans both hosts,e2ee-client/AGENTS.mdis already at its 1400 ceiling andcomponents/src/providers/AGENTS.mdis at 1250/1250.Visual explanation
Cold dispatch:
N × (refresh → readHead → readVersion)becomes1 × (refresh → readHead[+version]), shared by all concurrent callers; warm after workspace open.Before / after
e2ee_invalid-ledger, room stuckkey-pending→ refresh → plaintextTest plan
apps/electron:node --import tsx --test e2ee-runtime / e2ee-account-runtime / e2ee-cloud-client— 299/299. New: owner opens a new device's frame before ingesting its admission →key-pending, epoch +4 →key-pending, +5 and 999999 → terminal, plaintext after ingest (fails withinvalid-ledgerwithout the fix); pre-warm reads another member's machine once per entry;includeVersionsent, and after a refusal sent once then dropped.packages/e2ee-machine-sharing: vitest 33/33. New: three concurrent cold authorizes → onereadHead, then warm; inlined original skipsreadVersion, mismatched inline falls back, restart from the pin works (both new tests fail with singleflight/inline disabled).packages/components:desktop-e2ee-content+runtime-provider126/126; new spacing test with fake timers.packages/e2ee-client: unit 84/84; browserruntime.browser.test.ts38/40 — updated case: lagging epoch is nowkey-pending, forged far-future epoch is not. The 2 failures (control route reopens after confirmed close before stale …, chromium+webkit, 120 s timeout) fail identically onmainwithout this change.apps/cli:cloud-cli-port+e2ee-runtime210 passed. Typecheck: electron node+web, e2ee-machine-sharing, e2ee-client, cloud-api, components clean; CLI only reports the unbuiltacp-extension-claudesubmodule.oxlint,oxfmt --checkon changed files clean;pnpm run docs checkpasses. Not run: fullpnpm check, macOS E2E, a real skewed Convex deployment.Limits: the backend does not implement
includeVersionyet (private), so the inline saving lands only after it deploys; the "immediate catch-up for a strictly higher epoch inside the cooldown" variant was not implemented — the plain 3 s spacing is the rate cap.🤖 Generated with Claude Code