Skip to content

Fix native sync reconciliation latency - #156

Merged
tiensonqin merged 2 commits into
mainfrom
fix/sync-core-latency
Nov 9, 2026
Merged

tiensonqin merged 2 commits into
mainfrom
fix/sync-core-latency

Conversation

@tiensonqin

@tiensonqin tiensonqin commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Remove full-block materialization from synchronization reconciliation on the serial core executor shared by sync and native node interactions.

  • Reconcile only outstanding local UUIDs against indexed lookups in the confirmed graph; skip authoritative loading and legacy pending pumps when there are no eligible nodes. This also confirms nodes on ordinary pages and older journals instead of depending on the latest seven journals.
  • Query pending/failed nodes through the sync-status index. For reconciliation, filter indexed status metadata before reading blocks, preserving every status other than synced, including nonstandard retry states. Use hashed UUID matching rather than repeated list searches.
  • Preserve confirmation conditions, journal application, ACK/rebase, checkpoints/cursors, error propagation, and snapshot response format. Real changes still incur existing rebase/snapshot work.
  • Add one ordinary-page confirmation regression and extend the existing reconciliation contract to cover other unsynced states while protecting newer local edits.

Validation

  • opam exec --switch=5.5.0 -- dune build @shared/native/runtest: 1,296 passed on the final implementation, including encrypted sync, rejection/retry, persistence, ACK and rebase coverage. The nonstandard-status regression failed against the whitelist implementation and passed after the fix.
  • Native Swift sync-command, WebSocket recovery/cancellation checks and LogseqModel build passed. No swift test used.
  • Real local db-sync + native mobile core + CLI stress runs: seeds 20261109 / 20261110 / 20261111 / 20261112 / 20261113, with 320 / 500 / 320 / 160 / 320 generated steps. Covered offline concurrent edits, process crashes, lost WebSocket ACKs, server restart and asset byte verification. Each run ended with identical UUID-normalized synced datoms across mobile, CLI and a fresh server snapshot, zero pending work and no residual sync error.
  • macOS real-core probe with 10,000 journal nodes: legacy authoritative read 231ms, single-UUID read 0.039ms, empty pull 1.9ms. Final 10,000-synced-record cache probe: old full-block selection 190.7ms, metadata-only unsynced selection 0.224ms, indexed pending selection 0.015ms; an additional retry state remained selectable. Disposable probes removed.

Physical iPad latency and Instruments profiling are not yet verified; timings above are not device measurements. This PR shortens reconciliation but does not allow local operations to preempt an already-running sync task. Separating local interaction execution from synchronization preparation is a separate follow-up.

Link to Devin session: https://app.devin.ai/sessions/951fc198b3c140ad8b8cc8af5b531471
Open in Devin Desktop: https://app.devin.ai/desktop/session/951fc198b3c140ad8b8cc8af5b531471?variant=devin
Requested by: @tiensonqin


🟢 Devin fixed the finding · Follow Devin's work

Devin Review

Query only outstanding local UUIDs in the confirmed graph and skip empty reconciliation and pending pumps. Use indexed local sync statuses so settled captures cannot cause graph-sized work on the shared core executor.
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

Filter indexed sync-status metadata before reading eligible blocks, retaining confirmation for arbitrary non-synced statuses without materializing settled records. Extend the reconciliation contract to cover retrying states.
@tiensonqin
tiensonqin merged commit e07ce4c into main Nov 9, 2026
2 checks passed
@tiensonqin
tiensonqin deleted the fix/sync-core-latency branch November 9, 2026 13:16

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 finding. Devin automatically fixed it.

Fixed by Devin (1)
  • Other unsynced statuses skip confirmation — Preserved the original open-ended sync_status <> synced contract by filtering attribute-indexed status metadata before materializing matching blocks. The existing reconciliation test now covers retrying; red/green, all 1,296 core tests, and real three-client crash/reconnect/lost-ACK stress validation passed.

View findings in Devin Review · Follow Devin's work

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant