Skip to content

fix(events): jitter a Retry-After wait up, and pin a log's on-disk bytes - #426

Open
abelonogov-ld wants to merge 2 commits into
andrey/delivery-retry-extended-initial-delayfrom
andrey/retry-after-jitter-and-log-vector
Open

abelonogov-ld wants to merge 2 commits into
andrey/delivery-retry-extended-initial-delayfrom
andrey/retry-after-jitter-and-log-vector

Conversation

@abelonogov-ld

@abelonogov-ld abelonogov-ld commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Two changes, stacked on #425 (the RETRY 1.5.6 extended initial delay), which is stacked on #408 (tier 3).

A Retry-After wait is jittered up, never below what the service asked. It was jittered down by up to half, so an instance could post before the time the response asked for.

  • DeliveryRetryState.directedWaitMillis now waits between the asked duration and 1.5 times it, stopping at the one-hour cap on a server-directed wait (RETRY 1.11.4).
  • A wait asked for of an hour or more has no room above the cap, so it is still jittered down from the hour (30 to 60 minutes), which keeps a fleet told to wait an hour from posting together.
  • The same rule applies when a flush posts through a pending wait (recordFailureDuringWait), where a Retry-After can still only lengthen the wait.
  • The backoff without a Retry-After stays unjittered, as RETRY 1.4.3 allows.
  • Tests: the 10 s wait now falls in 10–15 s; a 45-minute one in 45–60 minutes; an hour or five hours in 30–60 minutes. aRetryAfterSetsTheWaitBeforeTheNextPost asks for 3 s instead of 4, so the retry still lands well inside its timeout.

The on-disk bytes of a log are pinned to the spec's worked example. The format tests were structural round trips, so a drift in the header or the framing still passed. The page-cache persistence spec's Appendix A log (two events, 108 bytes, checksums verified independently with zlib) is now a hex literal, EventLogFormatTest.WORKED_EXAMPLE, asserted for:

  • the file header constant;
  • what EventLogFormat and the store write for its two events;
  • what the reader makes of it (2 events, not damaged, the expected body);
  • a next run recovering it as one batch.

The iOS suite asserts the same bytes. Setting the format version to 2 fails all four new tests; the existing EventStoreTest tests all still pass with it.

Tested: :launchdarkly-android-client-sdk:testDebugUnitTest, 1264 tests, 0 failures.


Note

Overview
Event delivery: Retry-After handling now waits at least the server’s requested duration (jittered upward by up to half, capped at one hour) instead of jittering down, so clients do not retry early. Requests of an hour or more still jitter downward from the one-hour cap so fleets do not realign. Logic lives in new directedWaitMillis and is shared by recordFailure and recordFailureDuringWait.

Event log format: Tests pin on-disk bytes to the page-cache persistence spec’s Appendix A worked example (aligned with iOS), covering header, write/read, assembly, and store recovery—not just structural round-trips.

Reviewed by Cursor Bugbot for commit 96f97d6. Bugbot is set up for automated code reviews on this repo. Configure here.

abelonogov-ld and others added 2 commits November 10, 2026 11:25
A Retry-After wait was jittered down by up to half, so an instance could
post before the service asked. Jitter it up by up to half instead,
stopping at the hour a server-directed wait is capped at. A wait asked
for of an hour or more has no room above the cap, so it is still
jittered down from the hour.

Matches the iOS change.

Co-authored-by: Cursor <cursoragent@cursor.com>
The format tests were structural round trips, so a drift in the header or
the framing still passed. Assert the page-cache spec's Appendix A log as a
hex literal: the header, what the store writes for its two events, what
the reader makes of it, and that a next run delivers it. The iOS suite
asserts the same bytes.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant