[tanderson-ld]

What

Implements the EVTPROC event processor spec's Section 1.8 delivery cooldown (launchdarkly/sdk-specs#278) in the shared internal events code: on an unexpected delivery failure (a non-recoverable HTTP status such as 401 or 403), the event processor enters a bounded, self-recovering cooldown window instead of permanently disabling event delivery for the lifetime of the SDK instance.

• Windows run 5, 10, 20, 40, then 60 minutes (capped at the 1 hour ceiling), stepping once per failed probe. They are deterministic: RETRY 1.4.3 permits zero jitter for a component whose attempts are internally scheduled, and the flush schedule itself paces attempts, which is also why the windows are not floored at the flush interval.
• While a window is open, every flush (scheduled, flushAsync()/flushBlocking(), or the shutdown flush) is a no-op: it makes no network attempt and disposes of the batch exactly as a sent-and-refused payload would be disposed of. Two deliberate departures from pure refusal emulation: the diagnostic events-in-last-batch stat reports 0 (no batch was serialized), and the context deduplicator is flushed when the discarded batch contained an index or identify event, so a later payload can introduce those contexts again.
• Recording is never gated (1.8.8): evaluations keep updating summary counters and intake continues during a window.
• A successful probe returns delivery to the flush cadence immediately (1.8.3); retry state resets only after 2 consecutive successful deliveries (1.8.4). No cooldown state persists across restarts (1.8.10). The recoverable-failure retry inside DefaultEventSender (1.7.2) is unchanged.

This is a port of the reviewed Go implementation (launchdarkly/go-sdk-events#68), adapted to this module's threading model: delivery outcomes arrive on flush worker threads and the shared executor rather than on the dispatcher thread, so the cooldown tracker is synchronized pure state, and outcomes are correlated to the window era they were dispatched in via an opaque WindowToken (an outcome from a delivery in flight across a window boundary updates nothing).

API (internal module surface)

• EventSender.Result gains the unexpected classification with static factories success(Date), failure(), and unexpectedFailure(), and the accessor isUnexpectedFailure(), which also accepts the legacy flag so results from senders written against the old API still get cooldown behavior.
• The three-argument Result constructor and isMustShutDown() are deprecated. unexpectedFailure() keeps writing the legacy flag for consumers that have not migrated (the server SDK's EventSenderWrapper and the Android SDK).
• DefaultEventSender migrates off the deprecated isHttpErrorRecoverable/checkIfErrorIsRecoverableAndLog helpers to classifyHttpFailure (first commit, behavior-identical) and then returns the new classification (second commit). Transport-level IOExceptions keep their always-retry behavior.
• EventsConfiguration gains a package-private cooldownNanoTimeFn test seam (deliberately not part of any constructor); the tracker documents that the source must have System.nanoTime() semantics, and all window arithmetic uses overflow-safe difference comparisons.

Design notes

• Diagnostic event delivery is not gated by the window (matching the spec, which does not cover diagnostics). Diagnostic outcomes drive the cooldown only the way they drove the old permanent disable: an unexpected failure on a diagnostic payload opens a window, while a diagnostic success or ordinary failure has no effect on delivery state - in particular, a diagnostic success is not the probe and cannot count toward the reset threshold, since only analytics deliveries prove the service is accepting analytics payloads again. (Without that asymmetry, periodic diagnostic successes would hold the ladder at 5 minutes forever while analytics payloads kept failing.) A diagnostic dispatched inside a window carries no live token, so its outcome cannot move delivery state even if it lands after the window elapses. This coupling preserves this module's existing shape and is a deliberate divergence from the Go implementation, which excludes diagnostic results entirely; it is flagged for spec follow-up.
• Server-time calibration from the Date header remains unconditional, exactly as on main.
• The disabled flag remains as a shutdown-only latch; nothing sets it from delivery results anymore.
• A delivery that dies in an exception (in serialization, the sender, or a custom sender returning null) is recorded as an ordinary failure, breaking the consecutive-success streak without opening a window, and can no longer kill the delivery worker: outcome handling is guarded so the busy-worker count always comes back down and close() always returns.
• The cooldown check and the delivery token issue are one atomic tracker call, so a window opening concurrently on a worker thread either diverts the flush to the discard path or makes the dispatched payload's token stale. The once-per-window Warn is also claimed under the tracker's own lock.
• The tracker's clock seam is a package-private NanoTimeSource interface rather than java.util.function.LongSupplier, which does not exist on Android below API 24 (this module ships in the Android SDK at minSdk 21); the fix is verified with a d8 --min-api 21 dex sweep showing zero java.util.function references.
• Logging: one Error when a window opens (with its length), a Warn for the first discarded batch of each window (Debug thereafter), and an Info when delivery resumes.
• Java has no failed-send metrics surface, so the in-window discard fires no metric; the Warn/Debug discard logs carry the count and remaining window.

Testing

• DeliveryCooldownTrackerTest (24 tests): window ladder with every rung asserted past the shift-guard bound, reset semantics, boundary-round staleness on both probe success and recoverable probe failure, stale-outcome and in-window-outcome guards, null token, default clock path, boundary instant, the atomic gate, and the once-per-window discard notice.
• DefaultEventProcessorCooldownTest (18 tests): cooldown entry and resume across a fake clock, legacy shutdown-flag results, contradictory success+shutdown results, in-window disposal with context-introduction recovery through both the index and identify arms, summary-only discards leaving the deduplicator alone, empty in-window flushes doing nothing, shutdown during a window, warn-once-then-debug and resume logging, the diagnostic coupling (including a diagnostic held in flight across the window boundary), a null sender result neither stopping delivery nor wedging close(), a sender exception breaking the success streak, simultaneous failures across the full worker pool stepping the ladder exactly once, and the batch-size stat zeroing on discards.
• diagnosticEventsStopAfter401Error is rewritten as diagnosticEventsContinueAfter401Error; the permanent-disable test is renamed to describe the cooldown contract; a transport-failure sender test pins the deliberate never-unexpected classification of connection errors.
• Full module suite: 360 tests green; javadoc and checkstyle clean; javadoc warning count unchanged from main.
• Four multi-agent review rounds ran against this change before it was opened (two on the initial implementation, two on the fixed state), including adversarial reviews with runnable proofs on a real Android API 23 emulator image, a d8/dexdump Android compatibility sweep, mutation testing of the new tests, and behavioral-parity diffing against the spec and the Go reference.

Out of scope (follow-ups)

• The 413 carve-out: classifyHttpFailure has no locally-unrecoverable classification, so a 413 opens a window rather than dropping only the oversized payload. That is a pre-existing classifier gap with its own Jira bug (it also requires unsuppressing the corresponding contract test), deliberately not fixed here.
• The server SDK changes (dependency bump, public EventSender.Result.STOP doc reinterpretation, wrapper migration, event-delivery-cooldown contract-test capability, and the 413 test suppression) ship in a follow-up PR once this module releases.
• Android SDK adoption: the Android SDK's own DirectEventProcessor reads the dual-written legacy flag and keeps its current permanent-disable behavior byte-identical before and after this release; migrating it to the cooldown is its own coordinated change.
• Diagnostics RETRY alignment is separate spec work.

Release notes

• Minor (feat). Behavior change worth calling out: a custom event sender result carrying the legacy shutdown flag now causes a temporary cooldown rather than a permanent stop.

---

Note

Overview
Replaces permanent analytics shutdown after errors like HTTP 401 with a bounded, self-recovering cooldown driven by a new DeliveryCooldownTracker (5→10→20→40→60 minute windows, reset after two consecutive successes).

DefaultEventProcessor gates flushes through the tracker: in-window flushes discard buffered batches without network I/O (including context-deduplicator flush for index/identify discards), correlate outcomes via WindowToken, and keep diagnostics sending while only unexpected diagnostic failures open the window. Delivery workers now treat sender exceptions and null results as ordinary failures without wedging close().

EventSender.Result adds unexpected classification (success / failure / unexpectedFailure, isUnexpectedFailure()), with legacy mustShutDown deprecated but still honored. DefaultEventSender uses classifyHttpFailure so unexpected HTTP statuses backoff instead of implying permanent shutdown.

Extensive unit tests cover the tracker, processor cooldown behavior, and updated diagnostic/HTTP sender expectations.

^{Reviewed by Cursor Bugbot for commit 8358410. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8358410. Configure here.}

[cursor]

Transport errors log unsanitized exceptions

Medium Severity

The new IOException warn path logs e.toString(), so OkHttp transport failures can write hostnames, URLs, or other private request details into customer logs. Nearby cooldown handling already uses LogValues.exceptionSummary for Throwables, and classifyAndLogTransportFailure sanitizes the same class of errors.

 

^{Triggered by learned rule: Log exceptions with LogValues.exceptionSummary, not toString/getMessage}

^{Reviewed by Cursor Bugbot for commit 8358410. Configure here.}

[cursor]

Null sender result skips failure recording

Low Severity

A custom EventSender that returns null still only trips the outer catch around handleResponse. That catch keeps the worker alive but never calls recordFailure, so the consecutive-success streak is left intact. The PR contract treats a null result as an ordinary failure that should break the reset threshold the same way a thrown sender does.

Additional Locations (1)

• lib/shared/internal/src/main/java/com/launchdarkly/sdk/internal/events/DefaultEventProcessor.java#L687-L700

 

^{Reviewed by Cursor Bugbot for commit 8358410. Configure here.}

[beekld]

AI review: The description says the Android SDK's own DirectEventProcessor reads the legacy flag and keeps its permanent disable unchanged. Android's main branch has no DirectEventProcessor; it exists only on unmerged branches. Android main builds this module's DefaultEventSender and DefaultEventProcessor directly (ComponentsImpl.java), pinned at internal 1.9.0. So Android's next bump of this dependency picks up the cooldown and the ungated diagnostics without a coordinated change.

Relatedly, nothing in either repo reads the dual-written legacy flag. EventSenderWrapper only writes it, so it isn't a consumer that has not migrated.

[beekld]

AI review: A diagnostic failure can open a window, but a diagnostic success can never end one. Suppose a proxy or allowlist accepts /bulk but returns 403 or 404 for /diagnostic. Then every periodic diagnostic opens a new 5-minute window, and analytics that the service would accept are discarded in it. That's about a third of analytics at the default 15-minute diagnostic interval, and most of them at the 60-second minimum. The Go PR (launchdarkly/go-sdk-events#68) ignores diagnostic results, so it doesn't have this. Should a diagnostic failure open a window only while the analytics success streak is 0, or not at all?

[beekld]

AI review: Both catches above take only Exception. If the sender or formatter throws an Error (such as AssertionError, NoClassDefFoundError, or StackOverflowError), the thread dies before it reaches this decrement. The count stays raised, and close() blocks forever in waitUntilAllFlushWorkersInactive. Main has the same problem, but the description says close() always returns. Moving the decrement into a finally would make that true.

[beekld]

AI review: Because a stale failure returns here without touching the streak, the ladder can reset across a failure:

1. The window elapses. The probe P1 and a sibling P2 are dispatched before either returns, so they share a token.
2. P1 succeeds. That bumps the epoch, and the streak is 1.
3. P2 fails. Its token is stale, so this returns early.
4. P3 succeeds. The streak reaches 2, and attempts resets.

That's success, failure, success, in both dispatch and completion order. RETRY 1.8.3 says any failure restarts the reset-threshold count. The comment in recordSuccess assumes the next round fails; this is the case where it succeeds. recordUnexpectedFailure returns early the same way. Resetting consecutiveSuccesses before the staleness check in both methods would fix it, without letting a stale outcome open or step a window. The Go PR has the same code (launchdarkly/go-sdk-events#68).

[beekld]

AI review: This sends without checking the cooldown. payloadQueue can hold a payload whose flush ran before the window opened, while every worker was busy. If a 401 opens the window before a worker takes that payload, the payload is still sent inside the window. Its outcome is stale, so cooldown state stays right, but the request goes out. EVTPROC 1.8.1 says no delivery attempt starts during a window. 1.8.5 is about flushes during the window, so it's unclear whether it covers a payload handed off before it. The Go PR does the same (launchdarkly/go-sdk-events#68). Is this intended?

[beekld]

AI review: takeDiscardWarning() is atomic on its own, but not together with the cooldown check in triggerFlush, which takes the lock separately. A new window can open between them: the old window elapses, a diagnostic dispatched after that gets a 401, and the new window re-arms the Warn. This discard, gated by the old window, then takes the new window's Warn, and the new window's own discards log at Debug. In the Go PR, the check, the re-arm, and the check-and-set all run on the dispatcher goroutine, so this can't happen there. The impact is small, but returning the first-discard flag from the same tracker call as the cooldown check would close the gap.

[beekld]

AI review: failure() does affect the delivery schedule: it breaks the success streak, and on a probe it ends the elapsed window (DeliveryCooldownTracker.recordFailure).

[beekld]

AI review: Not only from the dispatcher thread: diagnostic deliveries get their tokens on the shared executor (createSendDiagnosticTask in DefaultEventProcessor).