Skip to content

Stop the aggregator stress test dropping events on a slow runner - #88

Merged
aleksandar-apostolov merged 1 commit into
developfrom
fix/flaky-aggregator-stress-test
Oct 6, 2026
Merged

aleksandar-apostolov merged 1 commit into
developfrom
fix/flaky-aggregator-stress-test

Conversation

@gpunto

@gpunto gpunto commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Goal

Closes AND-1605. StreamEventAggregatorImplTest > stress - 10K events with realistic type distribution fails intermittently in CI with 10K events not delivered in time. It has failed on develop (run), on a dependabot PR (run) and on #87 (run), and passed on rerun each time.

The cause is dropped events, not a slow runner. A slow runner only exposes the drop. The 60s timeout was never close to being needed:

  • The inbox is unlimited, so all 10K events are queued before the collector has to wait for any of them.
  • The dispatch queue keeps the default DEFAULT_DISPATCH_QUEUE_CAPACITY = 16. At a threshold of 50, the run produces about 200 batches.
  • When the dispatcher thread is not scheduled for a moment, the collector fills those 16 slots and trySend fails. As the class KDoc says, the batch is then dropped with a warning. The test has no logger, so the drop is silent.
  • After one dropped batch the delivered count can never reach 10K, so the latch waits out the full 60s.

Implementation

The stress test now sets dispatchQueueCapacity = totalEvents. Every queued item holds at least one event, so the queue can never fill up. I did not use totalEvents / threshold: batches can be smaller than the threshold when the 500ms window closes before 50 events arrive, so it is not a guaranteed bound.

The test still covers all three of its guarantees: every event delivered, batches no larger than the threshold, and fewer handler calls than events. The drop path is still covered by the existing full-dispatch-queue test.

Test-only, no production code touched. The drop under load is intended behaviour per the KDoc, but the test calls "all events delivered" a guarantee, and in production that only holds while the dispatcher keeps up.

Testing

  • Reproduced: with the test's scope switched to Dispatchers.Default.limitedParallelism(1) the collector never yields to the dispatcher, and the test fails every time.
  • Fix under the same single-thread setup: passes 3/3.
  • Final diff (original dispatcher and 60s timeout restored): ./gradlew :stream-android-core:testDebugUnitTest --tests '*StreamEventAggregatorImplTest*' is green, and spotless is clean.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Clarified the stress test’s event queue capacity to help ensure the full set of events can be processed without queue saturation.

The stress test floods 10K events into an unlimited inbox, but the
dispatch queue keeps its default capacity of 16. When the dispatcher
thread is not scheduled for a moment, the collector fills the queue and
drops whole batches by design, so the latch never reaches 10K and the
test fails after 60s. Reproducible every time on a single-thread
dispatcher.

Size the queue to the event count, an upper bound on queued items.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gpunto gpunto added the pr:test Test-only changes label Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

PR checklist ✅

All required conditions are satisfied:

  • Title length is OK (or ignored by label).
  • At least one pr: label exists.
  • Sections ### Goal, ### Implementation, and ### Testing are filled, or the PR is bot-authored.
  • An issue is linked (Linear ticket or GitHub issue), or the PR is bot-authored.

🎉 Great job! This PR is ready for review.

@sonarqubecloud

sonarqubecloud Bot commented Oct 2, 2026

Copy link
Copy Markdown

@gpunto
gpunto marked this pull request as ready for review October 2, 2026 15:41
@gpunto
gpunto requested a review from a team October 2, 2026 15:42
@coderabbitai

coderabbitai Bot commented Oct 2, 2026

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (1)
AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: ef62ef78-66ea-4050-a1a7-f15014d1a277

📥 Commits

Reviewing files that changed from the base of the PR and between 4c29da9 and 11f2ba0.

📒 Files selected for processing (1)
  • stream-android-core/src/test/java/io/getstream/android/core/internal/processing/StreamEventAggregatorImplTest.kt

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

The 10,000-event stress test now sets dispatchQueueCapacity to totalEvents. Comments explain that a full queue can drop events and describe why this capacity is expected not to fill.

Changes

Stress-test queue capacity

Layer / File(s) Summary
Stress-test queue policy
stream-android-core/src/test/java/io/getstream/android/core/internal/processing/StreamEventAggregatorImplTest.kt
The test policy sets dispatchQueueCapacity to totalEvents. Comments describe event dropping when the queue is full and the expected queue capacity.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 11f2b

This test-only change prevents the stress test from failing due to an undersized dispatch queue. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: preventing the aggregator stress test from dropping events when the dispatch queue fills.
Description check ✅ Passed The description includes complete Goal, Implementation, and Testing sections. It explains the failure cause, the test-only fix, validation steps, and scope. The Checklist section from the template is …
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the queue with care
Ten thousand events have room to spare
The comments note what fullness brings
The test hops on through queued-up things
Then nibbles clover as it springs

Comment @coderabbitai help to get the list of available commands.

@aleksandar-apostolov
aleksandar-apostolov merged commit 89288df into develop Oct 6, 2026
11 of 13 checks passed
@aleksandar-apostolov
aleksandar-apostolov deleted the fix/flaky-aggregator-stress-test branch October 6, 2026 13:36
@stream-public-bot stream-public-bot added the released Included in a release label Oct 7, 2026
@stream-public-bot

Copy link
Copy Markdown
Collaborator

🚀 Available in v5.1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr:test Test-only changes released Included in a release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants