Skip to content

Coalesce empty checkpoints to reduce API calls in high-concurrency map/wait scenarios#336

Open
AdityaAudi wants to merge 1 commit intoaws:mainfrom
AdityaAudi:feat/coalesce-empty-checkpoints-325
Open

Coalesce empty checkpoints to reduce API calls in high-concurrency map/wait scenarios#336
AdityaAudi wants to merge 1 commit intoaws:mainfrom
AdityaAudi:feat/coalesce-empty-checkpoints-325

Conversation

@AdityaAudi
Copy link
Copy Markdown

@AdityaAudi AdityaAudi commented Apr 5, 2026

Summary

Implements the empty-checkpoint coalescing optimization described in #325.

When many concurrent map/parallel branches resume from a wait operation simultaneously, each branch's resubmitter enqueues an empty checkpoint (no OperationUpdate). Previously, these counted toward the 250-operation batch limit, causing 300 concurrent branches to produce 2 API calls (250 + 50) instead of 1.

  • Tracks effective_operation_count separately from len(batch)
  • The first empty checkpoint in a batch counts as 1 effective operation
  • All subsequent empty checkpoints in the same batch are coalesced in without incrementing the effective count
  • Applied consistently across all three collection phases: overflow queue drain, first-op blocking wait, and time-window loop
  • Updated debug logging to clearly show real ops sent vs empties excluded

…ts (used by map/parallel branch resubmitters when resuming from timed waits) no longer count toward the 250-operation batch limit beyond the first. This prevents 300+ concurrent branch resumes from splitting across multiple API batches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant