Skip to content

perf(sse): avoid flattening fragmented block carry - #7498

Closed
Platinum-Scripts wants to merge 1 commit into
lidge-jun:devfrom
Platinum-Scripts:perf/responses-block-scan
Closed

Platinum-Scripts wants to merge 1 commit into
lidge-jun:devfrom
Platinum-Scripts:perf/responses-block-scan

Conversation

@Platinum-Scripts

@Platinum-Scripts Platinum-Scripts commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Avoid passing a growing unfinished SSE frame to the delimiter regex after a failed scan when appended fragments contain no CR/LF. Keep the existing scan cursor and delimiter handling when a new delimiter is possible, including queued appends and compaction.
  • Cache the final code unit from each appended fragment for surrogate-boundary accounting instead of indexing the growing carry on every append. Preserve that boundary across unsuccessful scans and reset it only when the buffer is emptied or cleared.
  • Preserve all existing buffer reservations, cap failures, output bytes, terminal behavior and rewrite policy. No additional payload buffer or budget operation is introduced.

Verification

  • Bun 1.4.3 focused SSE rewrite and file-size tests: 37 passed. Two deterministic work-bound regressions failed before their respective fixes, then passed. Existing CR/LF/CRLF, Unicode, drops/injection, EOF, cancellation, enqueue failure and overlap accounting cases pass. Added a failed-scan/split-surrogate regression.
  • Typecheck, structure SSOT, privacy scan and diff checks passed. The import-connected suite passed: 34,772 tests, 42 skips, zero failures across 1,566 files (324.25 seconds). This draft does not claim an unrun full suite.
  • 700 deterministic local compatibility scenarios matched the original module's complete outputs, tails, errors and budget snapshots, including 435 overflow scenarios.
  • Local actual Codex app-server → HTTP → actual npm OpenCodex 2.90 Responses route → simulated provider: 120 text turns and 120 dynamic-tool turns completed with exact hashes and real tool-result replay. Default tools/cache-only search retained. Twenty balanced alternating pairs per cell, plus two warmups per variant. Only this source module differed between runtime arms.
  • Paired tool-availability savings: 256 KiB frame / 512 B fragments 22.13 ms (exploratory bootstrap 95% interval 18.28–23.53); 1 MiB stress frame / 512 B fragments 175.53 ms (170.61–177.90). First-text savings were 23.52 and 173.28 ms respectively. Small-frame changes were submillisecond; small-frame tool results were inconclusive. These are simulated-provider client integration measurements, not vendor generation or GUI paint claims. No external inference.
  • Profiling located repeated work in the regex call and then in append's carry indexing; fixing only the regex moved the hotspot to append. The combined patch addresses both. The actual 2.90 module matched the upstream base byte-for-byte before patching.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. No credential, auth, dependency or release workflow changes.

Review readiness

  • Required local validation passed; commands, results, and any full-suite exception are documented.
  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • I resolved all correct Codex and CodeRabbit findings.
  • My PR is ready for review.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the enhancement New feature or request label Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

@lidge-jun

Copy link
Copy Markdown
Owner

Carried together with #7497/#7498 into #7541 on current dev, with authorship and a Co-authored-by trailer kept on every commit. The carry adds a small follow-up in createSseBlockBuffer.append() that updates the cached boundary state before the budget commit. The maintainer will close this PR.

lidge-jun added a commit that referenced this pull request Oct 23, 2026
…#7497, #7498) (#7541)

* perf(chat): avoid rescanning fragmented SSE carry

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* perf(chat): retain carry fast path within file-size cap

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* perf(sse): avoid flattening fragmented block carry

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* fix(sse): keep block-buffer boundary state in step with the carry

Record the cached final code unit and scan eligibility as soon as the
fragment joins the buffer, before the budget commit, so a throwing commit
cannot leave them describing an older carry.

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

---------

Co-authored-by: StreamClash Dev <dev@streamclash.local>
Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Triage note (2026-10-24, against dev at 2494848).

Superseded: already integrated through #7541 / c0d80ef869; source PR needs administrative closure only.

Still needed on dev: no. Carried by #7541 / c0d80ef869. Dev contains cached state and the carry follow-up that updates it before budget commit (src/server/sse-payload-rewrite.ts:127–129).

Exact-head CI at triage time: b37da6e — 25 success, 9 skipped; ci passes in run 39677364490. lane=all on exact head: n.a.

A maintainer will decide on closing this; no action needed from you.

@lidge-jun

Copy link
Copy Markdown
Owner

Closing as superseded: this change landed on dev through the combined carry #7541 (c0d80ef869); the cached carry state is in src/server/sse-payload-rewrite.ts:127. Thank you, @Platinum-Scripts — the carry keeps your authorship credit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants