Skip to content

Splice blink token stream in place to remove O(n^2) tail copies - #187

Merged
TomasVotruba merged 1 commit into
mainfrom
perf-token-stream
Oct 9, 2026
Merged

TomasVotruba merged 1 commit into
mainfrom
perf-token-stream

Conversation

@TomasVotruba

Copy link
Copy Markdown
Member

Problem

blink's token-stream mutations were super-linear. BenchmarkFixPipeline (lex + all rules + render) showed throughput dropping as file size grew, the signature of an O(n^2) cost. The culprit was blink/internal/tokens/stream.go:

  • ReplaceRange(start, end, repl) allocated a fresh copy of the entire tail on every call, then rebuilt the slice - even when the replacement was the same length as the range it replaced (the common case, e.g. reordering/normalising modifier keywords).
  • InsertAt / RemoveAt shifted the whole tail by hand.

These are called from ~90 sites across the rules, many inside loops over a single file, so repeated single-element edits were quadratic. A memory profile at 256 classes attributed 78% of all allocations to ReplaceRange alone (1.66 GB over the benchmark run).

Change

Replace the hand-rolled splices with the stdlib in-place operations:

  • ReplaceRange -> slices.Replace - no tail copy; when len(repl) equals the range it overwrites in place and the tail never moves.
  • InsertAt -> slices.Insert
  • RemoveAt -> slices.Delete

The public Stream API is unchanged (same signatures), so all 300+ rules and their tests are untouched. Output is byte-identical - the existing golden and parity suites cover this.

Benchmark (AMD Ryzen AI 9 HX 370, go1.26)

case before after speedup mem before mem after
1 class 111.7 us/op 116.5 us/op ~1x 21.0 KB/op 18.1 KB/op
8 classes 759.5 us/op 691.4 us/op 1.10x 213.8 KB/op 101.8 KB/op
64 classes 11.05 ms/op 7.11 ms/op 1.55x 7.16 MB/op 0.67 MB/op
256 classes 127.9 ms/op 47.9 ms/op 2.67x 102.5 MB/op 2.63 MB/op

At 256 classes: 2.67x faster and ~39x less allocated memory.

Remaining work (not in this PR)

A re-profile after the fix shows the residual cost is now memmove from genuinely single-element InsertAt/RemoveAt calls scattered across ~15 rules, each still shifting the tail once. Removing that last bit of super-linearity would need either a gap buffer for the token slice (adds index translation to the hot At path and carries real byte-parity risk across all rules) or retrofitting those rules to a batched edit API. Both are more invasive and better handled separately; this PR is the simplest safe change that already removes the dominant quadratic allocation cost.

@TomasVotruba
TomasVotruba enabled auto-merge (squash) October 9, 2026 11:42
@TomasVotruba
TomasVotruba disabled auto-merge October 9, 2026 13:31
@TomasVotruba
TomasVotruba merged commit 4bdc9b6 into main Oct 9, 2026
20 of 21 checks passed
@TomasVotruba
TomasVotruba deleted the perf-token-stream branch October 9, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant