Skip to content

fix: keep unclosed subscript recovery linear - #20

Open
everton-dgn wants to merge 1 commit into
webpro-nl:mainfrom
everton-dgn:fix/subscript-recovery-scaling
Open

everton-dgn wants to merge 1 commit into
webpro-nl:mainfrom
everton-dgn:fix/subscript-recovery-scaling

Conversation

@everton-dgn

Copy link
Copy Markdown

Repeated unclosed subscripts followed by a final ] make recovery scan the remaining input again for each command. This caches bracket results after a failed search, only at positions the lexer visited, so recovery can still reinterpret quotes and comments correctly.

For the 256-line case in the issue, character reads fall from 204,539 to 10,737. The regression checks growth in reads rather than elapsed time and preserves the later-assignment checks.

The full 2,060-test suite passed on Node 24. After the final syntax adjustment, 263 focused tests, type checks, lint, formatting and the runtime smoke test passed. I couldn't run the Node 14 smoke test on this arm64 Mac; CI still needs to cover it.

Fixes #14

@jecruz

jecruz commented Nov 5, 2026

Copy link
Copy Markdown

Thanks for the fix, @everton-dgn. I reviewed this against #14 and the results look good.

What I tested (head b071427 vs its merge-base 61b82c2, which is current main; Node 24, built with plain tsc -p tsconfig.build.json):

  1. Operation counts for Quadratic parsing of unclosed subscripts followed by a trailing ] #14. I counted String.prototype.charCodeAt calls during parse of 'a[x y\n'.repeat(n) + ']', n = 64..2048 (doubling), plus the 'a[$x y\n' variant.
    • Before: 14,267 reads at n=64 up to 12,646,395 at n=2048 (37 to 1,029 reads per character, doubling with n, i.e. quadratic).
    • After: 2,673 up to 86,001 reads, a constant ~7.0 reads per character. The $x variant is a constant ~8.1 (before: 46.5 to 1,322).
    • Without the closing ] the counts are identical before and after (5.0 and 5.6 reads per character), also linear.
  2. AST differential. I serialized full ASTs for both builds and compared them byte for byte: 5,258 rows (policy probes, 4,096 seeded fuzz inputs, scaling up to 16 KiB) and 4,121 further rows, plus 8,192 seeded cases built from subscript, quote, comment, $(, ${, [[, (( and += atoms and 20,000 longer cases biased toward subscripts, quotes and here-docs. 0 differences and 0 exceptions on either side.
  3. Recovery semantics. I checked the example from the issue (a[$x y # "\nb[$x z]=1\n# "), the single-quote variant, variants with an unmatched ] inside $(...), here-doc variants, an array body and a nested subscript. The ASTs are identical before and after, and the b[$x z]=1 assignment is still parsed as an assignment to b with index $x z.
  4. Test suite on the PR head: all non-benchmark tests pass (2050/2050, including the new ones). test/benchmark.test.ts fails the same 8 tests before and after in my setup because I did not install the benchmark dependencies, so I treat that as unrelated.

I did not find a concern. This is differential evidence over the corpora above, not a proof of equivalence. One reading note: the per-start-position cache is only filled from positions the search actually visited and only when no errors were recorded, which matches the comment about not caching inside skipped quote/comment syntax. The 28,192 targeted cases above did not find a case where that breaks down.

@everton-dgn

Copy link
Copy Markdown
Author

Thanks for the review and for the differential runs against #14. Let me know if anything else is needed before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Quadratic parsing of unclosed subscripts followed by a trailing ]

2 participants