Skip to content

docs: consolidate CHANGELOG for the perf audit and batch-2/batch-3 fixes; refresh performance baselines (#566) - #598

Merged
ajitpratap0 merged 5 commits into
mainfrom
docs/changelog-perf-audit
Oct 14, 2026
Merged

ajitpratap0 merged 5 commits into
mainfrom
docs/changelog-perf-audit

Conversation

@ajitpratap0

@ajitpratap0 ajitpratap0 commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

The CHANGELOG and baselines PR that the perf audit PRs promised; those PRs left CHANGELOG.md and performance_baselines.json alone so this could be one consolidated update.

CHANGELOG.md [Unreleased]

performance_baselines.json

  • Re-measured with BenchmarkPerformanceBaseline -count=10 -benchmem (median) on an Apple M4. Tolerances are unchanged.
  • Convention change: the baselines are now about 2.5× the M4 median, up from 1.5×.
    • The CI Benchmark job (ubuntu-latest) has measured these queries at about 1.5–3.2× the M4 numbers over the last 9 test runs on main (0715bb9..61ca6c8).
    • At 1.5× with the current tolerances, the limit would be about 2.4× M4, so CI's slowest runs would fail.
    • The cost: a local regression of up to about 4× would now pass.
Benchmark ns_per_op tolerance current_performance
SimpleSelect 800 60% ~318 ns/op on Apple M4 (median of 10 runs, re-measured 2026-10-13); ~467-918 ns/op typical on CI (GitHub ubuntu-latest Benchmark job, range over the last 9 Tests runs on main, 0715bb9..61ca6c8)
ComplexQuery 2900 65% ~1165 ns/op on Apple M4 (median of 10 runs, re-measured 2026-10-13); ~1842-3386 ns/op typical on CI (GitHub ubuntu-latest Benchmark job, range over the last 9 Tests runs on main, 0715bb9..61ca6c8)
WindowFunction 1500 60% ~585 ns/op on Apple M4 (median of 10 runs, re-measured 2026-10-13); ~940-1876 ns/op typical on CI (GitHub ubuntu-latest Benchmark job, range over the last 9 Tests runs on main, 0715bb9..61ca6c8)
CTE 1400 60% ~561 ns/op on Apple M4 (median of 10 runs, re-measured 2026-10-13); ~824-1692 ns/op typical on CI (GitHub ubuntu-latest Benchmark job, range over the last 9 Tests runs on main, 0715bb9..61ca6c8)
INSERT 950 60% ~382 ns/op on Apple M4 (median of 10 runs, re-measured 2026-10-13); ~605-1169 ns/op typical on CI (GitHub ubuntu-latest Benchmark job, range over the last 9 Tests runs on main, 0715bb9..61ca6c8)

Test Plan

  • go test -count=3 -run TestPerformanceRegression ./pkg/sql/parser/ passes locally.
  • performance_baselines.json is valid JSON.
  • An independent reviewer checked every new CHANGELOG bullet against the merged diffs and PR bodies over two rounds. That covered PR and issue numbers, API names and signatures, error codes, and performance figures, and every correction was applied. I read the Behavior changes section again before opening this PR.
  • The reviewer also checked the baselines against BenchmarkPerformanceBaseline medians and against CI Benchmark job output from main.

Breaking Changes

None. This PR changes documentation and test baselines only.

Part of #566

🤖 Generated with Claude Code

https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve

@vercel

vercel Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Oct 14, 2026 12:16pm UTC

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 32s —— View job


Review of PR #598: CHANGELOG consolidation and baseline refresh

  • Read performance_baselines.json diff
  • Read CHANGELOG.md diff, Behavior changes section (about 120 lines of the 337 added)
  • Checked that the baseline arithmetic is consistent
  • Not done: I did not re-read every CHANGELOG bullet against the merged PRs, and I did not run any tests. This is a docs and JSON change, and the PR says it was already reviewed against the diffs.

Verdict

No blocking issues. I'd approve this, but I can't submit a formal approval.

performance_baselines.json

  • The JSON parses. The diff touches only ns_per_op, updated, and the description and note strings. The tolerances are unchanged, as the PR says.

  • The 2.5× convention matches the numbers. Each baseline divided by its M4 median gives:

    • SimpleSelect: 800 / 318 = 2.52×
    • ComplexQuery: 2900 / 1165 = 2.49×
    • WindowFunction: 1500 / 585 = 2.56×
    • CTE: 1400 / 561 = 2.50×
    • INSERT: 950 / 382 = 2.49×
  • Every slowest CI run in the PR body stays under the new limit (baseline × (1 + tolerance)):

    Benchmark Limit (ns/op) Slowest CI run (ns/op)
    SimpleSelect 1280 918
    ComplexQuery 4785 3386
    WindowFunction 2400 1876
    CTE 2240 1692
    INSERT 1520 1169

    Headroom is about 25–40%.

  • Trade-off (already stated in the PR): a local regression of up to about 4× the M4 median now passes. TestPerformanceRegression is a coarse guard, not a precise one. A tighter local check, such as a benchstat comparison, would complement it. That would be separate work.

  • Nit, not blocking: "version": "1.14.0" is unchanged although the content now covers unreleased work. It makes sense as "last release baselined against". It may be worth bumping when the release is cut.

  • Nit, not blocking: the same long note string is repeated five times. It's harmless, but a single top-level note would be easier to maintain if the schema allows one.

  • Nit, not blocking: the "(up to ~3.2x)" parenthetical is redundant next to "1.5-3.2x".

CHANGELOG.md

  • Structure is good. The opening Behavior changes section lists what is observable on upgrade, with PR and issue references on each bullet. It follows Keep a Changelog conventions.
  • The zero-copy bullet is the most important one. It tells callers to use strings.Clone on cached token and AST strings, and it notes that Extract* and the DDL loader already clone. Consider also putting this in the release notes.
  • The "Oracle arr[1:n] now fails" bullet is a real change in behavior. It is documented clearly and is limited to DialectOracle.
  • The visitor-pointer bullet warns that mutating nodes now changes the AST and that the pointers are invalid after ReleaseAST. That is the right level of warning.
  • Wording nit: the intro says "Exported signatures are unchanged apart from additions", but the Behavior changes include ReleaseAST pooling semantics and error text changes. That's accurate for signatures, but you may want to say "source-compatible" so readers don't assume runtime behavior is unchanged.
  • Release-time check: the section header says [Unreleased]. When the release PR is cut (per the CLAUDE.md release workflow), make sure the version and date replace it. The baselines version field should be updated at the same time.
  • Perf figures are not independently verifiable here. The PR states they were copied from each PR's own measurements. I treated them as given.

Test plan

The PR reports TestPerformanceRegression passing locally and the JSON validating. I did not re-run the performance test. Race-detector runs skip it, and CI is the more meaningful signal given the runner variance this PR addresses.
· branch docs/changelog-perf-audit

@github-actions github-actions Bot added documentation Improvements or additions to documentation performance labels Oct 13, 2026
ajitpratap0 and others added 3 commits October 14, 2026 17:30
…resh performance baselines (#566)

CHANGELOG [Unreleased] now covers every PR merged since v1.14.0, with a
Behavior changes section for the observable changes in #597, #595, #584,
#585, #588, #581, #582/#570, #517/#518 and #511, plus Added, Changed,
Fixed, Performance, Security and Internal entries. The two existing
entries (#526 and the tokenizer-pool dialect fix) are kept verbatim.

performance_baselines.json is re-measured on an Apple M4 with
BenchmarkPerformanceBaseline -count=10 -benchmem (median). Baselines are
set at about 2.5x the M4 median rather than 1.5x: the CI Benchmark job on
ubuntu-latest measures these queries about 1.5-3.2x slower than M4 (last
9 Tests runs on main, 0715bb9..61ca6c8), so 1.5x with the unchanged
tolerances would fail CI's slowest runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
…e guards

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
@ajitpratap0
ajitpratap0 force-pushed the docs/changelog-perf-audit branch from 4da0296 to 5c44d7e Compare October 14, 2026 12:00
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 27s —— View job


Review of PR #598 (re-review after new commits)

Verdict

No blocking issues. I can't submit a formal approval.

performance_baselines.json

Unchanged since the last round. My earlier notes still apply:

  • The baselines are about 2.5× the M4 medians.
  • The slowest CI run is under each limit.
  • A local regression of up to about 4× now passes. The PR states this trade-off.

CHANGELOG.md (new commits)

Test plan

The PR reports TestPerformanceRegression passing locally. I did not re-run it. CI is the better signal for the runner variance this PR addresses.
· branch docs/changelog-perf-audit

…d costs

- #628: SQLite still rejects count(*) [cnt]; only SQL Server keeps it.
- #631: bare FORMAT / FORMAT <name> rules apply after SETTINGS.
- Performance: record the measured costs of the batch-3 correctness fixes
  (Identifier and TableReference growth, quoting in SQL(), lineStarts
  reallocation, sorted Extract results).
- #632: note the lıke, WITH CHECK OPTION and MINUS rendering changes.
- Fix the [Unreleased] compare link (v1.14.0...HEAD) and add [1.14.0].
- Keep each section a single tight list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 16s —— View job


Re-review of PR #598 (after fa39ca9)

  • Read the new commit fa39ca9 (CHANGELOG only, +19/-11)
  • Checked that the earlier nits were addressed
  • Not done: I did not check each new figure against the merged PRs, and I did not run any tests. This is docs only.

Verdict

No blocking issues. I can't submit a formal approval.

What the new commit fixes

Remaining items (all non-blocking)

performance_baselines.json is unchanged since my last review, so those notes still apply. The baselines are about 2.5× the M4 medians, and the slowest CI runs stay under each limit.
· branch docs/changelog-perf-audit

…me behaviour changes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UZSk1292jPc8ipDboNxeve
@ajitpratap0 ajitpratap0 changed the title docs: consolidate CHANGELOG for the perf audit and batch-2 fixes; refresh performance baselines (#566) docs: consolidate CHANGELOG for the perf audit and batch-2/batch-3 fixes; refresh performance baselines (#566) Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 20s —— View job


Re-review of PR #598 (after 36e4f43)

  • Checked the diff stat (CHANGELOG.md and performance_baselines.json only)
  • Checked that the "source-compatible" wording nit was addressed
  • Checked the compare links
  • Not done: I did not re-verify each bullet against the merged PRs, and I did not run any tests. The change is docs and JSON only. A sandbox restriction also stopped me from running a JSON parse check this round. The earlier reviews confirmed the file parses and that the diff touches only the baseline numbers and description strings.

Verdict

No blocking issues. I can't submit a formal approval.

What the new commit fixes

  • Intro wording: line 13 of CHANGELOG.md now reads "source-compatible: exported signatures are unchanged apart from additions". That closes the wording nit from my last two reviews.
  • Compare links: [Unreleased] points to v1.14.0...HEAD. [1.14.0] and [1.13.0] follow in the right order. I couldn't confirm here that the v1.13.0 and v1.14.0 tags exist. Please check that before the release.

Remaining items (all non-blocking)

performance_baselines.json is unchanged since my first review, so my earlier notes stand. The baselines are about 2.5× the M4 medians, and the slowest CI runs stay under each limit. A local regression of up to about 4× now passes, which the PR states.
· branch docs/changelog-perf-audit

@ajitpratap0

Copy link
Copy Markdown
Owner Author

Merge train (batch 3): #598, the last PR in the train

@ajitpratap0
ajitpratap0 merged commit d7338ce into main Oct 14, 2026
28 checks passed
@ajitpratap0
ajitpratap0 deleted the docs/changelog-perf-audit branch October 22, 2026 11:41

This branch was successfully deployed

1 active deployment
Preview — 36e4f439 Deployed Oct 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant