Skip to content

chore(docs): generate README metrics block and check version banners in CI - #710

Open
ajitpratap0 wants to merge 3 commits into
mainfrom
chore/readme-metrics-generation
Open

ajitpratap0 wants to merge 3 commits into
mainfrom
chore/readme-metrics-generation

Conversation

@ajitpratap0

Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/gen-readme-metrics (Go). It renders the README metrics table and the inline "318 ns" figure from scripts/gen-readme-metrics/metrics.json, between <!-- metrics:start/end --> and <!-- metrics-inline:start/end --> markers. Every number is labeled parse stage, hardware, Go version and run count. No competitor numbers are involved.
  • Rewrites version banners in docs/*.md to the latest release tag (v1.15.0), including banners that were stale at v1.6.0, v1.12.0, v1.13.0 and v1.14.0. sql99-compliance-analysis.md is a point-in-time snapshot and is skipped.
  • Checks (does not rewrite) that the newest doc.go version-history entry matches the tag. pkg/gosqlx/doc.go carries no version banner.
  • --check mode exits non-zero on drift. It is wired into .github/workflows/lint.yml as a new docs-drift job (fetch-depth 0 for tags) and into Taskfile as task docs:metrics / task docs:metrics:check.
  • CHANGELOG.md is intentionally not touched; mention for the next release: README metrics and doc version banners are now generated and CI-checked.

Note: after each release tag, the docs-drift job fails until go run ./scripts/gen-readme-metrics is run and doc.go history is updated. This is by design.

Test Plan

  • go run ./scripts/gen-readme-metrics --check passes on this branch
  • Seeded drift (parse_ns_per_op 318 -> 400) makes --check exit 1
  • go test -race ./scripts/gen-readme-metrics/ (banner regex and marker tests)
  • task check passes
  • actionlint on lint.yml could not be run in this sandbox; the new job mirrors the existing pinned-action steps

Breaking Changes

None.

Closes #658
Part of #634

🤖 Generated with Claude Code

https://claude.ai/code/session_01RkzVu3JY9Y6BRWfjesWts9

@vercel

vercel Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Oct 14, 2026 6:25pm UTC

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 25s —— View job


Review of PR #710: README metrics generation and version banner checks

The design is sound: a small stdlib-only generator, a --check mode, and markers in the README. I found no blockers. I read the diff only. I did not run the tests or the generator.

Bugs and robustness

  1. Every release tag turns CI red on every PR (main.go:97-108, lint.yml). You note this in the PR body as intentional. The cost is that unrelated PRs also fail until someone updates the docs. The release flow in CLAUDE.md creates the tag only after the docs PR merges, so the docs PR can't contain the new version before the tag exists. Options:

    • Make the check compare against the newest version found in doc.go or CHANGELOG.md, not the git tag.
    • Run the check only on push to main and on release PRs.
    • Leave it as is, but add the "run the generator after tagging" step to the Release Workflow section of CLAUDE.md.
  2. fetch-tags: true is redundant with fetch-depth: 0. It's harmless.

  3. --version plus --check can surprise. --version overrides the tag, but doc.go is still checked against it. That is probably what you want. Say so in the usage comment.

  4. The pkg/gosqlx/doc.go check is dead code today (main.go:192-199).

    • re is compiled inline and the file has no banner.
    • A missing file is silently ignored.
    • The case-insensitive regex ^//\s*version:? could match prose like // Version 2 of the API.... That would give a confusing hard failure.
    • Consider removing it, or anchoring it more tightly.
  5. Banner regex gaps (main.go:78-82).

    • Pattern 1, ^# .*\(v...\)\s*$, rewrites any H1 ending in (vX.Y.Z). This is fine now, but a doc whose title legitimately references an older version would be rewritten.
    • Patterns 2 and 3 are not anchored at the end, so **Version**: v1.2.3-beta becomes v1.15.0-beta.
    • Docs not matching any pattern are never checked. A new banner format would drift silently.
  6. Write/check ordering.

    • In non-check mode the files are written before the doc.go check, so a failure leaves the tree half-updated while still returning an error. That is acceptable, but running the doc.go check first would be cleaner.
    • Non-check mode also returns an error on a doc.go mismatch. The message is clear.
  7. Preserve file modes. os.WriteFile(path, ..., 0o644) only applies the mode to new files, so existing files keep theirs. This is not a problem.

Quality and maintainability

  • The table has hard-coded text (main.go:46-53).
    • "Complex queries take longer" and the performance_baselines.json reference are baked into the format string.
    • stage is in the JSON but never used. The table says "Parse stage only" regardless. Either use it or drop it.
    • Hardware and GoVersion are hand-entered, so nothing ties parse_ns_per_op to a real benchmark run. A short README in the scripts/gen-readme-metrics directory describing how to refresh metrics.json would help.
    • The go_version is "1.27" while go.mod/CLAUDE.md say 1.26+. If that is the benchmark machine's toolchain, it's fine.
  • parsesPerSec rounding. 1e9/318/1e6 = 3.14, which prints ~3.1M. Fine.
  • Marker handling (replaceBlock). The first :start is paired with the first :end. A duplicated marker pair would be silently half-handled. It is fine for the current README.
  • docs/*.md is not recursive. Subdirectories of docs are skipped. Confirm that is intended.

Tests (main_test.go)

  • Banner, block and rounding cases are covered.
  • Missing tests:
    • syncBanners with a pre-release suffix (item 5).
    • table() output.
    • latestTag parsing, which would need extracting the filtering into a pure function. That function is the part most likely to break, since it skips -rc tags and relies on --sort=-v:refname.
    • A check-mode integration test using a temp dir. run() reads relative paths and calls flag.Parse, so that would need a refactor.
  • The no v prefix case is good. A **Version**: v1.15.0 | x already-current case would confirm idempotency.

Security

  • No concerns. There is no user input and no network access. exec.Command uses fixed arguments.
  • The action SHAs are pinned, consistent with the existing steps.
  • The workflow has no explicit permissions:. This matches the existing jobs, but a contents: read block is good hygiene.

CI and workflow

  • The new job mirrors the existing steps. The missing trailing newline in lint.yml was already there.
  • You couldn't run actionlint. The YAML looks valid by inspection.
  • The go-version: '1.26' string should match the other jobs in the file.

Verdict: approve once item 1 (the policy for post-tag failures) is decided. Items 4 and 5 are nice-to-haves.
· branch chore/readme-metrics-generation

@ajitpratap0

Copy link
Copy Markdown
Owner Author

Fixed: bannerPatterns[1] now accepts an optional v (Version\*\*: v?), added a **Version**: 1.6.0 test case, and regenerated. docs/API_REFERENCE.md footer now reads 1.15.0; go run ./scripts/gen-readme-metrics --check and the package tests pass. sql99-compliance-analysis.md remains excluded and untouched.

@ajitpratap0

Copy link
Copy Markdown
Owner Author

Fixed in the latest push. The banner patterns now match both **Version:** v1.x and **Version**: v1.x forms, including Current and GoSQLX variants. TestSyncBanners covers the new forms. Regenerated docs/TROUBLESHOOTING.md, docs/COMPARISON.md and docs/PERFORMANCE_TUNING.md, which now read v1.15.0. I also updated the footer banner at COMPARISON.md:314. None of them are intentional snapshots, so snapshotDocs is unchanged. go run ./scripts/gen-readme-metrics --check fails on the old head and passes now.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 24s —— View job


Review of PR #710: README metrics generation and version banner checks

I read main.go, the workflow, the Taskfile and the README diff. I did not run the generator or the tests. The design is small and stdlib-only, and the --check mode is a good fit. I found no blockers. Below, a few points from the earlier review are confirmed against the code, and a few new ones are added.

Needs a decision

  1. Post-tag red CI (lint.yml, main.go:97-108). The release flow in CLAUDE.md tags only after the docs PR merges. latestTag() then returns a version the docs don't carry yet, so docs-drift fails on every open PR and on main until a fix lands. Pick one:
    • Compare against the newest doc.go history entry instead of the git tag. That entry is already hand-maintained.
    • Run the job only on push to main and on docs/v* branches.
    • Keep it as is and add "run task docs:metrics right after tagging" to the Release Workflow section of CLAUDE.md. This is the minimum.

Bugs and robustness

  1. pkg/gosqlx/doc.go check (main.go:192-199).
    • The PR body says this file has no banner, so the check does nothing today.
    • The (?mi) regex ^//\s*version:? would also match prose such as // Version 2 of the API…, and that would fail hard. Anchor it to ^//\s*(?:Current )?Version:\s*v?\d or remove the block.
    • The os.ReadFile error is dropped, so a missing file is ignored silently.
  2. Banner patterns 2 and 3 (main.go:80-81) are not end-anchored. **Version**: v1.2.3-beta becomes v1.15.0-beta. Pattern 1 also rewrites any H1 ending in (vX.Y.Z). Both are fine for the current docs. A cheap guard is a negative check such as (?:[^\w.-]|$) after the version.
  3. A new banner format would drift silently. Nothing flags a docs/*.md file that matches no pattern. A heuristic check would catch it, for example any line matching (?i)^\**(current )?version that no pattern rewrote.
  4. Write order. Non-check mode writes the files first and then runs the doc.go check. A mismatch leaves the tree updated and still exits 1. Running the doc.go check first is cleaner.
  5. --version with --check. --version also drives the doc.go comparison. Note that in the usage comment.
  6. replaceBlock (main.go:61-74). It pairs the first start marker with the first end marker. A duplicated pair would be half-handled silently. The README has one pair, so this is low priority.
  7. docs/*.md is not recursive. Confirm that is intended.

Quality

  • stage in metrics.json is never used, because the table hard-codes "Parse stage only". Use the field or drop it.
  • performance_baselines.json and "Complex queries take longer" are baked into the format string at main.go:49. That is fine, but it is hidden coupling.
  • hardware, go_version and parse_ns_per_op are entered by hand. Nothing ties them to a real run. A short README or a comment in metrics.json on how to refresh them would help. go_version: 1.27 differs from go.mod and CLAUDE.md (1.26+). That is fine if it is the benchmark machine's toolchain.
  • lint.yml: fetch-tags: true is redundant with fetch-depth: 0, but harmless. Adding permissions: contents: read to the job would be good practice. go-version: '1.26' should match the other jobs in the file.

Tests (main_test.go)

The banner, block and rounding cases are covered. Missing:

  • syncBanners with a pre-release suffix (item 3) and an already-current line, to confirm idempotency.
  • table() and inline() output.
  • latestTag filtering. Extract a pure pickTag([]string) function, because skipping -rc tags is the part most likely to break.
  • A --check integration test over a temp dir. run() uses relative paths and the global flag set, so it needs a small refactor to take a root and args.

Security

No concerns. There is no user input and no network access, and exec.Command takes fixed arguments. The action SHAs are pinned.

Verdict: Fine to merge once item 1 is decided. Items 2 to 4 are worth doing if cheap. The rest are nice-to-haves.
· branch chore/readme-metrics-generation

@github-actions github-actions Bot added documentation Improvements or additions to documentation ci-cd labels Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 21s —— View job


Review of PR #710: README metrics generation and version banner checks

I read main.go and main_test.go in full. I did not run the generator or the tests. I did not read the workflow diff or the docs changes, so nothing below covers those.

The design is sound: a small stdlib-only generator with a --check mode, markers in the README, and banner rewrites limited to docs/*.md. The **Version**: 1.6.0 fix from your last comment is in place, and its test case is there too. I found no blockers.

Issues worth addressing

  1. A new tag turns CI red on every PR.

    • main.go:97-108 takes the version from the newest git tag, and the docs-drift job runs on every PR.
    • CLAUDE.md's release flow merges the docs PR first and tags afterwards. So the docs PR can't contain the new version before the tag exists, and every other open PR fails until someone fixes the docs.
    • Options: compare against the newest doc.go entry instead of the tag; run the job only on push to main and on release PRs; or keep it and add a "run go run ./scripts/gen-readme-metrics after tagging" step to the Release Workflow section of CLAUDE.md.
    • Please decide on one before merging.
  2. The pkg/gosqlx/doc.go check is effectively dead code (main.go:192-199).

    • Your PR body says that file has no version banner, and a missing file is silently ignored.
    • The case-insensitive pattern ^//\s*(?:current )?version:?\s*v?\d+\.\d+\.\d+ could match prose such as // Version 2.0.0 of the API... and fail hard.
    • I'd remove the check or anchor it more tightly. The regex is also compiled inline on each run.
  3. Banner patterns 2 and 3 are not anchored at the end (main.go:80-81).

    • Pattern 2 ends in (), so **Version**: v1.2.3-beta becomes v1.15.0-beta.
    • Pattern 1 rewrites any H1 ending in (vX.Y.Z), including a title that deliberately names an older version.
    • A doc with a new banner format matches no pattern and drifts silently. A guard would help: fail if a docs file contains the word "Version" near the top but no pattern matched it.
  4. Write order in non-check mode (main.go:156-199).

    • Files are rewritten before the doc.go check, so a failure returns an error with the tree already modified.
    • Running the doc.go check first would be cleaner.

Smaller points

  • Unused stage field.
    • stage is parsed from metrics.json but never used. The table text hard-codes "Parse stage only (pre-tokenized input to AST)".
    • Either render it or drop it from the JSON.
  • Hand-entered benchmark metadata.
    • Hardware and GoVersion are hand-entered, so nothing ties parse_ns_per_op to a real benchmark run.
    • A short README in scripts/gen-readme-metrics/ on how to refresh metrics.json would help.
    • Please check that go_version matches the toolchain that produced the numbers (go.mod/CLAUDE.md say 1.26+).
  • --version with --check. --version overrides the tag, and doc.go is still checked against it. That is probably what you want, but say so in the usage comment.
  • replaceBlock pairing. It pairs the first :start with the first :end, and j < i is the only guard. A duplicated marker pair would be handled half-way.
  • docs/*.md is not recursive. Confirm that subdirectories are meant to be skipped.
  • fetch-tags: true. It is redundant with fetch-depth: 0. This is harmless.
  • Workflow hygiene. Consider an explicit permissions: contents: read on the new job.

Tests

The banner, block and rounding cases are good, including the v? fix.

Missing cases:

  • A pre-release suffix such as -beta on syncBanners, which locks in the behaviour from item 3.
  • An already-current, idempotent case such as **Version**: v1.15.0 | x.
  • table() output.
  • latestTag filtering. This is the part most likely to break, since it skips -rc tags and relies on --sort=-v:refname. Extract the filter into a pure function to test it.
  • A check-mode test over a temp dir. run() reads relative paths and calls flag.Parse, so it needs a small refactor first.

Security

No concerns. There is no user input and no network access, and exec.Command uses fixed arguments.

Verdict: approve once item 1 is decided. Items 2 to 4 are nice-to-haves.
· branch chore/readme-metrics-generation

Ajit Pratap Singh and others added 3 commits October 15, 2026 00:24
Make the v prefix optional so the API_REFERENCE.md footer banner is synced
and --check can catch it. Adds a test case and regenerates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkzVu3JY9Y6BRWfjesWts9
…etrics

Regenerate docs/TROUBLESHOOTING.md, COMPARISON.md and PERFORMANCE_TUNING.md
so no doc banner disagrees with the latest tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkzVu3JY9Y6BRWfjesWts9
@ajitpratap0
ajitpratap0 force-pushed the chore/readme-metrics-generation branch from 1ba5dad to c4eb175 Compare October 14, 2026 18:54
@vercel

vercel Bot commented Oct 14, 2026

Copy link
Copy Markdown

Deployment failed for project gosqlx-website with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/coolajitpratapsingh0-9932s-projects?upgradeToPro=build-rate-limit

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 20s —— View job


Review of PR #710: README metrics generation and version banner checks

I read scripts/gen-readme-metrics/main.go. I did not run the generator or the tests, and I did not re-read the workflow, docs or test diffs. The design is small and sensible, and I found no blockers. These points carry over from the earlier reviews and I confirmed them against the code.

Needs a decision

  1. Post-tag red CI (lint.yml, main.go:97-108). latestTag() returns the newest git tag. The CLAUDE.md release flow tags only after the docs PR merges. So docs-drift fails on every open PR and on main until someone updates the docs and doc.go. Pick one:
    • Compare against the newest doc.go history entry instead of the tag.
    • Run the job only on push to main and on docs/v* branches.
    • Keep it, and add "run task docs:metrics right after tagging" to the Release Workflow section of CLAUDE.md. This is the minimum.

Bugs and robustness
2. Pre-release suffixes (main.go:80-81). Patterns 2 and 3 are not end-anchored, so **Version**: v1.2.3-beta becomes v1.15.0-beta. Pattern 1 also rewrites any H1 ending in (vX.Y.Z). This is fine for the current docs.
3. Silent drift for new banner formats. A docs/*.md file that matches no pattern is never flagged. A cheap guard would be to fail if a line matching (?i)^\**(current )?version was not rewritten by any pattern.
4. pkg/gosqlx/doc.go check (main.go:192-199).

  • It does nothing today, because the file has no banner.
  • A missing file is silently ignored.
  • The (?mi) regex ^//\s*version:? would also match prose such as // Version 2 of the API… and fail hard.
  • Either remove the check or anchor it to Version:\s*v?\d. Compile the regex once at package level.
  1. Write order. In non-check mode the files are written first and the doc.go check runs afterwards. A mismatch exits 1 with the tree already modified. Running the doc.go check first is cleaner.
  2. --version with --check. --version also drives the doc.go comparison. Say so in the usage comment.
  3. replaceBlock (main.go:61-74). It pairs the first start marker with the first end marker. A duplicated pair would be handled half-way. This is low priority.
  4. docs/*.md is not recursive. Confirm that is intended.

Quality

  • stage in metrics.json is parsed but never used. The table hard-codes "Parse stage only". Use the field or drop it.
  • performance_baselines.json and "Complex queries take longer" are baked into the format string at main.go:49. That is hidden coupling.
  • hardware, go_version and parse_ns_per_op are entered by hand. Nothing ties them to a real run. A short README in scripts/gen-readme-metrics/ on how to refresh them would help. Check that go_version matches the toolchain that produced the numbers (go.mod and CLAUDE.md say 1.26+).
  • lint.yml:
    • fetch-tags: true is redundant with fetch-depth: 0, but harmless.
    • Add permissions: contents: read to the new job.
    • Keep go-version consistent with the other jobs.

Tests
Banner, block and rounding cases are covered, including the v? fix and the **Version**: forms. Missing:

  • syncBanners with a -beta suffix, to lock in the behaviour from item 2.
  • An already-current line, to confirm idempotency.
  • table() and inline() output.
  • latestTag filtering. Extract a pure pickTag([]string) function, because skipping -rc tags is the part most likely to break.
  • A --check test over a temp dir. run() uses relative paths and the global flag set, so it needs a small refactor to take a root and args.

Security
No concerns. There is no user input and no network access, and exec.Command uses fixed arguments.

Verdict: Fine to merge once item 1 is decided. Items 2 to 5 are worth doing if cheap. The rest are nice-to-haves.
· branch chore/readme-metrics-generation

This branch was successfully deployed

1 active (outdated) deployment
Preview — 1ba5dad9 Deployed Oct 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(docs): generate README metrics block and version-banner drift check in CI [1.17b]

1 participant