Skip to content

Check that the CLI's Go SDKs match the releases its generated files describe - #7823

Open
parthban-db wants to merge 1 commit into
mainfrom
parthban/check-go-sdks
Open

parthban-db wants to merge 1 commit into
mainfrom
parthban/check-go-sdks

Conversation

@parthban-db

@parthban-db parthban-db commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Adds task check-go-sdks, which runs in task checks and so in CI. It fails when the CLI's two Go SDKs and its generated files don't describe the same release:

  • .codegen/_openapi_sha, which .codegen/cli.json was generated from, must be the Universe commit that the classic Go SDK in go.mod was released from (its .codegen/_last_sha).
  • climod/.codegen/_last_sha names the Universe commit of the sdk-go release that climod/ was generated from. That must be a published (tagged) sdk-go release, and every github.com/databricks/sdk-go/... module in go.mod must be at its version in that release: the top release in the package's CHANGELOG.md at the release commit, whose tag must exist, read and tagged the way genkit apply-tag does.

Until climod/ is first generated, it has no _last_sha and only the first check runs, so the check passes on main today.

Why

The CLI uses both Go SDKs, so their versions and the generated cli.json and climod/ have to describe the same API. The daily Go SDK update keeps them aligned: https://github.com/databricks-eng/universe/pull/3168251 makes it generate climod/ from the newest sdk-go release at or before the classic release's commit and record that commit in climod/.codegen/_last_sha, and https://github.com/databricks-eng/universe/pull/3170943 makes it pin the sdk-go modules to that release the same way. This check catches a change that breaks the alignment, such as bumping an sdk-go module by hand. It can't check that the sdk-go release is at or before the classic release's commit, because that needs Universe's history; the update enforces it.

Tests

  • tools/test_check_go_sdks.py runs against a local repository: finding the tagged release for a commit skips an untagged release and a tagged commit without the marker; matching versions pass; a version that differs from the release's, a module whose release version has no tag yet, and a marker with no release fail.
  • ./task check-go-sdks passes on this branch.
  • With climod/.codegen/_last_sha set to the Universe commit of sdk-go's Nov 18 release, the check reports that go.mod pins files v0.3.0 while that release has v0.8.0, and passes auth, core and options. With a commit that no release came from, it reports that.

This pull request and its description were written by Isaac.

@parthban-db
parthban-db requested a review from a team as a code owner November 19, 2026 18:14

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 978b9e7. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds task check-go-sdks, a Python CI check that validates the CLI's Go SDK versions match the releases its generated files describe. The classic-SDK half of the check (comparing .codegen/_openapi_sha with the SDK's _last_sha) duplicates TestConsistentDatabricksSdkVersion, which already runs in CI. The sdk-go module half is well-structured but currently gated behind a file that doesn't exist yet. The implementation is correct; the duplication should be resolved.

Comment thread tools/check_go_sdks.py
classic_sha = f.read().strip()
with open(".codegen/_openapi_sha") as f:
openapi_sha = f.read().strip()
if openapi_sha != classic_sha:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The openapi_sha != classic_sha comparison checks the same invariant as TestConsistentDatabricksSdkVersion in internal/build/sdk_consistency_test.go (lines 20–52), which already runs in CI through test-unit-root. Both read the databricks-sdk-go version from go.mod, obtain its .codegen/_last_sha, and compare it with .codegen/_openapi_sha — the Go test via an HTTP fetch, this script via go mod download. Can one of them be removed, or does having both serve a purpose the existing test doesn't?

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 19288b1

Run: 44640203824

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 19 4:08
💚​ aws-windows-latest-4core-16gb 1 274 17 4:05
💚​ azure linux-2core-8gb 1 271 19 3:41
💚​ azure-windows-latest-4core-16gb 1 273 17 4:35
💚​ gcp linux-2core-8gb 1 272 19 3:59
💚​ gcp-windows-latest-4core-16gb 1 274 17 4:08
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 5 slowest tests (at least 2 minutes):
duration env testname
4:34 azure-windows-latest-4core-16gb TestAccept
4:07 gcp-windows-latest-4core-16gb TestAccept
4:04 aws-windows-latest-4core-16gb TestAccept
2:10 aws linux-2core-8gb TestAccept
2:01 gcp linux-2core-8gb TestAccept

@parthban-db
parthban-db force-pushed the parthban/check-go-sdks branch from 978b9e7 to a1c21a0 Compare November 19, 2026 20:34
…escribe

Co-authored-by: Isaac <no-reply@databricks.com>
@parthban-db
parthban-db force-pushed the parthban/check-go-sdks branch from a1c21a0 to 19288b1 Compare November 19, 2026 23:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants