Skip to content

Build PyPI wheels for the CLI in release-build - #7166

Closed
denik wants to merge 4 commits into
mainfrom
denik/databricks-wheel
Closed

denik wants to merge 4 commits into
mainfrom
denik/databricks-wheel

Conversation

@denik

@denik denik commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Build py3-none-<platform> wheels for the databricks PyPI project from the signed release binaries and upload them as a separate cli-wheels artifact. Nothing is published by this PR: the publish side is https://github.com/databricks/secure-public-registry-releases-eng/pull/272.

  • .github/scripts/build_cli_wheels.py wraps the 6 goreleaser archives (checksum-verified) into wheels that install only the binary into bin//Scripts/. No Python package, so uvx databricks runs the binary directly.
  • New cli-wheels job in release-build.yml: builds, twine check, installs the linux wheel and runs databricks --version, then uploads. The existing wheel (databricks-bundles) artifact is untouched.
  • Unit test wired into ./task checks.

This pull request and its description were written by Isaac.

Wrap the signed release binaries into py3-none-<platform> wheels for the
`databricks` PyPI project and upload them as the `cli-wheels` artifact,
separate from the `wheel` (databricks-bundles) artifact.

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 0bc9dac

Run: 39990791933

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 16 4:04
✅​ aws-windows-latest-4core-16gb 283 14 3:09
✅​ azure linux-2core-8gb 280 16 3:57
🔄​ azure-windows-latest-4core-16gb 4 278 14 3:37
✅​ gcp linux-2core-8gb 281 16 4:13
✅​ gcp-windows-latest-4core-16gb 283 14 4:20
Test Name azure-windows-latest-4core-16gb
🔄​ TestFsCpDirToDirWithOverwriteFlag 🔄​f
🔄​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_uc-volumes 🔄​f
🔄​ TestFsMkdirCreatesIntermediateDirectories 🔄​f
🔄​ TestFsMkdirCreatesIntermediateDirectories/uc-volumes 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:16 gcp-windows-latest-4core-16gb TestAccept
3:05 azure-windows-latest-4core-16gb TestAccept
2:55 aws-windows-latest-4core-16gb TestAccept

Package the CLI under test with build_cli_wheels.py and run it with
`uvx --find-links`, so uv picks the wheel for this platform and the test
fails if a wheel is not installable or does not run.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 0bc9dac56 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=0 unreleased=0 coverage=1 failing=0

test branch main (edebd84) latest (v1.22.1)
TestAccept/cmd/pypi-wheel ✅ ✅ ✅

denik and others added 2 commits October 23, 2026 18:36
Python helpers live in tools/. Also drop `uniq -c` from the acceptance
test: its padding differs between macOS and Linux/Windows.

Co-authored-by: Isaac <no-reply@databricks.com>
The archives come from the cli artifact of the same workflow run, so the
extra check adds nothing. The SHA256SUMS file name still carries the version.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

This PR has not received an update in a while. If you want to keep this PR open, please leave a comment below or push a new commit and auto-close will be canceled.

@denik

denik commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Continued in #7704 (rebased, plus macOS 13 tag and Metadata-Version 2.4 fixes).

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
Build `py3-none-<platform>` wheels for the `databricks` PyPI project
from the release-build `cli` archives and upload them as a separate
`cli-wheels` artifact. Nothing is published by this PR: the publish side
is
databricks/secure-public-registry-releases-eng#272.
Replaces databricks#7166, which was closed as stale.

- `tools/build_cli_wheels.py` wraps the 6 goreleaser archives into
wheels that install only the binary into `bin/`/`Scripts/`. There is no
Python package, so `uvx databricks` runs the binary directly. Builds are
deterministic.
- macOS wheels are tagged `macosx_13_0`, because the Go 1.27 binaries
have `minos 13.0`. METADATA is 2.4, the first version that allows
`License-File`, which PyPI enforces.
- New `cli-wheels` job in `release-build.yml`: builds the wheels, runs
`twine check`, installs the linux wheel and runs `databricks --version`,
then uploads. The existing `wheel` artifact (databricks-bundles) is
untouched.
- `tools/validate_cli_wheels.py` checks each wheel with `packaging`: the
filename, METADATA (stricter than `twine check`), WHEEL tags, RECORD
hashes and contents. It also requires one wheel per platform. It runs in
`release-build` and in the acceptance test.
- Unit tests for the builder and the validator are wired into `./task
checks`. The acceptance test `cmd/pypi-wheel` runs the built wheel
through `uvx` on Linux, macOS and Windows. Only the archive for the
test's own platform holds the real binary, so the test fails if uv picks
another platform's wheel.

Part of DECO-28996.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants