Repository navigation
Build PyPI wheels for the CLI in release-build #7704
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
bcf6dd5
Build PyPI wheels for the CLI in release-build
denik f82ce57
Add acceptance test running a built CLI wheel through uvx
denik 5b7776f
Move CLI wheel builder to tools/ and fix pypi-wheel test output
denik 5126238
Drop checksum verification from the CLI wheel builder
denik 050f0bc
CLI wheels: tag macOS 13, emit Metadata-Version 2.4, align action pins
denik 9b9c675
pypi-wheel test: fail if uv installs another platform's wheel; uv 0.8…
denik 06abe51
pypi-wheel test: record archives and wheel contents; skip gzip in fak…
denik ae1ecec
build_cli_wheels: print wheel paths with forward slashes
denik 58180a2
Validate CLI wheels with packaging in tests and release-build
denik 81c7826
CLI wheels: pin packaging in tools/requirements.txt; smoke test lets …
denik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| #!/usr/bin/env python3 | ||
| """Fake goreleaser output for the CLI wheel builder. | ||
|
|
||
| Only the archive for the platform running the test holds the CLI binary. The | ||
| others hold a placeholder that cannot be executed, so running the installed | ||
| `databricks` fails if uv picks another platform's wheel. | ||
| """ | ||
|
|
||
| import io | ||
| import platform | ||
| import sys | ||
| import tarfile | ||
| import zipfile | ||
| from pathlib import Path | ||
|
|
||
| GOOS = {"Linux": "linux", "Darwin": "darwin", "Windows": "windows"} | ||
| GOARCH = {"x86_64": "amd64", "amd64": "amd64", "aarch64": "arm64", "arm64": "arm64"} | ||
|
|
||
| cli, out = Path(sys.argv[1]), Path(sys.argv[2]) | ||
| host = (GOOS[platform.system()], GOARCH[platform.machine().lower()]) | ||
| placeholder = b"placeholder for a binary of another platform\n" | ||
|
|
||
| out.mkdir() | ||
| for goos in ("linux", "darwin", "windows"): | ||
| for goarch in ("amd64", "arm64"): | ||
| data = cli.read_bytes() if (goos, goarch) == host else placeholder | ||
| if goos == "windows": | ||
| path = out / f"databricks_cli_1.2.3_{goos}_{goarch}.zip" | ||
| with zipfile.ZipFile(path, "w") as z: | ||
| z.writestr("databricks.exe", data) | ||
| else: | ||
| path = out / f"databricks_cli_1.2.3_{goos}_{goarch}.tar.gz" | ||
| # Compression only costs test time: the builder reads the archive either way. | ||
| with tarfile.open(path, "w:gz", compresslevel=0) as t: | ||
| info = tarfile.TarInfo("databricks") | ||
| info.size = len(data) | ||
| info.mode = 0o755 | ||
| t.addfile(info, io.BytesIO(data)) | ||
| # Only the file name is used: it carries the version. | ||
| (out / "databricks_cli_1.2.3_SHA256SUMS").touch() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
|
|
||
| >>> python3 make_archives.py [CLI] archives | ||
|
|
||
| >>> ls archives | ||
| databricks_cli_1.2.3_SHA256SUMS | ||
| databricks_cli_1.2.3_darwin_amd64.tar.gz | ||
| databricks_cli_1.2.3_darwin_arm64.tar.gz | ||
| databricks_cli_1.2.3_linux_amd64.tar.gz | ||
| databricks_cli_1.2.3_linux_arm64.tar.gz | ||
| databricks_cli_1.2.3_windows_amd64.zip | ||
| databricks_cli_1.2.3_windows_arm64.zip | ||
|
|
||
| >>> python3 [TESTROOT]/../tools/build_cli_wheels.py archives wheels [TESTROOT]/../LICENSE | ||
| wheels/databricks-1.2.3-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl | ||
| wheels/databricks-1.2.3-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl | ||
| wheels/databricks-1.2.3-py3-none-macosx_13_0_x86_64.whl | ||
| wheels/databricks-1.2.3-py3-none-macosx_13_0_arm64.whl | ||
| wheels/databricks-1.2.3-py3-none-win_amd64.whl | ||
| wheels/databricks-1.2.3-py3-none-win_arm64.whl | ||
|
|
||
| >>> uv run -q --no-project --with packaging python [TESTROOT]/../tools/validate_cli_wheels.py wheels | ||
| valid: wheels/databricks-1.2.3-py3-none-macosx_13_0_arm64.whl | ||
| valid: wheels/databricks-1.2.3-py3-none-macosx_13_0_x86_64.whl | ||
| valid: wheels/databricks-1.2.3-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl | ||
| valid: wheels/databricks-1.2.3-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl | ||
| valid: wheels/databricks-1.2.3-py3-none-win_amd64.whl | ||
| valid: wheels/databricks-1.2.3-py3-none-win_arm64.whl | ||
|
|
||
| >>> python3 show_wheels.py wheels | ||
| == databricks-1.2.3-py3-none-macosx_13_0_arm64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-macosx_13_0_arm64 | ||
| == databricks-1.2.3-py3-none-macosx_13_0_x86_64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-macosx_13_0_x86_64 | ||
| == databricks-1.2.3-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-manylinux_2_17_aarch64 | ||
| Tag: py3-none-manylinux2014_aarch64 | ||
| Tag: py3-none-musllinux_1_1_aarch64 | ||
| == databricks-1.2.3-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-manylinux_2_17_x86_64 | ||
| Tag: py3-none-manylinux2014_x86_64 | ||
| Tag: py3-none-musllinux_1_1_x86_64 | ||
| == databricks-1.2.3-py3-none-win_amd64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks.exe | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-win_amd64 | ||
| == databricks-1.2.3-py3-none-win_arm64.whl | ||
| 100755 databricks-1.2.3.data/scripts/databricks.exe | ||
| 100644 databricks-1.2.3.dist-info/METADATA | ||
| 100644 databricks-1.2.3.dist-info/WHEEL | ||
| 100644 databricks-1.2.3.dist-info/licenses/LICENSE | ||
| 100644 databricks-1.2.3.dist-info/RECORD | ||
| Wheel-Version: 1.0 | ||
| Generator: databricks-cli build_cli_wheels.py | ||
| Root-Is-Purelib: false | ||
| Tag: py3-none-win_arm64 | ||
| == METADATA (same in every wheel) | ||
| Metadata-Version: 2.4 | ||
| Name: databricks | ||
| Version: 1.2.3 | ||
| Summary: Databricks CLI | ||
| Project-URL: Source, https://github.com/databricks/cli | ||
| Project-URL: Documentation, https://docs.databricks.com/dev-tools/cli/ | ||
| License-File: LICENSE | ||
| Description-Content-Type: text/markdown | ||
|
|
||
| # Databricks CLI | ||
|
|
||
| Command-line interface for Databricks. This package installs the prebuilt `databricks` binary. | ||
|
|
||
| Run it with `uvx databricks` or `pipx run databricks`, or install it with `uv tool install databricks`. | ||
|
|
||
| Documentation: https://docs.databricks.com/dev-tools/cli/ | ||
| Source: https://github.com/databricks/cli | ||
|
|
||
| >>> uvx -q --offline --no-index --find-links wheels --from databricks==1.2.3 databricks --version | ||
| Databricks CLI v[CLI_VERSION] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Package the CLI under test the way release-build does, then run it through uvx | ||
| # to check that a wheel built by build_cli_wheels.py is usable by Python tooling. | ||
| trace python3 make_archives.py $CLI archives | ||
| trace ls archives | ||
| trace python3 $TESTROOT/../tools/build_cli_wheels.py archives wheels $TESTROOT/../LICENSE | ||
| # Offline, so packaging comes from the vendored packages rather than the | ||
| # tools/requirements.txt pin (they can differ after a Dependabot bump). | ||
| trace uv run -q --no-project --with packaging python $TESTROOT/../tools/validate_cli_wheels.py wheels | ||
| trace python3 show_wheels.py wheels | ||
|
|
||
| # uv must pick the wheel for this platform: the other wheels hold a placeholder | ||
| # instead of the binary (see make_archives.py), so a wrong pick fails here. | ||
| trace uvx -q --offline --no-index --find-links wheels --from databricks==1.2.3 databricks --version | ||
|
|
||
| rm -fr archives wheels |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| #!/usr/bin/env python3 | ||
| """Print what each wheel in a directory installs, in a platform-independent form. | ||
|
|
||
| RECORD is not printed: its hashes cover the CLI binary, which differs per build. | ||
| """ | ||
|
|
||
| import sys | ||
| import zipfile | ||
| from pathlib import Path | ||
|
|
||
| wheels = sorted(Path(sys.argv[1]).glob("*.whl")) | ||
| for whl in wheels: | ||
| print(f"== {whl.name}") | ||
| with zipfile.ZipFile(whl) as z: | ||
| for info in z.infolist(): | ||
| print(f"{info.external_attr >> 16:o} {info.filename}") | ||
| wheel_file = next(n for n in z.namelist() if n.endswith(".dist-info/WHEEL")) | ||
| print(z.read(wheel_file).decode(), end="") | ||
|
|
||
| with zipfile.ZipFile(wheels[0]) as z: | ||
| metadata_file = next(n for n in z.namelist() if n.endswith(".dist-info/METADATA")) | ||
| print("== METADATA (same in every wheel)") | ||
| print(z.read(metadata_file).decode(), end="") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| Cloud = false | ||
|
|
||
| # uvx may need to create a Python environment. | ||
| Timeout = '180s' | ||
| TimeoutWindows = '240s' |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
intentionally not used and different from the script in release-build.yml?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is used:
test-cli-wheelsis inchecks(Taskfile.yml, thecheckslist), which runs in check.yml. It is different on purpose: the task unit-tests the builder and validator on fake input, while release-build runs the two scripts on the real release binaries. Clarified that in the task, and both now use the sameuv runoptions and the same requirements file.