Repository navigation
Conversation
For a working-tree snapshot, git ls-files silently ignores a pathspec that matches nothing, so a mistyped or stale include_paths entry was left out of the snapshot. The run then failed on the node after compute was provisioned. Commit-pinned snapshots already reject missing include_paths; working-tree snapshots now do the same before packaging. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Thanks for the contribution! This pull request was opened from a fork. If you're a member of the Databricks organization, please request access to the If you're an external contributor, no action is needed; a maintainer will review your pull request. |
Contributor
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
Member
Author
|
Superseded by #7643, which uses a branch in this repository so CI can run. |
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
…tabricks#7643) ## Changes - `air run`: for a working-tree snapshot (no `code_source.git`), reject `code_source.include_paths` entries that select no files, and name them, before packaging. Commit-pinned snapshots already reject missing include paths. Glob patterns, pathspec magic (`:!x`, `:(exclude)x`, `:/x`) and backslash escapes are left to git, which matches them as patterns. - Drop listing entries the tarball writer does not pack (directories such as a nested git repository or a submodule), so the empty and unmatched checks see what is uploaded. - Reject listed files outside `root_path`, which pathspec magic such as `:(top)x` can select. - The all-entries-empty error from databricks#7570 still fires first. Its messages now say that nested git repositories and submodules add no files, and both errors suggest `git check-ignore -v` only when `root_path` is in a git repository. ## Why `git ls-files` ignores a pathspec that matches nothing, so a mistyped or stale entry was dropped from the snapshot without a warning, `--dry-run` passed, and the run failed on the node after compute was provisioned. A realistic case is the AI Runtime cookbook's getting-started Ray recipes, which list the shared bootstrap and the recipe directory in `include_paths`: `include_paths: [ray_bootstrap.sh, my_job]` for a `my-job` directory uploaded only `ray_bootstrap.sh`. Replaces databricks#7639, opened from a fork where CI could not get its OIDC token. ## Tests - `TestSnapshotFiles_IncludePathMatching` accepts `.`, `./`, a file, `?` and `[...]` globs, and `:!`/`:(exclude)` exclusions with the expected files, and rejects `[data_v2, data]` naming only `data`. - `TestSnapshotFiles_NestedRepoSelectsNoFiles` and `TestSnapshotFiles_RejectsPathspecOutsideRoot` cover nested repositories and the containment check; `TestSnapshotFiles_AcceptsBackslashEscapedIncludePath` (not on Windows) and `TestSnapshotFiles_OmitsIgnoreRuleHintOutsideGit` cover backslash escapes and the hint. The earlier rejection and accepted-forms tests still pass. - Each behavior (the `.` shortcut, the path-component boundary, the glob, magic and backslash skips, the nested-repository skip, containment) fails its test when removed. - `acceptance/air/run` adds two dry-run cases: an `include_paths` entry that selects no files next to one that does, and a `:(top)` pathspec that selects a file outside `root_path`. Both fail against `main`'s `snapshot_package.go`, where the dry runs succeed. - `go test ./cmd/air/...` and `go test ./acceptance -run TestAccept/air` pass, and `golangci-lint run ./cmd/air/...` reports no issues. Locally `TestLogsCommandRepeatsRunLinkAfterStreaming` fails on `main` too when `DATABRICKS_WORKSPACE_ID` is exported; it passes with the variable unset. - Not covered here: for a non-git `root_path`, `ls-files` runs without `-C`, so the listing depends on the working directory. Run from a subdirectory of `root_path`, it lists the wrong files: v1.26.0 can upload an empty snapshot, and with this change an `include_paths` entry that exists can be reported as selecting no files. That predates this change; databricks#7646 adds `-C`. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
air run: for a working-tree snapshot (nocode_source.git), rejectcode_source.include_pathsentries that select no files, and name them, before packaging. Commit-pinned snapshots already reject missing include paths. Entries with glob characters are left to git, since it matches them as patterns.Why
git ls-filesignores a pathspec that matches nothing, so a mistyped or stale entry was dropped from the snapshot without a warning,--dry-runpassed, and the run failed on the node after compute was provisioned. A realistic case is the AI Runtime cookbook's getting-started Ray recipes, which list the shared bootstrap and the recipe directory ininclude_paths:include_paths: [ray_bootstrap.sh, my_job]for amy-jobdirectory uploaded onlyray_bootstrap.sh.Tests
TestSnapshotFiles_RejectsIncludePathMatchingNothingAlongsideOthers: one valid and one mistyped entry; the error names only the mistyped one.TestSnapshotFiles_RejectsIncludePathWhoseFilesAreIgnored: an entry whose files are all git-ignored is rejected with thecheck-ignorehint.TestSnapshotFiles_AcceptsIncludePathForms:./src/, a file, and a glob are accepted and select the expected files.go test ./cmd/air/...andgo test ./acceptance -run TestAccept/airpass;golangci-lint run ./cmd/air/...reports no issues.TestLogsCommandRepeatsRunLinkAfterStreamingfails onmaintoo whenDATABRICKS_WORKSPACE_IDis exported in the shell (it adds?o=<id>to the expected link); it passes with the variable unset.This pull request and its description were written by Isaac.