Skip to content

[air] Package code_source without git on PATH - #7625

Open
stevenchen-db wants to merge 3 commits into
mainfrom
schen/air-code-source-without-scm
Open

stevenchen-db wants to merge 3 commits into
mainfrom
schen/air-code-source-without-scm

Conversation

@stevenchen-db

Copy link
Copy Markdown
Contributor

Changes

databricks air run no longer needs git on PATH to package a code_source directory that isn't in a git repository.

  • Outside a repository, no git: the directory is packaged with libs/git's ignore rules (root and nested .gitignore, info/exclude, core.excludesFile). Symlinks are listed, not followed; a nested repository is one entry, as with git.
  • Inside a repository, or code_source.git set, no git: fails before validation, packaging, or any API call, with an actionable error.
  • git on PATH: the existing git-based listing is unchanged.
 $ PATH=<no git> databricks air run -f workload.yaml --dry-run    # code_source.root_path: . (not a repo)
-Error: failed to initialize temporary git metadata: exec: "git": executable file not found in $PATH:
+Dry run: code_source snapshot would include 2 file(s).

 $ PATH=<no git> databricks air run -f workload.yaml --dry-run    # code_source inside a git repo
-Error: failed to initialize temporary git metadata: exec: "git": executable file not found in $PATH:
+Error: code_source at /path/to/repo is in a git repository, so packaging it needs git on PATH; install git and retry

Known differences of the git-free listing (only used without git; pinned by tests): a root rule wins over a nested negation (*.log at the root excludes nested/keep.log even with !keep.log in nested/.gitignore), and include_paths globs use path.Match, where * doesn't cross /.

Why

Hosts without git (containers, CI images, agent sandboxes) couldn't use code_source at all, though git is only needed for its ignore rules outside a repository. The quickstart lists git as a requirement; a docs follow-up can narrow that to code inside a git repository.

No Jira ticket.

Tests

  • Unit: the git-free listing matches git ls-files for whole trees, directory and file include_paths, nested .gitignore, negation, ignored dirs, dotfiles and symlinks; nested repo as one entry; the two known differences; the early error for in-repo and pinned code (with an injected git lookup, also asserting no lookup without code_source).
  • Real workspace, no git on PATH: a code_source run succeeded and printed its output (run 892749685192587); the in-repo case failed with 0 API calls; a workload without code_source is unchanged. With git on PATH, the dry-run file count and request list match main.
  • TestAccept/air/ 20/20, lint clean, ./task checks passes. TestLogsCommandRepeatsRunLinkAfterStreaming fails the same way on main and is unrelated.

This pull request and its description were written by Isaac.

`air run` packaged every code_source with the git binary, so a host without git
failed with a raw exec error. A directory outside any repository now packages
without git: files come from libs/git's ignore rules, regular files and symlinks
are listed (symlinks not followed), and a nested repository is one "dir/" entry.
With git on PATH the git listing is unchanged.

Known differences of the git-free listing, only used without git:
- Rule precedence: the first matching rule from the root down wins, so a root
  "*.log" excludes nested/keep.log even with "!keep.log" in nested/.gitignore.
- include_paths globs use path.Match, where "*" does not cross "/".

A code_source inside a repository, or pinned with code_source.git, still needs
git. Without it, `air run` fails before config validation and packaging with an
actionable error.

Co-authored-by: Isaac <no-reply@databricks.com>
@stevenchen-db
stevenchen-db requested review from a team as code owners November 14, 2026 00:24
@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 14, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: e44a0bf

Run: 43917217074

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 8:53
💚​ aws-windows-latest-4core-16gb 1 272 13 7:34
💚​ azure linux-2core-8gb 1 269 15 8:32
💚​ azure-windows-latest-4core-16gb 1 271 13 7:17
💚​ gcp linux-2core-8gb 1 270 15 8:53
🔄​ gcp-windows-latest-4core-16gb 1 1 271 13 8:55
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncNestedSpacePlusAndHashAreEscapedSync ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
Top 11 slowest tests (at least 2 minutes):
duration env testname
3:50 azure-windows-latest-4core-16gb TestAccept
3:34 gcp-windows-latest-4core-16gb TestAccept
3:32 aws-windows-latest-4core-16gb TestAccept
3:00 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:58 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:36 azure-windows-latest-4core-16gb TestImportDirWithOverwriteFlag
2:25 gcp-windows-latest-4core-16gb TestFilerReadWrite/workspace_files
2:20 aws-windows-latest-4core-16gb TestImportDirDoesNotOverwrite
2:20 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:06 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:06 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsDelete

git ls-files lists only regular files and symlinks, and packaging a FIFO
would block. Also sharpen the fallback's doc comments.

Co-authored-by: Isaac <no-reply@databricks.com>
}

var names []string
err = filepath.WalkDir(repoPath, func(p string, d fs.DirEntry, err error) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If root_path is a symlink to a directory, WalkDir Lstats the root and never descends, so this returns no entries and the user sees the "directory is empty or all of its files are excluded by .gitignore" error. git -C follows the link, so the git path packages it fine. Could we filepath.EvalSymlinks(repoPath) before the walk? The emitted names are relative, so nothing else changes. A test with a symlinked root would pin it.

return true
}
for p := strings.TrimSuffix(name, "/"); p != ""; {
if ok, _ := path.Match(spec, p); ok {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include_paths: ["src/*.py"] selects src/x/y.py under git, since git's pathspec * crosses /. Here path.Match's * doesn't cross / and the ancestor loop only tries prefixes, so the file is dropped without any message. The doc comment calls this a known difference, but on a packaging path a silent omission is worse than an error. Could we either match git's semantics, or fail in the fallback when include_paths contain glob metacharacters? Either way, a row in TestSnapshotFilesWithoutGit_MatchesGitListing would catch it.

//
// Known differences from `git ls-files --others --exclude-standard`, which only matter
// on hosts without git (with git on PATH the git listing is used unchanged):
// - Rule precedence: libs/git returns the first matching rule from the root down, so

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root *.log plus nested !keep.log silently diverges: git includes nested/keep.log, while Repository.Ignore returns on the first positive match from the root down. The fixture in writeFallbackFixture has no conflicting root rule, so this is untested. Could we evaluate the deepest .gitignore first, or at least log a one-line warning when the fallback runs in a tree with nested .gitignore files, so users know the snapshot may differ from what git would select?

return false
}

// emptySnapshotError explains why no file was selected for the snapshot. An empty

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: in the non-repo branch below, the hint suggests git check-ignore -v, which the git-free path can't run, and it's shown whether or not git is installed. Could that branch give a git-agnostic hint, such as which .gitignore file to check?

@@ -0,0 +1,23 @@
//go:build unix

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Windows CI skips this whole file, so the fallback's separator handling (ToSlash on rel names, path.Match on include paths) has no Windows coverage. Would a case in TestSnapshotFilesWithoutGit_MatchesGitListing that runs on every platform be feasible?

Comment thread cmd/air/run.go

// checkCodeSourceGit fails before the config is validated or anything is packaged when
// git is missing and the code_source cannot be packaged without it. Runs for dry runs too.
func checkCodeSourceGit(ctx context.Context, cfg *runConfig, configPath string) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this re-runs resolveRootPath and gitRequiredError, which resolveSnapshotPlan repeats right after. The early check is justified by running before validation, so a short comment saying so would keep the duplication from looking accidental.

@pardis-beikzadeh-db pardis-beikzadeh-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The listing used when git isn't installed can be simplified a lot. Please fold these in when you rebase on main.


// matchesIncludePath reports whether name, or a directory containing it, matches one of
// the include_paths.
func matchesIncludePath(name string, includePaths []string) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building on Maggie's glob thread on line 320: if the fallback rejects glob include_paths (isPathspecPattern on main already detects them), it can walk each include_paths entry directly, the way libs/fileset.FileSet does. Then this function and the *.py known-difference test can go.

if ignored {
return fs.SkipDir
}
if hasGitMetadata(p) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On main, snapshotFiles drops directory entries (#7643), so after the rebase this dir/ entry never reaches the archive. Returning fs.SkipDir here is enough.

Comment thread cmd/air/snapshot_git.go

// insideGitRepository is the git-free stand-in for isRepository, used only when no git
// binary is available: repository metadata in dir or one of its ancestors marks a work tree.
func insideGitRepository(dir string) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

folders.FindDirWithLeaf(dir, git.GitDirectoryName) already does this upward walk; libs/git/info.go uses it the same way. FindDirWithLeaf treats an empty .git directory as a repository, which is fine here, so hasGitMetadata and its test can go.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants