Skip to content

air: suggest a working --output-dir when convert-to-dabs code is outside the bundle - #7640

Closed
dimk0 wants to merge 2 commits into
databricks:mainfrom
dimk0:air-convert-to-dabs-outside-bundle
Closed

dimk0 wants to merge 2 commits into
databricks:mainfrom
dimk0:air-convert-to-dabs-outside-bundle

Conversation

@dimk0

@dimk0 dimk0 commented Nov 15, 2026

Copy link
Copy Markdown
Member

Changes

  • air convert-to-dabs: when code_source.root_path resolves outside the bundle directory, the error names the resolved code directory and, for working-tree snapshots, the --output-dir that converts the workload (the code directory's parent). It no longer suggests setting sync.paths in databricks.yml.
  • Git-pinned snapshots still get only "move the code inside the bundle directory", matching the rule for the bundle-root case in air: explain root_path and default retries in convert-to-dabs #7579.

Why

The getting-started Ray recipes in the AI Runtime cookbook snapshot their parent directory (root_path: ..) to include a shared bootstrap. Running convert-to-dabs workload.yaml from the recipe directory failed with advice to set sync.paths in a databricks.yml that the command had not written, and the generated bundle only syncs generated_artifacts while the tgz artifact packages code inside the bundle anyway. The conversion works with --output-dir ../.., which no message mentioned.

Tests

  • TestConvertToDabsCodeOutsideBundleSuggestsOutputDir reproduces the Ray layout: the error names --output-dir <parent> and not sync.paths, and converting with that directory succeeds with the expected artifact include.
  • TestConvertToDabsGitPinOutsideBundleDoesNotSuggestOutputDir checks git-pinned snapshots get no --output-dir hint.
  • The first test fails without the change; TestConvertToDabsRejectsCodeOutsideBundle passes unchanged. go test ./cmd/air/... and go test ./acceptance -run TestAccept/air pass; golangci-lint run ./cmd/air/... reports no issues.

This pull request and its description were written by Isaac.

dimk0 and others added 2 commits November 14, 2026 23:49
…ide the bundle

When code_source.root_path resolved outside the bundle directory, for example
the getting-started Ray recipes that snapshot their parent directory with
root_path: .., convert-to-dabs suggested setting sync.paths in a databricks.yml
that it had not written. The error now names the resolved code directory and,
for working-tree snapshots, the --output-dir value that converts the workload.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! This pull request was opened from a fork.

If you're a member of the Databricks organization, please request access to the cli-contrib team and open your pull request from a branch in this repository instead of a fork — fork PRs don't run integration tests automatically and have reduced CI access.

If you're an external contributor, no action is needed; a maintainer will review your pull request.

@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 7640
  • Commit SHA: 54a6caddabc2ac1e7459b924b200c262632366ba

Checks will be approved automatically on success.

@dimk0

dimk0 commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

Superseded by #7644, which uses a branch in this repository so CI can run.

@dimk0 dimk0 closed this Nov 15, 2026
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…ide the bundle (databricks#7644)

## Changes
- `air convert-to-dabs`: when `code_source.root_path` resolves outside
the bundle directory, the error names the resolved code directory and,
for working-tree snapshots, the `--output-dir` that converts the
workload: the code directory's parent, quoted for the user's shell
(cmd.exe or PowerShell on Windows, a POSIX shell elsewhere). It no
longer suggests setting `sync.paths` in a `databricks.yml` it has not
written.
- When that parent already holds a bundle file (`databricks.yml`,
`databricks.yaml`, `bundle.yml` or `bundle.yaml`), the error says so
instead of suggesting it. Sibling workloads that share a code directory
would otherwise each be sent there and replace one another's bundle.
- A parent path that no quoting keeps literal, such as one with a
control character, gets generic wording instead of a printed path.
- `quoteArg` moves from `cmd/aitools` to `libs/shellquote` as `Arg`,
unchanged apart from the name, so both commands quote printed paths the
same way.
- Git-pinned snapshots still get only "move the code inside the bundle
directory", matching the rule for the bundle-root case in databricks#7579.

## Why
The getting-started Ray recipes in the AI Runtime cookbook snapshot
their parent directory (`root_path: ..`) to include a shared bootstrap.
Running `convert-to-dabs workload.yaml` from a recipe directory failed
with advice to set `sync.paths` in a `databricks.yml` the command had
not written; the conversion works with `--output-dir ../..`, which no
message mentioned. Four recipes share that parent, so the hint must not
steer a second conversion into the first one's bundle.

Replaces databricks#7640, opened from a fork where CI could not get its OIDC
token.

## Tests
- `TestConvertToDabsCodeOutsideBundleSuggestsOutputDir`: the error names
the resolved code directory, `--output-dir <parent>` and the fallback,
not `sync.paths`; converting with that directory succeeds with the
expected artifact `path` and `include`.
- `TestConvertToDabsCodeOutsideBundleSkipsParentWithBundle`: for each
bundle file name, the error says the parent holds a bundle and does not
suggest it.
- `TestConvertToDabsCodeOutsideBundleQuotesOutputDir`: a parent path
with a space is quoted for the platform's shell.
- `TestConvertToDabsGitPinOutsideBundleDoesNotSuggestOutputDir`:
git-pinned snapshots get only the move-the-code remedy. `TestArg` covers
the moved helper.
- Removing the bundle-file check or the quoting makes the corresponding
test fail.
- Ran the built CLI on a copy of the cookbook's Ray recipes: Ray Core
suggests the parent and converts there, and Ray Data then reports the
existing bundle.
- `acceptance/air/convert-to-dabs` adds a workload whose `root_path` is
outside its directory. Converted before the in-place bundle exists, the
error suggests `--output-dir` with the code directory's parent;
converted after it, the error reports the parent's `databricks.yml`.
Both cases fail against `main`'s `convert_to_dabs.go`, which prints the
old `sync.paths` advice.
- `go test ./cmd/air/... ./libs/shellquote/...`, `go test ./acceptance
-run TestAccept/air`, and `./task test-exp-aitools` pass, and
`golangci-lint run` reports no issues on the changed packages.
- Not covered here: re-running a workload whose own bundle is in the
parent is told to use another directory instead of `--force`, and
`--force` still replaces a bundle with a different `bundle.name`. Both
belong to a follow-up that compares bundle names. Git-pinned snapshots
also still get only the move-the-code remedy, although `--output-dir`
with the parent works for them unless the code directory is the
repository's top level.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant