Skip to content

Support accelerator_pool_id in AIR workload configuration - #7698

Merged
ben-hansen-db merged 8 commits into
mainfrom
ben/accelerator-pool-id
Nov 19, 2026
Merged

ben-hansen-db merged 8 commits into
mainfrom
ben/accelerator-pool-id

Conversation

@ben-hansen-db

@ben-hansen-db ben-hansen-db commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Changes

PRD has naming alignment

Merge blocked pending backend deployment and staging E2E validation.

Rename compute.pool_id to compute.accelerator_pool_id for AIR GPU Pool selection. This is a breaking YAML change: pool_id is rejected with a pointer to accelerator_pool_id, including when supplied alongside the new selector. Preflight and submission send accelerator_pool_id; bundle conversion uses provisioned_capacity_id, which its current schema supports.

Use consistent GPU Pool naming and describe pool selection without promising backend fallback behavior. Priority validation retains the existing supported values.

Why

Align AIR workload configuration with the task-scoped GPU Pool API. See the PRD for the naming decision. Backend support must be deployed before this CLI change merges or releases.

Tests

  • Targeted AIR unit tests cover pool ID validation, YAML parsing, preflight and submission payloads, bundle conversion, priority validation, and pool commands.
  • AIR acceptance tests cover help output and rejection of the removed YAML key, unsupported priority, and priority without a pool.
  • Formatting, lint, and changelog validation pass.
  • Staging E2E validation is pending backend deployment.

This PR was written with Codex.

@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 16, 2026
@ben-hansen-db
ben-hansen-db marked this pull request as ready for review November 16, 2026 19:37
@ben-hansen-db
ben-hansen-db requested review from a team as code owners November 16, 2026 19:37
Comment thread .nextchanges/air/accelerator-pool-id.md Outdated
@@ -0,0 +1 @@
* Add `compute.accelerator_pool_id` for GPU Pool selection in AIR workload YAML, retaining `compute.pool_id` as a compatibility alias. ([#7698](https://github.com/databricks/cli/pull/7698))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking on Tuesday deployment

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 894df8440 | Workflow run

Regression test report

Selected 5 changed tests (limit=20, 0 not selected)

5 tests: regression=5 unreleased=0 coverage=0 failing=0
test branch main (b8046e0) latest (v1.27.0)
TestAccept/air/config-help ✅ ❌ ❌
TestAccept/air/convert-to-dabs ✅ ❌ ❌
TestAccept/air/help ✅ ❌ ❌
TestAccept/air/run ✅ ❌ ❌
TestAccept/air/run-submit-deps ✅ ❌ ❌
TestAccept/air/config-help: branch ✅ | main (b8046e0) ❌ | latest (v1.27.0) ❌

main (b8046e0):

acceptance_test.go:1400: Diff:
        --- air/config-help/output.txt
        +++ /tmp/TestAcceptairconfig-help978181520/002/output.txt
        @@ -113,12 +113,12 @@
             GPU_1xRTX6000 - 1 accelerator per node
         
           Fields:
        -    num_accelerators     (required) Total number of GPUs to allocate.
        -    accelerator_type     (required) Which accelerator to run on, e.g. GPU_1xA10.
        +    num_accelerators  (required) Total number of GPUs to allocate.
        +    accelerator_type  (required) Which accelerator to run on, e.g. GPU_1xA10.
         
        -  GPU Pools (optional):
        -    accelerator_pool_id  GPU Pool to run on, by ID.
        -    priority_class       Scheduling priority within the GPU Pool: NORMAL.
        +  Provisioned capacity (optional):
        +    pool_id           GPU pool to run on, by id.
        +    priority_class    Scheduling priority within the pool: BEST_EFFORT (lowest, preemptable) or NORMAL.
         
         For details on a field, run "databricks air run -h config.compute.<field>".
         
        @@ -132,15 +131,13 @@
           Required: no
         
         >>> [CLI] air run -h config.compute.accelerator_pool_id
        -config.compute.accelerator_pool_id
        -  GPU Pool to run on, by ID. Must be 1-255 characters. List available pools with 'databricks air pools list'.
        +Error: unknown config field "config.compute.accelerator_pool_id"
         
        -  Type:     string
        -  Required: no
        +fields under "config.compute" are: accelerator_type, num_accelerators, pool_id, priority_class
         
         >>> [CLI] air run -h config.compute.priority_class
         config.compute.priority_class
        -  Scheduling priority within the GPU Pool: NORMAL. Requires accelerator_pool_id.
        +  Scheduling priority within the pool: BEST_EFFORT (lowest, preemptable) or NORMAL. Requires pool_id.
         
           Type:     string
           Required: no
        @@ -231,7 +224,7 @@
         >>> [CLI] air run -h config.compute.acclerator_type
         Error: unknown config field "config.compute.acclerator_type"; did you mean "accelerator_type"?
         
        -fields under "config.compute" are: accelerator_pool_id, accelerator_type, num_accelerators, priority_class
        +fields under "config.compute" are: accelerator_type, num_accelerators, pool_id, priority_class
         
         === free-form map keys are not schema fields
         
        @@ -241,6 +234,8 @@
         === removed pool field is not in the schema
         
         >>> [CLI] air run -h config.compute.pool_id
        -Error: unknown config field "config.compute.pool_id"; did you mean "accelerator_pool_id"?
        +config.compute.pool_id
        +  GPU pool to run on, by id. A GPU pool is a pre-provisioned accelerator reservation; contact your Databricks account team to provision one. Must be 1-255 characters. List available pools with 'databricks air pools list'.
         
        -fields under "config.compute" are: accelerator_pool_id, accelerator_type, num_accelerators, priority_class
        +  Type:     string
        +  Required: no

latest (v1.27.0): same output as main (b8046e0)

TestAccept/air/convert-to-dabs: branch ✅ | main (b8046e0) ❌ | latest (v1.27.0) ❌

main (b8046e0):

acceptance_test.go:1400: Diff:
        --- air/convert-to-dabs/output.txt
        +++ /tmp/TestAcceptairconvert-to-dabs2917425222/002/output.txt
        @@ -7,229 +7,6 @@
         === convert an AIR run YAML into a DABs bundle (in place, next to the source)
         
         >>> [CLI] air convert-to-dabs train.yaml
        -Wrote a Databricks Asset Bundle to .:
        -  databricks.yml
        -  generated_artifacts/training_config.yaml
        -  generated_artifacts/command.sh
        +Error: invalid config train.yaml: line 8: unknown field "compute.accelerator_pool_id"; run "databricks air run -h config.compute" for the list of fields
         
        -To deploy and run this workload as a bundle:
        -  1. [CLI] bundle validate
        -  2. [CLI] bundle deploy
        -  3. [CLI] bundle run torchrun-a10-smoke-test --no-wait
        -
        -bundle deploy uploads the code source and launch scripts automatically.
        -To see what it deployed and where: [CLI] bundle summary
        -
        -Unlike `air run` (which submits an ephemeral run), bundle deploy creates a
        -persistent job that is not garbage-collected. When you are done, remove the
        -job and its uploaded files with:
        -  [CLI] bundle destroy
        -
        -Note: the YAML does not set max_retries, so the job uses the `air run` default
        -(max_retries: 3, retry_on_timeout: true). Edit databricks.yml to change it.
        -
        -=== emitted databricks.yml (code_source packaged as a tgz artifact; no code is copied)
        -
        ->>> cat databricks.yml
        -bundle:
        -  name: torchrun-a10-smoke-test
        -sync:
        -  paths:
        -    - generated_artifacts
        -artifacts:
        -  code_source:
        -    type: tgz
        -    path: .
        -    include:
        -      - src
        -    files:
        -      - source: ./dist/code_source.tgz
        -targets:
        -  dev:
        -    mode: development
        -    default: true
        -resources:
        -  jobs:
        -    torchrun-a10-smoke-test:
        -      name: torchrun-a10-smoke-test
        -      tasks:
        -        - task_key: torchrun-a10-smoke-test
        -          environment_key: default
        -          max_retries: 3
        -          retry_on_timeout: true
        -          ai_runtime_task:
        -            experiment: torchrun-a10-smoke-test
        -            deployments:
        -              - command_path: ./generated_artifacts/command.sh
        -                compute:
        -                  accelerator_type: GPU_1xA10
        -                  accelerator_count: 1
        -                  provisioned_capacity_id: capacity-1
        -            code_source_path: ./dist/code_source.tgz
        -            mlflow_artifact_location: dbfs:/Volumes/main/default/artifacts
        -            priority_class: NORMAL
        -            unity_catalog_image_path: main.air.training:prod
        -      environments:
        -        - environment_key: default
        -          spec:
        -            environment_version: "6"
        -
        -=== the generated command.sh carries the run command
        -
        ->>> cat generated_artifacts/command.sh
        -cd $CODE_SOURCE_PATH
        -torchrun --nproc_per_node=1 train.py
        -
        -=== the emitted bundle validates
        -
        ->>> [CLI] bundle validate
        -Name: torchrun-a10-smoke-test
        -Target: dev
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/torchrun-a10-smoke-test/dev
        -
        -Validation OK!
        -
        -=== re-converting refuses to clobber the generated bundle
        -
        ->>> [CLI] air convert-to-dabs train.yaml
        -Error: databricks.yml already exists in .; pass --force to overwrite or remove it
        -
         Exit code: 1
        -
        -=== --force overwrites it
        -
        ->>> [CLI] air convert-to-dabs train.yaml --force
        -Wrote a Databricks Asset Bundle to .:
        -  databricks.yml
        -  generated_artifacts/training_config.yaml
        -  generated_artifacts/command.sh
        -
        -To deploy and run this workload as a bundle:
        -  1. [CLI] bundle validate
        -  2. [CLI] bundle deploy
        -  3. [CLI] bundle run torchrun-a10-smoke-test --no-wait
        -
        -bundle deploy uploads the code source and launch scripts automatically.
        -To see what it deployed and where: [CLI] bundle summary
        -
        -Unlike `air run` (which submits an ephemeral run), bundle deploy creates a
        -persistent job that is not garbage-collected. When you are done, remove the
        -job and its uploaded files with:
        -  [CLI] bundle destroy
        -
        -Note: the YAML does not set max_retries, so the job uses the `air run` default
        -(max_retries: 3, retry_on_timeout: true). Edit databricks.yml to change it.
        -
        -=== convert a B300 run using the current SDK
        -
        ->>> [CLI] air convert-to-dabs b300.yaml --output-dir generated-b300
        -Wrote a Databricks Asset Bundle to generated-b300:
        -  databricks.yml
        -  generated_artifacts/training_config.yaml
        -  generated_artifacts/command.sh
        -
        -To deploy and run this workload as a bundle:
        -  1. cd generated-b300
        -  2. [CLI] bundle validate
        -  3. [CLI] bundle deploy
        -  4. [CLI] bundle run b300-smoke-test --no-wait
        -
        -bundle deploy uploads the code source and launch scripts automatically.
        -To see what it deployed and where: [CLI] bundle summary
        -
        -Unlike `air run` (which submits an ephemeral run), bundle deploy creates a
        -persistent job that is not garbage-collected. When you are done, remove the
        -job and its uploaded files with:
        -  [CLI] bundle destroy
        -
        -Note: the YAML does not set max_retries, so the job uses the `air run` default
        -(max_retries: 3, retry_on_timeout: true). Edit databricks.yml to change it.
        -
        -=== the B300 bundle preserves its accelerator and AI Runtime version
        -
        ->>> cat generated-b300/databricks.yml
        -bundle:
        -  name: b300-smoke-test
        -sync:
        -  paths:
        -    - generated_artifacts
        -targets:
        -  dev:
        -    mode: development
        -    default: true
        -resources:
        -  jobs:
        -    b300-smoke-test:
        -      name: b300-smoke-test
        -      tasks:
        -        - task_key: b300-smoke-test
        -          environment_key: default
        -          max_retries: 3
        -          retry_on_timeout: true
        -          ai_runtime_task:
        -            experiment: b300-smoke-test
        -            deployments:
        -              - command_path: ./generated_artifacts/command.sh
        -                compute:
        -                  accelerator_type: GPU_8xB300
        -                  accelerator_count: 8
        -      environments:
        -        - environment_key: default
        -          spec:
        -            environment_version: "6"
        -
        -=== the current SDK recognizes B300 so bundle validation is clean
        -
        ->>> [CLI] bundle validate
        -Name: b300-smoke-test
        -Target: dev
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/b300-smoke-test/dev
        -
        -Validation OK!
        -
        -=== databricks_ai versions use a base environment
        -
        ->>> [CLI] air convert-to-dabs databricks-ai.yaml --force
        -Wrote a Databricks Asset Bundle to .:
        -  databricks.yml
        -  generated_artifacts/training_config.yaml
        -  generated_artifacts/command.sh
        -
... (34 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.27.0): same output as main (b8046e0)

TestAccept/air/help: branch ✅ | main (b8046e0) ❌ | latest (v1.27.0) ❌

main (b8046e0):

acceptance_test.go:1400: Diff:
        --- air/help/output.txt
        +++ /tmp/TestAcceptairhelp181652667/002/output.txt
        @@ -25,7 +25,7 @@
           images          Push container images for custom environments
         
         Provisioned capacity:
        -  pools           View provisioned GPU Pools, if any (not required to run)
        +  pools           View provisioned GPU pools, if any (not required to run)
         
         Flags:
           -h, --help[=true|false]   help for air
        @@ -116,19 +116,19 @@
         === pools help
         
         >>> [CLI] air pools --help
        -View GPU Pools provisioned for your workspace.
        +View GPU pools provisioned for your workspace.
         
        -A GPU Pool is dedicated accelerator capacity that your Databricks account team
        +A GPU pool is dedicated accelerator capacity that your Databricks account team
         provisions. Pools are optional: a run uses one only when its YAML sets
        -compute.accelerator_pool_id.
        +compute.pool_id.
         
         Usage:
           databricks air pools [flags]
           databricks air pools [command]
         
         Available Commands:
        -  get         Show a GPU Pool and its accelerator usage
        -  list        List the GPU Pools available to the current workspace
        +  get         Show a GPU pool and its accelerator usage
        +  list        List the GPU pools available to the current workspace
         
         Flags:
           -h, --help[=true|false]   help for pools

latest (v1.27.0):

acceptance_test.go:1400: Diff:
        --- air/help/output.txt
        +++ /tmp/TestAcceptairhelp1758932048/002/output.txt
        @@ -25,7 +25,7 @@
           images          Push container images for custom environments
         
         Provisioned capacity:
        -  pools           View provisioned GPU Pools, if any (not required to run)
        +  pools           View provisioned GPU pools, if any (not required to run)
         
         Flags:
           -h, --help[=true|false]   help for air
        @@ -77,7 +77,7 @@
               --all-users[=true|false]    Show runs from all users
               --filter stringArray        Filter runs by KEY=VALUE (repeatable)
           -h, --help[=true|false]         help for list
        -      --limit int                 Maximum number of runs to show; a run with several AI Runtime tasks shows one row per task (default 20)
        +      --limit int                 Maximum number of runs to show (default 20)
         
         Global Flags:
               --debug[=true|false]   enable debug logging
        @@ -92,20 +92,12 @@
         
         JOB_RUN_ID is the job run ID returned by "databricks air run".
         
        -A job may run other tasks around its AI Runtime task (for example a notebook
        -that prepares data first); the view shows the first AI Runtime task in task
        -order. When a job has several AI Runtime tasks, pass --task-key to show another
        -one. The status is that task's own, except that a task whose latest attempt
        -failed shows RUNNING while another task of the run is still active, since it
        -may yet be retried.
        -
         Usage:
           databricks air get JOB_RUN_ID [flags]
         
         Flags:
           -h, --help[=true|false]             help for get
               --include-config[=true|false]   Include stored config in JSON (downloads over 1 MiB are omitted; may contain plaintext env_variables and parameters)
        -      --task-key string               Show this AI Runtime task of the run (default: the first in task order)
         
         Global Flags:
               --debug[=true|false]   enable debug logging
        @@ -116,19 +102,19 @@
         === pools help
         
         >>> [CLI] air pools --help
        -View GPU Pools provisioned for your workspace.
        +View GPU pools provisioned for your workspace.
         
        -A GPU Pool is dedicated accelerator capacity that your Databricks account team
        +A GPU pool is dedicated accelerator capacity that your Databricks account team
         provisions. Pools are optional: a run uses one only when its YAML sets
        -compute.accelerator_pool_id.
        +compute.pool_id.
         
         Usage:
           databricks air pools [flags]
           databricks air pools [command]
         
         Available Commands:
        -  get         Show a GPU Pool and its accelerator usage
        -  list        List the GPU Pools available to the current workspace
        +  get         Show a GPU pool and its accelerator usage
        +  list        List the GPU pools available to the current workspace
         
         Flags:
           -h, --help[=true|false]   help for pools
        @@ -238,9 +216,6 @@
         Cancel one or more runs by job run ID, or use --all to cancel
         all active runs submitted by the current user.
         
        -Cancelling a run stops all of its tasks: every AI Runtime task in the job, and
        -any other task such as a notebook that prepares data.
        -
         --all prompts for confirmation. Use --yes to skip the prompt.
         
         Usage:
TestAccept/air/run: branch ✅ | main (b8046e0) ❌ | latest (v1.27.0) ❌

main (b8046e0):

acceptance_test.go:1400: Diff:
        --- air/run/output.txt
        +++ /tmp/TestAcceptairrun830024540/002/output.txt
        @@ -147,23 +147,8 @@
         === removed pool field is rejected
         
         >>> [CLI] air run -f removed-pool.yaml --dry-run
        -Error: invalid config removed-pool.yaml: line 6: unknown field "compute.pool_id"; did you mean "accelerator_pool_id"? Run "databricks air run -h config.compute" for the list of fields
        +Dry run: configuration for "smoke-test" is valid; not submitting.
         
        -Tip: use the --debug flag to see more details and a trace of this error:
        -  databricks --debug air run …
        +Unexpected success
         
        -=== unsupported priority is rejected
        -
        ->>> [CLI] air run -f invalid-priority.yaml --dry-run
        -Error: compute.priority_class: invalid priority_class "HIGH": valid values are: NORMAL
        -
        -Tip: use the --debug flag to see more details and a trace of this error:
        -  databricks --debug air run …
        -
        -=== priority without a pool is rejected
        -
        ->>> [CLI] air run -f priority-without-pool.yaml --dry-run
        -Error: compute.priority_class requires compute.accelerator_pool_id — priority applies only within a GPU Pool
        -
        -Tip: use the --debug flag to see more details and a trace of this error:
        -  databricks --debug air run …
        +Exit code: 1
        
FAIL

latest (v1.27.0): same output as main (b8046e0)

TestAccept/air/run-submit-deps: branch ✅ | main (b8046e0) ❌ | latest (v1.27.0) ❌

main (b8046e0):

acceptance_test.go:1400: Diff:
        --- air/run-submit-deps/output.txt
        +++ /tmp/TestAcceptairrun-submit-deps1831070875/002/output.txt
        @@ -2,71 +2,9 @@
         === submit with inline dependencies
         
         >>> [CLI] air run -f run.yaml
        -Submitting experiment: deps-smoke
        -Submitted workload with Job Run ID: 555
        -View job run at: [DATABRICKS_URL]/jobs/runs/555?o=[WORKSPACE_ID]
        +Error: invalid config run.yaml: line 6: unknown field "compute.accelerator_pool_id"; run "databricks air run -h config.compute" for the list of fields
         
        -Tip: use --watch when submitting a run to stream logs to your terminal.
        -Stream logs after submission using:
        -  databricks air logs 555 --follow
        -
        -=== only config + command are uploaded; no requirements.yaml
        -
        ->>> print_requests.py //api/2.0/workspace-files/import-file --oneline --sort --unique --keep
        -{"method": "POST", "path": "/api/2.0/workspace-files/import-file/Workspace/Users/[USERNAME]/.air/cli_launch/deps-smoke/deps-smoke_[RUN_ID]/command.sh", "q": {"overwrite": "true"}, "raw_body": "python train.py"}
        -{"method": "POST", "path": "/api/2.0/workspace-files/import-file/Workspace/Users/[USERNAME]/.air/cli_launch/deps-smoke/deps-smoke_[RUN_ID]/training_config.yaml", "q": {"overwrite": "true"}, "raw_body": "experiment_name: deps-smoke\ncommand: python train.py\ncompute:\n    accelerator_type: GPU_1xH100\n    num_accelerators: 1\n    accelerator_pool_id: capacity-123\nenvironment:\n    version: 5\n    dependencies:\n        - numpy\n        - torch==2.3.0\nmlflow_artifact_location: /Volumes/main/default/air-artifacts\n"}
        -
        -=== inline deps, artifact location, and capacity id reach the submit payload
        -
        ->>> print_requests.py //api/2.2/jobs/runs/submit
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/runs/submit",
        -  "body": {
        -    "environments": [
        -      {
        -        "environment_key": "default",
        -        "spec": {
        -          "dependencies": [
        -            "numpy",
        -            "torch==2.3.0"
        -          ],
        -          "environment_version": "5"
        -        }
        -      }
        -    ],
        -    "idempotency_token": "[UUID]",
        -    "run_name": "deps-smoke",
        -    "tasks": [
        -      {
        -        "ai_runtime_task": {
        -          "deployments": [
        -            {
        -              "command_path": "/Workspace/Users/[USERNAME]/.air/cli_launch/deps-smoke/deps-smoke_[RUN_ID]/command.sh",
        -              "compute": {
        -                "accelerator_count": 1,
        -                "accelerator_pool_id": "capacity-123",
        -                "accelerator_type": "GPU_1xH100"
        -              }
        -            }
        -          ],
        -          "experiment": "deps-smoke",
        -          "mlflow_artifact_location": "dbfs:/Volumes/main/default/air-artifacts"
        -        },
        -        "environment_key": "default",
        -        "max_retries": 3,
        -        "retry_on_timeout": true,
        -        "run_if": "ALL_SUCCESS",
        -        "task_key": "deps-smoke"
        -      }
        -    ]
        -  }
        -}
        -
        -=== a requirements.yaml file path is rejected; deps must be inline
        -
        ->>> [CLI] air run -f run-file.yaml
        -Error: invalid config run-file.yaml: environment.dependencies must be a list of packages or reference a requirements.txt (see https://docs.databricks.com/aws/en/machine-learning/ai-runtime/connecting/cli/yaml-config#reference). A direct file reference is not supported
        -
         Tip: use the --debug flag to see more details and a trace of this error:
           databricks --debug air run …
        +
        +Exit code: 1

latest (v1.27.0): same output as main (b8046e0)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 894df84

Run: 44597283345

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 24 10:29
💚​ aws-windows-latest-4core-16gb 1 274 22 7:39
💚​ azure linux-2core-8gb 1 271 24 12:08
💚​ azure-windows-latest-4core-16gb 1 273 22 7:29
💚​ gcp linux-2core-8gb 1 272 24 9:51
💚​ gcp-windows-latest-4core-16gb 1 274 22 9:03
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 18 slowest tests (at least 2 minutes):
duration env testname
3:55 gcp-windows-latest-4core-16gb TestAccept
3:51 aws-windows-latest-4core-16gb TestAccept
3:44 aws-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
3:28 azure linux-2core-8gb TestTelemetryEndpoint
3:27 gcp-windows-latest-4core-16gb TestImportDirWithOverwriteFlag
3:22 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsRead
3:21 azure-windows-latest-4core-16gb TestAccept
3:19 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
2:30 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:29 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsStat
2:23 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:18 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:15 gcp linux-2core-8gb TestImportDirWithOverwriteFlag
2:11 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
2:08 aws linux-2core-8gb TestImportDirDoesNotOverwrite
2:03 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:01 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
2:00 azure linux-2core-8gb TestAccept

Comment thread .nextchanges/air/accelerator-pool-id.md Outdated
@@ -0,0 +1 @@
* Rename `compute.pool_id` to `compute.accelerator_pool_id` for GPU Pool selection in AIR workload YAML. ([#7698](https://github.com/databricks/cli/pull/7698))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the new name accelerator_pool_id finalized? Since it's a user-facing YAML key that we'd have to support long-term, I'd like to confirm it before it ships.

If it is, this is a breaking change for released users: yaml:"pool_id" shipped in v1.26.0, and after this PR it fails with unknown field. The entry only says "Rename" and doesn't mention HIGH. Could we make it explicit, e.g.:

Breaking: compute.pool_id is no longer accepted; use compute.accelerator_pool_id. Also adds priority_class: HIGH.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/air/compute.go Outdated
PriorityClass *string `yaml:"priority_class" help:"Scheduling priority within the pool: BEST_EFFORT (lowest, preemptable) or NORMAL. Requires pool_id." group:"Provisioned capacity (optional)"`
NumAccelerators int `yaml:"num_accelerators" help:"Total number of GPUs to allocate. Must be a positive multiple of the accelerator type's per-node GPU count. See Docs below for GPUs per node." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorType string `yaml:"accelerator_type" help:"Which accelerator to run on, e.g. GPU_1xA10. Matched case-sensitively." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorPoolID *string `yaml:"accelerator_pool_id" help:"GPU Pool to run on, by ID. Must be 1-255 characters. List available pools with 'databricks air pools list'. An invalid or inaccessible pool is rejected. When omitted, workspace capacity selection applies." group:"GPU Pools (optional)"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users who still have pool_id in their YAML will get unknown field "compute.pool_id" with no pointer to the new name. closestConfigField (runconfig_load.go:178) only suggests within len/3 edits, and pool_id is 12 edits from accelerator_pool_id, so no "did you mean" fires. Could we add an explicit hint for the renamed key (pool_id -> accelerator_pool_id) and cover it in TestLoadRunConfigAcceleratorPool? Keeping the rejection itself is fine.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't reference pool_id, no one is using it. There's nothing legacy to do here. I am going to push back on this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG. Can you do a follow up PR to deprecate pool_id in YAML? (update all docs / examples /. skills, give warning for a few versions, then finally remove it)

Comment thread cmd/air/compute.go Outdated
const (
priorityClassBestEffort priorityClass = "BEST_EFFORT"
priorityClassNormal priorityClass = "NORMAL"
priorityClassHigh priorityClass = "HIGH"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH isn't in the contract this repo generates: bundle/schema/jsonschema.json (jobs.AiRuntimeTaskPriorityClass) and python/databricks/bundles/jobs/_models/ai_runtime_task_priority_class.py both list only BEST_EFFORT and NORMAL, while the comment above says these values match the AiRuntimeTask.PriorityClass proto enum. Could we confirm HIGH against the proto / AICM validator before merge, and regenerate the schema if it's stale?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sg, HIGH has been landed for a bit so it seems that we need to regenerate the schema

}
if cfg.Compute.PoolID != nil {
compute["provisioned_capacity_id"] = nv(*cfg.Compute.PoolID, 3)
// Bundles still use the legacy field supported by their current schema.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to the HIGH question: priority_class passes straight through into the bundle's ai_runtime_task (line 338), so with HIGH the generated bundle YAML would carry a value the bundle schema enum rejects. The DABs test only covers NORMAL (convert_to_dabs_test.go:197-200). Could we add a HIGH case, or note the gap if the bundle schema is expected to change alongside?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure I'll add a test

title "new submission fields are documented"
trace $CLI air run -h config.mlflow_artifact_location
trace $CLI air run -h config.compute.pool_id
trace $CLI air run -h config.compute.accelerator_pool_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pool_id rejection, an invalid priority_class, and priority_class without a pool are only covered by unit tests. Could we add trace $CLI air run -h config.compute.pool_id here (after the rename hint above lands), so the migration path is pinned in acceptance output?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, updated

Comment thread cmd/air/compute.go Outdated
PriorityClass *string `yaml:"priority_class" help:"Scheduling priority within the pool: BEST_EFFORT (lowest, preemptable) or NORMAL. Requires pool_id." group:"Provisioned capacity (optional)"`
NumAccelerators int `yaml:"num_accelerators" help:"Total number of GPUs to allocate. Must be a positive multiple of the accelerator type's per-node GPU count. See Docs below for GPUs per node." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorType string `yaml:"accelerator_type" help:"Which accelerator to run on, e.g. GPU_1xA10. Matched case-sensitively." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorPoolID *string `yaml:"accelerator_pool_id" help:"GPU Pool to run on, by ID. Must be 1-255 characters. List available pools with 'databricks air pools list'. An invalid or inaccessible pool is rejected. When omitted, workspace capacity selection applies." group:"GPU Pools (optional)"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "GPU Pool" here vs "GPU pool" in pools.go:23; could we pick one? Also, "An invalid or inaccessible pool is rejected" and "When omitted, workspace capacity selection applies" describe backend behavior we can't verify from the CLI. Worth confirming with the backend team or softening the wording.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can capitalize pool, we haven't finalized the backend behavior yet so it's kept intentionally vague

@pardis-beikzadeh-db pardis-beikzadeh-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes to back maggiewang-db's open threads. Two of them should block the merge: compute.pool_id (shipped in v1.18.0 through v1.26.0) now fails with no hint toward accelerator_pool_id, and HIGH passes air run validation but produces a convert-to-dabs bundle that the bundle schema's jobs.AiRuntimeTaskPriorityClass enum rejects.

@ben-hansen-db

Copy link
Copy Markdown
Contributor Author

Requesting changes to back maggiewang-db's open threads. Two of them should block the merge: compute.pool_id (shipped in v1.18.0 through v1.26.0) now fails with no hint toward accelerator_pool_id, and HIGH passes air run validation but produces a convert-to-dabs bundle that the bundle schema's jobs.AiRuntimeTaskPriorityClass enum rejects.

I'm going to push back strongly here. pool_id is not used by anyone and should not be supported. Wrt HIGH it seems that the proto needs to be regenerated. If there's some lag I'll leave it out of this pr. This is to support accelerator_pool_id

@pardis-beikzadeh-db pardis-beikzadeh-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for dropping HIGH. I'd still like a pointer for pool_id. It shipped in v1.18.0 through v1.26.0, so anyone who used it gets unknown field "compute.pool_id" and nothing naming accelerator_pool_id.

suggested fix: special-case pool_id in the unknown-field error with "renamed to accelerator_pool_id", and pin it in the removed-pool.yaml acceptance case.

Comment thread cmd/air/compute.go Outdated
NumAccelerators int `yaml:"num_accelerators" help:"Total number of GPUs to allocate. Must be a positive multiple of the accelerator type's per-node GPU count. See Docs below for GPUs per node." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorType string `yaml:"accelerator_type" help:"Which accelerator to run on, e.g. GPU_1xA10. Matched case-sensitively." required:"yes" docs:"/machine-learning/ai-runtime/connecting/cli/yaml-config#accelerator-types"`
AcceleratorPoolID *string `yaml:"accelerator_pool_id" help:"GPU Pool to run on, by ID. Must be 1-255 characters. List available pools with 'databricks air pools list'." group:"GPU Pools (optional)"`
PriorityClass *string `yaml:"priority_class" help:"Scheduling priority within the GPU Pool. Requires accelerator_pool_id." group:"GPU Pools (optional)"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

problem: -h config.compute.priority_class no longer lists the valid values (main did), so users only find them by hitting the error.

suggested fix: keep main's list with the new field name: Scheduling priority within the GPU Pool: BEST_EFFORT (lowest, preemptable) or NORMAL. Requires accelerator_pool_id.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still like a pointer for pool_id. It shipped in v1.18.0 through v1.26.0, so anyone who used it gets unknown field "compute.pool_id" and nothing naming accelerator_pool_id.

Sure. But no one is using it. It's not something folks used before. I would like a clean break rather than referencing the placeholder.

suggested fix: keep main's list with the new field name: Scheduling priority within the GPU Pool: BEST_EFFORT (lowest, preemptable) or NORMAL. Requires accelerator_pool_id.

No, some of those values are deprecated. BEST_EFFORT is already deprecated officially so we should really try not to reference it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Priorities are not enabled for users right now

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can list just NORMAL for now which is implicit anyways which is why I don't love even mentioning it. It's confusing

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But no one is using it. It's not something folks used before. I would like a clean break rather than referencing the placeholder.

aaaah... ok sorry I didn't know that. yes clean break sgtm.

sorry for the frustration caused. my agent proposed it and it looked like a reasonable concern but I didn't have the context that it hasn't been used at all... it was saying it was released before? but I guess it was hidden and / or flag gated somehow?

@pardis-beikzadeh-db pardis-beikzadeh-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding the pool_id hint. listing only NORMAL in the help is fine by me, since BEST_EFFORT is deprecated.

Comment thread acceptance/air/run/output.txt Outdated
=== unsupported priority is rejected

>>> [CLI] air run -f invalid-priority.yaml --dry-run
Error: compute.priority_class: invalid priority_class "HIGH": valid values are: BEST_EFFORT, NORMAL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking: the error still lists BEST_EFFORT, so anyone who mistypes the value (e.g. HIGH here) gets pointed at the deprecated one you dropped from the help.

suggested fix: have validPriorityClassesHint list only NORMAL, and keep accepting BEST_EFFORT so existing YAML still parses.

@pardis-beikzadeh-db

Copy link
Copy Markdown
Contributor

Requesting changes to back maggiewang-db's open threads. Two of them should block the merge: compute.pool_id (shipped in v1.18.0 through v1.26.0) now fails with no hint toward accelerator_pool_id, and HIGH passes air run validation but produces a convert-to-dabs bundle that the bundle schema's jobs.AiRuntimeTaskPriorityClass enum rejects.

I'm going to push back strongly here. pool_id is not used by anyone and should not be supported. Wrt HIGH it seems that the proto needs to be regenerated. If there's some lag I'll leave it out of this pr. This is to support accelerator_pool_id

see #7698 (comment)

@ben-hansen-db
ben-hansen-db added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit 06e25a1 Nov 19, 2026
28 checks passed
@ben-hansen-db
ben-hansen-db deleted the ben/accelerator-pool-id branch November 19, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants