Repository navigation
Support accelerator_pool_id in AIR workload configuration #7698
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
d95b1a3
d37e663
f1f7f8b
114791c
1744d1f
0c0e088
4448d3b
894df84
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * Breaking: `compute.pool_id` is no longer accepted in AIR workload YAML; use `compute.accelerator_pool_id`. ([#7698](https://github.com/databricks/cli/pull/7698)) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| experiment_name: smoke-test | ||
| command: python train.py | ||
| compute: | ||
| accelerator_type: GPU_1xH100 | ||
| num_accelerators: 1 | ||
| accelerator_pool_id: capacity-1 | ||
| priority_class: HIGH |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| experiment_name: smoke-test | ||
| command: python train.py | ||
| compute: | ||
| accelerator_type: GPU_1xH100 | ||
| num_accelerators: 1 | ||
| priority_class: NORMAL |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| experiment_name: smoke-test | ||
| command: python train.py | ||
| compute: | ||
| accelerator_type: GPU_1xH100 | ||
| num_accelerators: 1 | ||
| pool_id: capacity-1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -302,8 +302,9 @@ func buildBundleValue(ctx context.Context, cfg *runConfig, configPath, codeSourc | |
| "accelerator_type": nv(cfg.Compute.AcceleratorType, 1), | ||
| "accelerator_count": nv(cfg.Compute.NumAccelerators, 2), | ||
| } | ||
| if cfg.Compute.PoolID != nil { | ||
| compute["provisioned_capacity_id"] = nv(*cfg.Compute.PoolID, 3) | ||
| // Bundles still use the legacy field supported by their current schema. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Related to the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure I'll add a test |
||
| if poolID := cfg.Compute.AcceleratorPoolID; poolID != nil { | ||
| compute["provisioned_capacity_id"] = nv(*poolID, 3) | ||
| } | ||
| deployment := map[string]dyn.Value{ | ||
| "command_path": nv(localBundlePath(path.Join(generatedArtifactsDir, commandScriptName)), 1), | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
pool_idrejection, an invalidpriority_class, andpriority_classwithout a pool are only covered by unit tests. Could we addtrace $CLI air run -h config.compute.pool_idhere (after the rename hint above lands), so the migration path is pinned in acceptance output?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, updated