Skip to content

air: add a hidden air deploy command that registers and serves a model - #7642

Draft
erarpitagarwal wants to merge 11 commits into
databricks:mainfrom
erarpitagarwal:air-deploy
Draft

erarpitagarwal wants to merge 11 commits into
databricks:mainfrom
erarpitagarwal:air-deploy

Conversation

@erarpitagarwal

@erarpitagarwal erarpitagarwal commented Nov 15, 2026 •

Copy link
Copy Markdown

Changes

Adds a hidden databricks air deploy command: one flat YAML file serves a model on a Model Serving endpoint, registering it in Unity Catalog first when it comes from a checkpoint.

The fields a file sets select one of three kinds of deployment; there is no type key:

The file sets Deployment What air deploy does
checkpoint and command weights in a volume, served by the user's own server (vLLM, SGLang, a FastAPI app) registers a version, then serves it on serving compute
checkpoint, no command a supported Hugging Face LLM that Databricks serves with dedicated inference registers a version, then serves it on a dedicated endpoint
no checkpoint a model already logged and registered with MLflow serves model_version, or the newest READY version
  • Command: task and environment are required; environment.version has no default (6 for the Standard environment, databricks_ai_v6 for the Databricks AI environment, as for air run), and environment.dependencies adds pip packages. Any task Model Serving runs a command for is accepted, including llm/v1/audio/transcriptions; the server checks it, so the CLI keeps no task list.
  • Serving compute (command and registered): accelerator_type, num_accelerators and workload_size, with air run's accelerator names. Each supported pair maps to a serving workload type (GPU_1xA10 x1, x4 or x8, GPU_1xH100 x1, GPU_8xH100 x8, GPU_1xRTX6000 x1); anything else is refused with the list. Without compute, a command gets one GPU_1xA10 and a registered model gets CPU, both at size Small.
  • Dedicated compute: compute_profile (low_latency, fast, throughput or minimal_cost) and provisioned_model_units, both required. reservation_term is in the schema but refused before anything is registered, because dedicated endpoints don't accept a commitment term yet.
  • A field the file's kind doesn't use is an error that names the field, the kind, and why the file was read as that kind.
  • Registering is three calls: registerModelPackage once (with checkpoint or dedicated_checkpoint), checkModelDeploymentStagingStatus while the weights copy, then finalizeModelPackage. Every staging status must report a known state for both the weights and the serving environment; anything else fails at once instead of looking like progress until the timeout.
  • The endpoint is created or updated and waited on until it's ready. A dedicated endpoint is created and updated through the dedicated endpoints API, which the Go SDK doesn't model yet, so those requests and every endpoint read are JSON. Before registering, the endpoint is checked: one that serves another model, or is of the other kind (dedicated versus serving compute), fails before any weights are copied. It prints the URL to query: the audio route for an audio task, invocations otherwise.
  • Runs can resume: the idempotency key is printed first, and Ctrl-C or --timeout prints the command that continues. Transient errors are retried per request.
  • It reuses air run's YAML loader and idempotency helper. air run is unchanged apart from one error message, which now reads "config must be a YAML mapping".

Example, one per kind (also in databricks air deploy -h config):

# Your own server
registered_model_name: main.serving.qwen30b_sglang
endpoint_name: qwen3-30b-sglang
checkpoint: /Volumes/main/weights/qwen3_30b_a3b
command: python -m sglang.launch_server --model-path qwen3_30b_a3b --attention-backend triton --sampling-backend pytorch --host 0.0.0.0 --port 8080
task: llm/v1/chat
environment:
  version: 6
  dependencies: [sglang==0.5.23]
compute:
  accelerator_type: GPU_1xH100
  num_accelerators: 1
  workload_size: Small
# Dedicated inference
registered_model_name: main.serving.gemma4
endpoint_name: gemma4
checkpoint: /Volumes/main/weights/gemma4_e4b
compute:
  compute_profile: low_latency
  provisioned_model_units: 600
# A model already registered with MLflow
registered_model_name: main.ml.churn
endpoint_name: churn
model_version: 3

Why

Serving a checkpoint today means calling mlflow.<flavor>.log_model from Python: pickling a model, copying weights into it and packing an environment. With air deploy, the user hands over text and the files a training job already wrote to a volume; the server writes the MLmodel and builds the environment. A model that was already logged and registered is served directly.

The command stays hidden, with no changelog entry, until the server APIs are on everywhere.

Decisions:

  • No custom Python code for now. An earlier revision served a checkpoint with a Python file sent from the user's machine; it worked, and can come back as a fourth kind. Leaving it out keeps the first version to what needs no user code: a server command, dedicated inference, or an existing model.
  • The key is the only state the CLI keeps. A run resumes by re-sending the same request.
  • Fail before copying weights. Everything the CLI or a read-only call can check is checked before registration: the file, the endpoint's owner and kind, and fields the server would refuse later.
  • In JSON mode, errors are JSON envelopes with the server's error code, or one of INVALID_ARGS, INTERRUPTED, TIMEOUT, STAGING_FAILED and INTERNAL_ERROR.

Tests

  • go test ./cmd/air/, go vet and golangci-lint (0 issues) pass. New tests cover how each kind is inferred and every field a kind refuses, the compute table and its defaults, compute profiles and the refused reservation term, a required environment.version, the register request for a command and for a dedicated checkpoint, register/poll/finalize (retries, failures, malformed staging statuses, a version that isn't a number, an already-ready version, timeouts and cancellation), dedicated endpoint create and update (the update body as the server reads it), endpoints of the other kind refused before registering, picking the newest READY version, the query URL, resume commands and JSON error codes.
  • End to end against a staging workspace, on GPU endpoints that answered requests, with a test build of the server APIs and an earlier revision of this command: checkpoints served by a command (Qwen3 on vLLM and on SGLang, and Whisper behind its own FastAPI server), and serving a registered version. One server-side step of the test environment was done by hand; a fully automated run follows once the server change it needs is deployed. Dedicated inference and this revision's flat file haven't run end to end yet.
  • No acceptance tests: air has none yet, and the test server doesn't model these APIs.

This pull request and its description were written by Isaac.

`databricks air deploy -f deploy.yaml` registers a Unity Catalog model
version through the registerModelPackage API (a checkpoint in a volume,
a model from code, a model directory saved by MLflow) and serves it on a
Model Serving endpoint, or serves an existing version. Registration is
idempotent per key; the CLI repeats the same request until the version
is ready, and prints the command that continues after Ctrl-C or
--timeout. The command is hidden until the API is on everywhere.

The run YAML loader, --override, -h config.<field>, code_source file
selection and root path resolution are shared with `air run`.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! This pull request was opened from a fork.

If you're a member of the Databricks organization, please request access to the cli-contrib team and open your pull request from a branch in this repository instead of a fork — fork PRs don't run integration tests automatically and have reduced CI access.

If you're an external contributor, no action is needed; a maintainer will review your pull request.

@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 15, 2026
Arpit Agarwal and others added 10 commits November 15, 2026 06:27
…errors

- Register once, poll the read-only staging status API while a
  checkpoint's weights copy, then register again to finalize.
- Retry transient failures of the polled requests via libs/retry.
- Check the endpoint before registering: replace only a single version
  of the same model, and wait out another update first.
- Require code_source.include_paths, refuse pathspec magic and files
  git tracks although .gitignore matches them, and list the files on
  --dry-run. Fix the shared non-git listing, which was relative to the
  working directory instead of root_path.
- Print the idempotency key before the first request in every mode;
  reject an empty --idempotency-key; give JSON errors typed codes.
- Validate endpoint names, workload types and sizes, checkpoint tasks,
  and count length limits as the server does.

Co-authored-by: Isaac <no-reply@databricks.com>
- Poll the endpoint per GET with libs/retry, so transient failures
  spread over a long start don't end the wait.
- Don't resend a model from code that isn't ready after registering.
- Keep git's stderr in check-ignore failures; reject include_paths
  patterns and padding; don't mark a local client error retryable.

Co-authored-by: Isaac <no-reply@databricks.com>
The server now leaves a registered version pending: air deploy registers it
once, polls the read-only staging status until the weights are copied or fail,
then calls finalizeModelPackage. A status without a state is waited out, a
finalize failure prints the command that continues, and the finalize response
must name the same version.

Co-authored-by: Isaac <no-reply@databricks.com>
A checkpoint can declare llm/v1/audio/transcriptions or
llm/v1/audio/translations, which the registration API accepts. Their
requests are multipart uploads to the endpoint's audio route, so the
CLI prints that route instead of invocations.

Co-authored-by: Isaac <no-reply@databricks.com>
…hon code

A checkpoint is model.path plus exactly one of command or model.entry,
with model.model_config, model.streamable and the Databricks AI
environment (environment.version: databricks_ai_v6) for Python code. The
server decides which tasks a command can serve.

Co-authored-by: Isaac <no-reply@databricks.com>
Drop the code and saved_model_dir model types until the server handles them;
Python code now serves a checkpoint through model.entry, so the code
collection stays and builds the checkpoint's python_model.

Stop merging staging statuses: every registration, poll and finalize
response must report a known state for both the weights and the serving
environment, and a response that doesn't is an error rather than progress.
Each new status replaces the last one.

Co-authored-by: Isaac <no-reply@databricks.com>
…de files listed

model_config values YAML holds but JSON can't, such as .nan, .inf and mappings with keys that
aren't strings, were dropped silently, so the model got no config; they are now rejected when the
config loads. A code file replaced after it was listed, such as by a symlink, is refused instead
of read. A registered version that isn't a number is an error.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…t API does

The compute block's fields now all carry the served entity's API names: workload_type,
workload_size and scale_to_zero_enabled.

Co-authored-by: Isaac <no-reply@databricks.com>
…ents

Replace the nested model: block and the Python-code path with a flat YAML
whose fields select one of three variants:

- checkpoint and command: a checkpoint served by the command the file sets
- checkpoint without command: a checkpoint served with dedicated inference
- neither: a model version already registered in Unity Catalog

Compute is serving compute (accelerator_type, num_accelerators and
workload_size, mapped to a serving workload type) for a command or a
registered model, and dedicated compute (compute_profile,
provisioned_model_units and reservation_term) for a dedicated checkpoint.
A field the file's variant doesn't use is an error that names the field,
the variant and why the file is read as that variant.

Endpoints are read as JSON, since the SDK's served entity has no
compute_profile, and dedicated endpoints are created and updated over raw
JSON. A dry run prints the plan for each variant without making requests.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 7642
  • Commit SHA: c830022ecae18360c7176aa2a68b977411d420fc

Checks will be approved automatically on success.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant