Repository navigation
air: add a hidden air deploy command that registers and serves a model - #7642
Draft
erarpitagarwal wants to merge 11 commits into
Draft
erarpitagarwal wants to merge 11 commits into
erarpitagarwal wants to merge 11 commits into
Conversation
`databricks air deploy -f deploy.yaml` registers a Unity Catalog model version through the registerModelPackage API (a checkpoint in a volume, a model from code, a model directory saved by MLflow) and serves it on a Model Serving endpoint, or serves an existing version. Registration is idempotent per key; the CLI repeats the same request until the version is ready, and prints the command that continues after Ctrl-C or --timeout. The command is hidden until the API is on everywhere. The run YAML loader, --override, -h config.<field>, code_source file selection and root path resolution are shared with `air run`. Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Thanks for the contribution! This pull request was opened from a fork. If you're a member of the Databricks organization, please request access to the If you're an external contributor, no action is needed; a maintainer will review your pull request. |
…errors - Register once, poll the read-only staging status API while a checkpoint's weights copy, then register again to finalize. - Retry transient failures of the polled requests via libs/retry. - Check the endpoint before registering: replace only a single version of the same model, and wait out another update first. - Require code_source.include_paths, refuse pathspec magic and files git tracks although .gitignore matches them, and list the files on --dry-run. Fix the shared non-git listing, which was relative to the working directory instead of root_path. - Print the idempotency key before the first request in every mode; reject an empty --idempotency-key; give JSON errors typed codes. - Validate endpoint names, workload types and sizes, checkpoint tasks, and count length limits as the server does. Co-authored-by: Isaac <no-reply@databricks.com>
- Poll the endpoint per GET with libs/retry, so transient failures spread over a long start don't end the wait. - Don't resend a model from code that isn't ready after registering. - Keep git's stderr in check-ignore failures; reject include_paths patterns and padding; don't mark a local client error retryable. Co-authored-by: Isaac <no-reply@databricks.com>
The server now leaves a registered version pending: air deploy registers it once, polls the read-only staging status until the weights are copied or fail, then calls finalizeModelPackage. A status without a state is waited out, a finalize failure prints the command that continues, and the finalize response must name the same version. Co-authored-by: Isaac <no-reply@databricks.com>
A checkpoint can declare llm/v1/audio/transcriptions or llm/v1/audio/translations, which the registration API accepts. Their requests are multipart uploads to the endpoint's audio route, so the CLI prints that route instead of invocations. Co-authored-by: Isaac <no-reply@databricks.com>
…hon code A checkpoint is model.path plus exactly one of command or model.entry, with model.model_config, model.streamable and the Databricks AI environment (environment.version: databricks_ai_v6) for Python code. The server decides which tasks a command can serve. Co-authored-by: Isaac <no-reply@databricks.com>
Drop the code and saved_model_dir model types until the server handles them; Python code now serves a checkpoint through model.entry, so the code collection stays and builds the checkpoint's python_model. Stop merging staging statuses: every registration, poll and finalize response must report a known state for both the weights and the serving environment, and a response that doesn't is an error rather than progress. Each new status replaces the last one. Co-authored-by: Isaac <no-reply@databricks.com>
…de files listed model_config values YAML holds but JSON can't, such as .nan, .inf and mappings with keys that aren't strings, were dropped silently, so the model got no config; they are now rejected when the config loads. A code file replaced after it was listed, such as by a symlink, is refused instead of read. A registered version that isn't a number is an error. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…t API does The compute block's fields now all carry the served entity's API names: workload_type, workload_size and scale_to_zero_enabled. Co-authored-by: Isaac <no-reply@databricks.com>
…ents Replace the nested model: block and the Python-code path with a flat YAML whose fields select one of three variants: - checkpoint and command: a checkpoint served by the command the file sets - checkpoint without command: a checkpoint served with dedicated inference - neither: a model version already registered in Unity Catalog Compute is serving compute (accelerator_type, num_accelerators and workload_size, mapped to a serving workload type) for a command or a registered model, and dedicated compute (compute_profile, provisioned_model_units and reservation_term) for a dedicated checkpoint. A field the file's variant doesn't use is an error that names the field, the variant and why the file is read as that variant. Endpoints are read as JSON, since the SDK's served entity has no compute_profile, and dedicated endpoints are created and updated over raw JSON. A dry run prints the plan for each variant without making requests. Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Adds a hidden
databricks air deploycommand: one flat YAML file serves a model on a Model Serving endpoint, registering it in Unity Catalog first when it comes from a checkpoint.The fields a file sets select one of three kinds of deployment; there is no type key:
checkpointandcommandcheckpoint, nocommandcheckpointmodel_version, or the newest READY versiontaskandenvironmentare required;environment.versionhas no default (6for the Standard environment,databricks_ai_v6for the Databricks AI environment, as forair run), andenvironment.dependenciesadds pip packages. Any task Model Serving runs a command for is accepted, includingllm/v1/audio/transcriptions; the server checks it, so the CLI keeps no task list.accelerator_type,num_acceleratorsandworkload_size, withair run's accelerator names. Each supported pair maps to a serving workload type (GPU_1xA10x1, x4 or x8,GPU_1xH100x1,GPU_8xH100x8,GPU_1xRTX6000x1); anything else is refused with the list. Withoutcompute, a command gets oneGPU_1xA10and a registered model gets CPU, both at size Small.compute_profile(low_latency,fast,throughputorminimal_cost) andprovisioned_model_units, both required.reservation_termis in the schema but refused before anything is registered, because dedicated endpoints don't accept a commitment term yet.registerModelPackageonce (withcheckpointordedicated_checkpoint),checkModelDeploymentStagingStatuswhile the weights copy, thenfinalizeModelPackage. Every staging status must report a known state for both the weights and the serving environment; anything else fails at once instead of looking like progress until the timeout.invocationsotherwise.--timeoutprints the command that continues. Transient errors are retried per request.air run's YAML loader and idempotency helper.air runis unchanged apart from one error message, which now reads "config must be a YAML mapping".Example, one per kind (also in
databricks air deploy -h config):Why
Serving a checkpoint today means calling
mlflow.<flavor>.log_modelfrom Python: pickling a model, copying weights into it and packing an environment. Withair deploy, the user hands over text and the files a training job already wrote to a volume; the server writes the MLmodel and builds the environment. A model that was already logged and registered is served directly.The command stays hidden, with no changelog entry, until the server APIs are on everywhere.
Decisions:
INVALID_ARGS,INTERRUPTED,TIMEOUT,STAGING_FAILEDandINTERNAL_ERROR.Tests
go test ./cmd/air/,go vetandgolangci-lint(0 issues) pass. New tests cover how each kind is inferred and every field a kind refuses, the compute table and its defaults, compute profiles and the refused reservation term, a requiredenvironment.version, the register request for a command and for a dedicated checkpoint, register/poll/finalize (retries, failures, malformed staging statuses, a version that isn't a number, an already-ready version, timeouts and cancellation), dedicated endpoint create and update (the update body as the server reads it), endpoints of the other kind refused before registering, picking the newest READY version, the query URL, resume commands and JSON error codes.airhas none yet, and the test server doesn't model these APIs.This pull request and its description were written by Isaac.