Skip to content

serving-endpoints: add --json examples to help - #7666

Open
anshul0713 wants to merge 3 commits into
mainfrom
anshul0713/serving-endpoints-help-examples
Open

anshul0713 wants to merge 3 commits into
mainfrom
anshul0713/serving-endpoints-help-examples

Conversation

@anshul0713

@anshul0713 anshul0713 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Changes

--help for five serving-endpoints commands now ends with an example --json request body:

  • create: serve a Unity Catalog model version on CPU.
  • update-config: two versions with an 80/20 traffic split. It also notes that routes refer to served entity names.
  • put-ai-gateway: usage tracking, an inference table, a per-user rate limit and a PII input guardrail. It also says which settings an omitted field clears.
  • patch-telemetry-config: logs, traces and metrics exported to Unity Catalog tables.
  • query: dataframe_records, dataframe_split, inputs and chat messages, with a pointer to get-open-api for an endpoint's own schema.

Why

The generated help only lists --json, and most of these request bodies have no flags. In agent trials covering these commands, 4 of 8 trial reports flagged missing body shapes:

  • One agent introspected the Python SDK five times to learn the telemetry body.
  • Another put served_entities at the wrong level and only learned the config wrapper from unknown field warnings.

How the examples were checked

Each example was run against a CPU endpoint on a staging workspace, with only the names substituted: create, update-config (the traffic split was accepted), patch-telemetry-config (the config shows up in get), all three custom-model query formats, and put-ai-gateway.

That run also showed the AI Gateway behavior the help now describes:

  • Omitting rate_limits from a later request removed them.
  • Omitting inference_table_config kept it, until it was sent with "enabled": false.
  • Guardrails are rejected on custom model endpoints (AI Guardrails is not currently supported for this endpoint type).

The guardrail body itself is the one an agent applied to an external-model endpoint in a trial.

Backward compatibility

Help text only. There are no changes to requests, output, flags or exit codes.

Tests

  • Unit test: every example --json body is valid JSON.
  • Unit test: each command's help includes its example.
  • ./task lint passes.

This pull request and its description were written by Isaac.

The generated help for create, update-config, put-ai-gateway,
patch-telemetry-config and query only lists --json, and most of each
request body has no flags. In agent trials, users of these commands
guessed the body shape from errors or read the SDK source. Add one
example body per command, each checked against the API, plus the
AI Gateway field-removal behavior seen there.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Anshul Gupta <anshul.gupta@databricks.com>
@anshul0713
anshul0713 requested a review from a team as a code owner November 16, 2026 07:39
@anshul0713
anshul0713 requested a review from denik November 16, 2026 07:39
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Anshul Gupta <anshul.gupta@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 2809158

Run: 43985533376

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 9:54
💚​ aws-windows-latest-4core-16gb 1 272 13 6:28
🔄​ azure linux-2core-8gb 1 1 268 15 9:40
💚​ azure-windows-latest-4core-16gb 1 271 13 8:15
💚​ gcp linux-2core-8gb 1 270 15 8:14
💚​ gcp-windows-latest-4core-16gb 1 272 13 7:28
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncNestedFolderSync ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 11 slowest tests (at least 2 minutes):
duration env testname
3:49 azure-windows-latest-4core-16gb TestAccept
3:36 gcp-windows-latest-4core-16gb TestAccept
3:29 aws linux-2core-8gb TestSyncFullFileSync
3:19 aws-windows-latest-4core-16gb TestAccept
2:46 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:21 azure linux-2core-8gb TestFilerReadWrite/workspace_files_extensions
2:19 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:18 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:18 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:13 azure linux-2core-8gb TestExportDir
2:10 gcp linux-2core-8gb TestExportDir

The tensor-only inputs example failed for models with a column signature,
which reject a bare 2-D array. Checked against an MLflow scoring server.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Anshul Gupta <anshul.gupta@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants