Repository navigation
aitools: Hint the AI Runtime skill on air subcommands - #7622
artemy-kolesnikov wants to merge 11 commits into
Conversation
The Claude Code install hint fired on every non-aitools command with a
generic line ("Databricks skills are not installed..."). Coding agents
treat it as unrelated noise on commands like `auth profiles`, and it
never named the skill that helps with the task at hand.
MaybeHint now fires only on runnable `databricks air` subcommands and
names the databricks-ai-runtime skill, what it covers, the install
command, and a no-install preview (`aitools install --path`). Detection
looks for that skill by exact name in the home stores and the project
store (cwd up to the repository root), and counts the databricks plugin
only when it applies to cwd. Bare `air` and `air --help` render help
without the pre-run hook, so the air help footer now carries the same
benefit line and the preview command. Telemetry keeps INSTALL_TEXT and
the per-session throttle is unchanged.
Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/air/help: branch ✅ | main (73b204d) ❌ | latest (v1.26.0) ❌main (73b204d): latest (v1.26.0): TestAccept/air/image-push: branch ✅ | main (73b204d) ❌ | latest (v1.26.0) ❌main (73b204d): latest (v1.26.0): |
✅ Integration test reportCommit: 4fdad31
Top 9 slowest tests (at least 2 minutes):
|
Co-authored-by: Isaac <no-reply@databricks.com>
Windows paths contain backslashes, so interpolating them into the manifest string produced invalid JSON escapes and the plugin record was silently dropped, hinting when the test expected silence. Marshal the manifest instead. Co-authored-by: Isaac <no-reply@databricks.com>
…bricks#7652) Replaces databricks#7628, which was opened from a fork and couldn't run CI. Same change, rebased on `main`. ## Changes `databricks air run --help` now names the databricks-ai-runtime agent skill and gives the install command, below the `-h config` lookup lines: ``` databricks air run -h config.compute.accelerator_type one field To have a coding agent write the YAML and follow the run, install the databricks-ai-runtime skill: databricks aitools install --skills-only --skills databricks-ai-runtime ``` - The install command is now a constant, `skillInstallCommand`, in `cmd/air/runconfig.go`. The `-h config` skill note uses it as well. - The `air -h` footer in `cmd/air/air.go` still has its own copy of the command. I left that file alone because databricks#7622 edits the same footer. It can move to the constant after that PR lands. ## Why Coding agents read `air run --help` right before they write a workload YAML, so this page should point to the skill. - On v1.26.0 the skill appears only in `air run -h config`. - On `main`, databricks#7512 added it to the `air -h` footer too. `air run --help` still doesn't mention it. - databricks#7622 makes the install hint name this skill on air subcommands. That hint runs from `PersistentPreRunE`, and cobra doesn't run that for `--help`. Help text is the only way to show the skill on help pages. ## Tests - New unit test `TestRunHelpPointsToAgentSkill` in `cmd/air/air_test.go`. - Regenerated `acceptance/air/config-help/output.txt` with `-update`. The only diff is the 5 new lines in `air run --help`. - Passing: `go test ./acceptance -run 'TestAccept/air/'`, `go vet ./cmd/air/`, `./task lint-q` and `./task ws`. - `go test ./cmd/air/` passes except `TestLogsCommandRepeatsRunLinkAfterStreaming`. That test fails the same way on an unmodified `origin/main` checkout (bdb37ab) in this environment. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
It looks like you removed the generic Claude Code install hint across all non-air commands in the Databricks CLI.... are we sure we want to do that? regardless I'm not sure my stamp matters too much here, this is more of a broader DECO maintainers question.
| Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install | ||
| Databricks CLI v[CLI_VERSION] | ||
| >>> CLAUDE_CONFIG_DIR=missing [CLI] air list | ||
| Tip: the databricks-ai-runtime skill is not installed. It documents the air workload YAML, GPU types, secrets, and how to follow and debug runs. Install it: databricks aitools install --skills-only --skills databricks-ai-runtime (to read it first without installing: databricks aitools install --path <dir> --skills databricks-ai-runtime) |
There was a problem hiding this comment.
I suggest tightening this a little:
Tip: install the databricks-ai-runtime skill for workload configuration and run debugging: databricks aitools install --skills-only --skills databricks-ai-runtime. To preview it first, replace --skills-only with --path <dir>.
There was a problem hiding this comment.
Applied — the hint now reads exactly that (in libs/aitools/agents/hint.go, pinned by the air goldens such as acceptance/air/image-push/output.txt).
Note this file itself is back to its main golden since then: the change was narrowed so only runnable databricks air subcommands use this wording, and claude-config-dir exercises a non-air command, which keeps the generic install hint. As of ca03773.
Use the reviewer-suggested wording: name the skill and its benefit up front, then the install command, then the --path preview. Co-authored-by: Isaac <no-reply@databricks.com>
…tools-hint # Conflicts: # acceptance/cmd/aitools/claude-config-dir/output.txt # acceptance/cmd/aitools/claude-config-dir/script # acceptance/cmd/aitools/telemetry/hint/output.txt # acceptance/cmd/aitools/telemetry/hint/script # acceptance/cmd/version/script # libs/aitools/agents/calling.go # libs/aitools/agents/hint.go # libs/aitools/agents/hint_test.go # libs/aitools/agents/recommend.go
Narrow the change so non-air commands behave exactly as before: Claude Code keeps the generic install hint and the registry agents keep their per-agent install hints on every hinted command. Only on runnable 'databricks air' subcommands does Claude Code now get the AI Runtime skill recommendation instead of the generic hint, with detection keyed to exactly the databricks-ai-runtime skill in the locations Claude Code loads for the working directory. Co-authored-by: Isaac <no-reply@databricks.com>
|
Fair question — the change is now narrowed so the answer is no: nothing is removed. Non-air commands keep the generic Claude Code install hint and the per-agent install hints exactly as on The DECO-maintainers question remains, but only about the air wording and whether the skill-specific text should extend to the other agents — both small local changes if you'd prefer something else. As of ca03773. |
…anch Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Changes
The Claude Code install hint (
agents.MaybeHint, run from the rootPersistentPreRunEsince #5918) prints one generic line on every non-aitoolscommand:This change is deliberately narrow: behavior on non-air commands is unchanged. Only on runnable
databricks airsubcommands (run,get,list,logs,cancel,convert-to-dabs,images/images push,pools/pools list/pools get) does Claude Code's hint text change, to name the skill it can use there and give both the install and a no-install preview:main:auth profiles,version,jobs list,apps init(with its ownAPPS_INIT_TIP), and every other hinted command print the same lines as before. The native<claude-code-hint>plugin line (aitools: emit Claude Code's native plugin install hint #7585) and the per-agent install hints for Codex, Copilot, Cursor and the rest (aitools: install hints for non-Claude coding agents #7586) are untouched, on air and off it; theair run --helpskill pointer (air: Point to the databricks-ai-runtime skill in air run --help #7652) is untouched. Agents other than Claude Code keep their generic hint on air subcommands too: the installer has no per-agent AI Runtime variant to offer them.databricks-ai-runtimeskill in a location Claude Code loads for the working directory — Claude's own skills dir, the project skills dir of cwd or a parent up to the repository root, or the databricks plugin counted only when its record applies to cwd. Before this change, anydatabricks*skill (for example onlydatabricks-jobs) silenced the hint, even on air. The generic path's detection (claudeToolingInstalled) is unchanged.airandair --help: cobra renders help without runningPersistentPreRunE, so the hook can't fire there. Theairhelp footer now has the benefit line and the--pathpreview under the existing "Coding agent skill" line.CLAUDE_CODE_SESSION_IDthrottle (at most one hint per hour per session, shared between the generic and air text, so a hint shown on an earlier non-air command also covers the agent's first air command; with no session id the hint prints per command, as before), the telemetry kind (INSTALL_TEXT), the aitools-subtree and credential-helper exemptions, and the Claude-only gating. No new enum value;ExecutionContext.Commandalready tells air commands apart..nextchanges/cli/air-skill-hint.mdis added.Why
In a coding-agent run (Claude Code), the agent saw the generic hint on
databricks auth profiles. It read the hint as unrelated setup noise and went to docs instead of the skill. Rather than removing the generic hint anywhere (a product call for the DECO maintainers), this change adds the skill-specific recommendation only where the skill applies — an agent runningairhas an AI Runtime task for it to help with — and names the skill so the agent can tell the hint is about the task at hand. #7452's description makes the same point: calls with no task to apply the skills to "do not convert".Needs maintainer sign-off on the air wording. The generic hint is an existing feature and stays as is; this PR only swaps Claude Code's text on air subcommands and keys the air detection to the exact skill. If maintainers prefer a different wording, target set, or an extension of the skill-specific text to the other agents, that is a small local change.
Why this is based on
main, not stacked on #7452 or #7578:hint.go, so whichever lands second needs a small rebase.InteractiveModeFull, which needs a TTY. Coding agents run the CLI without one. We checked under the Claude Code Bash tool: stdin, stdout and stderr are all non-TTY,CLAUDECODE=1, andCLAUDE_CODE_SESSION_IDis set. So Suppress AI tooling hints in non-interactive sessions #7578 as written would also hide this hint from the agents it is meant for. The air-path tests run with non-TTY stderr and expect the hint, which pins that. If Suppress AI tooling hints in non-interactive sessions #7578 lands, the air hint needs to be exempt from its TTY gate.Tests
New unit tests:
libs/aitools/agents/hint_test.go:TestMaybeHint_NonAIRCommandsKeepGenericHint: outside air (includingauth profiles,version, bareairand the hiddenexperimental airalias) Claude Code gets the generic hint, never the skill line.TestMaybeHint_AIRSubcommandsHint: runnable air subcommands, including nested ones, get the skill line.TestMaybeHint_AIRSkillDetectionandTestMaybeHint_AIRSkillInRepositoryRoot: Claude's own skills dir, project skills dirs, repository root seen from a subdirectory; otherdatabricks*skills only; a file named like the skill; the shared canonical and legacy stores; a plugin for this project vs. an unrelated project.TestMaybeHint_AIRSessionThrottle: with a session id only the first hinted command is hinted, and a generic hint shown on a non-air command covers the session's air commands too.TestMaybeHint_OtherSessionStillHintedandTestAIRHintText.TestHintForCommand: the air/generic split per agent.TestMaybeHint_NativeHinttable gains air rows: child session + air prints the native line then the skill line, and an installed skill silences it.libs/aitools/agents/hint_agents_test.go:TestMaybeHint_AgentInstallHintUnchangedOnAIRpins that non-Claude agents keep their generic install hint on air subcommands.cmd/air/hint_test.go:TestRunnableAirCommandsGetSkillHintwalks the realairtree, so a new subcommand is covered automatically.TestBareAirAndHelpShowFooterWithoutPreRunruns bareairandair --help, and asserts the footer content and that the pre-run hook did not run.cmd/air/air_test.go:TestHelpFooterOnlyOnAirCommandalso asserts the skill line, the benefit line and the--pathpreview.Existing tests and goldens:
regularCmd, throttle, native-hint table,claudeToolingInstalleddetection) are unchanged frommainand still pass.acceptance/air/image-push: the hint on an air subcommand is now the skill line.acceptance/air/help: golden updated for the footer.cmd/aitools/*,cmd/version) are unchanged frommain.Results:
go test ./libs/aitools/... ./cmd/air/... ./cmd/root/... ./cmd/version/... ./libs/telemetry/...passes. AcceptanceTestAccept/cmd/aitools,TestAccept/cmd/versionandTestAccept/airpass.golangci-linton the touched packages reports 0 issues, andgofmtandvalidate_nextchanges.pyare clean.Manual end-to-end check: a local build in a fresh temp HOME, no workspace calls, run under the Claude Code Bash tool (non-TTY):
CLAUDECODE=1, no session id)mainstderrauth profilesversionjobs list(Codex)air listair --help/ bareair--pathpreviewaitools install --path $HOME/preview --skills databricks-ai-runtimeSKILL.md, no stateair liststill hintedair listwith onlydatabricks-jobsinstalledair listwithdatabricks-ai-runtimein<cwd>/.claude/skillsair listwith the plugin installed for another projectair list, thenair get 1, same session idair list/auth profileswithoutCLAUDECODEThis pull request and its description were written by Isaac.