Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .nextchanges/cli/aitools-hint-exempt-commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* Stop printing the Databricks AI tooling install hint on `auth`, `configure`, `current-user me`, `version`, `help`, and shell-completion commands. ([#7452](https://github.com/databricks/cli/pull/7452))
30 changes: 10 additions & 20 deletions acceptance/cmd/aitools/agent-hints/output.txt
Original file line number Diff line number Diff line change
@@ -1,46 +1,36 @@

=== Codex is hinted once per thread

>>> CODEX_CI=1 CODEX_THREAD_ID=thread-1 [CLI] version
>>> CODEX_CI=1 CODEX_THREAD_ID=thread-1 [CLI] jobs list
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents codex --scope global
Databricks CLI v[CLI_VERSION]

>>> CODEX_CI=1 CODEX_THREAD_ID=thread-1 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> CODEX_CI=1 CODEX_THREAD_ID=thread-1 [CLI] jobs list

>>> CODEX_CI=1 CODEX_THREAD_ID=thread-2 [CLI] version
>>> CODEX_CI=1 CODEX_THREAD_ID=thread-2 [CLI] jobs list
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents codex --scope global
Databricks CLI v[CLI_VERSION]

=== Cursor exposes no session id, so it is hinted once per machine

>>> CURSOR_AGENT=1 [CLI] version
>>> CURSOR_AGENT=1 [CLI] jobs list
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents cursor --scope global
Databricks CLI v[CLI_VERSION]

>>> CURSOR_AGENT=1 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> CURSOR_AGENT=1 [CLI] jobs list

=== Copilot CLI maps to the copilot registry agent

>>> COPILOT_CLI=1 [CLI] version
>>> COPILOT_CLI=1 [CLI] jobs list
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents copilot --scope global
Databricks CLI v[CLI_VERSION]

=== Gemini CLI with its own Databricks skills is not hinted

>>> GEMINI_CLI=1 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> GEMINI_CLI=1 [CLI] jobs list

=== Agents the CLI cannot install for are not hinted

>>> WINDSURF_AGENT=1 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> WINDSURF_AGENT=1 [CLI] jobs list

>>> VSCODE_AGENT=1 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> VSCODE_AGENT=1 [CLI] jobs list

=== Nested agents are not hinted

>>> CODEX_CI=1 CURSOR_AGENT=1 CODEX_THREAD_ID=thread-3 [CLI] version
Databricks CLI v[CLI_VERSION]
>>> CODEX_CI=1 CURSOR_AGENT=1 CODEX_THREAD_ID=thread-3 [CLI] jobs list
23 changes: 12 additions & 11 deletions acceptance/cmd/aitools/agent-hints/script
Original file line number Diff line number Diff line change
@@ -1,27 +1,28 @@
# aitools commands skip the install hint; version exercises it without auth or network.
# Hint-exempt commands such as aitools and version skip the install hint; jobs list
# exercises it against the fake server and prints nothing to stdout.
sethome ./home
unset XDG_CONFIG_HOME GEMINI_CLI_HOME CODEX_THREAD_ID

title "Codex is hinted once per thread"
trace CODEX_CI=1 CODEX_THREAD_ID=thread-1 $CLI version
trace CODEX_CI=1 CODEX_THREAD_ID=thread-1 $CLI version
trace CODEX_CI=1 CODEX_THREAD_ID=thread-2 $CLI version
trace CODEX_CI=1 CODEX_THREAD_ID=thread-1 $CLI jobs list
trace CODEX_CI=1 CODEX_THREAD_ID=thread-1 $CLI jobs list
trace CODEX_CI=1 CODEX_THREAD_ID=thread-2 $CLI jobs list

title "Cursor exposes no session id, so it is hinted once per machine"
trace CURSOR_AGENT=1 $CLI version
trace CURSOR_AGENT=1 $CLI version
trace CURSOR_AGENT=1 $CLI jobs list
trace CURSOR_AGENT=1 $CLI jobs list

title "Copilot CLI maps to the copilot registry agent"
trace COPILOT_CLI=1 $CLI version
trace COPILOT_CLI=1 $CLI jobs list

title "Gemini CLI with its own Databricks skills is not hinted"
sethome ./gemini-home
trace GEMINI_CLI=1 $CLI version
trace GEMINI_CLI=1 $CLI jobs list
sethome ./home

title "Agents the CLI cannot install for are not hinted"
trace WINDSURF_AGENT=1 $CLI version
trace VSCODE_AGENT=1 $CLI version
trace WINDSURF_AGENT=1 $CLI jobs list
trace VSCODE_AGENT=1 $CLI jobs list

title "Nested agents are not hinted"
trace CODEX_CI=1 CURSOR_AGENT=1 CODEX_THREAD_ID=thread-3 $CLI version
trace CODEX_CI=1 CURSOR_AGENT=1 CODEX_THREAD_ID=thread-3 $CLI jobs list
49 changes: 34 additions & 15 deletions acceptance/cmd/aitools/claude-config-dir/output.txt
Original file line number Diff line number Diff line change
@@ -1,38 +1,57 @@

=== Plugin installed only in the custom config directory

>>> CLAUDE_CONFIG_DIR=custom [CLI] version
Databricks CLI v[CLI_VERSION]
>>> CLAUDE_CONFIG_DIR=custom [CLI] clusters list
ID Name State
4321 some-test-cluster
9876 some-other-cluster

=== Custom directory takes precedence over the default installation

>>> CLAUDE_CONFIG_DIR=missing [CLI] version
>>> CLAUDE_CONFIG_DIR=missing [CLI] clusters list
<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
Databricks CLI v[CLI_VERSION]
ID Name State
4321 some-test-cluster
9876 some-other-cluster

=== Exempt commands never hint, even without the tooling

>>> CLAUDE_CONFIG_DIR=missing [CLI] current-user me
json.userName = "[USERNAME]";

=== Empty override uses the default installation

>>> CLAUDE_CONFIG_DIR= [CLI] version
Databricks CLI v[CLI_VERSION]
>>> CLAUDE_CONFIG_DIR= [CLI] clusters list
ID Name State
4321 some-test-cluster
9876 some-other-cluster

=== Unset override uses the default installation

>>> [CLI] version
Databricks CLI v[CLI_VERSION]
>>> [CLI] clusters list
ID Name State
4321 some-test-cluster
9876 some-other-cluster

=== Outside a Claude Code subprocess only the text hint prints

>>> CLAUDE_CODE_CHILD_SESSION= [CLI] version
>>> CLAUDE_CODE_CHILD_SESSION= [CLI] clusters list
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
Databricks CLI v[CLI_VERSION]
ID Name State
4321 some-test-cluster
9876 some-other-cluster

=== Within one session both lines print once, so 2>&1 pipelines see them only on the first command

>>> [CLI] version
>>> [CLI] clusters list
<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
Databricks CLI v[CLI_VERSION]

>>> [CLI] version
Databricks CLI v[CLI_VERSION]
ID Name State
4321 some-test-cluster
9876 some-other-cluster

>>> [CLI] clusters list
ID Name State
4321 some-test-cluster
9876 some-other-cluster
20 changes: 12 additions & 8 deletions acceptance/cmd/aitools/claude-config-dir/script
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,34 @@ unset CLAUDE_CODE_SESSION_ID
# CI and the CI/CD provider markers the SDK detects (GitHub Actions sets both).
unset CI GITHUB_ACTIONS GITLAB_CI JENKINS_URL TF_BUILD CIRCLECI TRAVIS BITBUCKET_BUILD_NUMBER PROJECT_ID CODEBUILD_BUILD_ARN TFC_RUN_ID

# aitools commands skip the install hint; version exercises it without auth or network.
# Hint-exempt commands (aitools, auth, one-shot lookups such as version and current-user me,
# completion) skip the install hint; clusters list exercises it against the fake server.
sethome ./empty-home
title "Plugin installed only in the custom config directory"
trace CLAUDE_CONFIG_DIR=custom $CLI version
trace CLAUDE_CONFIG_DIR=custom $CLI clusters list

sethome ./installed-home
title "Custom directory takes precedence over the default installation"
trace CLAUDE_CONFIG_DIR=missing $CLI version
trace CLAUDE_CONFIG_DIR=missing $CLI clusters list

title "Exempt commands never hint, even without the tooling"
trace CLAUDE_CONFIG_DIR=missing $CLI current-user me | gron.py | grep userName

title "Empty override uses the default installation"
trace CLAUDE_CONFIG_DIR= $CLI version
trace CLAUDE_CONFIG_DIR= $CLI clusters list

title "Unset override uses the default installation"
unset CLAUDE_CONFIG_DIR
trace $CLI version
trace $CLI clusters list

sethome ./empty-home
title "Outside a Claude Code subprocess only the text hint prints"
trace CLAUDE_CODE_CHILD_SESSION= $CLI version
trace CLAUDE_CODE_CHILD_SESSION= $CLI clusters list

# The session throttle lives in the file cache, which a host could disable.
export DATABRICKS_CACHE_ENABLED=true
title "Within one session both lines print once, so 2>&1 pipelines see them only on the first command"
export CLAUDE_CODE_SESSION_ID=session-1
trace $CLI version
trace $CLI version
trace $CLI clusters list
trace $CLI clusters list
unset CLAUDE_CODE_SESSION_ID
3 changes: 1 addition & 2 deletions acceptance/cmd/aitools/telemetry/hint/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ Databricks skills are not installed. To work with Databricks reliably, first run

=== hint printed on a command that resolves no auth: nothing is uploaded

>>> [CLI] version
>>> [CLI] bundle schema
<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
Databricks CLI v[CLI_VERSION]

>>> print_telemetry

Expand Down
2 changes: 1 addition & 1 deletion acceptance/cmd/aitools/telemetry/hint/script
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ trace $CLI jobs list
trace print_telemetry

title "hint printed on a command that resolves no auth: nothing is uploaded"
trace $CLI version
trace $CLI bundle schema > /dev/null

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] .agents/rules/testing.md asks for noisy output to go to LOG.<name>, not /dev/null, so it stays visible under go test -v. Use trace $CLI bundle schema > LOG.schema; the golden still shows only the hint on stderr.

trace print_telemetry

title "plugin installed: no hint, no event"
Expand Down
6 changes: 2 additions & 4 deletions acceptance/cmd/version/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,14 @@ Tip: coding agents detected without Databricks skills. Run 'databricks aitools i
"development_build": true
}

=== version --check from Claude Code prints only the root hint
=== version --check from Claude Code prints no hint

>>> [CLI] version --check
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
Databricks CLI v[CLI_VERSION]
This is a development build; skipping the update check.

=== version --check from Codex prints only the root hint
=== version --check from Codex prints no hint

>>> [CLI] version --check
Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install --agents codex --scope global
Databricks CLI v[CLI_VERSION]
This is a development build; skipping the update check.
7 changes: 4 additions & 3 deletions acceptance/cmd/version/script
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,16 @@ trace $CLI version --check
title "version --check --output json with a coding agent prints no tip"
trace $CLI version --check --output json

# Callers that agents.MaybeHint hints get only the root hint, not the tip.
title "version --check from Claude Code prints only the root hint"
# version is hint-exempt, and callers that agents.MaybeHint hints get no tip
# either, so they see no install nudge at all.
title "version --check from Claude Code prints no hint"
sethome ./claude-home
export CLAUDECODE=1
unset CLAUDE_CODE_SESSION_ID
trace $CLI version --check
unset CLAUDECODE

title "version --check from Codex prints only the root hint"
title "version --check from Codex prints no hint"
sethome ./codex-home
export CODEX_CI=1
unset CODEX_THREAD_ID
Expand Down
49 changes: 33 additions & 16 deletions libs/aitools/agents/hint.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,6 @@ const (
// unattended runs.
ciEnv = "CI"

// aiToolsCommandName is the top-level command whose subtree is never hinted:
// telling the agent to run `databricks aitools install` while it is already
// running an aitools command makes no sense.
aiToolsCommandName = "aitools"

// authCommandName and tokenCommandName name `databricks auth token`, the
// credential helper the SDKs and Terraform run. They embed its stderr
// mid-line in their own errors, where Claude Code can't strip the native
Expand Down Expand Up @@ -171,9 +166,10 @@ func (h *agentHint) nativeLine(ctx context.Context) string {

// MaybeHint recommends installing the Databricks AI tooling when a supported
// agent drives the CLI without it installed. It is a no-op for human callers,
// unsupported agents, aitools commands themselves, the `auth token` and `auth
// docker token` credential helpers, a CLI process the CLI started itself, when
// there is no output stream, and once the tooling is present.
// unsupported agents, hint-exempt commands (isHintExemptCommand), the `auth
// token` and `auth docker token` credential helpers, a CLI process the CLI
// started itself, when there is no output stream, and once the tooling is
// present.
// Both lines, the agent-native one (if any) and the model-facing text, and
// their hint-shown events are throttled together per session, or per agent on
// this machine for agents without a session id (fallbackKey).
Expand All @@ -187,7 +183,7 @@ func MaybeHint(ctx context.Context, cmd *cobra.Command, pluginRecorded PluginRec
if hint == nil {
return
}
if isAIToolsCommand(cmd) {
if isHintExemptCommand(cmd) {
return
}
if isCredentialHelperCommand(cmd) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Under the real databricks root, the auth entry in hintExemptCommands already returns before isCredentialHelperCommand runs, so this branch looks dead. The reason it stays (auth token and auth docker token must stay silent even if auth leaves the list) is only in the PR description and the test comment at hint_test.go:403. A later cleanup could drop the check as redundant; if auth is then removed from the list, the hint ends up inside SDK and Terraform error messages. .agents/rules/comments.md asks for a comment on branches kept as guards. Please add one line here, e.g. "Overlaps the auth exemption on purpose: the credential helpers must stay silent even if auth leaves hintExemptCommands."

Expand Down Expand Up @@ -278,11 +274,33 @@ func isCredentialHelperCommand(cmd *cobra.Command) bool {
return parent != nil && parent.Name() == authCommandName
}

// isAIToolsCommand reports whether cmd is the aitools command or one of its
// subcommands, walking the parent chain (mirrors versioncheck.isExemptCommand).
func isAIToolsCommand(cmd *cobra.Command) bool {
for c := cmd; c != nil; c = c.Parent() {
if c.Name() == aiToolsCommandName {
// hintExemptCommands are the commands, with their subcommands, on which no
// tooling hint prints. Paths are relative to the root command. The aitools
// trees are exempt because telling the agent to run aitools while it already
// runs aitools makes no sense. The rest are plumbing or one-shot lookups (auth
// checks, "who am I", version, help, shell completion): the agent has no task
// to apply the skills to. The nudge eval measured only "who am I": 0 of 20
// runs installed the tooling.
var hintExemptCommands = []string{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The entries in hintExemptCommands are hard-coded path strings, and TestIsHintExemptCommand builds fake trees with commandAt. Nothing checks that each path exists in the real CLI. If codegen renames current-user/me, or an aitools command moves, the exemption silently stops matching and the unit tests still pass; only the single current-user me acceptance step would catch it. Consider adding a test in an external package (or under cmd/) that builds cmd.New(ctx), resolves each entry with Find(strings.Fields(p)), and asserts an exact match.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not add this one. current-user me is the only generated entry, and the claude-config-dir step already runs it against the real tree; the rest are hand-written commands or cobra constants. The test would also need special cases for cobra's lazily registered help and __complete, and with the predicate now package-private it would have to live in the agents test binary and pull in cmd. Happy to revisit if you feel strongly.

"aitools",
"experimental aitools",
"auth",
"configure",
"current-user me",
"version",
"completion",
"help",
cobra.ShellCompRequestCmd,
}

// isHintExemptCommand reports whether cmd is one of hintExemptCommands or a
// subcommand of one.
func isHintExemptCommand(cmd *cobra.Command) bool {
path := cmd.CommandPath()
root := cmd.Root().CommandPath()
for _, exempt := range hintExemptCommands {
full := root + " " + exempt
if path == full || strings.HasPrefix(path, full+" ") {
return true
}
}
Expand Down Expand Up @@ -367,8 +385,7 @@ func SelfExecEnv(environ []string) []string {
// ExecutionContext.Command names the command the hint was shown on.
//
// The event is buffered and uploaded by cmd/root at exit, which needs resolved
// auth: a hint shown on a command that never resolves auth (e.g. `databricks
// version`) is not uploaded.
// auth: a hint shown on a command that never resolves auth is not uploaded.
func logHintShown(ctx context.Context, kind protos.AitoolsHintKind) {
telemetry.Log(ctx, protos.DatabricksCliLog{
AitoolsHintEvent: &protos.AitoolsHintEvent{HintKind: kind},
Expand Down
5 changes: 1 addition & 4 deletions libs/aitools/agents/hint_agents_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -259,10 +259,7 @@ func TestMaybeHint_AgentAuthTokenSilent(t *testing.T) {
func TestMaybeHint_AgentAIToolsCommandSilent(t *testing.T) {
setAgentEnv(t, "CODEX_CI")
ctx, stderr := agentCtx(t, t.TempDir(), t.TempDir())
parent := &cobra.Command{Use: aiToolsCommandName}
child := &cobra.Command{Use: "install"}
parent.AddCommand(child)
MaybeHint(ctx, child, noPluginRecorded)
MaybeHint(ctx, commandAt("aitools install"), noPluginRecorded)
assert.Empty(t, stderr.String())
}

Expand Down
Loading
Loading