Skip to content

aitools: install AI tools after the first command of a new CLI version - #7702

Merged
anton-107 merged 7 commits into
mainfrom
aitools/auto-install-first-run
Nov 17, 2026
Merged

anton-107 merged 7 commits into
mainfrom
aitools/auto-install-first-run

Conversation

@anton-107

@anton-107 anton-107 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #7701, the PR that stores the auto-install state and adds the opt-out commands and the cli-compat gate.

After a command succeeds, the CLI installs (or updates) Databricks AI tools for the coding agents on the machine, once per new CLI version. Ships off: nothing sets the gate yet.

Summary

root PersistentPostRunE (cmd/cmd.go; cobra skips it when the command failed)
  cmdio.Wait
  aitools.AutoInstall(cmd)                         # cmd/aitools/autoinstall_run.go; recovers panics, returns nothing
    Decide(command, version)                       # libs/aitools/autoinstall/decide.go (pure)
      skipped command | Databricks Runtime
      | cache disabled | development build         -> return (no state read, no log)
    Load state (under auto-install.json.lock)
    Decide again
      settled | retries exhausted
      | older than a CLI that already ran here     -> debug line, return
    TryLock auto-install.lock (busy -> skip); reload state
    Decide with the gate off
      CI                 -> settle, report SKIPPED_CI       # agents.InCI: CI=<anything but false>, or a provider the SDK detects
      opt-out            -> settle, report SKIPPED_OPTOUT   # env var, stored opt-out, or "no" in this process
      gate_off           -> look the gate up (cli-compat auto_install, 3s), Decide again
        still off        -> settle silently
        lookup failed    -> record attempt N, report nothing  # only the embedded manifest was reachable
      holdout (10%)      -> settle, report SKIPPED_HOLDOUT
      run
        record attempt N
        planTools
          global skills on disk, no install state,
            not left by our own unfinished install  -> leave them alone, settle, report KEPT
          no global install state:  mark "installing", buildPlan + executePlan  # as `aitools install --scope=global`,
                                                               # no prompts, agents whose config dir exists
          global install state:     runUpdate(--scope=global)  # as `aitools update`, edited skills kept,
                                                               # no raw-skills-to-plugin migration
          Windows: drop plugin installs; leave an install alone if its update touches plugins -> SKIPPED_PLATFORM
        (10s budget, installer output discarded, warnings demoted to debug)
        installed / changed -> settle, report INSTALLED, stderr: "Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable"
        update, no change -> settle, report UP_TO_DATE
        no agents -> settle, report SKIPPED_NO_AGENTS
        refused downgrade -> keep the installed tools, settle, report KEPT
        error     -> debug line, report FAILED (RETRIES_EXHAUSTED on attempt 10)
  • "Settle" writes last_version = current, attempts = 0 and clears installing: each machine reports at most one outcome per version, plus one per retry. A new version resets the retry budget.
  • Locks are OS file locks (flock / LockFileEx) on files that are never removed. They are only tried, never waited on. The state lock is retried for up to 1 s while another process holds it; any other lock error returns at once. A busy run lock skips the attempt.
  • Agent CLIs run in their own process group (process.BackgroundInGroup), so a timeout stops their descendants too. That takes them out of the terminal's foreground group, so while one runs, the first SIGINT/SIGTERM is caught, stops the group, and is raised again in the CLI: Ctrl-C ends aitools install/update as before, without orphans. Windows has no such stop, so there auto-install runs no agent CLI: a fresh install drops its plugin installs (skills are still installed), and an existing install whose update would touch plugins is left alone. A Windows machine with only plugin-capable agents settles as SKIPPED_NO_AGENTS.
  • Answering "no" at the apps init skills prompt opts out, and says so with the auto-install enable command. It is stored, honored in the process even if the write fails, and passed as DATABRICKS_AITOOLS_AUTO_INSTALL=false to the apps deploy / apps dev-remote children that apps init starts, next to their DATABRICKS_CLI_SELF_EXEC=1.
  • Skipped commands are the version notice's (version, help, completion), the credential helpers (auth token, auth docker token) and the aitools trees. Non-interactive and -o json runs are not skipped, unlike for the notice: coding agents run the CLI that way, and auto-install writes nothing to stdout. Databricks Runtime is skipped (ephemeral home, platform-managed CLI).
  • The gate is unknown, not off, when only the embedded manifest is reachable (clicompat.ResolveAutoInstall fails): the version is not settled, and the lookup uses up one of the version's 10 attempts, so an offline machine stops paying for it.
  • The holdout uses a random id stored in the state (the CLI has no machine id); only the arm (HOLDOUT/TREATMENT) is reported.
  • Development and snapshot builds never auto-install. Acceptance tests set the undocumented DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION to simulate releases and upgrades.
  • Telemetry: aitools_install_event.trigger = AUTO_FIRST_RUN (unset for the command's own installs, so existing goldens do not change) and a new aitools_auto_install_event {outcome, holdout_arm, interactive, error_category}, documented in libs/telemetry/protos/README.md. Outcomes: INSTALLED, UP_TO_DATE, KEPT, SKIPPED_CI, SKIPPED_OPTOUT, SKIPPED_HOLDOUT, SKIPPED_NO_AGENTS, SKIPPED_PLATFORM, FAILED, RETRIES_EXHAUSTED; the lumberjack proto needs the same values.
  • Includes a no-behavior-change refactor: aitools update's RunE body moves to runUpdate; agents.InCI, agents.IsCredentialHelperCommand and versioncheck.IsExemptCommand are exported for reuse; the acceptance helper print_telemetry moves up a directory.
  • No changelog fragment: nothing is user-visible while the gate is off. The PR that turns it on should add one.

Known limitations

Both must be fixed before the gate is turned on:

  • A global aitools uninstall that runs while an automatic install is in flight can see that attempt recreate the tools. Running uninstall again removes them.
  • A partial install (one agent fails) settles as INSTALLED, and later updates never add the missing agent. The same applies to the agents the Windows rule above leaves out. One fix is to record the targeted agents in auto-install.json. (An install cut short by the timeout is different: its installing marker makes the next run finish it.)

Review order

  1. libs/aitools/autoinstall/decide.go: the decision, pure.
  2. cmd/aitools/autoinstall_run.go: the hook, planTools, reporting.
  3. libs/aitools/autoinstall/run.go, lock_unix.go, lock_windows.go, state.go: locks, attempts, Decline / ChildEnv.
  4. cmd/cmd.go, cmd/apps/init.go, libs/aitools/agents/recommend.go: the wiring.
  5. Supporting: cmd/aitools/update.go (refactor), libs/aitools/installer/update.go (KeepModified, HasUntrackedSkills), libs/aitools/installer/plugin.go (process group), libs/telemetry/protos/.

Evidence

New acceptance tests in acceptance/cmd/aitools/auto-install/ run databricks jobs list against the fake server and a fake skills source, with a temporary HOME and agent dirs:

  • first-run:
    >>> [CLI] jobs list
    Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
    {"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],...,"trigger":"AUTO_FIRST_RUN"}}
    {"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
    === second run of the same version: nothing
    === uninstall turns it off: the next version installs nothing   -> SKIPPED_OPTOUT
    
  • upgrade: 1.30.0 installs skills release 0.3.0; 1.31.0 with release 0.4.0 prints Updated Databricks AI tools for Cursor., updates one skill and keeps a skill the user edited; 1.32.0 while the latest release is 0.2.0 refuses the downgrade (files unchanged, nothing printed, settled as KEPT, not retried); a 1.31.0 CLI run afterwards does nothing.
  • skipped: CI=true (reported once per version), DATABRICKS_AITOOLS_AUTO_INSTALL=false, auto-install disable, version/aitools/completion/__complete/help, DATABRICKS_CACHE_ENABLED=false, gate off (nothing reported, version settled), holdout machine, no agent config dir.
  • retries: nine FAILED attempts, RETRIES_EXHAUSTED on the tenth, nothing on the eleventh, a fresh install on the next version.
  • Unit tests: TestDecide (table-driven over every input); TestSkipsAutoInstall; TestPlanToolsRunsNoAgentCLIWhereTimeoutsCannotStopThem, TestPlanToolsReportsWhyNothingIsInstalled and TestPlanToolsKeepsSkillsWithoutInstallState (also: an unfinished install of ours is finished); TestChangedAgents; TestTryLockExcludesOtherRuns, TestLoadWaitsForStateLock, TestLoadFailsWhenStateLockStaysHeld, TestLockReturnsOtherErrorsAtOnce, TestRecordKeepsConcurrentOptOut, TestSetOptedOutWithoutLocks, TestRecordClearsInstallingWhenSettled; TestChildEnvCarriesTheRefusal, TestSelfExecEnvCarriesBothMarkers (the apps init children's env); TestResolveAutoInstallFailsOnTheEmbeddedFallback; TestBackgroundInGroupForwardsInterrupt (a helper process interrupted mid-run dies by SIGINT and its grandchild is reaped); TestRunAgentCmdTimeoutStopsDescendants; TestUpdateKeepModifiedKeepsEditedSkill.
  • Unit tests of cmd/... and libs/..., the aitools and apps acceptance suites, lint and whitespace checks pass on Linux; GOOS=windows/darwin build and vet pass. The gate-lookup failure has no acceptance test: the manifest URL has no override, so it is covered by the clicompat and state unit tests.

Merge Danger

Door: two-way while the gate is off

With the gate off it reads one small file and settles the version on the first run of each release, sends SKIPPED_CI/SKIPPED_OPTOUT events when those apply, and does nothing else. Turning the gate on (a cli-compat.json edit on main) is the one-way step: installs it causes are not rolled back by turning it off again.

Blast Radius: every command (post-run)

  • Runs after every successful command, except the skipped ones and on Databricks Runtime. Off the first run of a version it is one state-file read under a lock (at most 1 s more if another CLI holds the lock); on the first run, with the gate off, a cli-compat manifest lookup (usually the 1 h local cache, at most 3 s). A machine that cannot fetch the manifest repeats the lookup on up to 10 runs of the version.
  • Also changes the explicit aitools install/update today: agent CLIs run in their own process group, with Ctrl-C forwarded to it.
  • Two CLI versions on one machine: only the newest auto-installs; an older one does nothing.
  • With the gate on: up to about 20 s more after the first command of a version (the 10 s install budget plus up to 10 s for a timed-out agent CLI to release its output pipes), plus retries after failures; skill files in global agent dirs; and plugin installs through agent CLIs on PATH (not on Windows).
  • Writes ~/.databricks/aitools/auto-install.json and two lock files next to it.
  • Windows and macOS behavior is built and vetted, not run.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 334416885 | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0
test branch main (1a68ed9) latest (v1.26.0)
TestAccept/cmd/aitools/auto-install/first-run ✅ ❌ ❌
TestAccept/cmd/aitools/auto-install/retries ✅ ❌ ❌
TestAccept/cmd/aitools/auto-install/skipped ✅ ❌ ❌
TestAccept/cmd/aitools/auto-install/upgrade ✅ ❌ ❌
TestAccept/cmd/aitools/auto-install/first-run: branch ✅ | main (1a68ed9) ❌ | latest (v1.26.0) ❌

main (1a68ed9):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36895" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36895" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/first-run/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installfirst-run2635295704/002/output.txt
        @@ -2,20 +2,14 @@
         === first run of a new version: installs for the detected agents, one stderr line
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CLAUDE_CODE","error_category":"CLI_NOT_ON_PATH"}],"trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-test
        +(no skills)
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         === second run of the same version: nothing
         
        @@ -26,14 +14,6 @@
         === uninstall turns it off: the next version installs nothing
         
         >>> [CLI] aitools uninstall
        -Uninstalled 1 skill.
        +Error: no skills or plugins installed
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -(no skills)
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34433" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34433" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/first-run/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installfirst-run1300427957/002/output.txt
        @@ -2,20 +2,14 @@
         === first run of a new version: installs for the detected agents, one stderr line
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Claude Code, Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE","CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CLAUDE_CODE","error_category":"CLI_NOT_ON_PATH"}],"trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-test
        +(no skills)
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         === second run of the same version: nothing
         
        @@ -26,14 +14,6 @@
         === uninstall turns it off: the next version installs nothing
         
         >>> [CLI] aitools uninstall
        -Uninstalled 1 skill.
        +Error: no skills or plugins installed
         
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        -{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -(no skills)
        +Exit code: 1
TestAccept/cmd/aitools/auto-install/retries: branch ✅ | main (1a68ed9) ❌ | latest (v1.26.0) ❌

main (1a68ed9):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33365" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33365" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/retries/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installretries629193382/002/output.txt
        @@ -2,22 +2,13 @@
         === a failed attempt is silent and retried on the next run, up to 10 attempts
         
         >>> print_telemetry
        -9 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"FAILED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
        -9 {"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "attempts": 9,
        -  "installing": true,
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> DATABRICKS_SKILLS_REF=broken-ref [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"RETRIES_EXHAUSTED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
         
         === after 10 attempts the version gets no more
         
        @@ -31,14 +16,8 @@
         === a new version starts over
         
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.31.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39365" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39365" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/retries/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installretries2933552786/002/output.txt
        @@ -2,22 +2,13 @@
         === a failed attempt is silent and retried on the next run, up to 10 attempts
         
         >>> print_telemetry
        -9 {"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"FAILED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
        -9 {"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.30.0",
        -  "attempts": 9,
        -  "installing": true,
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> DATABRICKS_SKILLS_REF=broken-ref [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"RETRIES_EXHAUSTED","holdout_arm":"TREATMENT","interactive":false,"error_category":"SKILLS_FETCH_FAILED"}}
         
         === after 10 attempts the version gets no more
         
        @@ -31,14 +16,8 @@
         === a new version starts over
         
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> auto_install_state
        -{
        -  "last_version": "1.31.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
        
FAIL
TestAccept/cmd/aitools/auto-install/skipped: branch ✅ | main (1a68ed9) ❌ | latest (v1.26.0) ❌

main (1a68ed9):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33583" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33583" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/skipped/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installskipped2453074527/002/output.txt
        @@ -4,7 +4,6 @@
         >>> CI=true [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_CI","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> CI=true [CLI] jobs list
         
        @@ -18,7 +17,6 @@
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
         
         === auto-install disable: not installed
         
        @@ -28,7 +26,6 @@
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
         (no skills)
        @@ -51,10 +48,7 @@
         >>> print_telemetry
         
         >>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        +{}
         
         >>> cursor_skills
         (no skills)
        @@ -66,10 +60,7 @@
         >>> print_telemetry
         
         >>> auto_install_state
        -{
        -  "last_version": "1.33.0",
        -  "id": "treatment-machine"
        -}
        +{}
         
         >>> [CLI] jobs list
         
        @@ -83,7 +74,6 @@
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_HOLDOUT","holdout_arm":"HOLDOUT","interactive":false}}
         
         >>> cursor_skills
         (no skills)
        @@ -93,4 +83,3 @@
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_NO_AGENTS","holdout_arm":"TREATMENT","interactive":false}}
        
    acceptance_test.go:1158: 
        LOG.complete: jobs	The Jobs API allows you to create, edit, and delete jobs.
        LOG.complete: :4
        LOG.complete: Completion ended with directive: ShellCompDirectiveNoFileComp
    acceptance_test.go:1158: 
        LOG.completion: # bash completion V2 for databricks                           -*- shell-script -*-
        LOG.completion: 
        LOG.completion: __databricks_debug()
        LOG.completion: {
        LOG.completion:     if [[ -n ${BASH_COMP_DEBUG_FILE-} ]]; then
        LOG.completion:         echo "$*" >> "${BASH_COMP_DEBUG_FILE}"
        LOG.completion:     fi
        LOG.completion: }
        LOG.completion: 
        LOG.completion: # Macs have bash3 for which the bash-completion package doesn't include
        LOG.completion: # _init_completion. This is a minimal version of that function.
        LOG.completion: __databricks_init_completion()
        LOG.completion: {
        LOG.completion:     COMPREPLY=()
        LOG.completion:     _get_comp_words_by_ref "$@" cur prev words cword
        LOG.completion: }
        LOG.completion: 
        LOG.completion: # This function calls the databricks program to obtain the completion
        LOG.completion: # results and the directive.  It fills the 'out' and 'directive' vars.
        LOG.completion: __databricks_get_completion_results() {
        LOG.completion:     local requestComp lastParam lastChar args
        LOG.completion: 
        LOG.completion:     # Prepare the command to request completions for the program.
        LOG.completion:     # Calling ${words[0]} instead of directly databricks allows handling aliases
        LOG.completion:     args=("${words[@]:1}")
        LOG.completion:     requestComp="${words[0]} __complete ${args[*]}"
        LOG.completion: 
        LOG.completion:     lastParam=${words[$((${#words[@]}-1))]}
        LOG.completion:     lastChar=${lastParam:$((${#lastParam}-1)):1}
        LOG.completion:     __databricks_debug "lastParam ${lastParam}, lastChar ${lastChar}"
        LOG.completion: 
        LOG.completion:     if [[ -z ${cur} && ${lastChar} != = ]]; then
        LOG.completion:         # If the last parameter is complete (there is a space following it)
        LOG.completion:         # We add an extra empty parameter so we can indicate this to the go method.
        LOG.completion:         __databricks_debug "Adding extra empty parameter"
        LOG.completion:         requestComp="${requestComp} ''"
        LOG.completion:     fi
        LOG.completion: 
        LOG.completion:     # When completing a flag with an = (e.g., databricks -n=<TAB>)
        LOG.completion:     # bash focuses on the part after the =, so we need to remove
        LOG.completion:     # the flag part from $cur
        LOG.completion:     if [[ ${cur} == -*=* ]]; then
        LOG.completion:         cur="${cur#*=}"
        LOG.completion:     fi
        LOG.completion: 
        LOG.completion:     __databricks_debug "Calling ${requestComp}"
        LOG.completion:     # Use eval to handle any environment variables and such
        LOG.completion:     out=$(eval "${requestComp}" 2>/dev/null)
        LOG.completion: 
        LOG.completion:     # Extract the directive integer at the very end of the output following a colon (:)
        LOG.completion:     directive=${out##*:}
        LOG.completion:     # Remove the directive
        LOG.completion:     out=${out%:*}
        LOG.completion:     if [[ ${directive} == "${out}" ]]; then
        LOG.completion:         # There is not directive specified
        LOG.completion:         directive=0
        LOG.completion:     fi
        LOG.completion:     __databricks_debug "The completion directive is: ${directive}"
        LOG.completion:     __databricks_debug "The completions are: ${out}"
        LOG.completion: }
        LOG.completion: 
        LOG.completion: __databricks_process_completion_results() {
        LOG.completion:     local shellCompDirectiveError=1
        LOG.completion:     local shellCompDirectiveNoSpace=2
        LOG.completion:     local shellCompDirectiveNoFileComp=4
        LOG.completion:     local shellCompDirectiveFilterFileExt=8
        LOG.completion:     local shellCompDirectiveFilterDirs=16
        LOG.completion:     local shellCompDirectiveKeepOrder=32
        LOG.completion: 
        LOG.completion:     if (((directive & shellCompDirectiveError) != 0)); then
        LOG.completion:         # Error code.  No completion.
        LOG.completion:         __databricks_debug "Received error from custom completion go code"
        LOG.completion:         return
        LOG.completion:     else
        LOG.completion:         if (((directive & shellCompDirectiveNoSpace) != 0)); then
        LOG.completion:             if [[ $(type -t compopt) == builtin ]]; then
        LOG.completion:                 __databricks_debug "Activating no space"
        LOG.completion:                 compopt -o nospace
        LOG.completion:             else
        LOG.completion:                 __databricks_debug "No space directive not supported in this version of bash"
        LOG.completion:             fi
        LOG.completion:         fi
        LOG.completion:         if (((directive & shellCompDirectiveKeepOrder) != 0)); then
        LOG.completion:             if [[ $(type -t compopt) == builtin ]]; then
        LOG.completion:                 # no sort isn't supported for bash less than < 4.4
        LOG.completion:                 if [[ ${BASH_VERSINFO[0]} -lt 4 || ( ${BASH_VERSINFO[0]} -eq 4 && ${BASH_VERSINFO[1]} -lt 4 ) ]]; then
        LOG.completion:                     __databricks_debug "No sort directive not supported in this version of bash"
        LOG.completion:                 else
        LOG.completion:                     __databricks_debug "Activating keep order"
        LOG.completion:                     compopt -o nosort
        LOG.completion:                 fi
        LOG.completion:             else
        LOG.completion:                 __databricks_debug "No sort directive not supported in this version of bash"
        LOG.completion:             fi
        LOG.completion:         fi
        LOG.completion:         if (((directive & shellCompDirectiveNoFileComp) != 0)); then
        LOG.completion:             if [[ $(type -t compopt) == builtin ]]; then
        LOG.completion:                 __databricks_debug "Activating no file completion"
        LOG.completion:                 compopt +o default
        LOG.completion:             else
        LOG.completion: ... (326 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)
    acceptance_test.go:1158: 
        LOG.help: The Jobs API allows you to create, edit, and delete jobs.
        LOG.help: 
        LOG.help:   You can use a Databricks job to run a data processing or data analysis task in
        LOG.help:   a Databricks cluster with scalable resources. Your job can consist of a single
        LOG.help:   task or can be a large, multi-task workflow with complex dependencies.
        LOG.help:   Databricks manages the task orchestration, cluster management, monitoring, and
        LOG.help:   error reporting for all of your jobs. You can run your jobs immediately or
        LOG.help:   periodically through an easy-to-use scheduling system. You can implement job
        LOG.help:   tasks using notebooks, JARS, Spark Declarative Pipelines, or Python, Scala,
        LOG.help:   Spark submit, and Java applications.
        LOG.help: 
        LOG.help:   You should never hard code secrets or store them in plain text. Use the
        LOG.help:   [Secrets CLI] to manage secrets in the [Databricks CLI]. Use the [Secrets
        LOG.help:   utility] to reference secrets in notebooks and jobs.
        LOG.help: 
        LOG.help:   [Databricks CLI]: https://docs.databricks.com/dev-tools/cli/index.html
        LOG.help:   [Secrets CLI]: https://docs.databricks.com/dev-tools/cli/secrets-cli.html
        LOG.help:   [Secrets utility]: https://docs.databricks.com/dev-tools/databricks-utils.html#dbutils-secrets
        LOG.help: 
        LOG.help: Usage:
        LOG.help:   databricks jobs [flags]
        LOG.help:   databricks jobs [command]
        LOG.help: 
        LOG.help: Main Commands
        LOG.help:   cancel-all-runs       Cancel all runs of a job.
        LOG.help:   cancel-run            Cancel a run.
... (32 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42739" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42739" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/skipped/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installskipped3855326088/002/output.txt
        @@ -4,7 +4,6 @@
         >>> CI=true [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_CI","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> CI=true [CLI] jobs list
         
        @@ -18,32 +17,111 @@
         >>> DATABRICKS_AITOOLS_AUTO_INSTALL=false [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
         
         === auto-install disable: not installed
         
         >>> [CLI] aitools auto-install disable
        -Automatic installation of Databricks AI tools is disabled.
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
        +
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
        +
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_OPTOUT","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
         (no skills)
         
         >>> [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
        +
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
        +
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         === commands that never trigger it
         
         >>> [CLI] version
         Databricks CLI v[CLI_VERSION]
         
         >>> [CLI] aitools auto-install enable
        -Automatic installation of Databricks AI tools is enabled.
        +Install Databricks skills and plugins into your coding agent so it can work
        +effectively with Databricks resources (bundles, jobs, SQL, and more).
         
        +Supported agents: Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity, Pi, Gemini CLI, Goose, Kiro.
        +
        +Skills and plugins are sourced from
        +https://github.com/databricks/databricks-agent-skills
        +
        +Usage:
        +  databricks aitools [command]
        +
        +Available Commands:
        +  install     Install Databricks skills and plugins for coding agents
        +  list        List installed skills and plugins
        +  uninstall   Uninstall Databricks skills and plugins
        +  update      Update installed skills and plugins
        +  version     Show installed skills and plugins version
        +
        +Flags:
        +  -h, --help   help for aitools
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks aitools [command] --help" for more information about a command.
        +
         >>> print_telemetry
         
         >>> DATABRICKS_CACHE_ENABLED=false [CLI] jobs list
        @@ -51,10 +108,7 @@
         >>> print_telemetry
         
         >>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> cursor_skills
         (no skills)
        @@ -66,10 +120,7 @@
         >>> print_telemetry
         
         >>> auto_install_state
        -{
        -  "last_version": "1.33.0",
        -  "id": "treatment-machine"
        -}
        +{"id": "treatment-machine"}
         
         >>> [CLI] jobs list
         
        @@ -83,7 +134,6 @@
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_HOLDOUT","holdout_arm":"HOLDOUT","interactive":false}}
         
         >>> cursor_skills
         (no skills)
        @@ -93,4 +143,3 @@
         >>> [CLI] jobs list
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"SKIPPED_NO_AGENTS","holdout_arm":"TREATMENT","interactive":false}}
        
    acceptance_test.go:1158: 
        LOG.complete: jobs	The Jobs API allows you to create, edit, and delete jobs.
        LOG.complete: :4
        LOG.complete: Completion ended with directive: ShellCompDirectiveNoFileComp
    acceptance_test.go:1158: 
        LOG.completion: # bash completion V2 for databricks                           -*- shell-script -*-
        LOG.completion: 
        LOG.completion: __databricks_debug()
        LOG.completion: {
        LOG.completion:     if [[ -n ${BASH_COMP_DEBUG_FILE-} ]]; then
        LOG.completion:         echo "$*" >> "${BASH_COMP_DEBUG_FILE}"
        LOG.completion:     fi
        LOG.completion: }
        LOG.completion: 
        LOG.completion: # Macs have bash3 for which the bash-completion package doesn't include
        LOG.completion: # _init_completion. This is a minimal version of that function.
        LOG.completion: __databricks_init_completion()
        LOG.completion: {
        LOG.completion:     COMPREPLY=()
        LOG.completion:     _get_comp_words_by_ref "$@" cur prev words cword
        LOG.completion: }
        LOG.completion: 
        LOG.completion: # This function calls the databricks program to obtain the completion
        LOG.completion: # results and the directive.  It fills the 'out' and 'directive' vars.
        LOG.completion: __databricks_get_completion_results() {
        LOG.completion:     local requestComp lastParam lastChar args
        LOG.completion: 
        LOG.completion:     # Prepare the command to request completions for the program.
        LOG.completion:     # Calling ${words[0]} instead of directly databricks allows handling aliases
        LOG.completion:     args=("${words[@]:1}")
        LOG.completion:     requestComp="${words[0]} __complete ${args[*]}"
        LOG.completion: 
... (132 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)
TestAccept/cmd/aitools/auto-install/upgrade: branch ✅ | main (1a68ed9) ❌ | latest (v1.26.0) ❌

main (1a68ed9):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41783" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41783" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/upgrade/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installupgrade2712535154/002/output.txt
        @@ -2,65 +2,11 @@
         === first run of 1.30.0 installs skills release 0.3.0
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-dropped
        -databricks-edited
        -databricks-kept
        +(no skills)
        +script: line 299: [TEST_TMP_DIR]/home/.databricks/aitools/skills/databricks-dropped/SKILL.md: No such file or directory
         
        -=== upgrade to 1.31.0 with release 0.4.0: updates like aitools update, keeping the edited skills
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 DATABRICKS_CLI_GITHUB_API_URL=[DATABRICKS_URL]/newer [CLI] jobs list
        -Updated Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
        -
        ->>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -databricks-dropped
        -databricks-edited
        -databricks-kept
        -
        ->>> skill_file databricks-kept
        -kept 2.0.0
        -
        ->>> skill_file databricks-dropped
        -dropped 1.0.0
        -my notes
        -
        ->>> skill_file databricks-edited
        -edited 1.0.0
        -my notes
        -
        -=== upgrade to 1.32.0 while the latest release is 0.2.0: no downgrade, nothing printed, not retried
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.32.0 DATABRICKS_CLI_GITHUB_API_URL=[DATABRICKS_URL]/older [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"KEPT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> skill_file databricks-kept
        -kept 2.0.0
        -
        ->>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        -
        -=== an older CLI on the same machine (e.g. one an IDE ships) does nothing
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -
        ->>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:44079" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:44079" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: Diff:
        --- cmd/aitools/auto-install/upgrade/output.txt
        +++ /tmp/TestAcceptcmdaitoolsauto-installupgrade3366237713/002/output.txt
        @@ -2,65 +2,11 @@
         === first run of 1.30.0 installs skills release 0.3.0
         
         >>> [CLI] jobs list
        -Installed Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
         
         >>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED","trigger":"AUTO_FIRST_RUN"}}
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
         
         >>> cursor_skills
        -databricks-dropped
        -databricks-edited
        -databricks-kept
        +(no skills)
        +script: line 299: [TEST_TMP_DIR]/home/.databricks/aitools/skills/databricks-dropped/SKILL.md: No such file or directory
         
        -=== upgrade to 1.31.0 with release 0.4.0: updates like aitools update, keeping the edited skills
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 DATABRICKS_CLI_GITHUB_API_URL=[DATABRICKS_URL]/newer [CLI] jobs list
        -Updated Databricks AI tools for Cursor. Disable: databricks aitools auto-install disable
        -
        ->>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"INSTALLED","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> cursor_skills
        -databricks-dropped
        -databricks-edited
        -databricks-kept
        -
        ->>> skill_file databricks-kept
        -kept 2.0.0
        -
        ->>> skill_file databricks-dropped
        -dropped 1.0.0
        -my notes
        -
        ->>> skill_file databricks-edited
        -edited 1.0.0
        -my notes
        -
        -=== upgrade to 1.32.0 while the latest release is 0.2.0: no downgrade, nothing printed, not retried
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.32.0 DATABRICKS_CLI_GITHUB_API_URL=[DATABRICKS_URL]/older [CLI] jobs list
        -
        ->>> print_telemetry
        -{"command":"jobs_list","exit_code":0,"aitools_auto_install_event":{"outcome":"KEPT","holdout_arm":"TREATMENT","interactive":false}}
        -
        ->>> skill_file databricks-kept
        -kept 2.0.0
        -
        ->>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        -
        -=== an older CLI on the same machine (e.g. one an IDE ships) does nothing
        -
        ->>> DATABRICKS_AITOOLS_AUTO_INSTALL_CLI_VERSION=1.31.0 [CLI] jobs list
        -
        ->>> print_telemetry
        -
        ->>> auto_install_state
        -{
        -  "last_version": "1.32.0",
        -  "id": "treatment-machine"
        -}
        +Exit code: 1

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 3344168

Run: 44171323874

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 19 7:39
💚​ aws-windows-latest-4core-16gb 1 272 17 6:30
💚​ azure linux-2core-8gb 1 269 19 6:36
💚​ azure-windows-latest-4core-16gb 1 271 17 6:07
💚​ gcp linux-2core-8gb 1 270 19 7:21
💚​ gcp-windows-latest-4core-16gb 1 272 17 6:56
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 4 slowest tests (at least 2 minutes):
duration env testname
3:49 azure-windows-latest-4core-16gb TestAccept
3:29 gcp-windows-latest-4core-16gb TestAccept
3:21 aws-windows-latest-4core-16gb TestAccept
2:10 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 019d40c. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 2 blocking issue(s), tagged [blocking]; 1 open question about a change users could be hurt by.

This PR installs or updates Databricks AI tools after the first successful command of each new CLI version. It is gated off behind cli-compat and has opt-out, CI, holdout, retry and lock handling. The decision logic, locking and tests are careful, and CI is green. Two regressions ship even with the gate off: apps init drops the DATABRICKS_CLI_SELF_EXEC marker on its children, and the new process group stops Ctrl-C from reaching agent CLIs in the explicit aitools install/update. Several gaps should be closed before the gate is turned on: the skip list (auth docker token, CI providers, DBR/non-interactive runs), the gate-failure settle, a stuck partial first install, plugin migration hiding edits, and telemetry outcomes.

Questions

  1. [blocking] Auto-install skips only CI (CI=true), but the version-check notice and MaybeHint also skip Databricks Runtime, non-interactive runs and -o json. Scripts, IDE extensions and ephemeral homes (containers, DBR clusters) pay the gate lookup now: up to 3 s, once per version per fresh home. Once the gate is on, they also pay up to about 20 s, plus retries, after their command's output. Should auto-install reuse those skips, or is running there intended? If it is intended, a comment saying why would help.

Accepted answers

  • Changelog fragment for this change

Comment thread cmd/apps/init.go Outdated
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Stdin = os.Stdin
cmd.Env = autoinstall.ChildEnv()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] cmd.Env = autoinstall.ChildEnv() replaces the cmd.Env = agents.SelfExecEnv(os.Environ()) set four lines above (line 1968). The same happens in runPostCreateDev (lines 1998 and 2002). The apps deploy / apps dev-remote children therefore lose DATABRICKS_CLI_SELF_EXEC=1, and MaybeHint (libs/aitools/agents/hint.go:196) no longer suppresses the coding-agent hint in them. Under Claude Code and other agents, the child prints the hint again inside the parent's output and logs a second hint event. This ships even while the gate is off. Let ChildEnv take the environ slice and compose the two, e.g. cmd.Env = agents.SelfExecEnv(autoinstall.ChildEnv(os.Environ())), at both call sites. That also stops it from reading os.Environ() in library code. A test that checks both markers would cover these call sites, which currently have none.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. ChildEnv now takes the environ, and both call sites use a new selfExecEnv(environ) in cmd/apps/init.go = agents.SelfExecEnv(autoinstall.ChildEnv(environ)), so the children keep DATABRICKS_CLI_SELF_EXEC=1. TestSelfExecEnvCarriesBothMarkers checks both markers, before and after a "no".

Comment thread libs/aitools/installer/plugin.go Outdated
out, err := process.Background(cctx, argv)
// A timed-out agent CLI is stopped with its descendants: one left running
// would hold the output pipes open and block past the timeout.
out, err := process.Background(cctx, argv, process.WithProcessGroup())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] WithProcessGroup moves the agent CLI out of the terminal's foreground process group (here and in libs/aitools/agents/detect.go:115), so Ctrl-C no longer reaches it. Nothing in cmd/root or cmd/aitools catches SIGINT. Ctrl-C kills the CLI, the context is never cancelled, and claude plugin install / marketplace add keep running as orphans that modify ~/.claude. Before this PR, they were interrupted too. This changes the explicit aitools install / update today, for a feature that is off. cmd/environments/sync.go:92-101 documents the same effect for uv and adds a signal handler for that reason. With the gate on, it also releases the run lock while the orphan is still installing, which is the hazard used to exclude Windows. Either cancel the context on SIGINT/SIGTERM while agent CLIs run (see cmd/air/run.go:350 for restoring the default disposition), or use the process group only on the auto-install path.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f, with the signal route so the explicit commands and auto-install both get it. New process.BackgroundInGroup (libs/process/interrupt_unix.go) runs the child with WithProcessGroup and, while it runs, catches the first SIGINT/SIGTERM. It calls signal.Stop first, so a second signal kills at once (the same order as cmd/air/run.go), then cancels, which stops the group, and then raises the signal again in the CLI. The CLI therefore ends as it did before the group, and nothing is orphaned or keeps running past the run lock. runAgentCmd and ProbePlugin use it. On Windows the child stays in the console group, so nothing is forwarded there. TestBackgroundInGroupForwardsInterrupt interrupts a helper process mid-run, then checks that it died by SIGINT and that its grandchild was reaped.

Comment thread libs/aitools/autoinstall/decide.go Outdated
return true
}
}
return slices.Equal(path, []string{"auth", "token"})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] isSkippedCommand exempts only the exact path auth token. docker-credential-databricks runs databricks auth docker token (libs/dockercredentials/shim.go:68) on pulls and pushes. So auto-install runs inside Docker's credential lookup: up to 3 s for the gate now, and an install plus a stderr line once the gate is on. agents.isCredentialHelperCommand (hint.go:268-279) already covers both paths, and versioncheck.isExemptCommand covers the rest of this list. Reuse them instead of a parallel list. That also replaces the hard-coded "__complete" strings with cobra's constants.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. The command list is gone from Decide; it takes a SkippedCommand bool. skipsAutoInstall(cmd) in cmd/aitools is versioncheck.IsExemptCommand || agents.IsCredentialHelperCommand || in an aitools tree. Both helpers are now exported, so auth docker token is covered and the cobra constants are used. Test: TestSkipsAutoInstall.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Command: path,
CacheDisabled: autoinstall.CacheDisabled(ctx),
Version: version,
CI: versioncheck.InCI(ctx),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] versioncheck.InCI is env.GetBool(ctx, "CI"). It misses CI=<provider name>, which GetBool maps to false. It also misses providers that don't set CI, such as Azure Pipelines (TF_BUILD), Jenkins and CodeBuild, which useragent.CiCdProvider() detects. libs/aitools/agents/hint.go:253-266 (inCI) handles both cases, and its comment explains why GetBool can't be used. Such build agents are treated as user machines: the gate lookup today, and installs once the gate is on. Reuse inCI.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. It now uses agents.InCI (exported), so CI=<provider> and SDK-detected providers such as TF_BUILD count. versioncheck.InCI is reverted, so notice.go only has the IsExemptCommand export now.

Comment thread cmd/aitools/autoinstall_run.go Outdated
}
// Skills without install state have no checksums to tell the user's edits
// from ours, and an install would replace them.
if installer.HasUntrackedSkills(ctx, installer.ScopeGlobal) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] InstallSkillsForAgents writes skill directories one by one and saves .state.json only after the loop (installer.go:478). A first attempt that hits the 10 s installTimeout partway leaves skills on disk with no state, and counts as FAILED. On the next run, HasUntrackedSkills is true, so planTools takes the keep path and the run settles INSTALLED. Every later version takes the same path. The machine keeps a partial skill set forever and reports success. Tell the CLI's own interrupted install apart from a legacy one (for example, save state incrementally, or write a marker before the first skill), and retry it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f with a marker. Before a fresh install, the state gets installing: true, which only settling clears. planTools takes the keep path for untracked skills only when that marker is not set, so an install cut short by the timeout is finished (and retried) on a later run, while hand-made or legacy skills are still left alone. Tests: TestPlanToolsKeepsSkillsWithoutInstallState (second half) and TestRecordClearsInstallingWhenSettled; the retries golden shows the marker.

Comment thread cmd/aitools/autoinstall_run.go Outdated
return installResult{}, err
}
result := installResult{updated: true}
if after != nil && toolsChanged(before, after) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] With KeepModified, UpdateSkills still advances state.Release to the new tag (update.go:225) even when every outdated skill was kept. toolsChanged then sees a different Release, and the notice says the tools were updated when no file changed. It also lists every agent with a global install, not just the ones that changed. Compare skills and plugin versions only, or have UpdateSkills report whether anything changed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. toolsChanged became changedAgents, which ignores Release and names an agent only if it has skills and a skill version changed, or if its own plugin version changed. With no changed agent, the run reports UP_TO_DATE and prints nothing. Test: TestChangedAgents.

// explicit choice matters more than a broken record.
func SetOptedOut(ctx context.Context, optedOut bool) error {
s, err := Load(ctx)
unlock, err := lock(ctx, stateLockFileName, stateLockWait)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] SetOptedOut now requires the state lock. On a home directory whose filesystem rejects flock (some NFS/FUSE mounts), lock fails at once, so databricks aitools auto-install disable exits 1. In the base it just wrote the file. Auto-install can't run there either, because Load fails. Consider letting the explicit opt-out write without the lock when locking is unsupported (not merely held).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 879027f. SetOptedOut still returns errLockHeld when another process holds the lock. On any other lock error it writes without the lock (logged at debug), since auto-install cannot run there anyway. Test: TestSetOptedOutWithoutLocks.

Comment thread cmd/aitools/autoinstall_run.go Outdated
}
// Commands and builds that never auto-install skip before the state is
// read, and without a log line: they are most runs.
if reason := autoinstall.Decide(in); reason == autoinstall.ReasonSkippedCommand || reason == autoinstall.ReasonDevelopmentBuild {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The early return covers only skipped commands and development builds. With the cache disabled, the run still calls Load, which creates ~/.databricks/aitools/ and auto-install.json.lock. Add ReasonCacheDisabled to the early return.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. The early return now covers skipped command, Databricks Runtime, cache disabled and development build (stateFreeSkips), so none of them touches ~/.databricks/aitools/.

state, err := autoinstall.Load(ctx)
require.NoError(t, err)
assert.True(t, state.OptedOut)
assert.True(t, autoinstall.Declined())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This test leaves the package-global autoinstall.declined flag set, and the agents package can't reset it. Separately, TestChildEnvCarriesTheRefusal (libs/aitools/autoinstall/run_test.go:102) fails when the developer has DATABRICKS_AITOOLS_AUTO_INSTALL=false set, because ChildEnv reads os.Environ(). Either export a test reset or pass the environ in, and clear EnvVar in that test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. Added autoinstall.ResetDeclined(), which TestRecommendSkillsInstallInteractiveDecline and the new apps test pass to t.Cleanup. ChildEnv takes the environ, so TestChildEnvCarriesTheRefusal no longer reads the developer's environment.

Comment on lines +28 to +30
$CLI completion bash > /dev/null 2>&1
$CLI __complete jobs > /dev/null 2>&1
$CLI help jobs > /dev/null 2>&1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] .agents/rules/testing.md asks to route noisy output to LOG.<name> rather than /dev/null, and to capture a tolerated non-zero exit with errcode rather than || true. cursor_skills in script.prepare:63 uses || true; a [ -d ... ] check, as auto_install_state already does, avoids it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 879027f. cursor_skills checks [ -d ] instead of using || true, and the completion/__complete/help runs in skipped/script go to LOG.* instead of /dev/null.

First of three changes that install Databricks AI tools automatically on
the first run of a new CLI version. This one stores the state and the
opt-out and adds the remote gate; nothing installs yet.

- ~/.databricks/aitools/auto-install.json holds last_version, attempts
  and opted_out. It is a dedicated file, not a libs/cache entry, so an
  opt-out survives upgrades and `databricks cache clear`.
- `databricks aitools auto-install disable|enable` toggles the opt-out;
  enable warns when DATABRICKS_AITOOLS_AUTO_INSTALL=false is set.
- A global-scope `databricks aitools uninstall` (including --skills and
  --agents) opts out.
- cli-compat.json entries get an optional auto_install field, resolved by
  clicompat.ResolveAutoInstall. No entry sets it, and
  TestManifestLeavesAutoInstallOff keeps it off.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/auto-install-state-opt-out branch from 258418b to c5eef9c Compare November 17, 2026 10:30
After a command succeeds, the root PersistentPostRunE installs (or
updates) Databricks AI tools for the coding agents on the machine, once
per new CLI version. It ships off: no cli-compat entry sets the gate.

- autoinstall.Decide is a pure decision over the command, version, state,
  CI, opt-out, gate and holdout; skipped commands and development builds
  return before any state read.
- A fresh install runs as `aitools install --scope=global` without
  prompts for agents whose config dir exists; an existing global install
  runs `aitools update --scope=global`, keeping edited skills. The run has
  a 10s budget, discards installer output and prints one notice line.
- State is updated under OS file locks that are only tried, never waited
  on beyond 1s. Attempts are retried up to 10 times per version.
- Agent CLIs run in their own process group so a timeout stops their
  descendants. Windows cannot do that, so auto-install skips plugin
  installs there.
- Answering "no" at the `apps init` skills prompt opts out and is passed
  to the CLI children `apps init` starts.
- Telemetry: aitools_install_event.trigger = AUTO_FIRST_RUN and a new
  aitools_auto_install_event.
- Refactor: `aitools update`'s RunE body moves to runUpdate,
  versioncheck.InCI is exported, and the print_telemetry acceptance
  helper moves up a directory.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/auto-install-first-run branch from 019d40c to e5662ad Compare November 17, 2026 12:29
anton-107 and others added 2 commits November 17, 2026 14:26
- A global uninstall opts out only when it removed something, so a
  mistyped --skills or an empty scope no longer turns auto-install off.
- `aitools auto-install <typo>` now fails instead of printing help and
  exiting 0.
- Document auto_install in internal/build/README.md (format, the
  copy-forward step for new entries, the trust model).
- Say that any non-true DATABRICKS_AITOOLS_AUTO_INSTALL value opts out,
  and move EnvVar next to the other constant.
- Add unit tests for the state file; pin the env var in the acceptance
  test and log the reset steps instead of discarding them.

Co-authored-by: Isaac <no-reply@databricks.com>
- apps init children keep DATABRICKS_CLI_SELF_EXEC: ChildEnv takes the
  environ and composes with agents.SelfExecEnv.
- Agent CLIs run through process.BackgroundInGroup, which forwards the first
  SIGINT/SIGTERM to the child's process group and then re-raises it, so
  Ctrl-C stops `claude plugin install` again in aitools install/update.
- Skips reuse versioncheck.IsExemptCommand, agents.IsCredentialHelperCommand
  (covers `auth docker token`) and agents.InCI (CI=<provider>, TF_BUILD...),
  and add Databricks Runtime. Cache-disabled runs no longer read the state.
- A gate lookup that only reaches the embedded manifest fails instead of
  reading as off; it uses up an attempt and does not settle the version.
- An unfinished first install is marked in the state, so its skills are
  finished on a later run instead of being kept as untracked forever.
- Auto-install updates no longer migrate or remove raw skills.
- New outcomes UP_TO_DATE, KEPT and SKIPPED_PLATFORM; the update notice
  names only the agents whose skills or plugin changed.
- Saying "no" in apps init says that auto-install is off too.
- The explicit opt-out writes without the lock where locks are unsupported.
- Tests: ResetDeclined, environ passed in; acceptance helpers use LOG.* and
  no `|| true`.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All 14 threads are addressed in 879027f; there's a reply on each one. The PR description is updated too.

Re question 1 [blocking], the skips:

  • Databricks Runtime: now skipped (dbr.RunsOnRuntime). It is checked before the state is read, and nothing is reported. Your reasoning holds: the home doesn't outlive the cluster, and the platform owns the CLI version.
  • CI: broadened to agents.InCI (thread above), so CI=<provider>, Azure Pipelines, Jenkins and CodeBuild are covered.
  • Non-interactive and -o json: intentionally not skipped. Coding agents run the CLI that way, without a TTY and often with -o json, and they are the audience for the tools. Skipping those runs would leave only humans at a terminal, and aitools: ask before auto-installing AI tools in a terminal #7703 asks them before installing. Auto-install never writes to stdout, so JSON output stays clean. This is now a comment on skipsAutoInstall.
  • Lookup cost when the manifest can't be fetched: the lookup no longer settles the version. It uses one of the version's 10 attempts, so an offline machine, container or fresh home pays the 3 s lookup on at most 10 runs, not on every command.

Once the gate is on, the cost after a script's first command of a version is still there: up to about 20 s, plus retries. I've left that as is, because a script can't be told apart from an agent here. DATABRICKS_AITOOLS_AUTO_INSTALL=false turns it off, and so does CI detection. If you'd rather drop -o json runs anyway, I can add that.

Also: the lumberjack proto needs the new outcomes UP_TO_DATE, KEPT and SKIPPED_PLATFORM.

…nstall-first-run

Co-authored-by: Isaac <no-reply@databricks.com>
Base automatically changed from aitools/auto-install-state-opt-out to main November 17, 2026 14:59
…ests

agents.InCI also detects providers such as GITHUB_ACTIONS, so on CI runners
every case reported SKIPPED_CI.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the aitools AI tools (databricks aitools) related changes label Nov 17, 2026

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at eafb251. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 13 non-blocking comment(s) and 1 question(s) below.

This PR installs or updates Databricks AI tools after the first successful command of each new CLI version, behind a cli-compat gate that ships off. It also runs agent CLIs in their own process group and relays Ctrl-C to them. Both blocking issues from the last round are fixed, and nothing blocks merging while the gate stays off. The acceptance cases that failed CI on 15cf94d pass locally with the new unset list, but CI hasn't run on this head because the branch conflicts with main now that #7701 merged. Most comments below should be fixed before the gate is turned on for a version that includes this code: skills in agent folders get moved aside, some interrupted or partial installs never finish, and the gate can read off from a cache an older CLI rewrote.

Questions

  1. Each Reason is classified in three places that have to match Decide's order: Reason.Due, stateFreeSkips (which relies on Decide returning those reasons for a zero State) and skipOutcomes. Would one table per reason (due, state-free, reported outcome) be simpler, so a new reason can't be added to one place and missed in another? Or is the split intended? (non-blocking)

Accepted answers

Agreeing with existing threads

  • [nice to have] A first install that only keeps existing plugins is still reported as INSTALLED: #7702 (comment)

// Skills without install state have no checksums to tell the user's edits
// from ours, and an install would replace them. The ones an unfinished
// install of auto-install left are its own.
if !prior.Installing && installer.HasUntrackedSkills(ctx, installer.ScopeGlobal) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] HasUntrackedSkills only looks in ~/.databricks/aitools/skills and ~/.databricks/agent-skills, so this guard misses Databricks skills that live only in an agent's own folder. Once the gate is on, suppose a user copied or edited databricks-* skills into ~/.cursor/skills, by hand or with another skills tool. After an unrelated command, the fresh install's backupThirdPartySkill (installer.go:666) moves them to $TMPDIR/databricks-skill-backup-* and logs only a debug line. Also check each detected agent's global skills folder for databricks* entries that aren't symlinks to the CLI's copy, and settle as KEPT. Alternatively, skip the backup on the auto-install path.

continue
}

if wasInstalled && opts.KeepModified && !skillPrunable(ctx, baseDir, name, scope, cwd, state) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] UpdateSkills writes each changed skill's files and saves the state once, at the end. Say an automatic update writes skill A, then runs out the 10 s installTimeout on skill B. It returns before SaveState, so A's new files sit next to A's old version and checksums. On every later automatic update, skillPrunable sees that A's files differ from the record, and KeepModified keeps A as "modified" with a warning. A is never updated automatically again. Save the state after each skill, or record the written skills before returning the error, so a retry recognizes its own writes.

if err != nil {
return toolsPlan{}, err
}
if existing != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] A first install's skills batch can fail before it writes anything (the skills manifest returns 5xx, say) while a plugin install succeeds. executePlan then records the plugin, so the state holds plugins and no skills. On the retry existing != nil, so this takes the update path even though prior.Installing is set. runUpdate skips the skills reconcile for a plugins-only state, so the skills agents (Cursor here) never get skills and the version settles as UP_TO_DATE. The description says the installing marker makes the next run finish an install; that does not hold here. When prior.Installing is set, plan a fresh install for the agents that have no tools yet. Alternatively, don't record plugins on the auto-install path until the skills step succeeds.

// embedded manifest is available: that one predates any remote change of the
// gate, so it cannot tell off from not known yet.
func ResolveAutoInstall(ctx context.Context, cliVersion string) (bool, error) {
m, fallback, err := fetchManifest(ctx)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The gate trusts a fresh cached manifest. But writeLocalManifest re-marshals the parsed Manifest, and CLIs from before #7701 have no AutoInstall field, so they write compat-manifest.json without auto_install. With the gate on: a user runs databricks apps init or aitools update on the old CLI, upgrades, and runs any command within the 1 h TTL. Tier 1 returns the old CLI's copy, the gate reads off, and the version settles silently: no install and no event. The lookup runs once per version, so fetch the remote manifest for the gate first, and fall back to the cache only when the fetch fails.

// Runs without a terminal and runs with JSON output are not skipped, unlike
// for the notice: coding agents run the CLI that way, and they are what the
// tools are for. Auto-install writes nothing to stdout.
func skipsAutoInstall(cmd *cobra.Command) bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] Auto-install runs after any command that returns nil, including one the user stopped with Ctrl-C. apps dev-remote catches SIGINT, shuts down and returns nil (cmd/apps/dev.go:219), and never calls signal.Stop. With the gate on, its first run of a version then runs the gate lookup and an install of up to 10 s. Further Ctrl-C presses, and BackgroundInGroup's re-raised SIGINT, land in that stale channel, so the user can't stop it. Under apps init the parent already died from the Ctrl-C, so the orphaned child prints "Installed Databricks AI tools…" after the shell prompt is back. Skip auto-install when DATABRICKS_CLI_SELF_EXEC is set, as MaybeHint does; the parent's own post-run still sees Declined(). Also let auto-install cancel its own context on SIGINT/SIGTERM for its whole run, not only while an agent CLI runs.

}

// isExemptCommand suppresses the notice for commands where it would be noise or
// IsExemptCommand reports whether cmd gets no notice, as it for commands where it would be noise or

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The rename also rewrote this doc comment, which now reads "reports whether cmd gets no notice, as it for commands where…". .agents/rules/comments.md asks not to rewrite existing comments. Change only the identifier: // IsExemptCommand suppresses the notice for commands where it would be noise or….

Comment thread internal/build/README.md

`skills` is an exact pin for its CLI range or `"latest"` to install the skills repository's latest published GitHub release. When the latest-release lookup fails or the selected skills manifest is missing, ranges tracking latest use `AgentSkillsFallbackVersion` from `internal/build/clicompat.go`. Pinned ranges use their embedded manifest's skills version as the fallback. Changing either fallback takes effect in the next CLI build. Released binaries keep the fallback they shipped with.

An entry can also set `"auto_install": true` to turn on the automatic installation of Databricks AI tools on the first run of a new CLI version in its range (`aitools auto-install`). It is off when absent. A change reaches a machine on its first run of a CLI version after the change; machines that resolve to a cached or embedded manifest keep the value they have.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This sentence says machines that resolve to the embedded manifest keep the value they have. ResolveAutoInstall now fails on the embedded fallback, and the run counts an attempt; please update it. The PR description also has two errors. The hook is in cmd/root/root.go, wired in main.go, not cmd/cmd.go. And a Windows machine whose agents all install the plugin through their CLI settles as SKIPPED_PLATFORM (autoinstall_run.go:341), not SKIPPED_NO_AGENTS.

Comment thread cmd/root/root.go
// On the first run of a new CLI version, install Databricks AI tools.
// Cobra runs post-run hooks only when the command succeeded.
if autoinstall.Run != nil {
autoinstall.Run(cmd)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Auto-install runs inside ExecuteContextC, after startTime. So the gate lookup (up to 3 s now) and, once the gate is on, the install (up to about 20 s) count toward the command's execution_time_ms. The first run of each version then skews per-command latency after every release. Report the auto-install time on aitools_auto_install_event and leave it out of the command's time.

Comment on lines +39 to +49
for dir in $PATH; do
keep=1
for bin in claude cursor-agent codex opencode copilot pi gemini goose kiro; do
if [ -x "$dir/$bin" ]; then keep=; fi
for ext in $exts; do
if [ -f "$dir/$bin$ext" ]; then keep=; fi
done
done
if [ -n "$keep" ]; then filtered_path="$filtered_path:$dir"; fi
done
export PATH="${filtered_path#:}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] use_home drops every PATH directory that holds an agent CLI, but print_telemetry needs jq (and print_requests.py needs python3) from PATH. On a machine where jq comes only from the same directory as an agent CLI, every auto-install case fails. An example is Homebrew's /opt/homebrew/bin with an npm-installed claude. Resolve these first, as is done for mkdir/rm/ls. Line 9 also has a run of spaces between CIRCLECI and TRAVIS.

s.LastVersion = version
s.Attempts = attempts
if attempts == 0 {
s.Installing = false

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Installing is cleared only when a version settles. After a first install runs out of attempts, the marker stays set for later versions, and planTools then skips the untracked-skills guard. If skills without install state show up in ~/.databricks/aitools/skills after that (copied by hand, say), the next version's auto-install replaces them. Clear the marker when the attempts run out, or store the version that set it and trust it only for that version.

#7701 landed on main as a squash of this branch's base; its conflicts resolve
to this branch's side, as main has not changed those files since.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit b5bc9ed Nov 17, 2026
28 checks passed
@anton-107
anton-107 deleted the aitools/auto-install-first-run branch November 17, 2026 21:09
anton-107 added a commit that referenced this pull request Nov 17, 2026
Resolve the cmdio test conflict for #7699/#7700: Git Bash detection is
gone and NO_COLOR no longer turns off prompts, so the line-prompt test
checks TERM=dumb instead, and the docs name an MSYS2 pty pipe.

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: b5bc9ed

Run: 44177001355

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 9 1 2 10 1613 1359 243:47
🔄​ aws-windows-latest-4core-16gb 3 10 1541 1386 240:45
❌​ azure linux-2core-8gb 9 1 12 1593 1367 252:25
🔄​ azure-windows-latest-4core-16gb 3 12 1519 1394 240:22
❌​ gcp linux-2core-8gb 9 1 12 1600 1365 251:59
🔄​ gcp-windows-latest-4core-16gb 3 10 1528 1392 249:34
24 interesting tests: 9 FAIL, 9 RECOVERED, 5 flaky, 1 KNOWN
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🟨​ TestAccept 🟨​K 🔄​f 🟨​K 🔄​f 🟨​K 🔄​f
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true ❌​F ❌​F ❌​F
💚​ TestAccept/bundle/resources/apps/lifecycle-started 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate 💚​R 🔄​f 💚​R 💚​R 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS= 🔄​f 💚​R 💚​R 💚​R 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 🔄​f 🔄​f 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/templates/default-python/combinations/serverless ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=no/PY=yes/READPLAN=1 ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
13:42 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
13:28 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
12:29 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:51 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:37 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:33 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:04 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:50 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:41 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:41 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:36 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:22 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:21 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
9:11 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:01 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:55 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:54 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:51 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
8:45 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
8:43 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:37 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:34 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
8:25 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
8:16 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:16 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
8:06 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
7:44 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:26 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:22 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true
7:15 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:11 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:04 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:01 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
7:00 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:53 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:52 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:48 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
6:42 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:41 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:38 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:38 azure-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:37 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:33 azure linux-2core-8gb TestAccept/pipelines/run/refresh-flags
6:32 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:31 aws-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:29 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:21 gcp-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:18 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:15 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
5:53 aws linux-2core-8gb TestAccept/pipelines/run/refresh-flags

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…s#7703)

Stacked on databricks#7702, the PR that installs AI tools after the first command
of a new CLI version.

When a person runs the CLI in a terminal, auto-install asks before its
first install.

## Summary

```diff
 attempt(prior state)
   plan := planTools()                       # local only: install state, detected agents
+  if a person may be at the terminal && no "yes" stored for this version (accepted_version)
+    no existing install && some agent would get tools
+      cannot ask                            -> skip; save the machine ID, nothing installed,
+                                               version not settled (a later run asks)
+      discard type-ahead (cannot -> cannot ask)
+      settle the version                    # asked once per version, answered or not
+      "Install Databricks AI tools for Claude Code, Cursor? [Y/n]: "    (1 min)
+        Enter / y / yes -> store accepted_version (its retries don't ask), continue
+        n / no          -> store opted_out, report DECLINED,
+                           "Skipped. You won't be asked again; to install Databricks AI tools later, run: databricks aitools install"
+        anything else   -> report UNANSWERED, "Skipped. To install ... later, run: databricks aitools install"
+        no answer       -> report UNANSWERED (stdin closed, Ctrl+C, or 1 min passed); not opted out
   record attempt; install within 10s
+  failed after a "yes"                      -> "Could not install Databricks AI tools. To install them, run: databricks aitools install"
```

- "A person may be at the terminal": stdin, stdout or stderr is a
terminal and no coding agent is detected. The question is asked only
when it can be seen and answered (`cmdio.IsLinePromptSupported`): all
three streams are terminals (a redirected stream would hide the question
or take the answer from elsewhere, and a pager reading stdout would
share the terminal's keystrokes), the process is in the terminal's
foreground process group (a background job would stop on SIGTTIN), and
no goroutine still reads stdin (`apps dev-remote`, via
`cmdio.MarkStdinReadInBackground`). Runs with no terminal at all keep
the parent's non-interactive install.
- "Asked" is recorded explicitly: a "yes" stores `accepted_version`, and
the question and the early skip check it, not the attempt count. That
count also advances on a failed gate lookup, which never asks. A
planning failure is an ordinary failed attempt whose retry asks.
- A run that cannot ask returns before the lock and the gate lookup, and
saves the machine ID it drew. The locked path saves a drawn ID before
deciding, and `autoinstall.SaveID` returns the stored ID, so concurrent
runs agree on the holdout bucket. Holdout machines and gate-off versions
wait for a run that can ask, like treatment machines.
- A CLI process that the CLI started itself (`DATABRICKS_CLI_SELF_EXEC`,
now `agents.IsSelfExec`) does not ask; the parent asks after its own
command.
- Input typed while the command ran is discarded before asking
(`cmdio.DiscardInput`: tcflush on Linux and macOS,
FlushConsoleInputBuffer on Windows), and Ctrl+C at the question counts
as no answer, so the command keeps its exit status. Where input can't be
discarded (a pipe that Git Bash's mintty counts as a terminal), the
question is skipped. A timed-out or interrupted question also turns off
later line prompts in the process, since its read stays blocked on
stdin.
- The question is a plain-text line (`cmdio.AskCancellable`), so
`NO_COLOR`, `TERM=dumb` and Git Bash, which turn off the TUI prompts,
still get it. `cmdio.HasTerminal` and the new
`cmdio.IsLinePromptSupported` express the two checks.
- Only a fresh install asks. Updates of an existing installation and
retries after a "yes" do not.
- The question names only the agents that will get something (skills, or
a plugin whose CLI is on PATH); on Windows that excludes agents
installed through their CLI, which the plan already filters out. With
none, there is no question.
- An agent in a pseudo-terminal that the SDK does not recognize waits at
most a minute, once per version.
- `DECLINED` and `UNANSWERED` added to the auto-install outcome enum;
the Universe proto for the whole event is
https://github.com/databricks-eng/universe/pull/3127275. `auto-install
--help` mentions the prompt.

Review order: `cmd/aitools/autoinstall_run.go` (`autoInstall`'s early
skip and ID save, `attempt`, `canAsk`, `skipUnaskable`, `ask`,
`decline`), `libs/aitools/autoinstall` (`AcceptedVersion`, `Accept`,
`SaveID`), then `libs/cmdio/io.go` and `libs/cmdio/terminal_*.go`, then
`cmd/aitools/autoinstall_run_test.go`.

## Evidence

The prompt needs a terminal, which acceptance tests lack, so
`TestAutoInstallPrompt` (cmd/aitools) drives the hook through a test
terminal and checks the prompt and notice on stderr, the skill in
Cursor's dir, the stored state and the event:

```text
enter accepts                                     -> installed, INSTALLED, interactive=true
yes accepts                                       -> installed
no declines and opts out                          -> no skill, opted_out=true, DECLINED
no answer settles the version without opting out  -> no skill, last_version=1.30.0, UNANSWERED
another answer settles the version without opting out -> no skill, not opted out, UNANSWERED
a planning failure is a failed attempt, retried later -> no prompt, FAILED, attempts=1
a failed install after yes says so                -> "Could not install ...", FAILED, accepted_version=1.30.0
NO_COLOR still asks                               -> prompt, Enter installs
TERM=dumb still asks                              -> prompt, "n" opts out, DECLINED
redirected stderr skips without installing        -> no prompt, no skill, no last_version, no event
piped stdin skips without installing              -> no prompt, no skill, no last_version, no event
stdout to a pager skips without installing        -> no prompt, no skill, no last_version, no event
no terminal at all installs without asking        -> no prompt, installed
a retry after yes is not asked again              -> no prompt, installed
a retry that was never asked asks                 -> (attempts=1 from a gate failure) prompt, installed
a retry that was never asked skips when it cannot ask -> no install, attempts=1 kept
a CLI process started by the CLI does not ask     -> no prompt, no skill, version not settled
a holdout machine that cannot ask waits too       -> no event, version not settled
(every case: stdout empty)
```

`TestAutoInstallKeepsTheIDWhenItCannotAsk` runs a partial-terminal hook
twice on a machine with no stored ID: the ID is saved and stays the
same, and the version is not settled. `TestToolsPlanNewAgents` covers
which agents are named. `TestCapabilities_SupportsLinePrompt` and
`TestIsLinePromptSupported` (libs/cmdio) cover the stream checks and the
background stdin reader.

On a real terminal (tmux, Linux, `databricks auth profiles` with the
gate cached on):

```text
cli auth profiles &                    -> job ends "Done", no prompt, no "Stopped (tty input)", version not settled
sleep 2; cli ...  (typed "n⏎" in sleep) -> prompt still waits: the type-ahead was discarded
Ctrl+C at the prompt                   -> "^C", newline, EXIT=0, last_version=1.30.0, not opted out
state {last_version:1.30.0, attempts:1} -> the next foreground run asks (after the restack onto eafb251)
``` With Enter mapped to "no" (a temporary edit, not committed) `enter
accepts` fails, so the test catches the default.

Locally on Linux at 957cbf0 (after merging the base at eafb251): unit
tests of `cmd/aitools` (also shuffled), `cmd/apps`, `cmd/root`,
`libs/cmdio` (also shuffled), `libs/aitools/...`, `libs/apps/...`,
`libs/telemetry/...` and `libs/process/...` pass, and so do the
`cmd/aitools` and `cmd/apps` acceptance tests. golangci-lint, whitespace
and deadcode pass. `GOOS=windows`/`darwin` build and vet pass, and so
does a `GOOS=freebsd` vet of `libs/cmdio`. I did not run the full unit
and acceptance suites.

## Merge Danger

**Door:** two-way

Behind the same off gate as the parent.

**Blast Radius:** small

Once the gate is on, a person in a terminal sees one prompt after the
first command of a version that has nothing installed; the command waits
for the answer (at most a minute). A run with only some streams on a
terminal installs nothing until a run in a full terminal. On the same
first run, a Claude Code user without skills also sees the existing
install hint (unchanged) above the "Installed ..." line.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…cks#7802)

## Changes
`TestRecordKeepsConcurrentOptOut` starts 21 writers (20 `Record` + 1
`SetOptedOut`) that take the state lock one at a time. Each writer waits
at most `stateLockWait` (1s) for the lock. On slow runners, mostly
Windows, each locked read-and-rename takes long enough that the last
writers time out with `held by another CLI process`. The failing Windows
attempt ran for 3.83s, about 180ms per writer. The test was added in
databricks#7702.

This PR makes `stateLockWait` a package var and raises it to a minute in
this test. Production behavior is unchanged: the wait is still 1s, which
is plenty for the few CLI processes that contend for the lock in real
use.

Decision: I kept the 20 concurrent `Record` calls. Using fewer writers
would make the flake rarer without fixing it, and it would make the test
less likely to catch a lost opt-out.

## Why
Flaky on main and in unrelated PRs: 12 failures in 3 days, 9 of them on
Windows.

## Tests
- A temporary local test (not committed) wrapped `tryLock` so each
holder slept 60ms. It failed every run before this change (5 "held by
another CLI process" errors per run) and passed after.
- `go test -race -count=3 ./libs/aitools/autoinstall/` passes.

## Dashboard
-
[error/3669](http://go/cli/dashboard/error/3669?main=1&pr=1&end=2026-11-19T12:00:00Z&last=3d)
-
[libs/aitools/autoinstall.TestRecordKeepsConcurrentOptOut](http://go/cli/dashboard/test/libs/aitools/autoinstall.TestRecordKeepsConcurrentOptOut?main=1&pr=1&end=2026-11-19T12:00:00Z&last=3d)

```
Error:      	Received unexpected error:
            	[PATH]/.databricks/aitools/auto-install.json.lock: held by another CLI process
Test:       	TestRecordKeepsConcurrentOptOut
--- FAIL: TestRecordKeepsConcurrentOptOut (3.83s)
```

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants