Skip to content

aitools: ask before auto-installing AI tools in a terminal - #7703

Merged
anton-107 merged 12 commits into
mainfrom
aitools/auto-install-prompt
Nov 17, 2026
Merged

anton-107 merged 12 commits into
mainfrom
aitools/auto-install-prompt

Conversation

@anton-107

@anton-107 anton-107 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #7702, the PR that installs AI tools after the first command of a new CLI version.

When a person runs the CLI in a terminal, auto-install asks before its first install.

Summary

 attempt(prior state)
   plan := planTools()                       # local only: install state, detected agents
+  if a person may be at the terminal && no "yes" stored for this version (accepted_version)
+    no existing install && some agent would get tools
+      cannot ask                            -> skip; save the machine ID, nothing installed,
+                                               version not settled (a later run asks)
+      discard type-ahead (cannot -> cannot ask)
+      settle the version                    # asked once per version, answered or not
+      "Install Databricks AI tools for Claude Code, Cursor? [Y/n]: "    (1 min)
+        Enter / y / yes -> store accepted_version (its retries don't ask), continue
+        n / no          -> store opted_out, report DECLINED,
+                           "Skipped. You won't be asked again; to install Databricks AI tools later, run: databricks aitools install"
+        anything else   -> report UNANSWERED, "Skipped. To install ... later, run: databricks aitools install"
+        no answer       -> report UNANSWERED (stdin closed, Ctrl+C, or 1 min passed); not opted out
   record attempt; install within 10s
+  failed after a "yes"                      -> "Could not install Databricks AI tools. To install them, run: databricks aitools install"
  • "A person may be at the terminal": stdin, stdout or stderr is a terminal and no coding agent is detected. The question is asked only when it can be seen and answered (cmdio.IsLinePromptSupported): all three streams are terminals (a redirected stream would hide the question or take the answer from elsewhere, and a pager reading stdout would share the terminal's keystrokes), the process is in the terminal's foreground process group (a background job would stop on SIGTTIN), and no goroutine still reads stdin (apps dev-remote, via cmdio.MarkStdinReadInBackground). Runs with no terminal at all keep the parent's non-interactive install.
  • "Asked" is recorded explicitly: a "yes" stores accepted_version, and the question and the early skip check it, not the attempt count. That count also advances on a failed gate lookup, which never asks. A planning failure is an ordinary failed attempt whose retry asks.
  • A run that cannot ask returns before the lock and the gate lookup, and saves the machine ID it drew. The locked path saves a drawn ID before deciding, and autoinstall.SaveID returns the stored ID, so concurrent runs agree on the holdout bucket. Holdout machines and gate-off versions wait for a run that can ask, like treatment machines.
  • A CLI process that the CLI started itself (DATABRICKS_CLI_SELF_EXEC, now agents.IsSelfExec) does not ask; the parent asks after its own command.
  • Input typed while the command ran is discarded before asking (cmdio.DiscardInput: tcflush on Linux and macOS, FlushConsoleInputBuffer on Windows), and Ctrl+C at the question counts as no answer, so the command keeps its exit status. Where input can't be discarded (a pipe that Git Bash's mintty counts as a terminal), the question is skipped. A timed-out or interrupted question also turns off later line prompts in the process, since its read stays blocked on stdin.
  • The question is a plain-text line (cmdio.AskCancellable), so NO_COLOR, TERM=dumb and Git Bash, which turn off the TUI prompts, still get it. cmdio.HasTerminal and the new cmdio.IsLinePromptSupported express the two checks.
  • Only a fresh install asks. Updates of an existing installation and retries after a "yes" do not.
  • The question names only the agents that will get something (skills, or a plugin whose CLI is on PATH); on Windows that excludes agents installed through their CLI, which the plan already filters out. With none, there is no question.
  • An agent in a pseudo-terminal that the SDK does not recognize waits at most a minute, once per version.
  • DECLINED and UNANSWERED added to the auto-install outcome enum; the Universe proto for the whole event is https://github.com/databricks-eng/universe/pull/3127275. auto-install --help mentions the prompt.

Review order: cmd/aitools/autoinstall_run.go (autoInstall's early skip and ID save, attempt, canAsk, skipUnaskable, ask, decline), libs/aitools/autoinstall (AcceptedVersion, Accept, SaveID), then libs/cmdio/io.go and libs/cmdio/terminal_*.go, then cmd/aitools/autoinstall_run_test.go.

Evidence

The prompt needs a terminal, which acceptance tests lack, so TestAutoInstallPrompt (cmd/aitools) drives the hook through a test terminal and checks the prompt and notice on stderr, the skill in Cursor's dir, the stored state and the event:

enter accepts                                     -> installed, INSTALLED, interactive=true
yes accepts                                       -> installed
no declines and opts out                          -> no skill, opted_out=true, DECLINED
no answer settles the version without opting out  -> no skill, last_version=1.30.0, UNANSWERED
another answer settles the version without opting out -> no skill, not opted out, UNANSWERED
a planning failure is a failed attempt, retried later -> no prompt, FAILED, attempts=1
a failed install after yes says so                -> "Could not install ...", FAILED, accepted_version=1.30.0
NO_COLOR still asks                               -> prompt, Enter installs
TERM=dumb still asks                              -> prompt, "n" opts out, DECLINED
redirected stderr skips without installing        -> no prompt, no skill, no last_version, no event
piped stdin skips without installing              -> no prompt, no skill, no last_version, no event
stdout to a pager skips without installing        -> no prompt, no skill, no last_version, no event
no terminal at all installs without asking        -> no prompt, installed
a retry after yes is not asked again              -> no prompt, installed
a retry that was never asked asks                 -> (attempts=1 from a gate failure) prompt, installed
a retry that was never asked skips when it cannot ask -> no install, attempts=1 kept
a CLI process started by the CLI does not ask     -> no prompt, no skill, version not settled
a holdout machine that cannot ask waits too       -> no event, version not settled
(every case: stdout empty)

TestAutoInstallKeepsTheIDWhenItCannotAsk runs a partial-terminal hook twice on a machine with no stored ID: the ID is saved and stays the same, and the version is not settled. TestToolsPlanNewAgents covers which agents are named. TestCapabilities_SupportsLinePrompt and TestIsLinePromptSupported (libs/cmdio) cover the stream checks and the background stdin reader.

On a real terminal (tmux, Linux, databricks auth profiles with the gate cached on):

cli auth profiles &                    -> job ends "Done", no prompt, no "Stopped (tty input)", version not settled
sleep 2; cli ...  (typed "n⏎" in sleep) -> prompt still waits: the type-ahead was discarded
Ctrl+C at the prompt                   -> "^C", newline, EXIT=0, last_version=1.30.0, not opted out
state {last_version:1.30.0, attempts:1} -> the next foreground run asks (after the restack onto eafb2514)
``` With Enter mapped to "no" (a temporary edit, not committed) `enter accepts` fails, so the test catches the default.

Locally on Linux at 957cbf01d (after merging the base at eafb2514): unit tests of `cmd/aitools` (also shuffled), `cmd/apps`, `cmd/root`, `libs/cmdio` (also shuffled), `libs/aitools/...`, `libs/apps/...`, `libs/telemetry/...` and `libs/process/...` pass, and so do the `cmd/aitools` and `cmd/apps` acceptance tests. golangci-lint, whitespace and deadcode pass. `GOOS=windows`/`darwin` build and vet pass, and so does a `GOOS=freebsd` vet of `libs/cmdio`. I did not run the full unit and acceptance suites.

## Merge Danger

**Door:** two-way

Behind the same off gate as the parent.

**Blast Radius:** small

Once the gate is on, a person in a terminal sees one prompt after the first command of a version that has nothing installed; the command waits for the answer (at most a minute). A run with only some streams on a terminal installs nothing until a run in a full terminal. On the same first run, a Claude Code user without skills also sees the existing install hint (unchanged) above the "Installed ..." line.

This pull request and its description were written by Isaac.



@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 1fbc949

Run: 44177813921

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 8:25
💚​ aws-windows-latest-4core-16gb 1 272 13 7:08
💚​ azure linux-2core-8gb 1 269 15 7:30
💚​ azure-windows-latest-4core-16gb 1 271 13 6:03
💚​ gcp linux-2core-8gb 1 270 15 6:54
💚​ gcp-windows-latest-4core-16gb 1 272 13 6:17
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 5 slowest tests (at least 2 minutes):
duration env testname
4:09 aws-windows-latest-4core-16gb TestAccept
3:50 azure-windows-latest-4core-16gb TestAccept
3:32 gcp-windows-latest-4core-16gb TestAccept
2:07 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
2:06 aws linux-2core-8gb TestImportDirWithOverwriteFlag

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at a139569. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking].

This PR adds a [Y/n] question before auto-install's first install of a version, asked only when stdin, stdout and stderr are all terminals and no coding agent is detected. The approach is sound, but the new skip path for partial-terminal runs returns before the machine ID is saved, so new machines get a new holdout bucket on every such run and drift into the control group. That one needs a fix before merge. Smaller issues: typed-ahead input can answer the question and opt the user out for good, background jobs and Ctrl+C at the prompt behave badly, and partial-terminal runs repeat the gate lookup on every run. CI is green.

Questions

  1. DECLINED is a new value of AitoolsAutoInstallOutcome, which mirrors the Universe lumberjack proto in proto/logs/frontend/databricks_cli. Neither this PR nor #7702 links a Universe change. Is there one that adds DECLINED (or the whole aitools_auto_install_event)? Until the backend knows the value, declines may be dropped or read as unspecified. (non-blocking)

Accepted answers

  • Minute-long wait for unrecognized agents in a pseudo-terminal
  • Skip instead of install when only some streams are terminals

Comments outside the changed lines

  • [nice to have] cmd/aitools/autoinstall_run.go:123: When only some streams are terminals, attempt returns at line 175 without recording anything. Every later run of that kind repeats the two state loads, the machine-wide lock, this gate lookup and planTools, until the user runs one command with all three streams on a terminal. Before this PR that work happened once per version. The gate lookup refetches the manifest from GitHub whenever the 1-hour cache is stale, bounded by the 3s gateTimeout. A failed fetch reuses the stale file without refreshing its time. So on a network that drops the request, every such run after the first hour waits up to 3s at exit. Check the streams before taking the lock and resolving the gate: if autoInstallAsks(ctx) && !cmdio.IsLinePromptSupported(ctx) and there is no global install state, save the ID (see the comment on line 175) and return.
  • [nice to have] libs/telemetry/protos/aitools_events.go:218: The auto-install event's docs no longer match what the code sends. Interactive is documented as cmdio.IsPromptSupported with no agent, but it is now cmdio.HasTTY with no agent. It is true for runs that never prompt (updates, retries, NO_COLOR), so it can't tell a run that asked from one that didn't. FAILED (line 187) says a later run of the same version retries, but settleFailed reports FAILED and settles the version. An unanswered prompt (timeout or EOF) settles the version with no event at all, while the protos README says the event is logged once per version and machine, so a prompt nobody answered doesn't show up next to the holdout arm. Update the two comments and the README. Also consider reporting an outcome for an unanswered prompt, or a field that says the user was asked.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +174 to +175
log.Debugf(ctx, "auto-install: skipped: cannot ask, stdin, stdout or stderr is not a terminal")
return

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] On a machine with no stored ID, this early return throws away the ID that autoInstall just generated (NewID(), line 111). Only Record writes State.ID, and this path never calls it. So each later run with a piped or redirected stream draws a new ID and a new bucket. 90% of the draws return here again. The first draw that lands in the 10% holdout settles as SKIPPED_HOLDOUT and saves that ID for good. A new machine whose first 10 eligible runs look like $(databricks …) or databricks … | jq therefore ends up in holdout about 65% of the time instead of 10%. That skews the control group, and those users are never asked in any later version. The tests for this path seed {"id": "treatment-machine"}, so they miss it. Save the ID here without settling the version, for example with a small autoinstall.SaveID that sets s.ID only when it is empty. Also add a test case with no seeded state.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. A run that cannot ask now saves the ID it drew (new autoinstall.SaveID, which sets the ID only when none is stored and leaves the version unsettled), so later runs keep the bucket. Added TestAutoInstallKeepsTheIDWhenItCannotAsk, which runs twice with no seeded ID and checks that the ID stays the same and the version is not settled. I also took the out-of-diff suggestion: the check runs before the lock and the gate lookup (autoInstall, right after the first Decide), so these runs now cost one state read plus the local planTools, with no manifest fetch. The check under the lock stays, because the state can change between the two reads.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +259 to +263
switch strings.ToLower(strings.TrimSpace(ans)) {
case "", "y", "yes":
return true
}
r.decline(ctx)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The answer is read straight after a command that may have run for minutes, and nothing discards what was typed while it ran. Type-ahead therefore answers the question before the user sees it. Say the user types the next command during a bundle deploy, e.g. git status⏎. The prompt reads it as the answer. It is neither "" nor y/yes, so it stores a permanent opt-out and prints "You won't be asked again", and the shell never runs git status. A typo like yess has the same effect, and a stray Enter installs. Treat only n/no as a decline, and treat anything else as no answer (or ask again once). Also consider discarding pending input before asking: tcflush with TCIFLUSH on Unix, FlushConsoleInputBuffer on Windows. The doc comment on ask ("Only "no" stores the opt-out") already describes the safer rule.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. Only n/no decline now. Any other answer settles the version without opting out, reports the new UNANSWERED outcome and prints "Skipped. To install Databricks AI tools later, run: databricks aitools install". Pending input is also discarded before asking (cmdio.DiscardInput: TCFLSH/TCIFLUSH on Linux, TIOCFLUSH/FREAD on macOS, FlushConsoleInputBuffer on Windows), which covers the stray Enter too. Checked on a real terminal in tmux: sleep 2; databricks auth profiles with n⏎ typed during the sleep still waits at the prompt.

}
actx, cancel := context.WithTimeout(ctx, promptTimeout)
defer cancel()
ans, err := cmdio.AskCancellable(actx, "Install Databricks AI tools for "+strings.Join(names, ", ")+"? [Y/n]", "")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The prompt doesn't handle job control or Ctrl+C, so two common terminal actions now end badly after a command that succeeded. (1) databricks jobs list & from an interactive shell has the terminal on all three streams, so it prompts. A background process group that reads the terminal gets SIGTTIN and stops ("Stopped (tty input)"), and promptTimeout can't fire while it is stopped. (2) The CLI installs no SIGINT handler, so pressing Ctrl+C to dismiss the question kills the process. The shell sees exit status 130 for a command that succeeded, so a make target or && chain stops. The command's own telemetry upload in root.Execute also never runs. Ask only when the process is in the terminal's foreground process group (tcgetpgrp on stdin equals getpgrp). Consider treating SIGINT during the prompt as no answer, e.g. with signal.NotifyContext around AskCancellable, so the command keeps its exit status.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. (1) cmdio.IsLinePromptSupported now also requires the process to be in the terminal's foreground process group (TIOCGPGRP == getpgrp; always true on Windows). A background run takes the skip path, which saves the ID and doesn't settle the version. In tmux, databricks auth profiles & now ends as Done, with no prompt and no Stopped (tty input). (2) The prompt runs under signal.NotifyContext(..., os.Interrupt), so Ctrl+C counts as no answer (UNANSWERED, not opted out). In tmux, Ctrl+C at the prompt prints ^C and a newline, and the command exits 0.

Comment thread cmd/aitools/autoinstall_run.go Outdated
// per version even when it is never answered: the process is killed, stdin
// is closed, or nobody answers within promptTimeout (an unrecognized agent in
// a pseudo-terminal). Only "no" stores the opt-out.
func (r autoInstallRun) ask(ctx context.Context, plan []agentPlanItem) bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The prompt assumes nothing else is reading stdin. Without --auto-approve, apps dev-remote starts readStdinLines on os.Stdin and returns nil on Ctrl+C, and Bridge.Stop doesn't end that read. When it is the first eligible command of a version, the prompt runs while that goroutine is still blocked on stdin, so the first line the user types can go to the bridge's reader instead. Its signal.Notify also stays registered, so Ctrl+C does nothing at the prompt and the user waits out the minute. Skip the prompt after commands that read stdin in the background, or have the bridge stop its reader before returning.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0, using the first option. New cmdio.MarkStdinReadInBackground(), which the bridge calls before it starts readStdinLines. After that, IsLinePromptSupported is false for the rest of the process, so the run takes the skip path and a later command asks.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +270 to +274
if err := autoinstall.SetOptedOut(ctx, true); err != nil {
log.Warnf(ctx, "Could not save the answer, so a later CLI version asks again: %v", err)
}
r.settle(ctx, protos.AitoolsAutoInstallOutcomeDeclined)
cmdio.LogString(ctx, "Skipped. You won't be asked again; to install Databricks AI tools later, run: databricks aitools install")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] decline calls autoinstall.SetOptedOut instead of autoinstall.Decline, the helper apps init uses, which also sets the in-process flag. When saving fails, stderr shows Could not save the answer, so a later CLI version asks again at Warn, which is the default log level, followed by Skipped. You won't be asked again. Call autoinstall.Decline(ctx), and print the "won't be asked again" line only when saving succeeded.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. It now calls autoinstall.Decline. When saving fails, it prints the warning and "Skipped. To install Databricks AI tools later, run: databricks aitools install", without "won't be asked again". To run several declines in one test process I added autoinstall.ResetDeclined for t.Cleanup.

Comment thread libs/cmdio/io.go Outdated

// HasTTY reports whether stdin, stdout or stderr is a terminal, so that a
// person may be running the command.
func HasTTY(ctx context.Context) bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] HasTTY duplicates cmdio.HasTerminal (line 121 of this file, used by libs/aitools/agents/hint.go), which already reports whether stdin, stdout or stderr is a terminal. Likewise, NewTestIOWithInputTTY in testing.go does what cmdio.NewIO(ctx, flags.OutputText, cmdio.FakeTTYReader(in), out, err, "", "") already does. Use the existing helpers and drop the new ones, including Capabilities.HasTTY.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. I switched to cmdio.HasTerminal and NewIO with FakeTTYReader, and dropped HasTTY, Capabilities.HasTTY, their test and NewTestIOWithInputTTY.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +245 to +248
names := make([]string, len(plan))
for i, it := range plan {
names[i] = it.agent.DisplayName
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The question names every agent in the plan, including items that will get nothing. An agent whose CLI is not on PATH but that already has the plugin is planned as deliveryPlugin with cliMissing, and executePlan only records it as kept. If that is the only agent, the user is asked "Install Databricks AI tools for Claude Code?" for something already installed, and "n" opts them out for good. Name only the items that install something (skills, or a plugin whose CLI is present), and skip the question when none do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0. New toolsPlan.newAgents names only skills items and plugins whose CLI is on PATH. The question lists those, and when it is empty there is no question. Covered by TestToolsPlanNewAgents.

Comment thread cmd/aitools/autoinstall_run_test.go Outdated

// The interactive prompt needs a terminal, which acceptance tests do not
// have, so these cases drive the hook the root command runs with stdin as a
// test terminal; the rest of the terminal is detected as in a real run. Like the acceptance tests in acceptance/cmd/aitools/auto-install,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This comment line is 138 columns long: "Like the acceptance tests in acceptance/cmd/aitools/auto-install," was appended without rewrapping. Rewrap the paragraph.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cdf01c0 (rewrapped).

anton-107 and others added 3 commits November 17, 2026 10:22
First of three changes that install Databricks AI tools automatically on
the first run of a new CLI version. This one stores the state and the
opt-out and adds the remote gate; nothing installs yet.

- ~/.databricks/aitools/auto-install.json holds last_version, attempts
  and opted_out. It is a dedicated file, not a libs/cache entry, so an
  opt-out survives upgrades and `databricks cache clear`.
- `databricks aitools auto-install disable|enable` toggles the opt-out;
  enable warns when DATABRICKS_AITOOLS_AUTO_INSTALL=false is set.
- A global-scope `databricks aitools uninstall` (including --skills and
  --agents) opts out.
- cli-compat.json entries get an optional auto_install field, resolved by
  clicompat.ResolveAutoInstall. No entry sets it, and
  TestManifestLeavesAutoInstallOff keeps it off.

Co-authored-by: Isaac <no-reply@databricks.com>
After a command succeeds, the root PersistentPostRunE installs (or
updates) Databricks AI tools for the coding agents on the machine, once
per new CLI version. It ships off: no cli-compat entry sets the gate.

- autoinstall.Decide is a pure decision over the command, version, state,
  CI, opt-out, gate and holdout; skipped commands and development builds
  return before any state read.
- A fresh install runs as `aitools install --scope=global` without
  prompts for agents whose config dir exists; an existing global install
  runs `aitools update --scope=global`, keeping edited skills. The run has
  a 10s budget, discards installer output and prints one notice line.
- State is updated under OS file locks that are only tried, never waited
  on beyond 1s. Attempts are retried up to 10 times per version.
- Agent CLIs run in their own process group so a timeout stops their
  descendants. Windows cannot do that, so auto-install skips plugin
  installs there.
- Answering "no" at the `apps init` skills prompt opts out and is passed
  to the CLI children `apps init` starts.
- Telemetry: aitools_install_event.trigger = AUTO_FIRST_RUN and a new
  aitools_auto_install_event.
- Refactor: `aitools update`'s RunE body moves to runUpdate,
  versioncheck.InCI is exported, and the print_telemetry acceptance
  helper moves up a directory.

Co-authored-by: Isaac <no-reply@databricks.com>
When a person runs the CLI in a terminal, auto-install asks before its
first install of a version: "Install Databricks AI tools for <agents>?
[Y/n]" (1 minute).

- The question is asked only when stdin, stdout and stderr are all
  terminals and no coding agent is detected. If only some streams are
  terminals, nothing is installed and the version is not settled, so a
  later run asks. Runs with no terminal keep the non-interactive install.
- Enter/y/yes installs; any other answer stores the opt-out and reports
  DECLINED; no answer stops without opting out.
- Only a fresh install asks; updates and retries after an accepted,
  failed attempt do not. A planning failure on the first attempt settles
  the version and reports FAILED.
- The question is a plain-text line (cmdio.AskCancellable), so NO_COLOR,
  TERM=dumb and Git Bash still get it. New cmdio.IsLinePromptSupported
  and cmdio.HasTTY express the stream checks.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/auto-install-first-run branch from 019d40c to e5662ad Compare November 17, 2026 12:29
@anton-107
anton-107 force-pushed the aitools/auto-install-prompt branch from a139569 to 12bfe17 Compare November 17, 2026 12:31
anton-107 and others added 2 commits November 17, 2026 14:26
- A global uninstall opts out only when it removed something, so a
  mistyped --skills or an empty scope no longer turns auto-install off.
- `aitools auto-install <typo>` now fails instead of printing help and
  exiting 0.
- Document auto_install in internal/build/README.md (format, the
  copy-forward step for new entries, the trust model).
- Say that any non-true DATABRICKS_AITOOLS_AUTO_INSTALL value opts out,
  and move EnvVar next to the other constant.
- Add unit tests for the state file; pin the env var in the acceptance
  test and log the reset steps instead of discarding them.

Co-authored-by: Isaac <no-reply@databricks.com>
Address review on #7703:
- A run that cannot ask saves the machine ID it drew (autoinstall.SaveID),
  so partial-terminal runs no longer redraw the holdout bucket. It now skips
  before the lock and the gate lookup.
- Only "n"/"no" declines; any other answer, EOF, a timeout or Ctrl+C is
  reported as UNANSWERED and settles the version without opting out.
- Discard type-ahead before asking, ask only in the terminal's foreground
  process group, and treat Ctrl+C at the prompt as no answer.
- Skip the prompt when a goroutine still reads stdin (apps dev-remote).
- Use autoinstall.Decline and promise "won't be asked again" only when the
  opt-out was saved.
- Name only agents that would get something; reuse cmdio.HasTerminal and
  FakeTTYReader instead of new helpers.
- Fix the auto-install event docs.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107

Copy link
Copy Markdown
Contributor Author

@simonfaltum thanks for the review. Every inline comment is addressed in cdf01c0, with a reply on each thread. On the review body:

  • Universe proto: there's no Universe change yet for aitools_auto_install_event. That covers the whole event from aitools: install AI tools after the first command of a new CLI version #7702 and the two values this PR adds: DECLINED and now UNANSWERED. The gate stays off until the backend knows the event.
  • Repeated gate lookup on partial-terminal runs: a run that cannot ask now returns before the lock and the gate lookup. It saves the ID when none is stored and costs one state read plus the local planTools.
  • Event docs: Interactive now describes cmdio.HasTerminal with no agent. It says that DECLINED/UNANSWERED only come with true, and that INSTALLED with true may still have run without asking. FAILED mentions that a failure before the prompt settles the version. An unanswered prompt (EOF, timeout, Ctrl+C, or an answer that isn't yes or no) now reports UNANSWERED, so it shows up next to the holdout arm. The README and the event comment also say that a run that cannot ask, or a process killed at the prompt, logs nothing.

Real-terminal checks (tmux) are in the PR description under Evidence.

- apps init children keep DATABRICKS_CLI_SELF_EXEC: ChildEnv takes the
  environ and composes with agents.SelfExecEnv.
- Agent CLIs run through process.BackgroundInGroup, which forwards the first
  SIGINT/SIGTERM to the child's process group and then re-raises it, so
  Ctrl-C stops `claude plugin install` again in aitools install/update.
- Skips reuse versioncheck.IsExemptCommand, agents.IsCredentialHelperCommand
  (covers `auth docker token`) and agents.InCI (CI=<provider>, TF_BUILD...),
  and add Databricks Runtime. Cache-disabled runs no longer read the state.
- A gate lookup that only reaches the embedded manifest fails instead of
  reading as off; it uses up an attempt and does not settle the version.
- An unfinished first install is marked in the state, so its skills are
  finished on a later run instead of being kept as untracked forever.
- Auto-install updates no longer migrate or remove raw skills.
- New outcomes UP_TO_DATE, KEPT and SKIPPED_PLATFORM; the update notice
  names only the agents whose skills or plugin changed.
- Saying "no" in apps init says that auto-install is off too.
- The explicit opt-out writes without the lock where locks are unsupported.
- Tests: ResetDeclined, environ passed in; acceptance helpers use LOG.* and
  no `|| true`.

Co-authored-by: Isaac <no-reply@databricks.com>
anton-107 and others added 2 commits November 17, 2026 14:45
…nstall-first-run

Co-authored-by: Isaac <no-reply@databricks.com>
…ests

agents.InCI also detects providers such as GITHUB_ACTIONS, so on CI runners
every case reported SKIPPED_CI.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107

Copy link
Copy Markdown
Contributor Author

Universe proto for the auto-install event (including DECLINED and UNANSWERED), plus AitoolsInstallEvent.trigger: https://github.com/databricks-eng/universe/pull/3127275

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at cdf01c0. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking].

This PR asks before auto-install's first install of a CLI version, and only when a person can see and answer the question. Every fix from the earlier review is in and works, and CI is green. The PR now conflicts with its moved base branch, though. On that base a failed gate lookup uses up an attempt, so after the restack a run would install without asking, because the question is gated on attempt == 1. That needs an explicit "asked" record before merge. The other findings (Git Bash type-ahead, prompts in child CLI processes, a silent failure after "yes", an ID race outside the lock) are non-blocking.

Accepted answers

  • Does the Universe proto know DECLINED/UNANSWERED?: #7703 (comment)
  • Why does a planning failure before the prompt settle the version?

Comments outside the changed lines

  • [nice to have] cmd/aitools/autoinstall_run.go:220: After the user answers yes, a failed install shows nothing: attempt only writes the debug log and the event, and quietContext discards the installer's output. Scenario: a user presses Enter on a machine where the GitHub download fails. They wait up to installTimeout (10s), then get their shell back with no message, so they can't tell that nothing was installed. When the run asked, print one line on failure, e.g. "Could not install Databricks AI tools. To install them, run: databricks aitools install".

Comment thread cmd/aitools/autoinstall_run.go Outdated
// A terminal user is asked before the first install of a version, once:
// a retry was answered "yes" (or ran non-interactively), so it is not
// asked again.
if r.interactive && attempt == 1 {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] The base branch has moved to eafb251 and this PR now conflicts with it, so it needs a restack. On the new base, a failed gate lookup stores attempts = NextAttempt(...) without asking. So attempt == 1 here, and in.State.LastVersion != version in the early skip at line 102, no longer mean "not asked yet". Scenario: the first run of a version is offline, or the manifest fetch passes the 3s gateTimeout, so attempts=1 is stored. The next run in a terminal is attempt 2: it skips the question and installs. A run with piped stdout also gets past the early skip and installs. Record explicitly that this version was asked, e.g. a state field that ask sets before the question (or a "yes" answer), and gate the question and the early skip on that instead of the attempt count. A planning error could then also retry instead of settling the version.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0: I merged the new base (eafb251) into the branch and resolved the conflicts. Added State.AcceptedVersion. A "yes" at the prompt records it (autoinstall.Accept), and both the question and the early skip now check AcceptedVersion != version instead of attempt == 1 or LastVersion != version. So a retry after a failed gate lookup asks, and a retry after a "yes" doesn't. The version is still settled right before the question, so a process killed at the prompt doesn't install later. I also took the follow-on: a planning error is now an ordinary failed attempt (FAILED, attempts=1) whose retry asks, and settleFailed is gone. New cases in TestAutoInstallPrompt: "a retry that was never asked asks", "a retry that was never asked skips when it cannot ask" (piped stdout, attempts=1, no install), and "a retry after yes is not asked again". I also checked on a real terminal: with {"last_version":"1.30.0","attempts":1} stored, the next foreground run shows the question.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +273 to +275
if err := cmdio.DiscardInput(ctx); err != nil {
log.Debugf(ctx, "auto-install: could not discard the pending input: %v", err)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] In Git Bash under mintty, stdin is a pipe that isatty.IsCygwinTerminal counts as a terminal, so the question is asked. But FlushConsoleInputBuffer fails on a pipe handle, and the error is only logged. A stray Enter typed while the command ran is then read as the default "yes" and installs without the user seeing the question; a typed-ahead n opts out for good. Do the discard before Record, and treat a failure as "cannot ask" (skipUnaskable). Alternatively, on Windows, drain a non-console handle with PeekNamedPipe + ReadFile.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0, using the first option. ask now discards pending input before it settles the version, and if the discard fails it takes skipUnaskable (ID saved, version not settled, no question). So in mintty with a pipe stdin, the question is skipped and never answered by type-ahead. I didn't do the PeekNamedPipe drain: x/sys/windows doesn't wrap it, and I can't test a hand-rolled kernel32 call here. Happy to add it in a follow-up if you'd rather mintty users get the question.

Comment on lines +242 to +244
func autoInstallAsks(ctx context.Context) bool {
return cmdio.HasTerminal(ctx) && useragent.AgentProvider() == ""
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] autoInstallAsks does not skip CLI processes that the CLI starts itself, unlike MaybeHint, which checks DATABRICKS_CLI_SELF_EXEC. Scenario: on the first run of a version, databricks apps init --deploy runs in flags mode (or in a NO_COLOR or Git Bash terminal, where RecommendSkillsInstall only prints the tip). It runs the child apps deploy on the same terminal. When the deploy succeeds, the child asks "Install Databricks AI tools for …? [Y/n]" and waits up to a minute, before the parent prints "Deploy complete" and moves on to the run step. Skip auto-install, or at least the question, in a self-exec child, and leave it to the parent's own run. (On this PR's base, autoinstall.ChildEnv() also overwrites SelfExecEnv at those apps init call sites; the current base tip fixes that.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0, and it covers the question as well as the install: a self-exec child doesn't ask and doesn't install. canAsk now requires !agents.IsSelfExec(ctx), a new exported check beside SelfExecEnv, which MaybeHint now uses too. The child takes the cannot-ask skip: ID saved, version not settled, nothing installed. The parent then asks after its own command. New case: "a CLI process started by the CLI does not ask". The base's ChildEnv/SelfExecEnv fix comes in with the merge.

Comment thread cmd/aitools/autoinstall_run.go Outdated
// every such run until a run in a terminal asks.
if reason == autoinstall.ReasonGateOff && in.State.LastVersion != version && autoInstallAsks(ctx) && !cmdio.IsLinePromptSupported(ctx) {
if tools, err := planTools(quietContext(ctx)); err == nil && len(tools.newAgents()) > 0 {
skipUnaskable(ctx, in.State.ID, autoinstall.NewID())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] skipUnaskable saves the ID here without holding the run lock. Scenario: on a machine with no stored ID, another process holds the lock. It has drawn its own ID at lines 121–125 and decided the holdout arm with it. This SaveID stores a different ID first, and that run's Record keeps it, because s.ID is no longer empty. The run installs and reports as treatment, but every later run uses the saved ID's bucket, which can be holdout, so an installed machine counts as holdout. The window lasts as long as that run's question, up to a minute. In the locked path, persist the drawn ID with SaveID before deciding, and use the ID that is actually stored (for example, have SaveID return it).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0. SaveID now returns the stored ID. In the locked path, a machine with no ID saves the ID it draws with SaveID before deciding anything, and uses the ID that comes back for the bucket, the event and Record. The early skip also goes through SaveID, so whichever writes first, both runs end up with the same ID. TestSaveIDKeepsTheFirstID checks the returned value.

Comment thread cmd/aitools/autoinstall_run.go Outdated
Comment on lines +99 to +101
// A run that would ask before installing but cannot skips before the lock
// and the gate lookup: it would skip after them too, and it repeats on
// every such run until a run in a terminal asks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The comment says a run that cannot ask "would skip after them too". But after the gate lookup, a holdout machine settles as SKIPPED_HOLDOUT and a gate-off version settles silently, so the early return changes the outcome of those runs. That is probably intended: holdout and treatment machines both wait for a run that can ask, which keeps the arms comparable. The comment should say so. A test with a holdout-arm ID in a partial terminal (no event, version not settled) would pin it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0. The comment now says that holdout machines and gate-off versions wait for a run that can ask, like treatment machines, so the arms are counted alike. New case "a holdout machine that cannot ask waits too" (holdout-machine, redirected stderr) checks that there's no event and the version is not settled.

// succeeded.
actx, stop := signal.NotifyContext(actx, os.Interrupt)
defer stop()
ans, err := cmdio.AskCancellable(actx, "Install Databricks AI tools for "+strings.Join(names, ", ")+"? [Y/n]", "")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] When the question times out or gets Ctrl+C, AskCancellable leaves its read blocked on the terminal until the process exits, and root.Execute then uploads telemetry for up to 3s. A command the user types right after Ctrl+C is read by the abandoned goroutine and lost. Consider a read that stops on ctx.Done() (muesli/cancelreader is already in the module graph through bubbletea). At least have AskCancellable call MarkStdinReadInBackground when it returns on cancellation, so later prompts in the process know about the leftover read.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0, with the minimum: when AskCancellable returns on cancellation, it calls MarkStdinReadInBackground, so no later line prompt in the process can lose its answer to the abandoned read. TestAskCancellable_Cancelled checks this. I didn't switch to cancelreader. It would mean changing cmdio's unbuffered stdin handling, which other prompts share, and that's more than this PR should take on. The line typed right after Ctrl+C is still lost until the process exits; I can follow up if you think it's worth it.

Comment thread cmd/aitools/autoinstall_run.go Outdated
}
log.Debugf(ctx, "auto-install: skipped: the answer was neither yes nor no")
r.settle(ctx, protos.AitoolsAutoInstallOutcomeUnanswered)
cmdio.LogString(ctx, "Skipped. To install Databricks AI tools later, run: databricks aitools install")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] "Skipped. To install Databricks AI tools later, run: databricks aitools install" is written out at lines 299 and 309. .agents/rules/style-guide-go.md asks for a package-level constant when a value is used in more than one place, as this file already does for autoInstallDisable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0: added a skippedNotice constant.

Comment thread libs/cmdio/terminal_darwin.go Outdated
Comment on lines +12 to +15
func inForeground(f *os.File) bool {
pgrp, err := unix.IoctlGetInt(int(f.Fd()), unix.TIOCGPGRP)
return err == nil && pgrp == unix.Getpgrp()
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] inForeground is identical in terminal_darwin.go and terminal_linux.go. Move it to one file with //go:build linux || darwin, and keep only discardInput per OS.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 957cbf0. inForeground is now in terminal_foreground.go (//go:build linux || darwin), and only discardInput is per OS.

…ll-prompt

Restack onto the moved base and address the second review of #7703:
- Record a "yes" at the prompt (State.AcceptedVersion) and gate the
  question and the early skip on it instead of the attempt count, which a
  failed gate lookup now advances without asking. A planning failure is a
  failed attempt whose retry asks, instead of settling the version.
- Save a drawn machine ID before deciding (SaveID returns the stored ID),
  so a concurrent run that cannot ask cannot store a different one.
- Discard pending input before settling the version, and do not ask when
  that fails (a pipe that mintty counts as a terminal).
- Do not ask in a CLI process the CLI started itself (agents.IsSelfExec).
- Say so when the install fails after a "yes".
- A cancelled AskCancellable turns off later line prompts in the process.
- Share inForeground between Linux and macOS, name the repeated notice.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107

Copy link
Copy Markdown
Contributor Author

@simonfaltum second round is in 957cbf0: I merged the moved base (eafb251) and replied on each thread. Blocking item: a "yes" now stores accepted_version, and the question and the early skip check that instead of the attempt count. So a retry after a failed gate lookup asks, and a planning failure is an ordinary failed attempt whose retry asks. From the review body: a failed install after a "yes" now prints "Could not install Databricks AI tools. To install them, run: databricks aitools install". The Universe PR (https://github.com/databricks-eng/universe/pull/3127275) now also has the base's UP_TO_DATE, KEPT and SKIPPED_PLATFORM.

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 957cbf0. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 13 non-blocking comment(s) and 2 question(s) below.

This PR asks [Y/n] before auto-install's first install of a CLI version, and only when a person can see and answer the question. A "yes" is stored per version, only "no" opts out, and runs that cannot ask wait for one that can. Every fix from the earlier rounds is in place, and CI is green. Three non-blocking issues remain. A self-exec child with no terminal (e.g. air images push's auth docker configure) still installs without asking. Settling before the question clears the Installing marker. In mintty, the cannot-ask skip happens only after the gate lookup. There are also a few nits and two questions: whether the question should default to yes, and whether discarding type-ahead is acceptable.

Questions

  1. The question defaults to yes ([Y/n]), so an Enter pressed after it appears installs into the agents' config directories. The CLI's other consent prompts for installs, apps init's skills prompt (RecommendSkillsInstall in libs/aitools/agents/recommend.go) and confirmUvInstall in libs/localenv/uv.go, use AskYesOrNo, which defaults to No. So Enter means opposite things in two prompts for the same tools. Was default-yes chosen on purpose? An alternative is [y/N], where Enter counts as no answer (UNANSWERED, no opt-out) and only an explicit yes installs. (non-blocking)
  2. DiscardInput throws away whatever the user typed while the command ran. That is usually their next shell command (e.g. git push⏎ typed during a long bundle deploy), and nothing tells them it was dropped. Is losing that input acceptable once per version? An alternative is to check for pending input (FIONREAD/TIOCINQ on Unix, pending console input events on Windows). When there is some, skip the question without settling the version, so the shell still receives it and a later run asks. (non-blocking)

Accepted answers

Comments outside the changed lines

  • [nit] cmd/aitools/autoinstall_run.go:221: ask already settled the version with Record(v, 0). If this Record(v, attempt) then fails (the state lock is held for more than 1s, or a disk error), attempt returns with only a debug log. The state stays {last_version: v, attempts: 0}, so the version is never retried, and the user who just said yes sees nothing. When asked is set, print the "Could not install…" notice on this path too.
  • [nit] .github/PULL_REQUEST_TEMPLATE.md: The description uses ## Summary, ## Evidence and ## Merge Danger. The repo's pr-checklist skill (.agents/skills/pr-checklist/SKILL.md) asks agent-written PRs to use the template's ## Changes, ## Why and ## Tests headings in that order, and to keep the summary concise. Restructure the description under those headings.

// version: a "yes" covers that version's retries. A planning failure is
// a failed attempt, and its retry asks.
asked := false
if err == nil && r.interactive && prior.AcceptedVersion != r.version && len(tools.newAgents()) > 0 {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] agents.IsSelfExec is checked only in canAsk, which runs only when r.interactive is true. A self-exec child with no terminal therefore installs without asking. Scenario: databricks air images push runs auth docker configure as a self-exec child (configureImageDocker in cmd/air/image.go), with stdout and stderr in a buffer and stdin unset. That child has no terminal, so it installs and settles the version before the parent's hook runs. The user at the terminal is never asked, and the parent then prints the child's "Installed Databricks AI tools…" line. This contradicts the earlier answer that a self-exec child doesn't ask and doesn't install. Skip auto-install in a self-exec child before anything else, without settling the version, for example next to the state-free skips in autoInstall. (in reply to #7703 (comment))

skipUnaskable(ctx, r.id)
return false
}
if err := autoinstall.Record(ctx, r.version, 0, r.id); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] This Record with 0 attempts settles the version before the question, and it also clears Installing. Scenario: a first install that ran without asking (no terminal, e.g. an agent) times out after writing some skills, leaving Installing=true and no install state. A later run in a terminal plans a fresh install, because planTools skips the untracked-skills check while Installing is set, and asks. The user presses Ctrl+C, waits out the minute, or types something else. With Installing cleared, every later version finds untracked skills and settles as KEPT. The machine is never asked again and the partial install is never finished, contrary to UNANSWERED's "the next CLI version asks again". Keep Installing when settling before the question and for UNANSWERED/DECLINED, e.g. with a settle variant that leaves Installing alone.

// no. "Yes" is recorded so that the version's retries do not ask again; only
// "no" stores the opt-out.
func (r autoInstallRun) ask(ctx context.Context, names []string) bool {
if err := cmdio.DiscardInput(ctx); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] In Git Bash's mintty, stdin is a Cygwin pipe that isTTY counts as a terminal. So canAsk is true and the early skip at line 118 does not fire. On every command, a treatment machine takes the lock, resolves the gate (a manifest fetch of up to 3s once the hourly cache is stale), plans, and only then fails here in DiscardInput and takes skipUnaskable. The version is never settled and nothing is reported. A holdout machine on the same setup settles as SKIPPED_HOLDOUT after the gate lookup, so the arms are not counted alike, against the intent of the early skip. Make IsLinePromptSupported false on Windows when stdin is not a real console (for example, isatty.IsTerminal is false but IsCygwinTerminal is true, or GetConsoleMode fails), so the pre-lock skip catches it. Keep this error path as a fallback.

// no. "Yes" is recorded so that the version's retries do not ask again; only
// "no" stores the opt-out.
func (r autoInstallRun) ask(ctx context.Context, names []string) bool {
if err := cmdio.DiscardInput(ctx); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Pending input is discarded before Record, which can wait up to 1s for the state lock and then rewrites the file, and only then is the question printed. An Enter pressed in that window (e.g. tapping Enter while a deploy finishes) is read as the default "yes" before the question appears. Keep the discard here as the can-ask check, and discard again right before AskCancellable.

if err != nil {
log.Debugf(ctx, "auto-install: skipped: the prompt was not answered: %v", err)
// End the question's line.
cmdio.LogString(ctx, "")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] On a timeout, Ctrl+C or EOF, the user sees the question followed by an empty line, with no hint that it was skipped or how to install later. The "neither yes nor no" path prints skippedNotice, and this version won't ask again. Print skippedNotice here too.

// version: a "yes" covers that version's retries. A planning failure is
// a failed attempt, and its retry asks.
asked := false
if err == nil && r.interactive && prior.AcceptedVersion != r.version && len(tools.newAgents()) > 0 {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] tools.newAgents() is built here and again on line 213. The condition interactive && AcceptedVersion != version && len(newAgents()) > 0 is also written out again at line 118, and the two copies must stay in sync by hand. Compute the names once, and consider one helper for the "would ask" predicate used by both places.

if err == nil && r.interactive && prior.AcceptedVersion != r.version && len(tools.newAgents()) > 0 {
// autoInstall skips this case before the lock; the state it decided
// on may have changed since.
if !canAsk(ctx) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] No test reaches this second canAsk check, because the early skip catches every current case. The timeout and Ctrl+C paths of ask and the DiscardInput failure path are also untested, and promptTimeout is a const, so a test can't shorten it. The new libs/cmdio/terminal_*.go files have no tests either (.agents/rules/testing.md), and every test uses FakeTTYReader, so discardInput and inForeground never run. An os.Pipe test would check that inForeground is false and discardInput fails on a pipe. Consider making the timeout a package variable and adding cases for a cancelled question and a failed discard.

setupAutoInstallHome(t, "{}")
t.Setenv("TERM", "xterm")
var stderr bytes.Buffer
run := func() autoinstall.State {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] run := func() autoinstall.State {...} names a repeated operation as an inline closure, which .agents/rules/style-guide-go.md asks to extract as a regular named function. Make it a helper that takes t and the stderr buffer.

installTimeout = 10 * time.Second

// promptTimeout bounds the wait for an answer at the install prompt.
promptTimeout = time.Minute

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] ExecutionTimeMs in cmd/root/root.go is measured around ExecuteContextC, which includes the post-run hook. So the first command of each version can now report up to this 1-minute wait as command time. Before this PR it already included up to about 13s of install. Consider excluding the hook's time, or noting it where command latency is analyzed.

version to a later run that can ask; a process killed at the prompt logs nothing either. An install
with no tools installed yet also logs an `aitools_install_event` with `trigger` `AUTO_FIRST_RUN`;
installs by `databricks aitools install` leave `trigger` unset. The holdout arm comes from a random
machine id kept in

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The reflow left "machine id kept in" on its own line before the backticked path. Rewrap the paragraph.

#7701 landed on main as a squash of this branch's base; its conflicts resolve
to this branch's side, as main has not changed those files since.

Co-authored-by: Isaac <no-reply@databricks.com>
Base automatically changed from aitools/auto-install-first-run to main November 17, 2026 21:09
anton-107 and others added 2 commits November 17, 2026 21:13
Resolve the cmdio test conflict for #7699/#7700: Git Bash detection is
gone and NO_COLOR no longer turns off prompts, so the line-prompt test
checks TERM=dumb instead, and the docs name an MSYS2 pty pipe.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>

# Conflicts:
#	cmd/aitools/autoinstall.go
#	cmd/aitools/autoinstall_run.go
#	libs/aitools/autoinstall/run.go
#	libs/aitools/autoinstall/run_test.go
#	libs/aitools/autoinstall/state.go
#	libs/telemetry/protos/README.md
#	libs/telemetry/protos/aitools_events.go
@github-actions github-actions Bot added the aitools AI tools (databricks aitools) related changes label Nov 17, 2026
@anton-107
anton-107 enabled auto-merge November 17, 2026 21:14
@anton-107
anton-107 added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 304522b Nov 17, 2026
28 checks passed
@anton-107
anton-107 deleted the aitools/auto-install-prompt branch November 17, 2026 21:46
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 304522b

Run: 44183474720

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 9 1 2 10 1613 1360 222:57
💚​ aws-windows-latest-4core-16gb 13 1541 1387 220:10
❌​ azure linux-2core-8gb 9 1 12 1593 1368 226:36
💚​ azure-windows-latest-4core-16gb 13 1521 1395 227:29
❌​ gcp linux-2core-8gb 9 1 12 1600 1366 237:35
🔄​ gcp-windows-latest-4core-16gb 4 11 1526 1393 236:47
24 interesting tests: 9 FAIL, 9 RECOVERED, 5 flaky, 1 KNOWN
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🟨​ TestAccept 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/invariant/no_drift ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=secret.yml.tmpl/READPLAN= ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
💚​ TestAccept/bundle/resources/apps/lifecycle-started 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate 🔄​f 💚​R 💚​R 💚​R 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS= 🔄​f 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f
Top 50 slowest tests (at least 2 minutes):
duration env testname
11:42 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
11:11 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:51 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:28 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
10:26 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
10:00 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:46 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:44 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:38 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:28 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:26 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:21 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
9:09 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:46 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:16 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:14 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
8:03 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:58 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:11 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:09 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:00 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:58 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:58 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:40 azure-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:40 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:27 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:19 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:18 aws-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:13 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:11 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:06 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:06 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:05 gcp-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
6:02 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:02 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:59 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:56 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
5:55 aws linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=
5:52 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:36 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:35 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
5:31 azure linux-2core-8gb TestAccept/pipelines/run/refresh-flags
5:28 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
5:25 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:20 aws linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=
5:16 gcp linux-2core-8gb TestAccept/pipelines/run/refresh-flags
5:14 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:12 aws linux-2core-8gb TestAccept/pipelines/run/refresh-flags
5:06 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
4:52 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants