Repository navigation
Conversation
## Changes Automatic installation of Databricks AI tools now asks before a first install whenever the caller is not a coding agent, whether or not a terminal is attached. Before this change it asked only when stdin, stdout or stderr was a terminal. A run with no terminal at all installed without asking. - `autoInstallAsks` now depends only on whether a coding agent was detected. It no longer checks for a terminal. - A run that is not a coding agent's call and cannot show the question (`canAsk` is false) is skipped, the same way runs that have a terminal but cannot prompt are skipped today. It installs nothing, sends no telemetry and leaves the version unsettled, so a later run that can ask, or a coding agent's call, still installs. This skip happens before the lock and the gate lookup, so cron jobs and job pods don't wait on the lookup every time they run. - Coding agents still get the tools without a question, with or without a terminal. - The `interactive` field on the auto-install telemetry event is still true only when a terminal is attached and no coding agent is detected. It is now computed at the call site. ## Why Commands run over ssh without a terminal, under `nohup`, from cron or from an IDE extension have no terminal. Installing in those cases changed the user's agent configuration without asking a person who may have started the command. Asking is not possible there, so the run waits for one where it is. A coding agent's call is the case that is meant to install without a question, and it keeps doing so. ## Tests - Unit tests in `cmd/aitools/autoinstall_run_test.go`: - The "no terminal at all" case now expects a skip with the version left unsettled. - New cases check that a coding agent installs without asking, both with no terminal and in a terminal. - A new test checks that a run with no terminal and no coding agent skips before the gate lookup. - Existing tests that expect an install with no terminal now run as a coding agent's call. - New acceptance test `acceptance/cmd/aitools/auto-install/no-terminal`. A run with no terminal and no coding agent installs nothing and leaves the version unsettled. A later coding agent call then installs. - The shared `script.prepare` for the other auto-install acceptance cases sets `AGENT`. Those cases have no terminal, so they now run as a coding agent's call and their outputs stay the same. - Changelog fragment added under `.nextchanges/cli/`. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 2 tests: regression=1 unreleased=0 coverage=1 failing=0
TestAccept/cmd/aitools/auto-install/no-terminal: branch ✅ | main (8b7e509) ❌ | latest (v1.27.0) ❌main (8b7e509): latest (v1.27.0): |
Co-authored-by: Isaac <no-reply@databricks.com>
✅ Integration test reportCommit: 38e0e85
Top 3 slowest tests (at least 2 minutes):
|
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 96994c5. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 4 non-blocking comment(s) and 2 question(s) below.
This PR makes auto-install ask before a first install whenever no coding agent is detected, terminal or not. Runs that cannot ask now skip before the lock and the gate lookup, and coding agents still install without a question. The logic matches the description, the interactive telemetry field keeps its meaning, the new tests catch the old behavior, and CI passes. Nothing is blocking. The main follow-ups are wording: the help text and the changelog don't say that headless updates of an existing install still run. I also left two non-blocking questions, on per-run planning cost and on telemetry for runs that wait for consent.
Questions
- With this change, a run with no terminal and no coding agent never settles the version. Every cron, nohup or IDE-extension call on a machine with an agent config dir therefore runs
planToolsagain after its command: it reads the install state, scans for untracked skills, detects agents and looks up agent CLIs on PATH. Before, such a machine settled after one run. The cost is local I/O, but it now repeats on every call. Could the skip check the cheap conditions first (autoInstallAsks(),AcceptedVersion != version,!canAsk(ctx)) and read onlyinstaller.LoadStateto tell an update from a first install? Or is the current per-run cost acceptable? (non-blocking) - Headless runs without a coding agent used to report INSTALLED with
interactive=false. They now takeskipUnaskable, which emits nothing, so machines that wait for a run that can ask (IDE-only or cron-only users) disappear from the auto-install events. Dashboards then can't tell "waiting for consent" from "never ran". Is that intended, or should an unsettled skip outcome be reported once per version so this group stays measurable? (non-blocking)
Comments outside the changed lines
- [nice to have]
cmd/aitools/autoinstall.go:23: The help still says only "In a terminal it asks before the first install." With this PR, that reads as if the tools install without asking when there is no terminal. In fact a run with no terminal and no coding agent now skips and waits for a run that can ask, and a coding agent is never asked. Consider rewording it, for example: "It asks before the first install and skips when it cannot ask; a coding agent's call installs without a question."
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 38e0e85. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 5 non-blocking comment(s) and 1 question(s) below.
This PR makes auto-install skip a first install when there is no terminal and no coding agent, leaving the version for a run that can ask. Coding agents still install without a question, and headless updates still run. The logic matches the description and CI passes. I found nothing blocking. Open items: the auto-install help text still describes the old behavior, the headless retry after a "yes" has no test, there are a few test and comment nits, and one question about agents the SDK doesn't detect.
Questions
- Without a terminal, a first install now happens only when the SDK recognizes the coding agent from an environment variable (its known list, or
AGENT/AI_AGENT). An agent that sets none of these, such as Aider, Continue, Zed, Junie or Roo Code, used to get the tools when it ran the CLI without a terminal. Now it never does, unless a person later runs the CLI in a foreground terminal, and those skips are not reported. Is that trade-off intended? Or should the skip be counted, or the change wait for data on how manyinteractive=falseinstalls had no detected agent? (non-blocking)
Comments outside the changed lines
- [nit]
cmd/aitools/autoinstall_run.go:88: This comment says runs without a terminal are not skipped because coding agents run the CLI that way. That is still true for this command-level check. But a headless run with no coding agent now skips its first install later, inautoInstall, so a reader can come away thinking headless non-agent runs install. Consider adding a pointer, for example: "A first install without a terminal still needs a coding agent's call (see autoInstallAsks)." - [nit]
.agents/skills/pr-checklist/SKILL.md: The commits carryCo-authored-by: Isaac, but the PR body has no closing disclosure line..agents/skills/pr-checklist/SKILL.mdasks for one on the last line when an agent wrote or helped write the PR. Consider adding a single italic line at the end of the description. Separately, "Changelog fragment added" is listed under Tests and could move to Changes.
Agreeing with existing threads
- [nice to have] auto-install help text doesn't describe the headless skip: #7834 (review)
| wantOutcome: protos.AitoolsAutoInstallOutcomeInstalled, | ||
| }, | ||
| { | ||
| name: "a retry after yes is not asked again", |
There was a problem hiding this comment.
[nice to have] After this PR, the only first install a run with no terminal and no coding agent still makes is a retry of a version the user already accepted (accepted_version equals the version, so the checks at autoinstall_run.go:151 and :248 pass). No test covers it. This case runs with a TTY stdin, and the new headless cases have no accepted version. A later change that made the accepted-version check depend on canAsk would stop headless retries after a "yes" without failing any test. Consider adding a table case with stdinPiped, stdoutRedirected and stderrRedirected and the accepted state, expecting INSTALLED with interactive=false.
|
|
||
| // setCodingAgent makes the run the call of the coding agent whose env var is | ||
| // name. setupAutoInstallHome resets the SDK's cached agent after the test. | ||
| func setCodingAgent(t *testing.T, name string) { |
There was a problem hiding this comment.
[nit] setCodingAgent sets the env var with t.Setenv and clears the SDK cache, but it depends on running after setupAutoInstallHome. That call's isolateAgents clears the whole environment, and its cleanup resets the cache. Called earlier, the variable is wiped. Called without it, a cached agent can leak into later tests. All current callers run it in the right order. The existing pattern, passing the agent pair to isolateAgents as setupAgentCallerInstall in install_test.go does, would avoid the ordering dependency: for example, let setupAutoInstallHome accept extra env pairs.
|
|
||
| // A run with no terminal and no coding agent cannot ask, so it skips before the | ||
| // gate lookup: a cron job or a job pod does not wait for it on every run. | ||
| func TestAutoInstallSkipsTheGateLookupWhenItCannotAsk(t *testing.T) { |
There was a problem hiding this comment.
[nit] TestAutoInstallSkipsTheGateLookupWhenItCannotAsk repeats TestAutoInstallSkipsTheGateLookupWithoutAgents almost line for line: same setup, same failing gateOn fake, same assertions. The only differences are agent versus no agent and whether .cursor exists. .agents/rules/testing.md prefers a table for similar cases, and a table would make that difference explicit. It would also be a place to say why setCodingAgent is now needed in ...WithoutAgents: it keeps that test on the no-agents skip instead of the cannot-ask skip.
Changes
Automatic installation of Databricks AI tools now asks before a first install whenever the caller is not a coding agent, whether or not a terminal is attached. Before this change it asked only when stdin, stdout or stderr was a terminal. A run with no terminal at all installed without asking.
autoInstallAsksnow depends only on whether a coding agent was detected. It no longer checks for a terminal.canAskis false) is skipped, the same way runs that have a terminal but cannot prompt are skipped today. It installs nothing, sends no telemetry and leaves the version unsettled, so a later run that can ask, or a coding agent's call, still installs. This skip happens before the lock and the gate lookup, so cron jobs and job pods don't wait on the lookup every time they run.interactivefield on the auto-install telemetry event is still true only when a terminal is attached and no coding agent is detected. It is now computed at the call site.Why
Commands run over ssh without a terminal, under
nohup, from cron or from an IDE extension have no terminal. Installing in those cases changed the user's agent configuration without asking a person who may have started the command. Asking is not possible there, so the run waits for one where it is. A coding agent's call is the case that is meant to install without a question, and it keeps doing so.Tests
cmd/aitools/autoinstall_run_test.go:acceptance/cmd/aitools/auto-install/no-terminal. A run with no terminal and no coding agent installs nothing and leaves the version unsettled. A later coding agent call then installs.script.preparefor the other auto-install acceptance cases setsAGENT. Those cases have no terminal, so they now run as a coding agent's call and their outputs stay the same..nextchanges/cli/.