Repository navigation
aitools: Skip the apps init skills tip when Codex and Copilot have the plugin recorded - #7830
Conversation
…e plugin recorded
## Changes
`databricks apps init` decides whether to recommend Databricks skills by calling `agents.HasDatabricksToolingInstalled`. With this change, that check also uses the plugins recorded in the CLI install state (`installer.InstalledPluginAgents`). It is used for agents that have no plugin manifest the CLI can read, which today means Codex and Copilot.
```diff
for each detected agent:
if agent has no plugin manifest reader (Codex, Copilot):
- return false
+ if plugin not recorded in the CLI install state:
+ return false
+ continue
if databricks plugin not active for the working directory:
return false
return true
```
- `HasDatabricksToolingInstalled` and `RecommendSkillsInstall` now take an `installedPlugins func(context.Context) []string` argument. The `agents` package can't import the installer, so this follows the pattern `MaybeVersionCheckTip` already uses.
- `apps init` passes `installer.InstalledPluginAgents` in both paths: the flags-mode tip and the interactive install prompt.
- `MaybeVersionCheckTip` passes its existing `installedPlugins` through. Its behavior doesn't change, because it already returned early when any plugin was recorded.
- Claude Code is unchanged. It still needs the plugin in its own manifest, active for the working directory. A recorded Claude Code plugin without a manifest entry doesn't count.
- Added a changelog fragment under `.nextchanges/cli/`.
## Why
When Codex or Copilot was detected, `HasDatabricksToolingInstalled` returned false unless skills were in the shared skills directory. The CLI can't read a plugin manifest for those agents, so it never saw their plugin. As a result, users who had already run `databricks aitools install` to get the Databricks plugin for Codex or Copilot were still told to install skills by `apps init`, and the tip was logged in telemetry.
The CLI install state is the only plugin source the CLI has for these agents. One gap remains: a plugin installed with the agent's own CLI, not with `databricks aitools install`, still isn't detected.
## Tests
- `TestHasDatabricksToolingInstalledRecordedPlugins`, a new table test with these cases:
- Codex and Copilot both recorded: no recommendation.
- Only Codex recorded while Copilot is also detected: recommendation shown.
- Nothing recorded: recommendation shown.
- Codex recorded alongside Cursor, which has no tooling: recommendation shown.
- A Claude Code record without its manifest: recommendation shown.
- `TestRecommendSkillsInstallSkipsWhenPluginRecorded`: no prompt, no stderr output and no hint telemetry when Codex and Copilot plugins are recorded.
- Updated the existing `HasDatabricksToolingInstalled` and `RecommendSkillsInstall` tests for the new argument. They pass a stub that records no plugin, so their expectations don't change.
- Acceptance test `acceptance/cmd/aitools/telemetry/apps-init-tip`: a new case writes an install state that records Codex and Copilot plugins. It then checks that `apps init` prints no tip and logs no telemetry event.
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/aitools/telemetry/apps-init-tip: branch ✅ | main (0f23a05) ❌ | latest (v1.27.0) ❌main (0f23a05): latest (v1.27.0): |
✅ Integration test reportCommit: 231f85e
Top 3 slowest tests (at least 2 minutes):
|
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at abbcede. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 8 non-blocking comment(s) and 1 question(s) below.
This PR makes apps init stop recommending skills when the Databricks plugin for detected Codex and Copilot agents is recorded in the CLI install state. Claude Code still needs its own manifest entry. The logic is correct, and the new acceptance case fails on main and passes here. Nothing blocking turned up. The main fix is to isolate the two new unit tests and the new acceptance case from host env vars such as CLAUDE_CONFIG_DIR and XDG_CONFIG_HOME: they fail when those point at a real agent config. There is also an open, non-blocking question about sharing the per-agent plugin check with DatabricksTooling, plus a few small test, comment and efficiency nits.
Questions
HasDatabricksToolingInstallednow has its own copy of the per-agent rule thatAgent.DatabricksToolingalready applies: the manifest reader decides, otherwise the CLI record counts. This copy keys onpluginInstalls == nilrather thanPlugin != nil. It also reads records throughInstalledPluginAgents(global and project state plus the Claude cache), whileMaybeHintand the aitools User-Agent usePluginRecordedFunc/installer.PluginRecorded(ctx, ScopeGlobal, name). If either side changes later (an agent gains a manifest reader, or the record source changes), theapps inittip and the per-agent hint can disagree about the same agent. Would you consider sharing one per-agent check withDatabricksToolingand injecting the existingPluginRecordedFunc? Or is the separate source intentional? (non-blocking)
Comments outside the changed lines
- [nit]
libs/aitools/agents/recommend_test.go:171:recordedCodexPluginreturns"codex_0.2.6", theInstalledToolsformat, butHasDatabricksToolingInstallednow matches entries against the agent name. The case passes only becauselen(installedPlugins(ctx)) > 0short-circuits;slices.Contains(["codex_0.2.6"], "codex")is false. Return[]string{NameCodex}so the fixture matches whatInstalledPluginAgentsreturns.
…xture, read install state once - Use testutil.SetAgentEnv in the new unit tests and unset the agent config-dir overrides in the acceptance script, so host agent dirs can't add a detected agent. - Commit the recorded-plugin home as a fixture instead of creating it in the script, and explain why CLAUDECODE is unset. - Read the CLI install state lazily in HasDatabricksToolingInstalled and once in MaybeVersionCheckTip. - Note that a plugin removed with the agent's own CLI still counts while its record remains. - Add a case with a Claude Code manifest plugin and a recorded Codex plugin, and make the recorded fixture in TestMaybeVersionCheckTip match what InstalledPluginAgents returns. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
databricks apps initdecides whether to recommend Databricks skills by callingagents.HasDatabricksToolingInstalled. With this change, that check also uses the plugins recorded in the CLI install state (installer.InstalledPluginAgents). It is used for agents that have no plugin manifest the CLI can read, which today means Codex and Copilot.HasDatabricksToolingInstalledandRecommendSkillsInstallnow take aninstalledPlugins func(context.Context) []stringargument. Theagentspackage can't import the installer, so this follows the patternMaybeVersionCheckTipalready uses.apps initpassesinstaller.InstalledPluginAgentsin both paths: the flags-mode tip and the interactive install prompt.MaybeVersionCheckTippasses its existinginstalledPluginsthrough. Its behavior doesn't change, because it already returned early when any plugin was recorded..nextchanges/cli/.Why
When Codex or Copilot was detected,
HasDatabricksToolingInstalledreturned false unless skills were in the shared skills directory. The CLI can't read a plugin manifest for those agents, so it never saw their plugin. As a result, users who had already rundatabricks aitools installto get the Databricks plugin for Codex or Copilot were still told to install skills byapps init, and the tip was logged in telemetry.The CLI install state is the only plugin source the CLI has for these agents. One gap remains: a plugin installed with the agent's own CLI, not with
databricks aitools install, still isn't detected.Tests
TestHasDatabricksToolingInstalledRecordedPlugins, a new table test with these cases:TestRecommendSkillsInstallSkipsWhenPluginRecorded: no prompt, no stderr output and no hint telemetry when Codex and Copilot plugins are recorded.HasDatabricksToolingInstalledandRecommendSkillsInstalltests for the new argument. They pass a stub that records no plugin, so their expectations don't change.acceptance/cmd/aitools/telemetry/apps-init-tip: a new case writes an install state that records Codex and Copilot plugins. It then checks thatapps initprints no tip and logs no telemetry event.