Repository navigation
Conversation
## Changes `databricks aitools install --skills-only` now records which agents the user kept on raw skills, and `databricks aitools update` leaves those agents alone instead of moving them to the Databricks plugin. - `InstallState` has a new `skills_only_agents` field: a sorted list of agent names, left out of the file when it is empty. - `install --skills-only` sets the new `InstallOptions.SkillsOnly`. `InstallSkillsForAgents` then adds every target agent that supports a plugin and has its CLI on PATH to that list. - An agent whose CLI is missing is not added. `install` suggests `--skills-only` in that case too, so the user didn't turn the plugin down, and `update` should still migrate the agent once the CLI is installed. - `pluginMigrationCandidatesForScope` skips agents in the list, so `update` and `update --check` no longer migrate them or report that they would. Their raw skills still get refreshed as before. - `RecordPluginInstalls` takes an agent off the list once the plugin is installed for it. Running `install` again without `--skills-only` leaves the list as it is. ```diff pluginMigrationCandidatesForScope skip agents without a plugin skip agents already on the plugin + skip agents in state.SkillsOnlyAgents skip agents whose CLI is not on PATH skip agents without managed raw skills ``` ## Why For Claude Code, Codex and GitHub Copilot, `update` replaces CLI-managed raw skills with the plugin whenever the agent's CLI is on PATH. The state file couldn't tell skills installed with `--skills-only` apart from skills written by an older CLI or while the agent's CLI was missing. So the next `update` undid the user's `--skills-only` choice. Now that choice is saved, and only raw skills the user didn't choose get migrated. To move an agent to the plugin, run `databricks aitools install` without `--skills-only`. ## Tests New unit tests: - `TestInstallSkillsForAgentsRecordsSkillsOnlyAgents`: only agents with a plugin and their CLI on PATH are recorded, and a later install without `--skills-only` keeps the list. - `TestRecordPluginInstallsForgetsSkillsOnlyChoice`: installing the plugin for an agent takes only that agent off the list. - `TestPluginMigrationCandidatesSkipSkillsOnlyAgents`: agents in the list are not migration candidates, and other agents with raw skills still are. - `TestUpdateKeepsSkillsOnlyAgentsOnRawSkills`: `update --check` reports no migration, and `update` refreshes the raw skills without installing the plugin or cleaning up the skills. - The existing tests for `install --skills-only` and state round-tripping now also check the new option and field. `go test ./cmd/aitools/... ./libs/aitools/installer/...` passes. Adds the changelog fragment `.nextchanges/cli/aitools-update-keeps-skills-only.md`. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Automated review at b6e70db. Each finding was verified against the code.
No blocking issues found. 2 non-blocking comment(s) below.
The PR adds a SkillsOnlyAgents field to the install state so that update no longer migrates agents the user explicitly put on raw skills via install --skills-only. The lifecycle is clear for the happy path: install --skills-only records the preference, RecordPluginInstalls removes it on a successful plugin install, and pluginMigrationCandidatesForScope skips listed agents. Two edge cases in the preference lifecycle are worth tightening: install without --skills-only does not clear entries even when it targets the agent (so a transient plugin failure leaves the user stuck), and an older CLI that saves state will silently drop the field.
An install that tries the plugin for an agent the user had put on raw skills now drops that agent from the --skills-only record even when the plugin install fails, so update tries the plugin again. Also note on the field that an older CLI rewriting the state drops it. Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 06c8e20. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 7 non-blocking comment(s) and 1 question(s) below.
This PR records which agents install --skills-only put on raw skills, and makes update skip migrating those agents to the plugin. A recorded or failed plugin install clears the entry. The core change is small and well tested, and CI is green; I found nothing blocking. The main open point is that several CLI-suggested --skills-only runs (plugin-failure fallbacks, legacy-state rebuild advice, --skills cherry-picks, the deprecated alias) now pin agents to raw skills permanently. Smaller fixes: the changelog overstates the guarantee, a targeted uninstall leaves stale entries, and two existing tests can now rewrite a developer's real state file.
Questions
--skills-onlynow also records "this agent declined the plugin", but the CLI itself tells users to run--skills-onlyfor other reasons while the agent's CLI is on PATH: every plugin-failure fallback inpluginFailureText(recovery.go:436-461: CLI too old forplugin, timeout, shim error, failed install), the legacy-state rebuild advice (installer.go:379, uninstall.go:131), and--skillscherry-picks, which require--skills-only(install.go:206). Each of these now pins the agent to raw skills for every laterupdate, even after the plugin problem is fixed; before this PR,updatewould migrate it. The CLI-missing exception is justified by the same reasoning ("install suggests --skills-only for that case too"). Should these runs also not be recorded? Or should--skills-onlyalways count as a choice, with its help text and the fallback messages saying thatupdatekeeps it? (non-blocking)
Accepted answers
- Should a failed plugin install during install clear the skills-only choice?: #7831 (comment)
- An older CLI rewriting state drops skills_only_agents: #7831 (comment)
- Why aren't agents whose CLI is missing recorded?
- The deprecated experimental skills install alias no longer records the choice: it adds --skills-only itself, so its users did not decline the plugin. - A targeted uninstall without --skills ends the choice of those agents. - Scope the changelog entry to installs made from this release on. - Keep the executePlan tests with a failing plugin off the real HOME, and stub agents.LookPath instead of writing a binary on PATH. - Cover install --skills-only then update with an agent CLI on PATH in the install-non-interactive acceptance test. Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/experimental/aitools/skills/install-non-interactive: branch ✅ | main (8b7e509) ❌ | latest (v1.27.0) ❌main (8b7e509): latest (v1.27.0): |
Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 245f98f. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 6 non-blocking comment(s) below.
This PR records which agents install --skills-only put on skill files, so update stops moving them to the plugin. A plugin install for the agent, a failed attempt at one, or a targeted uninstall clears the entry. CI is green, the earlier review threads are addressed, and I found nothing blocking. Smaller points: a plain uninstall that can't remove a plugin leaves stale entries, plus a few nits (an acceptance step that can pass without a stand-in claude, test isolation, changelog wording, a duplicated helper, a stale comment, the outdated description). One earlier question, whether the CLI's own --skills-only suggestions should keep agents off the plugin, is still unanswered.
Comments outside the changed lines
- [nit]
cmd/aitools/install.go:421: TheinstallOutputIsJSONcomment says the legacyskills installalias "builds a NewInstallCmd", butnewLegacySkillsInstallCmdnow callsnewInstallCmd(false), which deliberately turns off recording. Change the comment to namenewInstallCmd.
Agreeing with existing threads
- [nit] PR description still says install without --skills-only leaves the list as it is: #7831 (comment)
| // CLI scope. Install calls it for agents whose plugin install it attempted and | ||
| // that failed, so update tries the plugin again. Agents not in the list, or a | ||
| // scope without state, are left as they are. | ||
| func ForgetSkillsOnlyChoice(ctx context.Context, cliScope string, agentNames []string) error { |
There was a problem hiding this comment.
[nit] ForgetSkillsOnlyChoice repeats the skillsDir/LoadState/nil-check/change-detect/SaveState skeleton of ForgetPluginInstalls directly above it (plugin.go:544-563), and the same slices.DeleteFunc name filter also appears in RecordPluginInstalls and uninstall.go:125. An install with both successful and failed plugin agents now also writes the state twice. Optional: one helper that drops the named agents from a scope's state, or a single ForgetSkillsOnlyChoice call for all attempted plugin agents, would remove the duplication.
There was a problem hiding this comment.
I'd rather leave this as is in this PR. A shared helper would mean refactoring ForgetPluginInstalls, which this PR doesn't otherwise touch, and the two functions filter different fields (a map and a sorted slice). The second state write only happens when one install has both successful and failed plugin agents, and it is a small local file. Happy to fold the three name filters into one helper in a follow-up if you think it's worth it.
There was a problem hiding this comment.
Leaving this as is here, for the reasons above. The shared helper would mean changing ForgetPluginInstalls, which is outside this change. I'll fold the name filters into one helper in a follow-up.
Co-authored-by: Isaac <no-reply@databricks.com>
A plain uninstall now ends every --skills-only choice, also when the state file stays for a plugin it could not remove. An uninstall that could not remove a skill keeps the choice, since the files are still there. The install-non-interactive acceptance test stops when the agent CLI stand-ins were not built, and sets FAKE_AGENT_LOG for them. Two uninstall tests no longer follow the host's agent config dir overrides. Reword the changelog fragment and fix a stale comment. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 1e1539b. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 4 non-blocking comment(s) below.
This PR records which agents install --skills-only put on skill files, and makes update stop moving them to the plugin. A plugin install, a failed plugin attempt or an uninstall clears the entry. The core change is small and well tested, CI is green, and I found nothing blocking. Two state gaps are worth closing: a plain install's CLI-missing fallback keeps the choice, and a multi-agent uninstall with one failure keeps every named agent's choice. There are also a few nits. The earlier question about the CLI's own --skills-only suggestions is still open.
Accepted answers
- One shared helper or one state write for dropping agents from SkillsOnlyAgents: #7831 (comment)
Comments outside the changed lines
- [nice to have]
cmd/aitools/install.go:955: A plaindatabricks aitools install(no--agents) can find Claude Code's config dir withoutclaudeon PATH. The agent then becomes a fallback item and gets skill files, but nothing callsForgetSkillsOnlyChoicefor it. So an agent recorded by an earlier--skills-onlyrun stays inskills_only_agents, and onceclaudeis back on PATH,updatenever migrates it. The same install with--agents claude-codebehaves differently: it attempts the plugin, fails, and clears the choice at line 1011.cliMissingOutcomealready reports the fallback as a failed plugin attempt. Either callinstaller.ForgetSkillsOnlyChoiceforfallbackItemsnext toforgetStalePluginRecords, or document that a fallback keeps the choice. In the second case, also adjust theSkillsOnlyAgentsdoc: it says update migrates raw skills that were written while the agent's CLI was missing.
Agreeing with existing threads
- [nit] PR description is out of date: #7831 (comment)
Co-authored-by: Isaac <no-reply@databricks.com>
With several --agents, a removal failure for one agent no longer keeps every named agent's choice; agents whose entries were all removed lose theirs. Trim the SkillsOnlyAgents doc comment to its contract and restore an unrelated blank line. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
databricks aitools install --skills-onlynow records which agents the user kept on raw skills, anddatabricks aitools updateleaves those agents alone instead of moving them to the Databricks plugin.InstallStatehas a newskills_only_agentsfield: a sorted list of agent names, left out of the file when it is empty.install --skills-onlysets the newInstallOptions.SkillsOnly.InstallSkillsForAgentsthen adds every target agent that supports a plugin and has its CLI on PATH to that list.installsuggests--skills-onlyin that case too, so the user didn't turn the plugin down, andupdateshould still migrate the agent once the CLI is installed.experimental aitools skills installalias always adds--skills-only, so it does not record the choice.pluginMigrationCandidatesForScopeskips agents in the list, soupdateandupdate --checkno longer migrate them or report that they would. Their raw skills still get refreshed as before.RecordPluginInstallsrecords the plugin for it;installwithout--skills-onlytries the plugin for it and the plugin install fails (ForgetSkillsOnlyChoice), so the nextupdatetries the plugin again;uninstallwithout--skillsremoves all of its skills. A plainuninstallclears the whole list,uninstall --agentsonly the named agents. An uninstall that could not remove a skill keeps the list, since the files are still there.installwithout--skills-onlythat does not try the plugin for an agent leaves the list as it is.pluginMigrationCandidatesForScope skip agents without a plugin skip agents already on the plugin + skip agents in state.SkillsOnlyAgents skip agents whose CLI is not on PATH skip agents without managed raw skillsWhy
For Claude Code, Codex and GitHub Copilot,
updatereplaces CLI-managed raw skills with the plugin whenever the agent's CLI is on PATH. The state file couldn't tell skills installed with--skills-onlyapart from skills written by an older CLI or while the agent's CLI was missing. So the nextupdateundid the user's--skills-onlychoice. Now that choice is saved, and only raw skills the user didn't choose get migrated. To move an agent to the plugin, rundatabricks aitools installwithout--skills-only.Tests
New unit tests:
TestInstallSkillsForAgentsRecordsSkillsOnlyAgents: only agents with a plugin and their CLI on PATH are recorded, and a later install without--skills-onlykeeps the list.TestRecordPluginInstallsForgetsSkillsOnlyChoice: installing the plugin for an agent takes only that agent off the list.TestForgetSkillsOnlyChoice,TestForgetSkillsOnlyChoiceWithoutState: only the named agents come off the list, and a scope without state is left alone.TestExecutePlanFailedPluginForgetsSkillsOnlyChoice: a failed plugin install takes the agent off the list.TestPluginMigrationCandidatesSkipSkillsOnlyAgents: agents in the list are not migration candidates, and other agents with raw skills still are.TestUpdateKeepsSkillsOnlyAgentsOnRawSkills:update --checkreports no migration, andupdaterefreshes the raw skills without installing the plugin or cleaning up the skills.TestUninstallTargetedAgentForgetsSkillsOnlyChoice,TestUninstallTargetedAgentFailureKeepsSkillsOnlyChoice,TestUninstallAllForgetsSkillsOnlyChoice: a targeted uninstall clears only the named agents and keeps the choice when a skill could not be removed, and a plain uninstall clears the list even when the state file stays for a plugin.install --skills-only, the legacyskills installalias and state round-tripping now also check the new option and field.The
install-non-interactiveacceptance test now also runsinstall --skills-onlyandupdatewith a Claude Code CLI stand-in on PATH and checks that the skill files stay../task aitools-fullpasses.Adds the changelog fragment
.nextchanges/cli/aitools-update-keeps-skills-only.md.