Skip to content

aitools: test that deprecated experimental aliases mirror stable commands - #7826

Merged
anton-107 merged 4 commits into
mainfrom
deprecated-aitools-alias-tests
Nov 20, 2026
Merged

anton-107 merged 4 commits into
mainfrom
deprecated-aitools-alias-tests

Conversation

@anton-107

Copy link
Copy Markdown
Contributor

Changes

Adds unit tests for the hidden databricks experimental aitools command group in experimental/aitools/cmd/aitools_test.go. Only tests change.

  • TestExperimentalAitoolsSubcommands checks that the group is hidden and has exactly these subcommands: install, list, skills, tools, uninstall, update, version.
  • TestDeprecatedAliasesMirrorStableCommands builds both databricks aitools and databricks experimental aitools. For each deprecated alias (install, update, uninstall, list, version) it checks that:
    • the alias is hidden and its deprecation notice reads use "databricks aitools <name>" instead.
    • the stable command is visible and not deprecated
    • Use, Short and Long are the same on both
    • both have the same flags, compared by name, shorthand, type, default, hidden and deprecated state, and usage text

Why

The skills-management commands moved from databricks experimental aitools <name> to databricks aitools <name>. The old paths still work as hidden, deprecated aliases. These tests fail if the two drift apart, for example when a flag is added to only one of them or the deprecation notice points to the wrong command. Without them, scripts that still use the old paths could break with no test catching it.

Tests

The new tests pass when run with go test ./experimental/aitools/cmd/.

…ands

## Changes

Adds unit tests for the hidden `databricks experimental aitools` command group in `experimental/aitools/cmd/aitools_test.go`. Only tests change.

- `TestExperimentalAitoolsSubcommands` checks that the group is hidden and has exactly these subcommands: `install`, `list`, `skills`, `tools`, `uninstall`, `update`, `version`.
- `TestDeprecatedAliasesMirrorStableCommands` builds both `databricks aitools` and `databricks experimental aitools`. For each deprecated alias (`install`, `update`, `uninstall`, `list`, `version`) it checks that:
  - the alias is hidden and its deprecation notice reads `use "databricks aitools <name>" instead.`
  - the stable command is visible and not deprecated
  - `Use`, `Short` and `Long` are the same on both
  - both have the same flags, compared by name, shorthand, type, default, hidden and deprecated state, and usage text

## Why

The skills-management commands moved from `databricks experimental aitools <name>` to `databricks aitools <name>`. The old paths still work as hidden, deprecated aliases. These tests fail if the two drift apart, for example when a flag is added to only one of them or the deprecation notice points to the wrong command. Without them, scripts that still use the old paths could break with no test catching it.

## Tests

The new tests pass when run with `go test ./experimental/aitools/cmd/`.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 19, 2026 19:42
@github-actions github-actions Bot added aitools AI tools (databricks aitools) related changes test-only-fix PRs that update failing of flaky tests without changing CLI code. labels Nov 19, 2026

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 36112ed. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds test coverage for the hidden databricks experimental aitools command group: one test asserts the subcommand set and visibility, and another checks that deprecated aliases mirror the stable commands in metadata and flags. The tests are straightforward and correctly written. One repo convention is violated.

Co-authored-by: Isaac <no-reply@databricks.com>

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at bb41534. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 4 non-blocking comment(s) and 1 question(s) below.

This PR adds unit tests that pin the hidden databricks experimental aitools subcommands and check that each deprecated alias stays hidden, has the right notice, and matches its stable databricks aitools command. The tests are correct, CI is green, and the earlier _test package nit is fixed. Nothing is blocking. The main suggestion is to compare persistent flags too: both paths share constructors, so a persistent flag on the stable parent is the one drift these checks miss. I also left an open question on whether acceptance steps for the remaining aliases would protect old scripts more directly.

Questions

  1. Both paths build install, update, uninstall, list and version with the same aitoolscmd.New*Cmd constructors (experimental/aitools/cmd/aitools.go:25-29, cmd/aitools/aitools.go:29-33). So the Use/Short/Long/flag checks can't fail when a constructor changes. That includes the description's example of a flag added to only one of them. The checks fail only if a command is changed after it is built. The tests also build both trees standalone instead of through the real databricks experimental aitools wiring, and they don't run the commands. Old scripts depend on these paths still running and printing the notice. Would trace $CLI experimental aitools uninstall|list|version steps in the existing acceptance/experimental/aitools/skills/ tests cover that more directly, with or without these unit tests? Those tests already pin the install and update notices, and .agents/rules/testing.md prefers acceptance tests for user-visible output. If the unit tests stay, could the description say which drift they guard against? (non-blocking)

Comments outside the changed lines

  • [nit] experimental/aitools/cmd/aitools_test.go: Both commits carry Co-authored-by: Isaac trailers. .agents/skills/pr-checklist/SKILL.md:54 asks for a single italic disclosure line at the end of the PR body when an agent wrote or helped write the PR, for example _This PR was written by Claude Code._. Consider adding one.

anton-107 and others added 2 commits November 19, 2026 23:04
Co-authored-by: Isaac <no-reply@databricks.com>
Compare local and inherited flags (with NoOptDefVal) so a persistent flag
added to only one parent is caught, keep the alias names in one list used
by both tests, and build the experimental group under an experimental
parent so failures name the tree that lacks a subcommand.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit 9d1c8ef Nov 20, 2026
23 checks passed
@anton-107
anton-107 deleted the deprecated-aitools-alias-tests branch November 20, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aitools AI tools (databricks aitools) related changes test-only-fix PRs that update failing of flaky tests without changing CLI code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants