Repository navigation
Add a hidden --skills-invoked flag for experimental skill tracking - #7783
Conversation
Agents prefixing CLI calls with DATABRICKS_SKILLS_INVOKED=... no longer match permission allowlists such as Claude Code's Bash(databricks:*), so the command needs approval or is denied in headless runs. A flag keeps the command starting with "databricks". The flag is hidden from help and completion, wins over the env var when set, and is ignored when telemetry is disabled. Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/skills-invoked: branch ✅ | main (e822544) ❌ | latest (v1.26.0) ❌main (e822544): latest (v1.26.0): |
simonfaltum
left a comment
There was a problem hiding this comment.
No blocking issues found in 2ca4742c5. The focused skills unit tests and acceptance test pass. CI has no failing checks; Windows and integration checks are still running.
Please address the two small inline comments before merging.
For the agent-skills rollout, how will flag usage be gated on CLI support? Older CLIs reject --skills-invoked, as the regression report shows. The existing min_cli_version support or a version check could cover this.
This review was generated with GitHub MCP.
| if f := cmd.Flag(skillsInvokedFlag); f.Changed { | ||
| raw = f.Value.String() | ||
| } | ||
| for _, skill := range parseSkills(ctx, raw) { |
There was a problem hiding this comment.
[nit] Both debug messages in parseSkills still name DATABRICKS_SKILLS_INVOKED. With --skills-invoked='bad name!' and no env var set, the diagnostic points at the wrong source. Could these say "skill entry" now that the helper parses either input?
There was a problem hiding this comment.
Fixed in e4d58d0: both messages now say "skill entry" without naming the source.
| // skillsInvokedEnvVar (experimental tracking), unless the user opted out of | ||
| // telemetry. | ||
| func withSkillsInUserAgent(ctx context.Context) context.Context { | ||
| func initSkillsInvokedFlag(cmd *cobra.Command) { |
There was a problem hiding this comment.
[nit] Please add a brief doc comment for initSkillsInvokedFlag, as required by .agents/rules/comments.md (starting with the function name and ending with a period).
✅ Integration test reportCommit: e4d58d0
Top 10 slowest tests (at least 2 minutes):
|
Debug messages no longer name DATABRICKS_SKILLS_INVOKED, since the value can come from the flag too. Add a doc comment to initSkillsInvokedFlag. Co-authored-by: Isaac <no-reply@databricks.com>
|
@simonfaltum thanks. Both nits are fixed in e4d58d0. On gating: the databricks-core skill (universe#3145534) tells the agent to add |
❌ Integration test reportCommit: 6109237
36 interesting tests: 15 FAIL, 12 RECOVERED, 6 flaky, 3 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
Changes
Adds a hidden persistent
--skills-invokedflag. It feeds the experimentalskill/<name>user-agent pairs added in #7730, using the same parsing, the same 225-byte budget, and the sameDATABRICKS_CLI_DISABLE_TELEMETRYopt-out.DATABRICKS_SKILLS_INVOKEDwhenever it is set, including set to empty (it checksChanged). The env var still works.--helpand shell completion. There is no changelog entry, since this is internal, experimental tracking rather than a supported interface.Why
#7730 had agent skills prefix CLI calls with
DATABRICKS_SKILLS_INVOKED=... databricks .... A prefixed command no longer starts withdatabricks, so permission allowlists like Claude Code'sBash(databricks:*)stop matching it. I checked this with headless Claude Code: with onlyBash(databricks:*)allowed, the prefixed command was refused as "requires approval" and the CLI never ran. The agent-skills plugin's own/databricks:setupand/databricks:doctoruse that allowlist too. A flag keeps the command starting withdatabricks.Tests
cmd/root_test): env var only, flag only, flag over env var, an explicit empty flag over the env var, and the telemetry opt-out for both.acceptance/cmd/skills-invoked): the flag adds the pairs, the flag wins over the env var, the opt-out applies to the flag, and the flag is absent from--helpand__completeoutput.This pull request and its description were written by Isaac.