Skip to content

acc: share the CI env unset in an unset_ci_env helper - #7691

Merged
anton-107 merged 2 commits into
mainfrom
aitools/claude-code-native-hint-ci-env
Nov 18, 2026
Merged

anton-107 merged 2 commits into
mainfrom
aitools/claude-code-native-hint-ci-env

Conversation

@anton-107

@anton-107 anton-107 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Adds an unset_ci_env helper to acceptance/script.prepare. It replaces the three copies of the CI and CI/CD provider unset line in cmd/aitools/claude-config-dir, cmd/aitools/telemetry/hint and air/image-push. The helper's comment points at listCiCdProviders in the Go SDK's useragent/cicd.go so the list stays in sync. The helper is also listed in the built-in helpers section of .agents/rules/testing.md.

Why

Addresses simonfaltum's review comment on #7585 about the duplicated line. The helper lives in the root script.prepare because that is the only prepare file that covers both cmd/aitools and air. After this change, unset_ci_env is the only copy of the list.

Tests

  • Rebased on current main, which includes aitools: throttle the native Claude Code hint on its own shorter clock #7692. go test ./acceptance -run TestAccept/cmd/aitools (14 dirs) and -run TestAccept/air (24 dirs) both pass with no golden changes. They pass in a clean env and with CI=true GITHUB_ACTIONS=true exported.
  • Negative control: I turned the helper into a no-op and exported the CI vars. cmd/aitools/telemetry/hint, cmd/aitools/claude-config-dir and air/image-push all fail.

This pull request and its description were written by Isaac.

@anton-107
anton-107 requested review from a team as code owners November 16, 2026 16:03
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 13578d5ca | Workflow run

Regression test report

Selected 3 changed tests (limit=20, 0 not selected)

3 tests: regression=0 unreleased=3 coverage=0 failing=0
test branch main (3db96fa) latest (v1.26.0)
TestAccept/air/image-push ✅ ✅ ❌
TestAccept/cmd/aitools/claude-config-dir ✅ ✅ ❌
TestAccept/cmd/aitools/telemetry/hint ✅ ✅ ❌
TestAccept/air/image-push: branch ✅ | main (3db96fa) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- air/image-push/output.txt
        +++ /tmp/TestAcceptairimage-push1923287822/002/output.txt
        @@ -150,18 +150,18 @@
           databricks air images push [flags]
         
         Flags:
        -      --artifact string     Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        -      --catalog string      Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -h, --help[=true|false]   help for push
        -      --pull[=true|false]   Pull the source image even when it is already available locally
        -      --schema string       Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -s, --source string       Source container image to push (NAME[:TAG] or NAME@DIGEST)
        +      --artifact string   Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        +      --catalog string    Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -h, --help              help for push
        +      --pull              Pull the source image even when it is already available locally
        +      --schema string     Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -s, --source string     Source container image to push (NAME[:TAG] or NAME@DIGEST)
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
         
         >>> musterr databricks air images push --image artifact
        @@ -171,18 +169,18 @@
           databricks air images push [flags]
         
         Flags:
        -      --artifact string     Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        -      --catalog string      Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -h, --help[=true|false]   help for push
        -      --pull[=true|false]   Pull the source image even when it is already available locally
        -      --schema string       Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        -  -s, --source string       Source container image to push (NAME[:TAG] or NAME@DIGEST)
        +      --artifact string   Destination ARTIFACT[:TAG]; defaults to the source image name and tag (latest if untagged); maximum 255 characters for catalog.schema.artifact and 128 for the tag
        +      --catalog string    Destination Unity Catalog catalog; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -h, --help              help for push
        +      --pull              Pull the source image even when it is already available locally
        +      --schema string     Destination Unity Catalog schema; use lowercase letters and digits separated by one or two underscores or one or more hyphens
        +  -s, --source string     Source container image to push (NAME[:TAG] or NAME@DIGEST)
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
         
         === reject oversized catalog.schema.artifact before Docker operations
        @@ -217,10 +213,10 @@
         === in a Claude Code session the auth docker configure child adds no install hint
         
         >>> databricks air images push -p docker-test -s example/image:v1 --catalog main --schema training
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
         Pushing example/image:v1 as main.training.image:v1 using profile docker-test
        +Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Configured Docker credential helper for [WORKSPACE_ID].container.us-west-2.localhost
         Updated Docker config: [TEST_TMP_DIR]/home/docker/config.json
         Installed Docker credential helper: [TEST_TMP_DIR]/home/bin/docker-credential-databricks
        @@ -232,8 +225,8 @@
         Now you can set `environment.unity_catalog_image = main.training.image:v1` in your YAML config and create a workload with `databricks air run -f config.yaml`.
         
         >>> musterr databricks air images push -p pat-test -s example/image:v1 --catalog main --schema training
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Warning: This feature is in Preview. APIs may change, and the workspace must enable the Preview features.
         Pushing example/image:v1 as main.training.image:v1 using profile pat-test
        -Error: configure Docker authentication: run auth docker configure: exit status 1: Error: profile "pat-test" uses auth_type "pat". Docker credential helper requires a profile created by databricks auth login
        +Error: configure Docker authentication: run auth docker configure: exit status 1: Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        +Error: profile "pat-test" uses auth_type "pat". Docker credential helper requires a profile created by databricks auth login
        
FAIL
TestAccept/cmd/aitools/claude-config-dir: branch ✅ | main (3db96fa) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- cmd/aitools/claude-config-dir/output.txt
        +++ /tmp/TestAcceptcmdaitoolsclaude-config-dir822137310/002/output.txt
        @@ -9,7 +9,6 @@
         === Custom directory takes precedence over the default installation
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster   
        @@ -18,6 +17,7 @@
         === Exempt commands never hint, even without the tooling
         
         >>> CLAUDE_CONFIG_DIR=missing [CLI] current-user me
        +Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         json.userName = "[USERNAME]";
         
         === Empty override uses the default installation
        @@ -45,7 +45,6 @@
         === Within one session the text line prints at most once per hour and the native line at most once per 10 minutes, so a 2>&1 pipeline on the next command sees neither
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         ID    Name                State
         4321  some-test-cluster   
        @@ -65,7 +64,6 @@
         9876  some-other-cluster  
         
         >>> [CLI] clusters list
        -<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
         ID    Name                State
         4321  some-test-cluster   
         9876  some-other-cluster
TestAccept/cmd/aitools/telemetry/hint: branch ✅ | main (3db96fa) ✅ | latest (v1.26.0) ❌

latest (v1.26.0):

acceptance_test.go:1248: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36919" ("[DATABRICKS_HOST]")
    acceptance_test.go:1248: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36919" ("[DATABRICKS_HOST]")
    acceptance_test.go:1380: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1380
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1121
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:666
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event per printed line is uploaded\n\n>>> [CLI] jobs list\n<claude-code-hint v=\"1\" type=\"plugin\" value=\"databricks@claude-plugins-official\" />\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"CLAUDE_CODE_NATIVE\"}}\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] bundle schema\n<claude-code-hint v=\"1\" type=\"plugin\" value=\"databricks@claude-plugins-official\" />\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event per printed line is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] bundle schema\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,2 @@
        	            	 >>> [CLI] jobs list
        	            	-<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
        	            	 Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        	            	@@ -8,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"CLAUDE_CODE_NATIVE"}}
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	            	@@ -14,3 +11,2 @@
        	            	 >>> [CLI] bundle schema
        	            	-<claude-code-hint v="1" type="plugin" value="databricks@claude-plugins-official" />
        	            	 Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint1402229880/002/output.txt

@anton-107 anton-107 added aitools AI tools (databricks aitools) related changes ai-auto Deprecated: use factory-generated instead. labels Nov 16, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 13578d5

Run: 44207364905

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 18 5:32
💚​ aws-windows-latest-4core-16gb 1 272 16 3:50
💚​ azure linux-2core-8gb 1 269 18 4:42
💚​ azure-windows-latest-4core-16gb 1 271 16 3:57
💚​ gcp linux-2core-8gb 1 270 18 5:08
💚​ gcp-windows-latest-4core-16gb 1 272 16 3:55
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:53 gcp-windows-latest-4core-16gb TestAccept
3:48 aws-windows-latest-4core-16gb TestAccept
3:44 azure-windows-latest-4core-16gb TestAccept

Base automatically changed from aitools/claude-code-native-hint to main November 16, 2026 16:40
@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 16, 2026 16:40

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 0486b26. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking].

This PR moves the CI / CI/CD-provider unset line, which was copied into three acceptance scripts, into a shared unset_ci_env helper in the root script.prepare. The helper commit itself is correct and follows the repo's conventions: its variable list matches listCiCdProviders in the pinned SDK, and the root prepare file is the right home for it. However, the branch still contains an older, pre-review revision of #7585 rather than the merged one, so it conflicts with main and its diff drags in stale #7585 code. Resolving those conflicts toward the branch would revert merged fixes, such as accepting CI=no/off. Please rebase so that only the helper commit sits on main, and refresh the description; note that #7692 edits the same three scripts.

Comment thread libs/aitools/agents/hint_test.go Outdated
"github.com/stretchr/testify/require"
)

// hintEnvVars are the non-detection env vars MaybeHint reads. They are cleared

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] This branch is built on 07a831d, an older revision of #7585 from before its review feedback, not on the squash-merged #7585 (10fe968). GitHub reports the PR as not mergeable, with conflicts in six files: .nextchanges/cli/claude-code-native-hint.md, the three acceptance scripts, hint.go and hint_test.go. The PR diff also shows 21 files instead of the 5 the helper touches. If a conflict is resolved toward this branch, merged #7585 fixes come back undone. For example, inCI would go back to strconv.ParseBool, so CI=no or CI=off would count as CI again and drop the native hint. Also returning: hintEnvVars/unsetEnv (main now relies on testutil.SetAgentEnv clearing the env), the inline closure in place of the shared noRecordedPlugin, the extra plugin-manifest read, and the older .nextchanges entry. The description is also stale: #7585 is already merged, and main has no hintEnvVars, so after a rebase unset_ci_env is the only copy of the list. Please rebase with git rebase --onto origin/main 07a831d8a, which keeps only 0486b26 (it applies cleanly), and update the description.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto current main (now including #7692). The branch is the single helper commit (3de5e07), and the diff is 5 files. I also refreshed the description: it no longer mentions the stacking or hintEnvVars, and unset_ci_env is now the only copy of the list. I re-ran TestAccept/cmd/aitools and TestAccept/air in a clean env and with CI=true GITHUB_ACTIONS=true, plus the no-op negative control.

@anton-107
anton-107 force-pushed the aitools/claude-code-native-hint-ci-env branch from 0486b26 to 193c480 Compare November 17, 2026 11:15
@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 17, 2026
Three acceptance scripts unset the same list of CI and CI/CD provider
variables so the native Claude Code hint line also prints in CI. Move the
list into one unset_ci_env helper in the root script.prepare, the only
prepare file that covers both cmd/aitools and air, with a pointer to the
SDK's useragent/cicd.go so it stays in sync.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/claude-code-native-hint-ci-env branch from 193c480 to 3de5e07 Compare November 17, 2026 15:49

@pardis-beikzadeh-db pardis-beikzadeh-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

unset CLAUDE_CODE_SESSION_ID
# CI and the CI/CD provider markers the SDK detects (GitHub Actions sets both).
unset CI GITHUB_ACTIONS GITLAB_CI JENKINS_URL TF_BUILD CIRCLECI TRAVIS BITBUCKET_BUILD_NUMBER PROJECT_ID CODEBUILD_BUILD_ARN TFC_RUN_ID
unset_ci_env

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this duplicates the comment on unset_ci_env in acceptance/script.prepare. I'd drop it here and in acceptance/cmd/aitools/telemetry/hint/script.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 13578d5: I dropped the comment from both aitools scripts. I left air/image-push alone because its comment explains why the CI vars matter in that test specifically.

The helper in acceptance/script.prepare already documents the list.

Co-authored-by: Isaac <no-reply@databricks.com>

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 13578d5. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 1 non-blocking comment(s) and 2 question(s) below.

This PR moves the CI / CI/CD-provider unset line, which was copied into three acceptance scripts, into a shared unset_ci_env helper in the root script.prepare, and lists the helper in testing.md. The variable list matches listCiCdProviders in the pinned SDK, all three call sites keep their behavior, no other copy remains, and CI is green. The earlier rebase request and the duplicated-comment suggestion are both handled. One nit is open (link the SDK file in the sync note), plus two non-blocking questions: whether to clear the CI markers in the harness, and how to keep the list in sync on SDK bumps.

Questions

  1. The Go harness already clears the AI-agent markers and CLAUDE_CODE_CHILD_SESSION for every test (acceptance/acceptance_test.go:253-289). The CI markers are cleared only in tests that call unset_ci_env. A new native-hint test that forgets the call would record the hint locally and lose it on GitHub Actions; one that asserts no hint would pass there without testing anything. Did you consider clearing the CI markers in the harness instead? That would also make the cicd/... Repl in acceptance/cmd/api/test.toml unnecessary, though it could change other goldens that depend on CI. Keeping it opt-in is fine if that's deliberate. (non-blocking)
  2. unset_ci_env is a hand copy of the SDK's unexported listCiCdProviders, and only its own comment asks for it to be kept in sync. .agents/skills/bump-sdk/SKILL.md doesn't mention it, and it doesn't mention the agent list in acceptance_test.go either. Is the comment enough, or would a line in the bump-sdk skill that names these hand-copied SDK lists be worth adding (here or in a follow-up)? (non-blocking)

Comment thread acceptance/script.prepare

# unset_ci_env clears CI and the CI/CD provider markers, so a test behaves the
# same in CI as on a laptop. The provider list mirrors listCiCdProviders in the
# Go SDK's useragent/cicd.go (read by useragent.CiCdProvider()); keep it in

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The sync note names useragent/cicd.go but doesn't link it. .agents/rules/comments.md asks for a source URL when code detects environment variables, and the sibling aitools code does that (libs/aitools/agents/hint.go:26 links useragent/agent.go). Without a link, whoever syncs the list on the next SDK bump has to find the repo and file themselves. Add # https://github.com/databricks/databricks-sdk-go/blob/main/useragent/cicd.go to the comment.

@anton-107
anton-107 added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit e378b8a Nov 18, 2026
27 checks passed
@anton-107
anton-107 deleted the aitools/claude-code-native-hint-ci-env branch November 18, 2026 12:02
@denik denik added factory-generated and removed ai-auto Deprecated: use factory-generated instead. labels Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI aitools AI tools (databricks aitools) related changes factory-generated

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants