Skip to content

Run pipelines test against the pipeline's testing.sources - #7429

Merged
tharunbhuk-db merged 7 commits into
mainfrom
tharun/pipelines-test-j1-testing-sources
Nov 18, 2026
Merged

tharunbhuk-db merged 7 commits into
mainfrom
tharun/pipelines-test-j1-testing-sources

Conversation

@tharunbhuk-db

@tharunbhuk-db tharunbhuk-db commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

🥞 Stacked PR


  • databricks pipelines test runs without an environment variable. Its help marks it Beta, so it stays unstable under the stability policy.
  • A run of all tests sends test_options as an empty object, so a workspace that does not support it returns a clear error.
  • The pipelines test acceptance bundles declare testing, since test runs need testing.sources.
  • When Start Update fails with PIPELINE_TEST_NO_TEST_SOURCES and the bundle sets testing, the error says to deploy first.
  • Adds the changelog entry for the command and the testing property.

Tests: TestNewStartTestUpdateRequest, TestWithDeployHint; acceptance pipelines/test/not-enabled, pipelines/test/no-test-sources, pipelines/databricks-cli-help, and the pipelines/test goldens without the gate cases.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 7ca3bea75 | Workflow run

Regression test report

Selected 8 changed tests (limit=20, 0 not selected)

8 tests: regression=8 unreleased=0 coverage=0 failing=0
test branch main (73a2b68) latest (v1.26.0)
TestAccept/pipelines/databricks-cli-help ✅ ❌ ❌
TestAccept/pipelines/test/arguments ✅ ❌ ❌
TestAccept/pipelines/test/immutable ✅ ❌ ❌
TestAccept/pipelines/test/no-test-sources ✅ ❌ ❌
TestAccept/pipelines/test/not-enabled ✅ ❌ ❌
TestAccept/pipelines/test/run-control/timeout ✅ ❌ ❌
TestAccept/pipelines/test/selection ✅ ❌ ❌
TestAccept/pipelines/test/submit-error ✅ ❌ ❌
TestAccept/pipelines/databricks-cli-help: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: Diff:
        --- pipelines/databricks-cli-help/output.txt
        +++ /tmp/TestAcceptpipelinesdatabricks-cli-help1139207626/002/output.txt
        @@ -30,7 +30,7 @@
           run                   Run a pipeline
           show                  Preview a table's columns and sample rows
           stop                  Stop a pipeline
        -  test                  *Beta* Run the tests of a pipeline
        +  test                  *Experimental* Run the tests of a pipeline
         
         Management Commands
           apply-environment     *Public Preview* Apply the latest environment to the pipeline.

latest (v1.26.0):

acceptance_test.go:1384: Diff:
        --- pipelines/databricks-cli-help/output.txt
        +++ /tmp/TestAcceptpipelinesdatabricks-cli-help1593511623/002/output.txt
        @@ -30,7 +30,7 @@
           run                   Run a pipeline
           show                  Preview a table's columns and sample rows
           stop                  Stop a pipeline
        -  test                  *Beta* Run the tests of a pipeline
        +  test                  *Preview* Run the tests of a pipeline
         
         Management Commands
           apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        @@ -52,13 +52,13 @@
           update-permissions    Update pipeline permissions.
         
         Flags:
        -  -h, --help[=true|false]   help for pipelines
        -      --var strings         set values for variables defined in project config. Example: --var="foo=bar"
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
         Use "databricks pipelines [command] --help" for more information about a command.
TestAccept/pipelines/test/arguments: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments2363175690/002/output.txt
        @@ -2,6 +2,8 @@
         >>> [CLI] pipelines test --help
         Run the tests of a pipeline in a test-only update.
         
        +This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.
        +
         KEY is the unique name of the pipeline, as defined in its YAML file.
         It is required when test selections are given. If there is only one pipeline
         in the project and no arguments are given, it is selected automatically.
        @@ -62,52 +64,52 @@
         Error: -k is a pytest argument: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -- -k orders
        -Error: the -- separator is not supported: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === Invalid --pytest-arg values fail before any request
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=-x
        -Error: unsupported pytest argument "-x": only -k and -m are supported
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=
        -Error: --pytest-arg requires a non-empty value
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === The pipeline ID must be a pipeline ID
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234
        -Error: --pipeline-id "1234" is not a pipeline ID
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === JSON output is not supported
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -o json
        -Error: --output json is not supported by pipelines test
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === Incompatible run-control flags
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 tests --resume 5678
        -Error: --resume cannot be used with test selections
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678 --pytest-arg=-m --pytest-arg=smoke
        -Error: --resume cannot be used with --pytest-arg
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678 --no-wait
        -Error: --resume cannot be used with --no-wait
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --no-wait --timeout 10m
        -Error: --timeout cannot be used with --no-wait
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678' OR '1'='1
        -Error: --resume "5678' OR '1'='1" is not an update ID; pass the update ID that pipelines test printed
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 
        -Error: --resume "" is not an update ID; pass the update ID that pipelines test printed
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === --junit-xml needs the results
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --no-wait --junit-xml out.no-wait.xml
        -Error: --junit-xml cannot be used with --no-wait because no results are available when the command returns
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === An invalid flag replaces an older report
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=-x --junit-xml out.flag-error.xml
        -Error: unsupported pytest argument "-x": only -k and -m are supported
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        
    acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1137
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"1234\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"unsupported pytest argument &#34;-x&#34;: only -k and -m are supported\">unsupported pytest argument &#34;-x&#34;: only -k and -m are supported</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	actual  : "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"1234\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,3 @@
        	            	     <testcase name="pipelines test">
        	            	-      <error message="unsupported pytest argument &#34;-x&#34;: only -k and -m are supported">unsupported pytest argument &#34;-x&#34;: only -k and -m are supported</error>
        	            	+      <error message="pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>
        	            	     </testcase>
        	Test:       	TestAccept/pipelines/test/arguments
        	Messages:   	pipelines/test/arguments/out.flag-error.xml vs /tmp/TestAcceptpipelinestestarguments2363175690/001/out.flag-error.xml
    acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1137
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"1234\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"--junit-xml cannot be used with --no-wait because no results are available when the command returns\">--junit-xml cannot be used with --no-wait because no results are available when the command returns</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	actual  : "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"1234\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,3 @@
        	            	     <testcase name="pipelines test">
        	            	-      <error message="--junit-xml cannot be used with --no-wait because no results are available when the command returns">--junit-xml cannot be used with --no-wait because no results are available when the command returns</error>
        	            	+      <error message="pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>
        	            	     </testcase>
        	Test:       	TestAccept/pipelines/test/arguments
        	Messages:   	pipelines/test/arguments/out.no-wait.xml vs /tmp/TestAcceptpipelinestestarguments2363175690/001/out.no-wait.xml

latest (v1.26.0):

acceptance_test.go:1384: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments4129819716/002/output.txt
        @@ -2,6 +2,8 @@
         >>> [CLI] pipelines test --help
         Run the tests of a pipeline in a test-only update.
         
        +This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.
        +
         KEY is the unique name of the pipeline, as defined in its YAML file.
         It is required when test selections are given. If there is only one pipeline
         in the project and no arguments are given, it is selected automatically.
        @@ -21,40 +23,22 @@
         
           databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
         
        -The command waits for the results. It exits with code 0 only if at least one
        -test was collected and none failed or errored, so a run whose tests were all
        -skipped succeeds; otherwise it exits with code 1. --no-wait returns after the
        -update starts.
        -
        ---timeout and Ctrl+C stop waiting but not the update; pass its ID to --resume
        -to read the results later. To stop the update, run 'pipelines stop'.
        -
        ---junit-xml writes one entry per test with its outcome and duration. An
        -incomplete run, a run without tests, or a timeout adds a failing entry that
        -names the cause. The report replaces the file on every exit except a flag that
        -cannot be parsed. With workspace.sync_mode immutable, test names keep the path
        -of the deployed snapshot, which changes with each deployment.
        -
         Usage:
           databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
         Pipeline Test Flags:
        -      --junit-xml string         Write the results as a JUnit XML report to this file, replacing it.
        -      --no-wait[=true|false]     Don't wait for the tests to complete.
               --pipeline-id string       ID of the pipeline to test, instead of a KEY.
               --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        -      --resume string            Read the results of an existing test update instead of starting one.
        -      --timeout duration         Stop waiting for results after this duration, for example 30m. The update keeps running.
         
         Flags:
        -  -h, --help[=true|false]   help for test
        +  -h, --help   help for test
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        -      --var strings          set values for variables defined in project config. Example: --var="foo=bar"
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
         === pytest arguments must be passed with --pytest-arg
         
        @@ -62,52 +36,196 @@
         Error: -k is a pytest argument: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -- -k orders
        -Error: the -- separator is not supported: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === Invalid --pytest-arg values fail before any request
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=-x
        -Error: unsupported pytest argument "-x": only -k and -m are supported
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --pytest-arg=
        -Error: --pytest-arg requires a non-empty value
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === The pipeline ID must be a pipeline ID
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234
        -Error: --pipeline-id "1234" is not a pipeline ID
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === JSON output is not supported
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -o json
        -Error: --output json is not supported by pipelines test
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
         === Incompatible run-control flags
         
         >>> musterr [CLI] pipelines test --pipeline-id 1234 tests --resume 5678
        -Error: --resume cannot be used with test selections
        +Error: unknown flag: --resume
         
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
        +
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678 --pytest-arg=-m --pytest-arg=smoke
        -Error: --resume cannot be used with --pytest-arg
        +Error: unknown flag: --resume
         
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
        +
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678 --no-wait
        -Error: --resume cannot be used with --no-wait
        +Error: unknown flag: --resume
         
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
        +
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --no-wait --timeout 10m
        -Error: --timeout cannot be used with --no-wait
        +Error: unknown flag: --no-wait
         
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
        +
         >>> musterr [CLI] pipelines test --pipeline-id 1234 --resume 5678' OR '1'='1
        -Error: --resume "5678' OR '1'='1" is not an update ID; pass the update ID that pipelines test printed
        +Error: unknown flag: --resume
         
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
        +
... (96 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)
TestAccept/pipelines/test/immutable: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable3415898093/002/output.txt
        @@ -4,53 +4,16 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Relative test paths are rejected with workspace.sync_mode immutable
         
         >>> musterr [CLI] pipelines test my_pipeline tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": relative test paths need synced local files, but workspace.sync_mode is immutable; pass the /Workspace path
        -
         === Absolute workspace paths and running all tests still work
         
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        -
        ->>> [CLI] pipelines test my_pipeline --no-wait
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {}
        -  }
        -}
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1384: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable3289270457/002/output.txt
        @@ -1,56 +1,41 @@
         
         >>> [CLI] pipelines deploy
        +Warning: unknown field: testing
        +  at resources.pipelines.my_pipeline
        +  in databricks.yml:14:7
        +
         Created internal_immutable_snapshots.immutable
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Relative test paths are rejected with workspace.sync_mode immutable
         
         >>> musterr [CLI] pipelines test my_pipeline tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": relative test paths need synced local files, but workspace.sync_mode is immutable; pass the /Workspace path
        -
         === Absolute workspace paths and running all tests still work
         
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> [CLI] pipelines test my_pipeline --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Flags:
        +  -h, --help   help for test
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {}
        -  }
        -}
        +Exit code: 1
TestAccept/pipelines/test/no-test-sources: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1125
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] pipelines deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-no-test-sources/default/files...\nCreated pipelines.my_pipeline\nFiles: 3 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\nView your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]\n\n=== The bundle now sets testing, so the service error comes with a hint to deploy\n\n>>> musterr [CLI] pipelines test --no-wait\nRecommendation: This command runs the last deployed version of the code\n\nIf you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.\n\nError: [PIPELINE_TEST_NO_TEST_SOURCES] The pipeline has no test sources. Set `testing.sources` to the test files or folders to run.\nThe bundle sets testing for my_pipeline, but the deployed pipeline has none. Run 'databricks pipelines deploy' first\n"
        	            	actual  : "\n>>> [CLI] pipelines deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-no-test-sources/default/files...\nCreated pipelines.my_pipeline\nFiles: 3 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\nView your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]\n\n=== The bundle now sets testing, so the service error comes with a hint to deploy\n\n>>> musterr [CLI] pipelines test --no-wait\nError: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -11,8 +11,3 @@
        	            	 >>> musterr [CLI] pipelines test --no-wait
        	            	-Recommendation: This command runs the last deployed version of the code
        	            	+Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        	            	 
        	            	-If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        	            	-
        	            	-Error: [PIPELINE_TEST_NO_TEST_SOURCES] The pipeline has no test sources. Set `testing.sources` to the test files or folders to run.
        	            	-The bundle sets testing for my_pipeline, but the deployed pipeline has none. Run 'databricks pipelines deploy' first
        	            	-
        	Test:       	TestAccept/pipelines/test/no-test-sources
        	Messages:   	pipelines/test/no-test-sources/output.txt vs /tmp/TestAcceptpipelinestestno-test-sources2719890357/002/output.txt

latest (v1.26.0):

acceptance_test.go:1384: Diff:
        --- pipelines/test/no-test-sources/output.txt
        +++ /tmp/TestAcceptpipelinestestno-test-sources1469165618/002/output.txt
        @@ -9,9 +9,22 @@
         === The bundle now sets testing, so the service error comes with a hint to deploy
         
         >>> musterr [CLI] pipelines test --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Error: [PIPELINE_TEST_NO_TEST_SOURCES] The pipeline has no test sources. Set `testing.sources` to the test files or folders to run.
        -The bundle sets testing for my_pipeline, but the deployed pipeline has none. Run 'databricks pipelines deploy' first
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        +
        +Flags:
        +  -h, --help   help for test
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        +
TestAccept/pipelines/test/not-enabled: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1125
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "\n=== Where the testing API is off, a run of all tests is rejected, not run as an older test-only update\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml\nError: Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.\n\n>>> print_requests.py //updates\n{\n  \"method\": \"POST\",\n  \"path\": \"/api/2.0/pipelines/[UUID]/updates\",\n  \"body\": {\n    \"test_only\": true,\n    \"test_options\": {}\n  }\n}\n"
        	            	actual  : "\n=== Where the testing API is off, a run of all tests is rejected, not run as an older test-only update\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml\nError: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\n\n>>> print_requests.py //updates\nFile [OUTPUT_DIR]/out.requests.txt not found\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,13 +4,8 @@
        	            	 >>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml
        	            	-Error: Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.
        	            	+Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        	            	 
        	            	 >>> print_requests.py //updates
        	            	-{
        	            	-  "method": "POST",
        	            	-  "path": "/api/2.0/pipelines/[UUID]/updates",
        	            	-  "body": {
        	            	-    "test_only": true,
        	            	-    "test_options": {}
        	            	-  }
        	            	-}
        	            	+File [OUTPUT_DIR]/out.requests.txt not found
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/pipelines/test/not-enabled
        	Messages:   	pipelines/test/not-enabled/output.txt vs /tmp/TestAcceptpipelinestestnot-enabled3268619313/002/output.txt
    acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1137
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"[UUID]\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.\">Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	actual  : "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"[UUID]\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,3 @@
        	            	     <testcase name="pipelines test">
        	            	-      <error message="Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.">Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.</error>
        	            	+      <error message="pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>
        	            	     </testcase>
        	Test:       	TestAccept/pipelines/test/not-enabled
        	Messages:   	pipelines/test/not-enabled/out.not-enabled.xml vs /tmp/TestAcceptpipelinestestnot-enabled3268619313/001/out.not-enabled.xml

latest (v1.26.0):

acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1125
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "\n=== Where the testing API is off, a run of all tests is rejected, not run as an older test-only update\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml\nError: Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.\n\n>>> print_requests.py //updates\n{\n  \"method\": \"POST\",\n  \"path\": \"/api/2.0/pipelines/[UUID]/updates\",\n  \"body\": {\n    \"test_only\": true,\n    \"test_options\": {}\n  }\n}\n"
        	            	actual  : "\n=== Where the testing API is off, a run of all tests is rejected, not run as an older test-only update\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml\nError: unknown flag: --junit-xml\n\nUsage:\n  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]\n\nPipeline Test Flags:\n      --pipeline-id string       ID of the pipeline to test, instead of a KEY.\n      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.\n\nFlags:\n  -h, --help   help for test\n\nGlobal Flags:\n      --debug            enable debug logging\n  -o, --output type      output type: text or json (default text)\n  -p, --profile string   ~/.databrickscfg profile\n  -t, --target string    bundle target to use (if applicable)\n      --var strings      set values for variables defined in project config. Example: --var=\"foo=bar\"\n\n\n>>> print_requests.py //updates\nFile [OUTPUT_DIR]/out.requests.txt not found\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,13 +4,26 @@
        	            	 >>> musterr [CLI] pipelines test --pipeline-id [UUID] --junit-xml out.not-enabled.xml
        	            	-Error: Pipeline testing is not enabled for this pipeline. Contact your Databricks account team to join the preview.
        	            	+Error: unknown flag: --junit-xml
        	            	+
        	            	+Usage:
        	            	+  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        	            	+
        	            	+Pipeline Test Flags:
        	            	+      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        	            	+      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        	            	+
        	            	+Flags:
        	            	+  -h, --help   help for test
        	            	+
        	            	+Global Flags:
        	            	+      --debug            enable debug logging
        	            	+  -o, --output type      output type: text or json (default text)
        	            	+  -p, --profile string   ~/.databrickscfg profile
        	            	+  -t, --target string    bundle target to use (if applicable)
        	            	+      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        	            	+
        	            	 
        	            	 >>> print_requests.py //updates
        	            	-{
        	            	-  "method": "POST",
        	            	-  "path": "/api/2.0/pipelines/[UUID]/updates",
        	            	-  "body": {
        	            	-    "test_only": true,
        	            	-    "test_options": {}
        	            	-  }
        	            	-}
        	            	+File [OUTPUT_DIR]/out.requests.txt not found
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/pipelines/test/not-enabled
        	Messages:   	pipelines/test/not-enabled/output.txt vs /tmp/TestAcceptpipelinestestnot-enabled670331469/002/output.txt
    acceptance_test.go:1355: Missing output file: out.not-enabled.xml
TestAccept/pipelines/test/run-control/timeout: branch ✅ | main (73a2b68) ❌ | latest (v1.26.0) ❌

main (73a2b68):

acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1125
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "\n=== --timeout stops waiting but not the update, and prints the command that reads the results\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml\nPipeline: [UUID][0]\nUpdate: [UUID][1]\nRunning tests...\nJUnit report: out.report.xml\nError: stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]\n"
        	            	actual  : "\n=== --timeout stops waiting but not the update, and prints the command that reads the results\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml\nError: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,7 +4,3 @@
        	            	 >>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml
        	            	-Pipeline: [UUID][0]
        	            	-Update: [UUID][1]
        	            	-Running tests...
        	            	-JUnit report: out.report.xml
        	            	-Error: stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]
        	            	+Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        	            	 
        	Test:       	TestAccept/pipelines/test/run-control/timeout
        	Messages:   	pipelines/test/run-control/timeout/output.txt vs /tmp/TestAcceptpipelinestestrun-controltimeout379938245/002/output.txt
    acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1137
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"[UUID][0]\" tests=\"2\" failures=\"0\" errors=\"2\" skipped=\"0\">\n    <testcase classname=\"tests.test_orders\" name=\"test_slow\">\n      <error message=\"no result received\"></error>\n    </testcase>\n    <testcase name=\"pipelines test\">\n      <error message=\"stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]\">stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]&#xA;Update: [UUID][1]</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	actual  : "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<testsuites>\n  <testsuite name=\"[UUID]\" tests=\"1\" failures=\"0\" errors=\"1\" skipped=\"0\">\n    <testcase name=\"pipelines test\">\n      <error message=\"pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it\">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>\n    </testcase>\n  </testsuite>\n</testsuites>\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,8 +2,5 @@
        	            	 <testsuites>
        	            	-  <testsuite name="[UUID][0]" tests="2" failures="0" errors="2" skipped="0">
        	            	-    <testcase classname="tests.test_orders" name="test_slow">
        	            	-      <error message="no result received"></error>
        	            	-    </testcase>
        	            	+  <testsuite name="[UUID]" tests="1" failures="0" errors="1" skipped="0">
        	            	     <testcase name="pipelines test">
        	            	-      <error message="stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]">stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]&#xA;Update: [UUID][1]</error>
        	            	+      <error message="pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it">pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it</error>
        	            	     </testcase>
        	Test:       	TestAccept/pipelines/test/run-control/timeout
        	Messages:   	pipelines/test/run-control/timeout/out.report.xml vs /tmp/TestAcceptpipelinestestrun-controltimeout379938245/001/out.report.xml

latest (v1.26.0):

acceptance_test.go:1384: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1384
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1125
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:670
        	Error:      	Not equal: 
        	            	expected: "\n=== --timeout stops waiting but not the update, and prints the command that reads the results\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml\nPipeline: [UUID][0]\nUpdate: [UUID][1]\nRunning tests...\nJUnit report: out.report.xml\nError: stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]\n"
        	            	actual  : "\n=== --timeout stops waiting but not the update, and prints the command that reads the results\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml\nError: unknown flag: --resume\n\nUsage:\n  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]\n\nPipeline Test Flags:\n      --pipeline-id string       ID of the pipeline to test, instead of a KEY.\n      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.\n\nFlags:\n  -h, --help   help for test\n\nGlobal Flags:\n      --debug            enable debug logging\n  -o, --output type      output type: text or json (default text)\n  -p, --profile string   ~/.databrickscfg profile\n  -t, --target string    bundle target to use (if applicable)\n      --var strings      set values for variables defined in project config. Example: --var=\"foo=bar\"\n\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,7 +4,21 @@
        	            	 >>> musterr [CLI] pipelines test --pipeline-id [UUID][0] --resume [UUID][1] --timeout 3s --junit-xml out.report.xml
        	            	-Pipeline: [UUID][0]
        	            	-Update: [UUID][1]
        	            	-Running tests...
        	            	-JUnit report: out.report.xml
        	            	-Error: stopped waiting for update [UUID][1] after 3s; the update keeps running, to get its results run: databricks pipelines test --pipeline-id [UUID][0] --junit-xml out.report.xml --resume [UUID][1]
        	            	+Error: unknown flag: --resume
        	            	 
        	            	+Usage:
        	            	+  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        	            	+
        	            	+Pipeline Test Flags:
        	            	+      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        	            	+      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        	            	+
        	            	+Flags:
        	            	+  -h, --help   help for test
        	            	+
        	            	+Global Flags:
        	            	+      --debug            enable debug logging
        	            	+  -o, --output type      output type: text or json (default text)
        	            	+  -p, --profile string   ~/.databrickscfg profile
        	            	+  -t, --target string    bundle target to use (if applicable)
        	            	+      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
        	            	+
        	            	+
        	Test:       	TestAccept/pipelines/test/run-control/timeout
        	Messages:   	pipelines/test/run-control/timeout/output.txt vs /tmp/TestAcceptpipelinestestrun-controltimeout2923980557/002/output.txt
    acceptance_test.go:1355: Missing output file: out.report.xml

2 more failure details omitted, see the full report in the run.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 453fa56

Run: 43092531796

@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from ed5750c to 75f2713 Compare November 10, 2026 01:16
@tharunbhuk-db
tharunbhuk-db changed the base branch from tharun/pipelines-test-j1-base to tharun/pipelines-test-b3-dependency-checks November 10, 2026 01:16
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from 75f2713 to ccdd807 Compare November 10, 2026 10:07
@tharunbhuk-db
tharunbhuk-db changed the base branch from tharun/pipelines-test-b3-dependency-checks to tharun/pipelines-test-b4-testing-deploy November 10, 2026 10:07
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch 2 times, most recently from 2e9b82e to a149708 Compare November 10, 2026 11:11
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch 2 times, most recently from 490849a to 6c31046 Compare November 10, 2026 17:14
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from 6c31046 to bbb4adb Compare November 10, 2026 17:22
@tharunbhuk-db
tharunbhuk-db marked this pull request as ready for review November 10, 2026 18:11
@tharunbhuk-db
tharunbhuk-db requested review from a team as code owners November 10, 2026 18:11
@tharunbhuk-db
tharunbhuk-db removed this pull request from stack #7758 November 18, 2026 10:10
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from 8b33ae8 to 6983923 Compare November 18, 2026 10:10
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from 6983923 to 5986de8 Compare November 18, 2026 10:23
@tharunbhuk-db

Copy link
Copy Markdown
Contributor Author

@pietern thanks for the review. Reworked the stack, ready for another look: it is now 3 PRs (#7427 → #7437 → #7429).

@tharunbhuk-db
tharunbhuk-db added this pull request to the merge queue Nov 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to invalid changes in the merge commit Nov 18, 2026
Base automatically changed from tharun/pipelines-test-b3-testing-paths to main November 18, 2026 12:10
tharunbhuk-db and others added 7 commits November 18, 2026 04:10
The pipelines test acceptance bundles declare testing, as the service requires
testing.sources for a test-only update.

When Start Update fails with PIPELINE_TEST_NO_TEST_SOURCES and the bundle sets
testing for the pipeline, the error says to deploy first: the bundle changed
after its last deploy.

Add a live acceptance test that deploys a bundle with testing, including a local
test dependency after -e, and runs its tests. Locally it uses stubs for the
test run. Against a real workspace it runs only when
TEST_PIPELINES_TESTING_ENABLED is set.

Co-authored-by: Isaac <no-reply@databricks.com>
The command no longer needs DATABRICKS_CLI_PIPELINES_TEST=true. Whether a
workspace can run pipeline tests is decided by the service, so a run of all
tests now always sends test_options, empty when nothing is selected. Where the
testing API is off, the service rejects such a request with a clear error,
instead of running its older test-only update.

Co-authored-by: Isaac <no-reply@databricks.com>
The command no longer needs an environment variable to run.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-j1-testing-sources branch from 5986de8 to 7ca3bea Compare November 18, 2026 12:10
@tharunbhuk-db

Copy link
Copy Markdown
Contributor Author

/merge

@tharunbhuk-db
tharunbhuk-db added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit 76bf50d Nov 18, 2026
25 checks passed
@tharunbhuk-db
tharunbhuk-db deleted the tharun/pipelines-test-j1-testing-sources branch November 18, 2026 12:38
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 76bf50d

Run: 44305882577

Env 💥​PANIC ❌​FAIL 🤯​MISS 💚​RECOVERED ✅​pass 🙈​skip Time
💥​ aws linux-2core-8gb 1 3 796 9 818 1378 320:00
💥​ aws-windows-latest-4core-16gb 1 3 752 14 773 1406 320:03
💥​ azure linux-2core-8gb 1 3 632 14 957 1386 320:00
💥​ azure-windows-latest-4core-16gb 1 3 715 14 790 1414 320:03
💥​ gcp linux-2core-8gb 1 715 9 889 1384 320:00
💥​ gcp-windows-latest-4core-16gb 1 2 623 9 895 1412 320:03
1188 interesting tests: 1170 MISS, 9 RECOVERED, 6 PANIC, 3 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🤯​ TestAccept 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/basic 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/basic/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/recreation/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
Top 50 slowest tests (at least 2 minutes):
duration env testname
19:55 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
16:39 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
16:06 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
16:03 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
15:06 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
13:51 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
13:39 aws-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
13:15 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
12:47 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
12:33 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:27 azure linux-2core-8gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
12:26 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:26 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
11:09 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:08 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
11:05 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:42 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
10:41 gcp linux-2core-8gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=true
10:41 azure linux-2core-8gb TestAccept/bundle/resources/materialization_sets/batch/DMS=
10:39 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
10:34 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
10:33 azure-windows-latest-4core-16gb TestAccept/bundle/quiet-levels/DMS=true
10:30 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
10:27 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
10:09 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
10:05 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
10:03 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:53 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:51 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:48 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
9:33 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
9:27 azure linux-2core-8gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=
9:24 aws-windows-latest-4core-16gb TestAccept/bundle/resources/registered_models/basic/DMS=
9:11 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:09 aws linux-2core-8gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=no/DMS=/NBOOK=yes/PY=no/READPLAN=1
9:09 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:08 aws-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
9:08 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
9:05 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:04 aws-windows-latest-4core-16gb TestAccept/bundle/resources/registered_models/basic/DMS=true
9:03 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:02 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=true
8:59 azure-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
8:52 gcp linux-2core-8gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=
8:51 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
8:47 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=no/DMS=/NBOOK=no/PY=no/READPLAN=1
8:43 gcp linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=registered_model.yml.tmpl/READPLAN=
8:42 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=true
8:40 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=no/DMS=/NBOOK=yes/PY=yes/READPLAN=
8:31 gcp linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=volume.yml.tmpl/READPLAN=
(1039 table rows omitted to keep the report under 60000 bytes)

deco-sdk-tagging Bot added a commit that referenced this pull request Nov 19, 2026
## Release v1.27.0

### Notable Changes

 * `databricks aitools install`, `update` and `uninstall` now exit with code 1 when they do nothing they were asked to. `install` fails when it sets up nothing: no coding agent is detected, every agent is skipped or fails to install, or no skill in the release can be installed (also with `--path`). `update` fails when every update or check it attempts fails, and `uninstall` when every removal it attempts fails. Scripts that ran these commands as a best-effort step and relied on exit code 0 should handle the new exit code. ([#7481](#7481))
 * Add `file:<path>` selectors to `pipelines run` and pipeline `bundle run`. `--refresh` and `--full-refresh` now use space-separated lists, such as `--refresh "orders payments"`, instead of comma-separated lists. Single names containing spaces now require backticks, e.g. ``--refresh 'table:`order details`'``. ([#7184](#7184))
 * The Databricks CLI is now available on PyPI as `databricks`: run it with `uvx databricks`, or install it with `uv tool install databricks`, `pipx install databricks` or `pip install databricks`. The package contains only the prebuilt CLI binary. Uninstall the legacy `databricks-cli` package first, since both install a `databricks` command. ([#7704](#7704), [#7777](#7777))

### CLI

 * Add `databricks sandbox fs` commands (`cp`, `ls`, `cat`, `mkdir`, `rm`) to transfer files between your local filesystem and sandboxes using `<sandbox-id>:/path`. ([#7372](#7372))
 * `databricks fs` commands retry reads (stat, list, open) and deletes on DBFS and Volumes when the server returns a transient error or does not respond within the HTTP timeout. ([#7556](#7556))
 * Retry UC Volumes file operations (`fs cp`, `fs ls`, `fs rm`, bundle uploads to Volumes) when the Files API returns HTTP 408, 429, 502, 503 or 504 (for example "AWS S3 is throttling requests"), or on a transient network error; this regressed in v1.10.0. ([#7559](#7559))
 * Stop warning when the host metadata request (`/.well-known/databricks-config`) times out; the CLI already falls back to your config, and the message is still shown with `--debug`. ([#7752](#7752))
 * Add `databricks aitools auto-install disable` and `enable` to turn off (or back on) the automatic installation of Databricks AI tools on CLI upgrades. A global-scope `databricks aitools uninstall` also turns it off. ([#7701](#7701))
 * Stop printing the Databricks AI tooling install hint on `auth`, `configure`, `current-user me`, `version`, `help`, and shell-completion commands. ([#7452](#7452))
 * When Codex, Cursor, GitHub Copilot CLI, OpenCode, Gemini CLI, Antigravity, Pi, Goose or Kiro run the CLI without Databricks skills installed for that agent, the CLI now prints a one-line hint to install them with `databricks aitools install --agents <agent> --scope global`, previously shown only to Claude Code. ([#7586](#7586))
 * `databricks aitools install` now rejects an unknown `--agents` name before it shows the scope prompt, instead of after. ([#7469](#7469))
 * `databricks aitools install` no longer asks "Proceed?" after the agent picker; the picker selection confirms the plan. The picker title now shows the install scope, its hint says Enter installs, `--skills-only` runs label plugin agents "skills" and pre-check every agent whose config dir exists, and the plan summary after it is reduced to the scope line. ([#7478](#7478))
 * `databricks aitools install --output json` reports more specific `error_category` values: `SKILLS_FETCH_FAILED`, `RELEASE_LOOKUP_FAILED`, `RATE_LIMITED`, `MANIFEST_INVALID`, `STATE_INVALID` and `PLUGIN_CLI_TIMEOUT`. A coding agent CLI that does not answer in time is now reported as `timed out after <duration>`, followed by any stderr it wrote before it was stopped. ([#7475](#7475))
 * `databricks aitools install` no longer prompts when a coding agent runs it, even in a pseudo-terminal; it uses the non-interactive defaults instead and prints a note saying so. ([#7477](#7477))
 * `databricks aitools install` now prints the directory it installed skills into (for example `Installed 1 skill in ~/.databricks/aitools/skills.`), so agents can open the installed `SKILL.md` files without searching for them. ([#7668](#7668))
 * `databricks aitools install` now ends a failed install with a step to recover, in the text output and in a new `recovery` field of `--output json`. ([#7480](#7480))
 * `databricks aitools install` now installs plugins for the other coding agents even when installing skill files fails. ([#7479](#7479))
 * `databricks aitools install` now installs skill files for a detected coding agent whose CLI is not on PATH (for example a Claude Code desktop or IDE install), unless it already has the plugin, says how to switch to the plugin later, and warns when the current project also has the plugin. `databricks aitools update` no longer tries to install the plugin for such an agent, and no longer writes skill files next to a plugin that such an agent already has. ([#7482](#7482))
 * Fix `databricks aitools install` waiting a long time on an unreachable or slow workspace while it resolves authentication for telemetry; that wait is now limited to a few seconds. ([#7468](#7468))
 * `databricks aitools install`, `update` and `list` now refuse a skills manifest that names a skill or file outside the skills directory, before writing any skill files. A skill whose name or file paths are not portable is skipped with a warning, and `update` keeps its installed copy. ([#7722](#7722))
 * `databricks aitools install` and `databricks aitools update` refresh only the marketplace the Databricks plugin is listed in, so another unreachable marketplace no longer blocks them. When that refresh fails, both commands warn, install or update the plugin from the local copy of the marketplace, and record the plugin version as unknown. ([#7473](#7473))
 * `databricks aitools install` and `databricks aitools update` now wait up to 20 seconds instead of 5 for a coding agent's CLI to answer the plugin capability check, so a slow first start is less likely to report the agent as unavailable. ([#7472](#7472))
 * `databricks aitools install` no longer fails a plugin-only install when the lookup of the latest skills release on GitHub fails or is rate-limited. The plugin is installed and its version is recorded as unknown. ([#7474](#7474))
 * `databricks aitools` commands now send usage telemetry, and the CLI records when it recommends installing Databricks AI tools. When the configured workspace is slow or unreachable, `aitools install`, `update` and `uninstall` can take up to about 6 seconds longer, and `list`, `version` and runs that stop early up to about 3 seconds. Set `DATABRICKS_CLI_DISABLE_TELEMETRY=1` to opt out. ([#7467](#7467), [#7469](#7469), [#7470](#7470))
 * `databricks aitools install` and `databricks aitools update` now report a coding agent CLI shim (`.cmd` or `.bat`) on Windows that prints its plugin help and then exits with an error as `shim-exited-with-error`, with the command to run to see the error, instead of reporting the CLI as not on PATH. ([#7476](#7476))
 * `apps init` no longer suggests installing skills when every detected coding agent already has the Databricks plugin. ([#7467](#7467))
 * `databricks auth login` now gives up on the optional workspace lookup after about 5 seconds instead of retrying for about 5 minutes, and explains the fallback to the manual workspace ID prompt. ([#7522](#7522))
 * Throttle Claude Code's native plugin install hint separately from the text hint: it prints at most once per 10 minutes per session, so the install dialog still comes up when a subagent, a hook, or a command outside a Claude Code subprocess runs the session's first `databricks` command. ([#7692](#7692))
 * Print Claude Code's native plugin install hint alongside the text hint, so Claude Code offers a one-click install of the Databricks plugin. Claude Code now counts as having the tooling only when it loads it: skills only in `~/.databricks/aitools/skills` don't count, and a project- or local-scope plugin counts only when the CLI runs inside that project. `auth token`, `auth docker token` and CLI subprocesses no longer print install hints. ([#7585](#7585))
 * Interactive prompts and pagers now work in Git for Windows' bash and other MSYS2 shells when they run under ConPTY (the default since Git for Windows 2.47) or directly in a Windows console. ([#7699](#7699))
 * Add the `databricks feedback` command to send feedback about the CLI to Databricks. ([#7724](#7724))
 * Fix shell completion in `databricks fs` commands for local paths typed with a leading `./` or repeated separators, which previously offered no suggestions. ([#7679](#7679))
 * Fix a panic in `databricks labs` commands when a non-string SDK setting such as `DATABRICKS_DEBUG_HEADERS` or `DATABRICKS_RATE_LIMIT` is set in the environment. ([#7673](#7673))
 * Fix a panic in `databricks labs install` for projects whose `labs.yml` has no `install` section. ([#7673](#7673))
 * `NO_COLOR` no longer disables prompts, spinners or the list pager; they run without color. ([#7700](#7700))
 * Add the Beta `databricks pipelines test` command, which runs the tests of a pipeline, and the `testing` property of pipelines in bundles. ([#7429](#7429))
 * Add `databricks sandbox forward`, which serves an app running in a sandbox at a local URL. ([#7395](#7395))
 * Bound the whole telemetry upload at exit, including config resolution and authentication, by its timeout, so a stalled auth or telemetry endpoint no longer keeps a command from exiting. ([#7467](#7467))
 * `databricks version --check` now suggests `databricks aitools install` on stderr when a coding agent is detected without the Databricks plugin or skills. ([#7471](#7471))

### AI Runtime

 * `air get -o json` now reports `PENDING` while the run waits for compute, matching `air list`, and includes the run's `status_message`. ([#7455](#7455))
 * Wrap long failure reasons and log lines in the `air list` info and logs panes, keep the MLflow link (labeled with its retry) when a run's last retry failed before starting, and point `air logs` at earlier retries when the last one has no logs. ([#7456](#7456))
 * `databricks air logs --node` now reports an out-of-range node index when it can determine the run's node count, instead of looking like a successful run with no logs. ([#7516](#7516))
 * Add experiment, duration, accelerator, and dashboard metadata to `air list --output json`. For retried runs, `started_at` now reports the latest attempt, matching `air get`. ([#7568](#7568))
 * `air list` shortens MLflow run names and unnamed-run IDs by keeping their end behind a leading `…`, and its columns no longer overflow on wide (for example CJK) text; `air get` shows an unnamed run's full MLflow run ID. ([#7582](#7582))
 * Shrink the Experiment and User columns of the interactive `air list` table to fit the terminal width, keeping the end of a shortened experiment name. ([#7583](#7583))
 * Show the experiment, accelerators and configuration in the `air list` info pane for runs where it showed `N/A`. ([#7587](#7587))
 * Strip terminal control characters from the MLflow run, experiment and user names shown by `air list` and `air get`, so a run name cannot clear or retitle the terminal of whoever lists it. ([#7601](#7601))
 * `air logs`, `air get`, and `air run --watch` no longer make an extra credential-check request before fetching the run, and `air get` reuses the configured workspace ID instead of looking it up. ([#7613](#7613))
 * `air list` no longer waits on extra current-user requests before listing runs, which makes the default list about 150ms faster. ([#7615](#7615))
 * `air list --all-status` requests larger index pages, so it makes far fewer round trips once the server supports `page_size`. ([#7616](#7616))
 * Fit the interactive `air list` table to narrow terminals: when the Experiment and User columns are at their floors, shorten Started to `MM-DD HH:MM` and size the MLflow column to its labels, then hide Accelerators, User, MLflow and Progress in that order, naming the hidden columns in the hint line. ([#7629](#7629))
 * `air get -o json` now includes `mlflow_run_id` and `mlflow_experiment_id`, so a run can be handed to MLflow-based steps without parsing `mlflow_url`. They identify the MLflow run `mlflow_url` links to, including one from an earlier retry that `mlflow_retry` names. ([#7636](#7636))
 * `air get` and the `air list` info pane now show the environment of runs on the Databricks AI environment (`databricks_ai_v*`) instead of `N/A`. ([#7637](#7637))
 * `air run` now rejects a `code_source.include_paths` path that selects no files instead of uploading a snapshot without it; glob patterns and pathspec magic are left to git. ([#7643](#7643))
 * `air convert-to-dabs` now suggests the `--output-dir` that converts a working-tree workload whose `code_source.root_path` is outside the bundle directory, instead of suggesting `sync.paths`. ([#7644](#7644))
 * `air run` now lists a `code_source.root_path` outside a git repository relative to `root_path`, so running from a subdirectory of `root_path` no longer uploads an empty or partial snapshot. ([#7646](#7646))
 * `air list --all-status` now fills `--limit` past purged or inaccessible indexed runs. ([#7743](#7743))
 * `air run` now warns about missing PyTorch when a pinned snapshot's `include_paths` selects a Python file. ([#7744](#7744))
 * `air cancel JOB_RUN_ID` now rejects run IDs that are not AI Runtime runs instead of cancelling unrelated Jobs runs. In `-o json` output, the `failed[].error` text for a run that does not exist or that you cannot view now starts with `failed to get run <id>:` instead of the cancel API error. ([#7569](#7569))
 * `air get -o json` now reports `mlflow_retry` when the MLflow link comes from an earlier retry because the run's last retry never reached MLflow. ([#7620](#7620))
 * `databricks air run --help` now shows how to install the databricks-ai-runtime coding agent skill. ([#7652](#7652))
 * `air run --watch -o json` now reports each lifecycle `STATUS` transition once and never reports a non-terminal state after the run has ended, including when logs are served from MLflow artifacts. ([#7621](#7621))
 * `air convert-to-dabs` now explains the `--output-dir` workaround when `code_source.root_path` resolves to the bundle root, and notes when the generated job carries the default `max_retries`. ([#7579](#7579))
 * `air run` now refuses to upload a `code_source` snapshot that selects no files (for example when `root_path` is git-ignored) and explains the cause, and `--dry-run` runs the snapshot file selection and reports the file count. ([#7570](#7570))
 * Repeat the Jobs run link after live log streaming ends. ([#7571](#7571))
 * `air get -o json` includes run details including flat `accelerator_type` and `num_accelerators`. Use `--include-config` to include stored config, which may contain plaintext `env_variables` and parameters; downloads over 1 MiB are omitted, and config-dependent `max_retries` is otherwise null. ([#7624](#7624))
 * `air get` and the `air list` info pane no longer show `Max Retries 0` when the Jobs API leaves the task's retry limit out. For a run submitted with `air run`, the limit now comes from the run's uploaded config, shown as `3 (CLI default)` when the config doesn't set `max_retries`; it shows `N/A` when the limit can't be known (job runs, multi-container runs, legacy runs with no explicit value, or an unreadable config). ([#7623](#7623))
 * Reorganize `databricks air` help: subcommands are grouped by task, `air run` has examples, and YAML fields are listed under headings, with pool fields marked optional and `command`, `compute.num_accelerators`, and `compute.accelerator_type` correctly marked required. ([#7512](#7512))
 * `air run -h config` now lists the environment variables AI Runtime sets on every node (such as `MLFLOW_RUN_ID` and `CODE_SOURCE_PATH`), and `air logs --help` notes that `--download-to` cannot be combined with `--tail` or `--minutes`. ([#7655](#7655))
 * Support full workspace accelerator types in `air list` filters while preserving family matching, and reject invalid filter values. ([#7515](#7515))
 * `databricks air list --all-status` now warns when the AI Runtime run index is unavailable and it falls back to scanning Jobs runs. That fallback scan is capped at 250 runs, and a stuck index request fails after 15 seconds instead of retrying for minutes. `--all-status --filter user=<your email>` now uses the run index instead of scanning the workspace's full run history. ([#7619](#7619))
 * Return structured JSON errors for invalid `air list --filter` values when `-o json` is used. ([#7612](#7612))
 * `air list --all-status` no longer fails when a listed run has been purged. ([#7523](#7523))
 * **Behavior change:** `databricks air logs` now prints the logs available now and exits, also for an active run, matching `kubectl logs`, `docker logs` and `databricks apps logs`. Pass `--follow` (`-f`) to stream new lines until the run completes; `--tail N --follow` prints the last N lines first. `databricks air run --watch` still submits and streams. ([#7658](#7658))
 * `databricks air logs -o json` and `databricks air run --watch -o json` now emit a `WAITING` event, for example `{"type":"WAITING","message":"Waiting for accelerator compute capacity..."}`, when the reported waiting status changes. ([#7695](#7695))
 * `databricks air logs` on a run that ended in the last two minutes now waits for the log index to settle before printing, instead of sometimes dropping lines that were indexed late. ([#7663](#7663))
 * `air logs` and `air run --watch` now print the waiting status (for example `Waiting for accelerator compute capacity...`) as a line on stderr when stderr is not a terminal, instead of printing nothing until the run produces logs. ([#7657](#7657))
 * Route AIR log links to the MLflow Logs page instead of the hidden artifact path. ([#7491](#7491))
 * Invalid `compute.accelerator_type` errors and `air run -h config.compute` no longer list the CLI's built-in accelerator types, which could differ from the workspace's. ([#7520](#7520))
 * `databricks air --help` now notes that GPU pools are not required to run workloads. ([#7581](#7581))
 * `air run` now prints the job run URL with the `?o=<workspace-id>` suffix, matching `air get` and `air list`; `dashboard_url` in `air run -o json` and the `--watch` events gains it too. When `air list` cannot resolve the workspace ID, its links now omit `?o=` instead of ending in `?o=0`. ([#7573](#7573))
 * `databricks air run` now reports an unknown workload YAML key by its field path, suggests the closest valid field, and points to `-h config` instead of naming an internal Go type. ([#7574](#7574))
 * `air run` (including `--dry-run`) now warns before submitting when the `code_source` imports `torch` but PyTorch is not declared: the environment is the Standard environment (version 5 or later) with no `torch` dependency, so the run may fail after it waits for a GPU. ([#7656](#7656))

### Bundles

 * Add `isolation_mode` to the `catalogs` resource, so a catalog can be deployed as `OPEN` or `ISOLATED`. ([#7463](#7463))
 * `bundle validate` no longer warns when resource permissions and top-level or target permissions name the same principal. ([#7291](#7291))
 * Retry bundle file uploads, reads and deletes when the server does not respond within the HTTP timeout ("request timed out after ... of inactivity"). ([#7521](#7521))
 * Show how to recover when `bundle deploy` fails because an app's service principal does not exist or was deleted. ([#7533](#7533))
 * Retry transient server errors when creating Lakebase (`postgres_*`) resources in the direct deployment engine. ([#7557](#7557))
 * Fix `bundle plan` reporting the `spark_conf` and `custom_tags` the backend derives from `is_single_node: true` as drift on clusters. ([#7577](#7577))
 * Stop planning a spurious update for Lakeflow Connect pipelines whose `ingestion_definition.data_staging_options` is filled in by the backend. ([#7603](#7603))
 * Retry the root path check in `bundle destroy` on transient errors, so a dropped connection no longer fails the destroy. ([#7597](#7597))
 * Use the Lakebase branch when its create request times out but the branch was created, instead of failing the deploy, in the direct deployment engine. ([#7602](#7602))
 * Retry transient server errors when deleting resources in the direct deployment engine. ([#7708](#7708))
 * Use the Lakebase project when its create request times out but the project was created, instead of failing the deploy with "already exists", in the direct deployment engine. ([#7747](#7747))
 * Use the feature, model serving endpoint or dashboard that a create request created despite timing out, instead of failing the deploy, in the direct deployment engine. ([#7757](#7757))
 * Use the Lakebase role when its create request fails with a transient error but the role was created, instead of failing the deploy, in the direct deployment engine. ([#7767](#7767))
 * Use the MCP service, model service or model provider service that a create request created despite failing with a timeout, a 499 or a reset stream, instead of failing the deploy, in the direct deployment engine. Create recovery for features, serving endpoints and dashboards now also covers 499 and reset streams. ([#7782](#7782))
 * Retry app updates and deletes in the direct deployment engine when they fail with 400 BAD_REQUEST because the app compute is `UPDATING`, after waiting for the compute to settle. ([#7606](#7606))
 * Enable drift detection for `source_code_path` and `git_source` on apps. ([#6399](#6399))
 * `bundle destroy` and `bundle deploy` no longer fail with "internal error, missing in state" when the state holds an empty-ID entry left by an interrupted recreate in an older CLI. ([#7676](#7676))
 * Fix a feature whose `full_name` contains uppercase letters being recreated on every deploy. ([#7398](#7398))
 * Fix `bundle deployment bind` (and `bundle generate --bind`) failing with `no such node "resources.internal_immutable_snapshots.immutable"` when `workspace.sync_mode` is `immutable`, as reported in [#7306](#7306). ([#7329](#7329))
 * Retry creating the deployment lock (without `--force-lock`) when the request fails with a transient server error, times out ("request timed out after ... of inactivity") or loses its connection ("unexpected EOF"). ([#7594](#7594))
 * Fix a materialization set member whose `feature_name` contains uppercase letters failing to deploy, or recreating the set on every deploy. ([#7398](#7398))
 * Delete the old resource when its name follows a renamed schema. ([#7718](#7718))
 * Fix `pipelines dry-run` starting a real run when given a job KEY, and a crash in its shell completion. ([#7672](#7672))
 * Rename schemas in place; schemas and volumes follow a renamed catalog or schema. ([#7343](#7343))
 * Fix a panic when validating or deploying jobs with local libraries and serverless job clusters. ([#7725](#7725))
 * Fix `tgz` artifacts with `git` set producing an empty tarball or failing when the bundle is in a subdirectory of the git repository. ([#7674](#7674))
 * Apply `presets.trigger_pause_status` and the development mode pause default to `triggers[]` entries on jobs. ([#7450](#7450))

### Dependency Updates

 * Bump dependencies with known vulnerabilities. ([#7750](#7750))
 * Update the Go SDK to [v0.198.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.198.0). ([#7530](#7530))
 * Update the Go SDK to [v0.199.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.199.0). ([#7584](#7584))
 * Update the Go SDK to [v0.200.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.200.0). ([#7653](#7653))
 * Update the Go SDK to [v0.201.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.201.0). ([#7769](#7769))
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…oyed folder (databricks#7427)

## 🥞 Stacked PR
-
[**stack/pipelines-test-b1-editable-folder**](databricks#7427)
[[Files changed](https://github.com/databricks/cli/pull/7427/files)] ←
_this PR_
-
[stack/pipelines-test-b3-testing-paths](databricks#7437)
[[Files changed](https://github.com/databricks/cli/pull/7437/files)]
-
[stack/pipelines-test-j1-testing-sources](databricks#7429)
[[Files changed](https://github.com/databricks/cli/pull/7429/files)]

---------
- `databricks pipelines test` runs without an environment variable. Its
help marks it Beta, so it stays unstable under the stability policy.
- A run of all tests sends `test_options` as an empty object, so a
workspace that does not support it returns a clear error.
- The `pipelines test` acceptance bundles declare `testing`, since test
runs need `testing.sources`.
- When Start Update fails with `PIPELINE_TEST_NO_TEST_SOURCES` and the
bundle sets `testing`, the error says to deploy first.
- Adds the changelog entry for the command and the `testing` property.

Tests: `TestNewStartTestUpdateRequest`, `TestWithDeployHint`; acceptance
`pipelines/test/not-enabled`, `pipelines/test/no-test-sources`,
`pipelines/databricks-cli-help`, and the `pipelines/test` goldens
without the gate cases.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants