Repository navigation
Add --resume and --timeout to databricks pipelines test - #7426
Merged
Merged
Conversation
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 10, 2026 00:25
23e0bb0 to
b45969f
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 10, 2026 00:25
cd64420 to
17f12d7
Compare
This was referenced Nov 10, 2026
Contributor
|
Commit: Regression test reportSelected 7 changed tests (limit=20, 0 not selected) 7 tests: regression=7 unreleased=0 coverage=0 failing=0
TestAccept/pipelines/test/arguments: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/results/incomplete: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/run-control/canceled: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/run-control/not-test-update: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/run-control/resume: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/run-control/timeout: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): TestAccept/pipelines/test/submit-error: branch ✅ | main (4e503ea) ❌ | latest (v1.26.0) ❌main (4e503ea): latest (v1.26.0): same output as main (4e503ea) |
Collaborator
✅ Integration test reportCommit: 17f12d7
Top 3 slowest tests (at least 2 minutes):
|
tharunbhuk-db
added a commit
that referenced
this pull request
Nov 10, 2026
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 10, 2026 04:14
b45969f to
e8a772a
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 10, 2026 04:14
17f12d7 to
2f8b6d9
Compare
This was referenced Nov 10, 2026
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 10, 2026 06:26
e8a772a to
e7de143
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 10, 2026 06:57
2f8b6d9 to
e225070
Compare
tharunbhuk-db
marked this pull request as ready for review
November 10, 2026 07:08
tharunbhuk-db
requested review from
pranshupand-db and
renaudhartert-db
and removed request for
a team and
renaudhartert-db
November 10, 2026 07:08
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 10, 2026 09:08
e225070 to
1059c51
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 17, 2026 00:17
cdc73e0 to
86122f3
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 17, 2026 00:17
37575dd to
3b224ed
Compare
tharunbhuk-db
added a commit
that referenced
this pull request
Nov 17, 2026
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 17, 2026 07:00
86122f3 to
ae6fcdc
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 17, 2026 07:01
3b224ed to
4a3c384
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-pr2-output
branch
from
November 17, 2026 07:56
ae6fcdc to
cfe688b
Compare
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 17, 2026 08:19
4a3c384 to
3554c4f
Compare
--timeout and Ctrl+C end only the local wait and print the command that reads the results later; the update keeps running. That command keeps the KEY or --pipeline-id, --target, --profile and every --var. --resume <update-id> reads the results of an existing test update instead of starting one, and rejects an update that the service reports is not a test update. The signal handler is installed before the update starts, but only the wait listens to it, so a Ctrl+C during Start Update still prints the update ID. The first signal releases the handler, so a second one ends the process. Co-authored-by: Isaac <no-reply@databricks.com>
The Go SDK now has UpdateInfo.test_only, so --resume checks the update with w.Pipelines.GetUpdate instead of sending the request directly. As before, it rejects the update only when the response has test_only set to false; an update without the field is accepted. Co-authored-by: Isaac <no-reply@databricks.com>
…ing a run When the update or its events cannot be read, the error ends with the --resume command, as on a timeout. An incomplete result prints the --resume command before the summary, because events can arrive after the update ends. When Start Update says the pipeline already has an active update, the error says to wait, to stop it (for a bundle pipeline), or to read it with --resume. The changelog entry for --resume and --timeout moves to this PR. Co-authored-by: Isaac <no-reply@databricks.com>
The timeout was checked only between polls, so a slow GetUpdate, event read or SDK retry could finish after it and report the update as completed. All reads now use a context that the timeout cancels. It is a cancel, not a deadline, because the SDK rate limiter fails early on a deadline with an error that does not name the timeout. The timeout error, resume hint and exit code are unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
The entry is added with the other pipeline testing entries in #7439. Co-authored-by: Isaac <no-reply@databricks.com>
tharunbhuk-db
force-pushed
the
tharun/pipelines-test-a4-run-control
branch
from
November 17, 2026 08:47
3554c4f to
69a059e
Compare
Collaborator
❌ Integration test reportCommit: 95d08a4
169 interesting tests: 111 FAIL, 45 MISS, 8 RECOVERED, 5 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
## 🥞 Stacked PR - [stack/pipelines-test-pr1-command](databricks#7297) [[Files changed](https://github.com/databricks/cli/pull/7297/files)] - [stack/pipelines-test-bundle-key](databricks#7431) [[Files changed](https://github.com/databricks/cli/pull/7431/files)] - [**stack/pipelines-test-read-events**](databricks#7432) [[Files changed](https://github.com/databricks/cli/pull/7432/files)] ← _this PR_ - [stack/pipelines-test-print-results](databricks#7433) [[Files changed](https://github.com/databricks/cli/pull/7433/files)] - [stack/pipelines-test-pr2-output](databricks#7298) [[Files changed](https://github.com/databricks/cli/pull/7298/files)] - [stack/pipelines-test-a4-run-control](databricks#7426) [[Files changed](https://github.com/databricks/cli/pull/7426/files)] - [stack/pipelines-test-pr3-junit](databricks#7299) [[Files changed](https://github.com/databricks/cli/pull/7299/files)] - [stack/pipelines-test-junit-flag](databricks#7434) [[Files changed](https://github.com/databricks/cli/pull/7434/files)] --------- - Read every page of an update's test events with the SDK's `ListPipelineEventsRequest` and `buildPipelineEventFilter`. Existing code is not changed. - Collect one result per test case and check that the results are complete. - Not used by the command yet. - Start test-only updates with `w.Pipelines.StartUpdate`: the Go SDK now has `test_only` and `test_options`. The body has the same fields and values; only the key order inside `test_options` follows the SDK type (one golden file). - The List Pipeline Events request stays a direct call: the SDK's `PipelineEvent` has no `details` or `origin.test_case_id`, which the test results come from. - The short help says *Experimental*, like the long help and the gate. A selection outside the synced files, and a relative selection with `workspace.sync_mode: immutable`, now say which /Workspace path to pass. Tests: unit tests, including 200 cases over 3 pages; acceptance `pipelines/test/*`. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
## 🥞 Stacked PR - [stack/pipelines-test-pr1-command](databricks#7297) [[Files changed](https://github.com/databricks/cli/pull/7297/files)] - [stack/pipelines-test-bundle-key](databricks#7431) [[Files changed](https://github.com/databricks/cli/pull/7431/files)] - [stack/pipelines-test-read-events](databricks#7432) [[Files changed](https://github.com/databricks/cli/pull/7432/files)] - [**stack/pipelines-test-print-results**](databricks#7433) [[Files changed](https://github.com/databricks/cli/pull/7433/files)] ← _this PR_ - [stack/pipelines-test-pr2-output](databricks#7298) [[Files changed](https://github.com/databricks/cli/pull/7298/files)] - [stack/pipelines-test-a4-run-control](databricks#7426) [[Files changed](https://github.com/databricks/cli/pull/7426/files)] - [stack/pipelines-test-pr3-junit](databricks#7299) [[Files changed](https://github.com/databricks/cli/pull/7299/files)] - [stack/pipelines-test-junit-flag](databricks#7434) [[Files changed](https://github.com/databricks/cli/pull/7434/files)] --------- - Print test results: `<i> of <n> <STATUS> <test ID> ....... [<STATUS> in <s>s]`, failure details, and a last line `Done. PASSED=… TOTAL=…`. - Not used by the command yet. Tests: unit tests. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
…ks#7298) ## 🥞 Stacked PR - [stack/pipelines-test-pr1-command](databricks#7297) [[Files changed](https://github.com/databricks/cli/pull/7297/files)] - [stack/pipelines-test-bundle-key](databricks#7431) [[Files changed](https://github.com/databricks/cli/pull/7431/files)] - [stack/pipelines-test-read-events](databricks#7432) [[Files changed](https://github.com/databricks/cli/pull/7432/files)] - [stack/pipelines-test-print-results](databricks#7433) [[Files changed](https://github.com/databricks/cli/pull/7433/files)] - [**stack/pipelines-test-pr2-output**](databricks#7298) [[Files changed](https://github.com/databricks/cli/pull/7298/files)] ← _this PR_ - [stack/pipelines-test-a4-run-control](databricks#7426) [[Files changed](https://github.com/databricks/cli/pull/7426/files)] - [stack/pipelines-test-pr3-junit](databricks#7299) [[Files changed](https://github.com/databricks/cli/pull/7299/files)] - [stack/pipelines-test-junit-flag](databricks#7434) [[Files changed](https://github.com/databricks/cli/pull/7434/files)] --------- - Wait for the test update, print the results, and exit 0 only when at least one test ran and none failed or errored. - `--no-wait` returns after the update starts. - The wait follows the bundle pipeline runner's loop. - "No tests were collected" names the selection, or the pipeline's test sources, and points to the `--pytest-arg` filters when they were given. - The changelog entry for the results output is in this PR. Tests: unit tests; acceptance in `acceptance/pipelines/test/results/*`, with the service stubbed per case. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
) ## 🥞 Stacked PR - [stack/pipelines-test-pr1-command](databricks#7297) [[Files changed](https://github.com/databricks/cli/pull/7297/files)] - [stack/pipelines-test-bundle-key](databricks#7431) [[Files changed](https://github.com/databricks/cli/pull/7431/files)] - [stack/pipelines-test-read-events](databricks#7432) [[Files changed](https://github.com/databricks/cli/pull/7432/files)] - [stack/pipelines-test-print-results](databricks#7433) [[Files changed](https://github.com/databricks/cli/pull/7433/files)] - [stack/pipelines-test-pr2-output](databricks#7298) [[Files changed](https://github.com/databricks/cli/pull/7298/files)] - [stack/pipelines-test-a4-run-control](databricks#7426) [[Files changed](https://github.com/databricks/cli/pull/7426/files)] - [**stack/pipelines-test-pr3-junit**](databricks#7299) [[Files changed](https://github.com/databricks/cli/pull/7299/files)] ← _this PR_ - [stack/pipelines-test-junit-flag](databricks#7434) [[Files changed](https://github.com/databricks/cli/pull/7434/files)] --------- - Build a JUnit XML report from a test run: one entry per case, plus a failing entry for an incomplete run. - The file is replaced atomically. - Not used by the command yet. - Every failing entry that the command adds is named `pipelines test`; the update ID is in the entry's text. Tests: unit tests. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
) ## 🥞 Stacked PR - [stack/pipelines-test-pr1-command](databricks#7297) [[Files changed](https://github.com/databricks/cli/pull/7297/files)] - [stack/pipelines-test-bundle-key](databricks#7431) [[Files changed](https://github.com/databricks/cli/pull/7431/files)] - [stack/pipelines-test-read-events](databricks#7432) [[Files changed](https://github.com/databricks/cli/pull/7432/files)] - [stack/pipelines-test-print-results](databricks#7433) [[Files changed](https://github.com/databricks/cli/pull/7433/files)] - [stack/pipelines-test-pr2-output](databricks#7298) [[Files changed](https://github.com/databricks/cli/pull/7298/files)] - [stack/pipelines-test-a4-run-control](databricks#7426) [[Files changed](https://github.com/databricks/cli/pull/7426/files)] - [stack/pipelines-test-pr3-junit](databricks#7299) [[Files changed](https://github.com/databricks/cli/pull/7299/files)] - [**stack/pipelines-test-junit-flag**](databricks#7434) [[Files changed](https://github.com/databricks/cli/pull/7434/files)] ← _this PR_ --------- - Add `--junit-xml <path>`, which writes the report on every exit after the flags parse. - The `JUnit report:` line prints before the summary line. - The help has examples: all tests, one file, one test by ID, CI with `--junit-xml`, `--timeout` then `--resume`, and `--pipeline-id`. - The changelog entry for `--junit-xml` is in this PR. Tests: acceptance reports (`out.*.xml`) in `acceptance/pipelines/test/*`. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
… is sent (databricks#7820) ## Changes In `waitForTestUpdate` (`databricks pipelines test`), a read that fails with `context.Canceled` now also carries the wait context's cause (`errTestWaitTimeout`). The read's own error stays wrapped. Other read errors are returned unchanged, even if the context was canceled by then. The command's caller still checks `context.Cause` as a fallback. ## Why `TestWaitForTestUpdateTimeout` failed once on main on Windows: ``` Target error should be in err chain: expected: "test wait timed out" in chain: "failed to fetch pipeline events: failed in rate limiter: context canceled" ``` The 20ms timeout can pass between two reads. Windows timers are coarse, so this happens there. A request that is already in flight gets the cancel cause from `net/http`. But the SDK rate limiter checks the context before sending, and it fails with a bare `context canceled`, which hides the timeout. The command itself reported this case correctly, because it checked `context.Cause` separately. The function didn't, so the test was flaky. This is not a regression from [databricks#7800](databricks#7800): that PR changed `pipelines run` and `bundle run` (`cmd/pipelines/run.go`, `bundle/run/`). It did not change `pipelines test`'s wait loop (`cmd/pipelines/test.go`, `test_events.go`). The flaky test came with [databricks#7426](databricks#7426). ## Tests New `TestWaitReadError` covers: a real error is kept, and a context error gets the cause and keeps its message. New `TestWaitForTestUpdateTimeoutBeforeRead`: the wait context is already canceled with the timeout cause, so the first read fails in the rate limiter. Without the fix it fails with the same message as on CI; with the fix it passes. The other `TestWaitForTestUpdate*` tests and the `pipelines` acceptance tests pass. Decision: this does not change one existing behavior. If the timeout fires during the final events read after the update has already ended, the run is still reported as timed out. That is unchanged from main and rare, since it needs the timeout to hit that exact read. ## Dashboard | error | verbatim snippet | |---|---| | [error/3790](http://go/cli/dashboard/error/3790?main=1&pr=1&end=2026-11-19T16:00:00Z&last=2d) | `Target error should be in err chain: expected "test wait timed out", in chain: "failed to fetch pipeline events: failed in rate limiter: context canceled"` | Test: [cmd/pipelines.TestWaitForTestUpdateTimeout](http://go/cli/dashboard/test/cmd/pipelines.TestWaitForTestUpdateTimeout?main=1&pr=1&end=2026-11-19T16:00:00Z&last=2d) This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
--timeoutand Ctrl+C stop the local wait and print the--resumecommand; the update keeps running.--resume <update-id>reads an existing test update.--resumereadstest_onlywithw.Pipelines.GetUpdate: the Go SDK now hasUpdateInfo.test_only. As before, only an explicittest_only: falseis rejected; an update without the field is accepted.--resumecommand, as a timeout does. An incomplete result prints the--resumecommand before the summary.--resume.--resumeand--timeoutis in this PR.Tests: unit tests; acceptance in
acceptance/pipelines/test/run-control/*.This pull request and its description were written by Isaac.