Repository navigation
Show status and endpoint for API errors without a message; retry pipeline events reads - #7800
Merged
Merged
Conversation
…line events reads
An error response whose JSON body has no "message" field leaves the SDK's
APIError.Message empty, so the CLI printed a bare "Error: ". Print the HTTP
status, error_code and endpoint instead.
`pipelines run` / `bundle run` aborted when the events API returned a transient
500 ("An unexpected error happened.") while polling an update. Retry that read
with libs/retry, as other read paths do.
aws: TestAccept/pipelines/run/refresh-flags TestAccept/bundle/deployment/bind/alert
Co-authored-by: Isaac <no-reply@databricks.com>
aws: TestAccept/pipelines/run/refresh-flags TestAccept/bundle/deployment/bind/alert Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 3 tests: regression=2 unreleased=0 coverage=1 failing=0
TestAccept/cmd/empty-error-message: branch ✅ | main (70d24e3) ❌ | latest (v1.27.0) ❌main (70d24e3): latest (v1.27.0): same output as main (70d24e3) TestAccept/pipelines/run/events-500: branch ✅ | main (70d24e3) ❌ | latest (v1.27.0) ❌main (70d24e3): latest (v1.27.0): same output as main (70d24e3) |
Don't branch on the text of wrapping errors: only a bare API error with an empty message is described. Also retry the update status read in the poll loop and the events read after the update completes. aws: TestAccept/pipelines/run/refresh-flags TestAccept/bundle/deployment/bind/alert Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
✅ Integration test reportCommit: 265587f
Top 13 slowest tests (at least 2 minutes):
|
aws: TestAccept/pipelines/run/refresh-flags TestAccept/bundle/deployment/bind/alert Co-authored-by: Isaac <no-reply@databricks.com>
Generated commands return the API error wrapped without extra text, so a bare type check missed it. aws: TestAccept/pipelines/run/refresh-flags TestAccept/bundle/deployment/bind/alert Co-authored-by: Isaac <no-reply@databricks.com>
denik
marked this pull request as ready for review
November 19, 2026 12:26
denik
requested review from
Sankalp-Mittal,
andrewnester and
maksim-zuev-db
and removed request for
maksim-zuev-db
November 19, 2026 12:26
andrewnester
approved these changes
Nov 19, 2026
Sankalp-Mittal
left a comment
Contributor
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
…reads - Share the status and endpoint formatting between libs/retry and cmd/root via libs/diag. - Retry the error-events read and the post-run GetUpdate too. - Assert the retry warning in pipelines/run/events-500. - Mention update status reads in the changelog entry. aws: TestAccept/pipelines/run/refresh-flags Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
❌ Integration test reportCommit: ac6b85d
1354 interesting tests: 1339 MISS, 6 KNOWN, 5 PANIC, 4 FAIL
Top 50 slowest tests (at least 2 minutes):
|
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
… is sent (databricks#7820) ## Changes In `waitForTestUpdate` (`databricks pipelines test`), a read that fails with `context.Canceled` now also carries the wait context's cause (`errTestWaitTimeout`). The read's own error stays wrapped. Other read errors are returned unchanged, even if the context was canceled by then. The command's caller still checks `context.Cause` as a fallback. ## Why `TestWaitForTestUpdateTimeout` failed once on main on Windows: ``` Target error should be in err chain: expected: "test wait timed out" in chain: "failed to fetch pipeline events: failed in rate limiter: context canceled" ``` The 20ms timeout can pass between two reads. Windows timers are coarse, so this happens there. A request that is already in flight gets the cancel cause from `net/http`. But the SDK rate limiter checks the context before sending, and it fails with a bare `context canceled`, which hides the timeout. The command itself reported this case correctly, because it checked `context.Cause` separately. The function didn't, so the test was flaky. This is not a regression from [databricks#7800](databricks#7800): that PR changed `pipelines run` and `bundle run` (`cmd/pipelines/run.go`, `bundle/run/`). It did not change `pipelines test`'s wait loop (`cmd/pipelines/test.go`, `test_events.go`). The flaky test came with [databricks#7426](databricks#7426). ## Tests New `TestWaitReadError` covers: a real error is kept, and a context error gets the cause and keeps its message. New `TestWaitForTestUpdateTimeoutBeforeRead`: the wait context is already canceled with the timeout cause, so the first read fails in the rate limiter. Without the fix it fails with the same message as on CI; with the fix it passes. The other `TestWaitForTestUpdate*` tests and the `pipelines` acceptance tests pass. Decision: this does not change one existing behavior. If the timeout fires during the final events read after the update has already ended, the run is still reported as timed out. That is unchanged from main and rare, since it needs the timeout to hit that exact read. ## Dashboard | error | verbatim snippet | |---|---| | [error/3790](http://go/cli/dashboard/error/3790?main=1&pr=1&end=2026-11-19T16:00:00Z&last=2d) | `Target error should be in err chain: expected "test wait timed out", in chain: "failed to fetch pipeline events: failed in rate limiter: context canceled"` | Test: [cmd/pipelines.TestWaitForTestUpdateTimeout](http://go/cli/dashboard/test/cmd/pipelines.TestWaitForTestUpdateTimeout?main=1&pr=1&end=2026-11-19T16:00:00Z&last=2d) This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
cmd/root: when an API error response has nomessagefield, the SDK leavesAPIError.Messageempty and the CLI printed a bareError:. Now it prints the HTTP status,error_codeand endpoint, e.g.Error: API request failed with 500 Internal Server Error (INTERNAL_ERROR) from POST /api/2.0/lakeview/dashboards. This applies only when the whole error would print as nothing. Errors whose wrappers add their own text are left as they are (e.g.genie askalready explains a 500 with no details).pipelines run/bundle run: the reads made while waiting for an update (the progress events, the update status, and the events read after completion) are now retried on transient errors (libs/retry.IsTransientRead) instead of aborting the run.Why
Two main-branch integration test failures with opaque messages:
lakeview create(dashboard bind tests, before the first traced command) andalerts-v2 create-alertfailed with an empty error message, so the logs don't show which request failed or why. This PR doesn't fix the backend failure itself. It makes the next occurrence show the status, code and endpoint.pipelines runfailed withError: An unexpected error happened.right after the update reached INITIALIZING. That text is the message of a 500INTERNAL_ERRORfrom the pipeline events list API. It is a read, so retrying is safe. The update itself kept running.Decision: both fixes are in one PR because they come from the same investigation (generic or empty errors on main) and each is small.
Tests
cmd/empty-error-message: errors without a message (with and withouterror_code) and a normal error.pipelines/run/events-500: one injected 500 on the events API is retried and the run completes.describeEmptyAPIError.pipelines/run/refresh-flagsandbundle/deployment/bind/alerton AWS.Dashboard
>>> [CLI] alerts-v2 create-alert --json @alert.json/Error:/Exit code: 1update_progress INFO "Update 535c12 is INITIALIZING."/Error: An unexpected error happened.Tests: TestAccept/bundle/deployment/bind/alert/DMS=, TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN=, TestAccept/bundle/deployment/bind/dashboard/recreation/DMS=/READPLAN=1, TestAccept/pipelines/run/refresh-flags
This pull request and its description were written by Isaac.