Repository navigation
Show status and endpoint for API errors without a message; retry pipeline events reads #7800
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
a6aa644
Show status and endpoint for API errors without a message; retry pipe…
denik 89458de
Add changelog fragments
denik 27309ac
Describe only bare API errors; retry the other pipeline polling reads
denik 7a0f596
Keep the bare-error type assertion that lint --fix rewrote to errors.As
denik df9f9a5
Describe an API error only when the printed message would be empty
denik 265587f
Address review: share API error formatting, retry remaining pipeline …
denik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * `bundle run` and `pipelines run` now retry transient errors from the pipeline events and update status reads while waiting for an update. ([#7800](https://github.com/databricks/cli/pull/7800)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * Show the HTTP status, error code and endpoint when an API error response has no message, instead of a bare `Error: `. ([#7800](https://github.com/databricks/cli/pull/7800)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
|
|
||
| >>> errcode [CLI] lakeview create --display-name test | ||
| Error: API request failed with 500 Internal Server Error (INTERNAL_ERROR) from POST /api/2.0/lakeview/dashboards | ||
|
|
||
| Exit code: 1 | ||
|
|
||
| >>> errcode [CLI] lakeview get 1234 | ||
| Error: API request failed with 404 Not Found from GET /api/2.0/lakeview/dashboards/1234 | ||
|
|
||
| Exit code: 1 | ||
|
|
||
| >>> errcode [CLI] secrets list-scopes | ||
| Error: no access | ||
|
|
||
| Exit code: 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| trace errcode $CLI lakeview create --display-name test | ||
| trace errcode $CLI lakeview get 1234 | ||
| # A non-empty message is printed as is. | ||
| trace errcode $CLI secrets list-scopes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| Cloud = false | ||
|
|
||
| # Error responses without a "message" field: the CLI must still say which request failed. | ||
|
|
||
| [[Server]] | ||
| Pattern = "POST /api/2.0/lakeview/dashboards" | ||
| Response.Body = '{"error_code": "INTERNAL_ERROR"}' | ||
| Response.StatusCode = 500 | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /api/2.0/lakeview/dashboards/{dashboard_id}" | ||
| Response.Body = '{}' | ||
| Response.StatusCode = 404 | ||
|
|
||
| [[Server]] | ||
| Pattern = "GET /api/2.0/secrets/scopes/list" | ||
| Response.Body = '{"error_code": "PERMISSION_DENIED", "message": "no access"}' | ||
| Response.StatusCode = 400 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| bundle: | ||
| name: test-pipeline-run-flags | ||
|
|
||
| resources: | ||
| pipelines: | ||
| my_pipeline: | ||
| name: test-pipeline-flags | ||
| libraries: | ||
| - file: | ||
| path: pipeline_file.py |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
|
|
||
| >>> [CLI] pipelines deploy | ||
| Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-pipeline-run-flags/default/files... | ||
| Created pipelines.my_pipeline | ||
| Files: 2 uploaded, 0 deleted | ||
| Resources: 1 created, 0 changed, 0 deleted, 0 unchanged | ||
| View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID] | ||
|
|
||
| === A 500 from the events API while polling the update is retried | ||
|
|
||
| >>> [CLI] pipelines run | ||
| Recommendation: This command runs the last deployed version of the code | ||
|
|
||
| If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included. | ||
|
|
||
| Update URL: [DATABRICKS_URL]/#joblist/pipelines/[UUID][0]/updates/[UUID][1] | ||
|
|
||
| Warn: retrying after 500 Internal Server Error from GET /api/2.0/pipelines/[UUID][0]/events | ||
| Update ID: [UUID][1] | ||
|
|
||
| Update for pipeline completed successfully. | ||
|
|
||
| Pipeline configurations for this update: | ||
| • All tables are refreshed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| import dlt | ||
| from pyspark.sql import SparkSession | ||
|
|
||
|
|
||
| @dlt.table | ||
| def my_table(): | ||
| return spark.range(10) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| trace $CLI pipelines deploy | ||
|
|
||
| title "A 500 from the events API while polling the update is retried" | ||
| fault.py "GET /api/2.0/pipelines/*/events" 500 0 1 INTERNAL_ERROR | ||
| trace $CLI pipelines run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| Env.DATABRICKS_BUNDLE_RETRY_INTERVAL_MS = "100" | ||
| # Assert the retry warning, which acceptance/test.toml suppresses by default. | ||
| SuppressLines.warn_retry = "" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| package root | ||
|
|
||
| import ( | ||
| "errors" | ||
| "strings" | ||
|
|
||
| "github.com/databricks/cli/libs/diag" | ||
| "github.com/databricks/databricks-sdk-go/apierr" | ||
| ) | ||
|
|
||
| // describedError replaces the text of an error while keeping it unwrappable. | ||
| type describedError struct { | ||
| msg string | ||
| err error | ||
| } | ||
|
|
||
| func (e *describedError) Error() string { return e.msg } | ||
| func (e *describedError) Unwrap() error { return e.err } | ||
|
|
||
| // describeEmptyAPIError fills in an API error whose response had no "message" field. | ||
| // The SDK then leaves the message empty and the CLI would print a bare "Error: ", | ||
| // which says nothing about which request failed or how. | ||
| // | ||
| // Only an error that would print as nothing at all is described; wrappers that add | ||
| // their own text already say something about the failure. | ||
| func describeEmptyAPIError(err error) error { | ||
| if strings.TrimSpace(err.Error()) != "" { | ||
| return err | ||
| } | ||
| apiErr, ok := errors.AsType[*apierr.APIError](err) | ||
| if !ok { | ||
| return err | ||
| } | ||
|
|
||
| desc := "API request failed with " + diag.FormatAPIErrorStatus(apiErr) | ||
| // The SDK formats a missing error_code as "<nil>". | ||
| if code := apiErr.ErrorCode; code != "" && code != "<nil>" { | ||
| desc += " (" + code + ")" | ||
| } | ||
| if endpoint := diag.FormatAPIErrorEndpoint(apiErr); endpoint != "" { | ||
| desc += " from " + endpoint | ||
| } | ||
| return &describedError{msg: desc, err: err} | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| package root | ||
|
|
||
| import ( | ||
| "errors" | ||
| "fmt" | ||
| "net/http" | ||
| "net/url" | ||
| "testing" | ||
|
|
||
| "github.com/databricks/databricks-sdk-go/apierr" | ||
| "github.com/databricks/databricks-sdk-go/common" | ||
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| func newAPIError(message, code string) *apierr.APIError { | ||
| req := &http.Request{Method: http.MethodPost, URL: &url.URL{Path: "/api/2.0/x", RawQuery: "q=1"}} | ||
| return &apierr.APIError{ | ||
| Message: message, | ||
| ErrorCode: code, | ||
| StatusCode: 500, | ||
| ResponseWrapper: &common.ResponseWrapper{Response: &http.Response{Request: req}}, | ||
| } | ||
| } | ||
|
|
||
| func TestDescribeEmptyAPIError(t *testing.T) { | ||
| for _, tc := range []struct { | ||
| name string | ||
| err error | ||
| want string | ||
| }{ | ||
| {"not an API error", errors.New("boom"), "boom"}, | ||
| {"has a message", newAPIError("no access", "PERMISSION_DENIED"), "no access"}, | ||
| {"empty message", newAPIError("", "INTERNAL_ERROR"), "API request failed with 500 Internal Server Error (INTERNAL_ERROR) from POST /api/2.0/x"}, | ||
| {"no error code", newAPIError("", "<nil>"), "API request failed with 500 Internal Server Error from POST /api/2.0/x"}, | ||
| {"whitespace message", newAPIError("\n", ""), "API request failed with 500 Internal Server Error from POST /api/2.0/x"}, | ||
| {"wrapped with own text", fmt.Errorf("creating thing: %w", newAPIError("", "")), "creating thing: "}, | ||
| {"wrapped without own text", fmt.Errorf("%w", newAPIError("", "INTERNAL_ERROR")), "API request failed with 500 Internal Server Error (INTERNAL_ERROR) from POST /api/2.0/x"}, | ||
| } { | ||
| t.Run(tc.name, func(t *testing.T) { | ||
| got := describeEmptyAPIError(tc.err) | ||
| assert.Equal(t, tc.want, got.Error()) | ||
| assert.ErrorIs(t, got, tc.err) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func TestDescribeEmptyAPIErrorWithoutRequest(t *testing.T) { | ||
| err := &apierr.APIError{StatusCode: 499} | ||
| assert.Equal(t, "API request failed with 499", describeEmptyAPIError(err).Error()) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.