Repository navigation
Conversation
apply retries DoUpdate on transient errors. When the app start waiter's
GET returned 504, the retry sent POST /apps/{name}/start again, and the
backend rejected it with 400 because the first start had already brought
the compute to ACTIVE.
On a 400 from Start, read the app once and continue if its compute is
ACTIVE. libs/testserver now rejects a start of an ACTIVE app as the
backend does.
Co-authored-by: Isaac <no-reply@databricks.com>
cloud: TestAccept/bundle/resources/apps/lifecycle-started-toggle Co-authored-by: Isaac <no-reply@databricks.com>
✅ Integration test reportCommit: 99b39f6
Top 24 slowest tests (at least 2 minutes):
|
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
andrewnester
left a comment
There was a problem hiding this comment.
No blocking findings from a full pass; I'll take a manual look at the direct changes before approving.
A retried start can arrive while the first start is still bringing the compute up. Wait for it to become ACTIVE, as bundle run does, instead of failing with the 400. Add a local acceptance test that loses the first start's response with a 504 after the handler ran, so the retried start reaches the testserver's ACTIVE rejection. Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/bundle/resources/apps/lifecycle-start-retried/DMS=: branch ✅ | main (61f1fc5) ❌ | latest (v1.26.0) ❌main (61f1fc5): latest (v1.26.0): TestAccept/bundle/resources/apps/lifecycle-start-retried/DMS=true: branch ✅ | main (61f1fc5) ❌ | latest (v1.26.0) ❌main (61f1fc5): latest (v1.26.0): |
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Follow-up review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
Each DMS variant waited the default 15s before the retried DoUpdate. Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Follow-up review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
No new issues found.
Resolve the conflict in bundle/direct/dresources/app.go: keep startApp and take main's updated hasAppChanges comment from #6399. Co-authored-by: Isaac <no-reply@databricks.com>
Resolve conflicts with #7606 in bundle/direct/dresources: keep both the libs/retry import and log import in app.go, and keep both sets of tests in app_test.go. #7606 refreshes the compute state before manageLifecycle; startApp still handles a rejected start, so the two changes compose. Co-authored-by: Isaac <no-reply@databricks.com>
|
AI agent (opus5.5, isaac_claude): merged main twice to clear conflicts (no rebase or force-push). The second merge (d769cae) overlapped #7606 in |
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
If the GET after a rejected start failed, startApp returned only the start's 400, so apply did not retry a transient read failure (such as a 504). Return the read error, with the 400 as context. Co-authored-by: Isaac <no-reply@databricks.com>
andrewnester
left a comment
There was a problem hiding this comment.
No blocking findings from a full pass; I'll take a manual look at the direct changes before approving.
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Follow-up review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
No new issues found.
Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
| startCalls := 0 | ||
| server.Handle("POST", "/api/2.0/apps/{name}/start", func(req testserver.Request) any { | ||
| startCalls++ | ||
| return testserver.Response{ |
There was a problem hiding this comment.
[AI Generated] nit: nit: this builds the 400 response and the client inline, while the next test uses appBadRequest and newAppForServer from this file. Can this one use them too?
| // is in ACTIVE state"). That happens when apply retries DoUpdate after a transient error in the | ||
| // start waiter: the first start already went through, so the compute is ACTIVE, or still | ||
| // STARTING if the retry came before it finished. | ||
| func (r *ResourceApp) startApp(ctx context.Context, id string) (*apps.App, error) { |
There was a problem hiding this comment.
We have a similar logic in bundle/run/app.go function start, can we consolidate in single helper function maybe?
Changes
In the direct deployment engine, starting an app whose compute is already
ACTIVEnow counts as success. On a 400 fromPOST /api/2.0/apps/{name}/start, the app is read once. If its compute isACTIVE, the deploy continues to the app deployment. If it isSTARTING, the CLI waits forACTIVEwithApps.WaitGetAppActive, asbundle rundoes. Otherwise the original error is returned.libs/testservernow rejects a start of anACTIVEapp with the same 400 the backend returns.Why
DeploymentUnit.Updateinbundle/direct/apply.goretries the wholeDoUpdateon transient errors (from #7129). For an app toggled fromlifecycle.started: falsetotrue,DoUpdatecalls Start and then waits for the compute with GETs. On GCP, one of those GETs returned 504 Gateway Timeout. The retry ranDoUpdateagain with the plan-time state (compute stopped) and sent Start a second time. The backend rejected it because the first start had already gone through:Decisions:
ACTIVEcompute, or aSTARTINGone that then becomesACTIVE, turns the 400 into success. A retry comes 15s after the failure, so the first start may still be in progress. Any other state, such asDELETINGorSTOPPING, fails at once, as it did before. This PR is independent of direct: retry app update and delete while app compute is UPDATING #7606, which retries update and delete while the compute isUPDATING,STARTINGorSTOPPING.Tests
TestAppStartApp_RejectedStart: a rejected start succeeds forACTIVE, waits and succeeds forSTARTING, and fails forSTOPPED,DELETINGand a missing compute status.bundle/resources/apps/lifecycle-start-retried:fault.py --after-handleranswers the first start with a 504 after the handler ran, so the retried start is rejected because the compute is alreadyACTIVE. The deploy succeeds. Without the fix, the test fails with the error from GCP.bundle/resources/apps/lifecycle-started-togglepasses on AWS withtestme-aws. The 504 itself cannot be reproduced on demand.Dashboard:
Error: cannot update resources.apps.mykey: updating id=[UNIQUE_NAME]: Cannot start app [UNIQUE_NAME] as its compute is in ACTIVE state. App compute needs to be STOPPED to start. (400 BAD_REQUEST)Test: TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= (2 flaky runs on GCP main, 2026-11-13)
This pull request and its description were written by Isaac.