Repository navigation
Fail instead of panicking when the terraform state disappears before migration - #7778
Merged
Merged
Conversation
Contributor
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/bundle/migrate/user-agent/race/DMS=: branch ✅ | main (bc2ce9e) ❌ | latest (v1.27.0) ❌main (bc2ce9e): latest (v1.27.0): |
Collaborator
✅ Integration test reportCommit: bd0f4e3
Top 3 slowest tests (at least 2 minutes):
|
andrewnester
approved these changes
Nov 18, 2026
denik
force-pushed
the
denik/engine-ua-terraform-tests
branch
from
November 18, 2026 17:52
db32498 to
2495c84
Compare
denik
force-pushed
the
denik/migrate-missing-tfstate
branch
2 times, most recently
from
November 18, 2026 18:21
1c1af2f to
dbd6a73
Compare
Sankalp-Mittal
left a comment
Contributor
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
denik
force-pushed
the
denik/migrate-missing-tfstate
branch
2 times, most recently
from
November 19, 2026 10:12
c360095 to
7fa63ee
Compare
Sankalp-Mittal
left a comment
Contributor
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.
No issues found.
…migration When the pulled state was terraform but the local terraform state file was gone by the time the auto-migration ran (e.g. a concurrent deploy migrated it), Migrate returned "nothing to migrate" and the deploy continued on the terraform engine with no direct state open, then panicked in CalculatePlan. Migrate only runs for a terraform state, so treat the missing file as an error, and drop its bool return, which is now always true on success. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
direct_migrate_error is documented as "state conversion itself errored"; a state removed after the pull is a race, not a conversion failure, so report it as direct_migrate_state_missing. The race test now requires the failure with musterr and asserts the metric. Co-authored-by: Isaac <no-reply@databricks.com>
The fix covers every command that migrates the state, not only deploy, and a retry only succeeds when a concurrent deploy did the migration. Co-authored-by: Isaac <no-reply@databricks.com>
denik
force-pushed
the
denik/migrate-missing-tfstate
branch
from
November 19, 2026 10:56
7fa63ee to
bd0f4e3
Compare
Collaborator
❌ Integration test reportCommit: ab7c938
892 interesting tests: 863 MISS, 12 RECOVERED, 8 KNOWN, 6 PANIC, 3 FAIL
Top 50 slowest tests (at least 2 minutes):
|
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
…abricks#7690) ## Changes Commands that read and need the state (deploy, plan, destroy, run, summary) now tag the user agent with `engine/direct` before the bundle is loaded, instead of after the state is pulled. Pull-only commands (bind, unbind, `debug states`, `deployment migrate`) keep tagging the pulled state's engine. ## Why Requests made while loading the bundle, initializing, pulling the state and resolving `lookup` variables (`scim/v2/Me`, `workspace/export`, `workspace/get-status`, `jobs/list`, `pipelines`, ...) went out without an engine tag, as did the telemetry upload of runs that failed early. On CLI >= 1.20 that's 2.3% of `bundle_deploy` requests and 0.78% of deploy runs over the last 30 days, and dashboards that classify by engine count them as terraform. These commands always run on the direct engine since v1.20.0: a terraform state is migrated first, and a failed migration (including a state removed after the pull, databricks#7778) is fatal. ## Tests In `acceptance/bundle/user_agent`, the opening requests flip from MISS to OK; the only untagged ones left are `/.well-known/databricks-config`, which is sent before there is a command context, and `bundle validate`, which doesn't read state. `state_present` and `migrate/auto/run-no-commit` only change the tag order. `migrate/auto/plan-failure` asserts that every request of a deploy whose migration fails is tagged `engine/direct`; its deploy telemetry still reports `state_engine: terraform`, the state it started from. Stacked on databricks#7778, which is stacked on databricks#7775. In databricks#7775's tests, the `race` deploy is now tagged `engine/direct`, and deploys rejected for a terraform engine pinned via `DATABRICKS_BUNDLE_ENGINE` or `bundle.engine` now tag their telemetry upload `engine/direct`. `bind-unbind` is unchanged: those commands still tag `engine/terraform`. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
statemgmt.Migratenow returns an error when the local terraform state is missing, instead of reporting that there is nothing to migrate. Itsboolreturn was only false on that path, so it is dropped. The failure is recorded as a new deploy telemetry key,direct_migrate_state_missing, separate fromdirect_migrate_error(a conversion failure).Why
Migration only runs when the pulled state is terraform, and the pull always leaves that state on disk, so a missing file means it changed after the pull, e.g. a concurrent deploy migrated it. Before, the deploy then continued on the terraform engine with no direct state open: it took the deploy lock, uploaded files, sent
engine/terraformin the user agent, and panicked inCalculatePlanwithDeploymentState must be opened first. This is what one production deploy on v1.24.0 looked like.Tests
The
migrate/user-agent/racegolden from #7775 (this PR is stacked on it) changes from the panic to the error, and the deploy now stops after the state pull.This pull request and its description were written by Isaac.