Repository navigation
Tag bundle requests with engine/direct before loading the bundle - #7690
Conversation
|
Commit: Regression test reportSelected 7 changed tests (limit=20, 0 not selected) 7 tests: regression=6 unreleased=0 coverage=1 failing=0
TestAccept/bundle/migrate/auto/plan-failure/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): same output as main (9d39813) TestAccept/bundle/migrate/auto/run-no-commit/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): TestAccept/bundle/migrate/user-agent/engine-pinned/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): TestAccept/bundle/migrate/user-agent/race/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): TestAccept/bundle/state/state_present/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): same output as main (9d39813) TestAccept/bundle/user_agent/simple/DMS=: branch ✅ | main (9d39813) ❌ | latest (v1.27.0) ❌main (9d39813): latest (v1.27.0): |
✅ Integration test reportCommit: a566bc8
Top 6 slowest tests (at least 2 minutes):
|
3edfd44 to
64ce49b
Compare
db32498 to
2495c84
Compare
db8778a to
a257262
Compare
7fa63ee to
bd0f4e3
Compare
Commands that read and operate on the state (deploy, plan, destroy, run, summary) always run on the direct engine, but the engine user-agent tag was only added after the state was pulled. Requests made during initialization and the state pull (scim/v2/Me, workspace/export, lookup variable resolution) went out without it. Co-authored-by: Isaac <no-reply@databricks.com>
Commands that need the state are tagged before initialization, so the post-migration tag is only reachable for pull-only commands, which never migrate. Tag those once, right after the pull, and drop the comments that described the old two-stage tagging. Assert in migrate/auto/plan-failure that every request of a deploy whose migration fails is tagged engine/direct. Co-authored-by: Isaac <no-reply@databricks.com>
- race: the deploy whose migration finds no local terraform state is now tagged engine/direct instead of engine/terraform; it still panics. - engine-pinned: a deploy rejected for bundle.engine: terraform now tags its telemetry upload engine/direct, and summary tags its first request. Co-authored-by: Isaac <no-reply@databricks.com>
The tag was set after several early returns (loading the bundle, --var, the engine setting check), so runs that failed there uploaded their deploy telemetry untagged. shouldReadState and needsState depend only on opts, so compute them and tag first. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Reuse the shared helper instead of an inline print_requests.py | jq copy; print_engines also clears the recorded requests, so the rm after it goes. Co-authored-by: Isaac <no-reply@databricks.com>
a257262 to
a566bc8
Compare
Sankalp-Mittal
left a comment
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
No blocking or major issues found.
| # request it made, including those before the failure and the telemetry upload, is tagged | ||
| # engine/direct. An untagged request would show up as MISSING. print_engines also clears | ||
| # the recorded requests for the next step. | ||
| trace print_engines |
There was a problem hiding this comment.
[AI Generated] nit: The process.go comment says a failed migration's deploy telemetry still reports state_engine: terraform, but no golden prints it. Can this script print resources_metadata.state_engine before print_engines clears the request log?
❌ Integration test reportCommit: 058be51
1218 interesting tests: 1190 MISS, 10 RECOVERED, 7 KNOWN, 6 PANIC, 5 FAIL
Top 50 slowest tests (at least 2 minutes):
|
…te (databricks#7775) ## Changes Acceptance tests under `acceptance/bundle/migrate/user-agent/` that record the `engine/` user-agent tag of every request when a bundle still has a terraform state: - `race`: the local terraform state disappears between the state pull and the auto-migration, as when two deploys run at once. The deploy tags `engine/terraform` and then panics with `DeploymentState must be opened first`. - `bind-unbind`: `deployment bind` and `unbind` only pull the state, so they tag `engine/terraform` before rejecting it. - `engine-pinned`: a terraform engine pinned through `DATABRICKS_BUNDLE_ENGINE` or `bundle.engine`. The shared `acceptance/bundle/migrate/script.prepare` gains the `print_engines` helper and keeps `LOG.*` files out of the bundle upload, for all migrate tests. ## Why Telemetry classifies bundle requests by this tag, and `engine/terraform` still shows up from CLI >= 1.20. These tests pin down current behavior so changes to the tagging (databricks#7690) and a fix for the panic show up as golden diffs. ## Tests Test-only change; goldens record what main does today. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
Changes
Commands that read and need the state (deploy, plan, destroy, run, summary) now tag the user agent with
engine/directbefore the bundle is loaded, instead of after the state is pulled. Pull-only commands (bind, unbind,debug states,deployment migrate) keep tagging the pulled state's engine.Why
Requests made while loading the bundle, initializing, pulling the state and resolving
lookupvariables (scim/v2/Me,workspace/export,workspace/get-status,jobs/list,pipelines, ...) went out without an engine tag, as did the telemetry upload of runs that failed early. On CLI >= 1.20 that's 2.3% ofbundle_deployrequests and 0.78% of deploy runs over the last 30 days, and dashboards that classify by engine count them as terraform. These commands always run on the direct engine since v1.20.0: a terraform state is migrated first, and a failed migration (including a state removed after the pull, #7778) is fatal.Tests
In
acceptance/bundle/user_agent, the opening requests flip from MISS to OK; the only untagged ones left are/.well-known/databricks-config, which is sent before there is a command context, andbundle validate, which doesn't read state.state_presentandmigrate/auto/run-no-commitonly change the tag order.migrate/auto/plan-failureasserts that every request of a deploy whose migration fails is taggedengine/direct; its deploy telemetry still reportsstate_engine: terraform, the state it started from.Stacked on #7778, which is stacked on #7775. In #7775's tests, the
racedeploy is now taggedengine/direct, and deploys rejected for a terraform engine pinned viaDATABRICKS_BUNDLE_ENGINEorbundle.enginenow tag their telemetry uploadengine/direct.bind-unbindis unchanged: those commands still tagengine/terraform.This pull request and its description were written by Isaac.