Repository navigation
acc: ignore postgres project compute_last_active_time in update_display_name plans - #7745
Merged
Merged
Conversation
…ay_name plans The backend sets status.compute_last_active_time only once the project's compute has run, so it appears in remote_state nondeterministically. cloud: TestAccept/bundle/resources/postgres_projects/update_display_name Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 2 tests: regression=0 unreleased=0 coverage=2 failing=0
|
Collaborator
✅ Integration test reportCommit: bed4614
Top 3 slowest tests (at least 2 minutes):
|
denik
marked this pull request as ready for review
November 18, 2026 05:43
denik
requested review from
Sankalp-Mittal,
andrewnester,
anton-107 and
rugpanov
and removed request for
anton-107
November 18, 2026 05:43
denik
enabled auto-merge
November 18, 2026 05:43
Sankalp-Mittal
left a comment
Contributor
There was a problem hiding this comment.
[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
remote_state is the full GetProject payload; backend-computed status fields come and go, so recording it is unstable. The create plan has no remote_state, so it is left unfiltered. cloud: TestAccept/bundle/resources/postgres_projects/update_display_name Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
left a comment
Contributor
There was a problem hiding this comment.
[AI Generated] Follow-up review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.
No new issues found.
andrewnester
approved these changes
Nov 18, 2026
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
…n goldens (databricks#7764) ## Changes New acceptance helper `acceptance/bin/plan_filter.py`: reads `bundle plan -o json` on stdin, drops `remote_state` from every plan entry and removes the deployment stamp (reusing `nostamp.render`). With a resource key it prints just that entry, and fails if the key isn't in the plan. So ``` $CLI bundle plan -o json | nostamp | jq '(.plan."resources.X.y" // .) | del(.remote_state)' ``` becomes ``` $CLI bundle plan -o json | plan_filter.py resources.X.y ``` It's covered by a selftest (`acceptance/selftest/plan_filter`) and documented in `.agents/rules/testing.md`, along with a rule to use it unless a test checks `remote_state` itself. Applied to the cloud postgres plan tests: `postgres_branches/update_protected`, `postgres_databases/update`, `postgres_roles/update`, `postgres_projects/add_default_endpoint_settings`, plus `postgres_projects/update_display_name` and `postgres_endpoints/update_autoscaling`, which already dropped `remote_state` with jq. The golden updates only remove `remote_state` blocks (3 insertions, all from trailing-comma changes, 193 deletions). ## Why `remote_state` is the backend's GET response copied into the plan as is: status, timestamps, sizes, UIDs, and any field the backend starts returning. If a golden records it, the test fails whenever the backend changes, even when the CLI's decision (`action`, `new_state`, `changes`) is the same, and the failure shows up on every cloud at once. `changes[*].remote` still records the remote value of each field the plan compares. Measured on main before this PR: - 115 goldens record `remote_state`, 42 of them in `Cloud = true` tests. - Over 30 days of main runs, these "Plan differs" errors had every diff line inside `remote_state`, so this filter would have prevented them: [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) (13 failures), [error/2893](http://go/cli/dashboard/error/2893?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) (10), [error/2964](http://go/cli/dashboard/error/2964?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3587](http://go/cli/dashboard/error/3587?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3526](http://go/cli/dashboard/error/3526?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d). databricks#7745 and databricks#7495 fixed some of these test by test. - Not prevented: the postgres GET change from last week ([error/3534](http://go/cli/dashboard/error/3534?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3535](http://go/cli/dashboard/error/3535?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3537](http://go/cli/dashboard/error/3537?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) and siblings, fixed in databricks#7596). Once the backend echoed spec fields back, `changes` changed too (`skip`/`spec:input_only` entries disappeared and `remote` values appeared), so those tests would have failed with or without `remote_state`. That's a real change in the plan, and the goldens should catch it. | error | snippet (inside `remote_state`) | |---|---| | [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) | `+ "compute_last_active_time": "[TIMESTAMP]",` | | [error/2893](http://go/cli/dashboard/error/2893?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) | `Unexpected output "status": "MONITOR_STATUS_PENDING",` | ## Decisions - Tests opt in with a helper; the runner doesn't strip `remote_state` by default. Some tests check `remote_state` on purpose (dashboards' `serialized_dashboard`, grants' `__embed__` principals, `apps/config-env-no-drift`, `quality_monitors`' allow-list), and a filter hidden in the runner would make those goldens misleading. - `changes[*].remote` is kept. It's the input to the CLI's drift decision, and out-of-band tests depend on it. - When the resource key is missing, `plan_filter.py` fails. The old `jq '.plan."KEY" // .'` silently wrote the whole plan instead. None of the converted goldens relied on that fallback, and a missing entry should fail loudly. - For `skip` entries (the `no_change` goldens), only `action` is left, because a skip has no `changes`. That's intended: the plan's decision is the assertion. If a broken read changed what the CLI sees, the action would turn into update or recreate, and that still shows up. - It's a separate script rather than a flag on `nostamp.py`. `nostamp` is also used for states and request bodies, where `remote_state` doesn't apply. `plan_filter.py` reuses `nostamp.render`, so stamp handling stays in one place. - Scope is the postgres family, which has churned the most. The other cloud goldens that record `remote_state` (dashboards, grants, quality_monitors) check it on purpose or already filter it. Local-only tests can adopt the helper when they're next touched. ## Tests - `selftest/plan_filter` (local): whole plan, one entry, missing entry, non-plan and empty input. - All 12 variants (DMS= and DMS=true) of the six postgres tests pass on AWS (`testme-aws`). The last commit message requests the same AWS run in CI. ## Dashboard [error/3534](http://go/cli/dashboard/error/3534?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3535](http://go/cli/dashboard/error/3535?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3537](http://go/cli/dashboard/error/3537?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d): these motivated the change. They were already fixed on main by databricks#7596, and this PR wouldn't have prevented them (see above). It does prevent the class above: [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3587](http://go/cli/dashboard/error/3587?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/2964](http://go/cli/dashboard/error/2964?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3526](http://go/cli/dashboard/error/3526?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d). ``` Plan differs (out.plan.update.json): Unexpected output "is_protected": false, "no_expiry": true, Plan differs (out.plan.restore.json): Unexpected output "compute_last_active_time": "[TIMESTAMP]", ``` This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
postgres_projects/update_display_namerecorded the full plan, includingremote_state, which is the full GetProject payload. The backend addsstatus.compute_last_active_timeto the project once its compute has run, so the field shows up in some runs and not others. Other backend-computed fields could do the same.This drops
remote_statefrom the no_change, update and restore plans (del(.remote_state)), aspostgres_endpoints/update_autoscalingdoes. The create plan has noremote_state, so it is left unfiltered.changes.display_name.remotestill records the remote value, andget-project | project_fieldsstill checks the remote fields.I chose a test-only fix: the field is output-only (
REMOTEin refschema) and the direct engine never diffs it, since it never appears inchanges. The flakiness comes from recordingremote_state, not from real drift.Tests
Passed locally and on AWS (
testme-aws, DMS= and DMS=true). The commit message asks CI to run the test on every cloud.Dashboard
Plan differs (out.plan.restore.json): Unexpected output "compute_last_active_time": "[TIMESTAMP]",Plan differs (out.plan.update.json): Unexpected output "compute_last_active_time": "[TIMESTAMP]",Tests: TestAccept/bundle/resources/postgres_projects/update_display_name/DMS=, TestAccept/bundle/resources/postgres_projects/update_display_name/DMS=true
This pull request and its description were written by Isaac.