Repository navigation
acc: match postgres GET now echoing spec - #7596
Merged
Merged
Conversation
Since 2026-11-12 the Lakebase postgres API returns spec on GET for projects, branches, endpoints, databases and roles, not only status. Plans now record remote values for spec fields, and fields that match no longer appear as "skip spec:input_only" changes. The testserver now echoes spec the way the cloud does: as sent on create, merged by update_mask on update (a masked field missing from the body keeps its value, and one oneof member replaces the others), without fields that stay input-only (project pg_version, branch source_branch*, role postgres_role/auth_method/identity_type). Goldens regenerated on AWS and checked locally. cloud: TestAccept/bundle/resources/postgres_projects TestAccept/bundle/resources/postgres_branches TestAccept/bundle/resources/postgres_endpoints TestAccept/bundle/resources/postgres_databases TestAccept/bundle/resources/postgres_roles Co-authored-by: Isaac <no-reply@databricks.com>
Review follow-ups: clear a stored oneof sibling only when the update mask covers the group; name the input-only spec field lists once; document the spec map helpers; say in the dresources comments that the input_only marks live in the generated configs and still suppress remote changes. cloud: TestAccept/bundle/resources/postgres_projects TestAccept/bundle/resources/postgres_branches TestAccept/bundle/resources/postgres_endpoints TestAccept/bundle/resources/postgres_databases TestAccept/bundle/resources/postgres_roles Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 15 changed tests (limit=20, 0 not selected) 90 tests: regression=0 unreleased=0 coverage=90 failing=0
|
Collaborator
✅ Integration test reportCommit: e86837f
Top 19 slowest tests (at least 2 minutes):
|
pietern
approved these changes
Nov 13, 2026
pietern
added a commit
that referenced
this pull request
Nov 13, 2026
- Classify materialized_features[*].notifications as immutable; the new spec no longer marks it output-only. - Regenerate postgres and bundle/invariant goldens against #7596's testserver, dropping the drift the bumper recorded. Co-authored-by: Isaac <no-reply@databricks.com>
pietern
added a commit
that referenced
this pull request
Nov 13, 2026
…oldens The bumper recorded 'Exit code: N' for these two tests instead of their real output; main's goldens are the correct ones. Postgres goldens regenerated with -update for the spec echo fix from #7596. Co-authored-by: Isaac <no-reply@databricks.com>
This was referenced Nov 13, 2026
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 16, 2026
…atabricks#7641) ## Changes Record postgres requests with `print_requests.py --unique` in `bundle/resources/postgres_endpoints/update_autoscaling`, so a retried request (an identical consecutive request) is recorded once. ## Why On GCP the backend returned a transient 500 for `GET .../branches/main` during the no-change deploy. The CLI retried the read on purpose (`libs/retry`, `IsTransientRead` in `bundle/direct`), and the retry showed up as an extra request in `out.requests.no_change.json`. The retry is correct behavior, so the recording should tolerate it. Same approach as databricks#7511 and databricks#7547 for apps. Not a regression from databricks#7557 (that change retries create-operation polling) or from databricks#7596 (testserver spec echo). The failing request is a plain remote-state read that has been retried for a while. The current goldens have no consecutive duplicate requests, so `--unique` changes only the traced command line in output.txt. ## Dashboard - [TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DMS=true](http://go/cli/dashboard/test/acceptance.TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DMS=true?main=1&pr=1&end=2026-11-15T03:00:00Z&last=2d) - [error/3462](http://go/cli/dashboard/error/3462?main=1&pr=1&end=2026-11-15T03:00:00Z&last=2d) ``` Suppressed from output.txt: Warn: retrying after 500 Internal Server Error from GET /api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]/branches/main ... +{ + "method": "GET", + "path": "/api/2.0/postgres/projects/test-pg-proj-[UNIQUE_NAME]/branches/main" +} Messages: bundle/resources/postgres_endpoints/update_autoscaling/out.requests.no_change.json ``` ## Tests Passed locally and on AWS (both DMS variants). The last commit asks CI to run the test on GCP and AWS. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc
pushed a commit
to idelafuente-oc/cli
that referenced
this pull request
Nov 20, 2026
…n goldens (databricks#7764) ## Changes New acceptance helper `acceptance/bin/plan_filter.py`: reads `bundle plan -o json` on stdin, drops `remote_state` from every plan entry and removes the deployment stamp (reusing `nostamp.render`). With a resource key it prints just that entry, and fails if the key isn't in the plan. So ``` $CLI bundle plan -o json | nostamp | jq '(.plan."resources.X.y" // .) | del(.remote_state)' ``` becomes ``` $CLI bundle plan -o json | plan_filter.py resources.X.y ``` It's covered by a selftest (`acceptance/selftest/plan_filter`) and documented in `.agents/rules/testing.md`, along with a rule to use it unless a test checks `remote_state` itself. Applied to the cloud postgres plan tests: `postgres_branches/update_protected`, `postgres_databases/update`, `postgres_roles/update`, `postgres_projects/add_default_endpoint_settings`, plus `postgres_projects/update_display_name` and `postgres_endpoints/update_autoscaling`, which already dropped `remote_state` with jq. The golden updates only remove `remote_state` blocks (3 insertions, all from trailing-comma changes, 193 deletions). ## Why `remote_state` is the backend's GET response copied into the plan as is: status, timestamps, sizes, UIDs, and any field the backend starts returning. If a golden records it, the test fails whenever the backend changes, even when the CLI's decision (`action`, `new_state`, `changes`) is the same, and the failure shows up on every cloud at once. `changes[*].remote` still records the remote value of each field the plan compares. Measured on main before this PR: - 115 goldens record `remote_state`, 42 of them in `Cloud = true` tests. - Over 30 days of main runs, these "Plan differs" errors had every diff line inside `remote_state`, so this filter would have prevented them: [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) (13 failures), [error/2893](http://go/cli/dashboard/error/2893?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) (10), [error/2964](http://go/cli/dashboard/error/2964?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3587](http://go/cli/dashboard/error/3587?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3526](http://go/cli/dashboard/error/3526?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d). databricks#7745 and databricks#7495 fixed some of these test by test. - Not prevented: the postgres GET change from last week ([error/3534](http://go/cli/dashboard/error/3534?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3535](http://go/cli/dashboard/error/3535?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3537](http://go/cli/dashboard/error/3537?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) and siblings, fixed in databricks#7596). Once the backend echoed spec fields back, `changes` changed too (`skip`/`spec:input_only` entries disappeared and `remote` values appeared), so those tests would have failed with or without `remote_state`. That's a real change in the plan, and the goldens should catch it. | error | snippet (inside `remote_state`) | |---|---| | [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) | `+ "compute_last_active_time": "[TIMESTAMP]",` | | [error/2893](http://go/cli/dashboard/error/2893?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d) | `Unexpected output "status": "MONITOR_STATUS_PENDING",` | ## Decisions - Tests opt in with a helper; the runner doesn't strip `remote_state` by default. Some tests check `remote_state` on purpose (dashboards' `serialized_dashboard`, grants' `__embed__` principals, `apps/config-env-no-drift`, `quality_monitors`' allow-list), and a filter hidden in the runner would make those goldens misleading. - `changes[*].remote` is kept. It's the input to the CLI's drift decision, and out-of-band tests depend on it. - When the resource key is missing, `plan_filter.py` fails. The old `jq '.plan."KEY" // .'` silently wrote the whole plan instead. None of the converted goldens relied on that fallback, and a missing entry should fail loudly. - For `skip` entries (the `no_change` goldens), only `action` is left, because a skip has no `changes`. That's intended: the plan's decision is the assertion. If a broken read changed what the CLI sees, the action would turn into update or recreate, and that still shows up. - It's a separate script rather than a flag on `nostamp.py`. `nostamp` is also used for states and request bodies, where `remote_state` doesn't apply. `plan_filter.py` reuses `nostamp.render`, so stamp handling stays in one place. - Scope is the postgres family, which has churned the most. The other cloud goldens that record `remote_state` (dashboards, grants, quality_monitors) check it on purpose or already filter it. Local-only tests can adopt the helper when they're next touched. ## Tests - `selftest/plan_filter` (local): whole plan, one entry, missing entry, non-plan and empty input. - All 12 variants (DMS= and DMS=true) of the six postgres tests pass on AWS (`testme-aws`). The last commit message requests the same AWS run in CI. ## Dashboard [error/3534](http://go/cli/dashboard/error/3534?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3535](http://go/cli/dashboard/error/3535?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3537](http://go/cli/dashboard/error/3537?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d): these motivated the change. They were already fixed on main by databricks#7596, and this PR wouldn't have prevented them (see above). It does prevent the class above: [error/2961](http://go/cli/dashboard/error/2961?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3587](http://go/cli/dashboard/error/3587?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/2964](http://go/cli/dashboard/error/2964?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d), [error/3526](http://go/cli/dashboard/error/3526?main=1&pr=1&end=2026-11-18T11:00:00Z&last=14d). ``` Plan differs (out.plan.update.json): Unexpected output "is_protected": false, "no_expiry": true, Plan differs (out.plan.restore.json): Unexpected output "compute_last_active_time": "[TIMESTAMP]", ``` This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Since 2026-11-12 the postgres API returns
specon GET for projects, branches, endpoints, databases and roles, where before it returned onlystatus. The CLI already reads the embedded spec into the remote state, so plans now show remote values for spec fields. Fields that match the config no longer show up as"action": "skip", "reason": "spec:input_only"changes. This is a backend change: nothing postgres-related changed in the CLI on main in that window, and SDK v0.197.0→v0.198.0 has no change toservice/postgres/model.go.libs/testserver: postgres GET now echoes spec the way the cloud does:update_mask: a masked field missing from the body keeps its value, and setting one oneof member clears the others;pg_version; branchsource_branch,source_branch_lsn,source_branch_time; rolepostgres_role,auth_method,identity_type).-updateagainst AWS. The two local-only tests (postgres_branches/update_expire_time,recreate_source_branch_time) were updated locally.bundle/direct/dresourcesupdated.This also unblocks the SDK bump PRs (#7530, #7584). Their regenerated configs drop
spec:input_onlyfor these fields. Against a testserver that still strips spec, that turns into phantom drift, which is the "Unexpected action='update'" ininvariant/no_drift. With this PR's testserver plus #7530's*.generated.yml, allpostgres_*,invariantanddeploy/readplanacceptance tests pass on this PR's goldens with no further changes.Decision: I updated the goldens rather than recording less, because the change is a permanent backend fix (spec echo), and the remote values make the plans more precise.
Not in this PR: on main, the generated configs (
bundle/direct/dresources/configs/postgres_*.generated.yml) still mark these fieldsspec:input_only, so the CLI keeps ignoring remote-only changes to them. Those files are generated from the OpenAPI annotations, and the SDK bump PRs regenerate them. This PR only makes the tests match the new API and keeps them passing both before and after that regeneration.Tests
testme-aws -updateforpostgres_{projects,branches,endpoints,databases,roles}: pass.libs/testserver/bundle/direct/...unit tests: pass. New unit tests cover the spec merge rules.cloud:so CI runs them on all clouds.Dashboard
"new": "259200s", + "remote": "604800s"(postgres_branches/update_ttl)"new": "600s", "remote": "300s"Plan differs (out.plan.json): Unexpected output "display_name": ..., "history_retention_duration"Unexpected output "attributes": { "createdb": ...(postgres_roles/update)"old": "300s", "remote": "300s"Unexpected output "default_endpoint_settings": { "autoscaling_limit_max_cu": 4,"new": "16000", "remote": "8000""new": 4, "remote": 8(postgres_endpoints/update_autoscaling)+ "postgres_database": "initial_db_name", + "role": ".../roles/app-owner"Unexpected output "is_protected": false, "no_expiry": true,This pull request and its description were written by Isaac.