Skip to content

Detect remote permission changes on undeployed permissions blocks in config-remote-sync - #7716

Merged
ilyakuz-db merged 6 commits into
mainfrom
config-remote-sync-undeployed-permissions
Nov 20, 2026
Merged

ilyakuz-db merged 6 commits into
mainfrom
config-remote-sync-undeployed-permissions

Conversation

@ilyakuz-db

@ilyakuz-db ilyakuz-db commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Changes

bundle config-remote-sync now also reads the remote ACL of a deployed resource whose permissions: block was never deployed, and writes back per-principal differences the same way it does for deployed permissions.

In that window the state has no .permissions entry, so the plan creates it without reading the ACL (no remote state), and neither the declared nor the unmanaged path looked at it. The ACL is now read with the existing readPermissions, diffed against the declared entries with the adapter's keyed-slice comparison, and fed through addDeclaredPermissionsChanges, so only entries written in the config change and the current user's FixPermissions entry is left alone. When the config has no entry for the current user, their default grants (such as a cluster policy creator's CAN_USE) are left out of the comparison, as for an unmanaged import. The run_as owner lookup uses the same read instead of fetching the ACL again. Deploy is unchanged.

Why

After a sync imports an unmanaged ACL and the user saves it, a grant made in the UI before the next deploy was reported as "No changes", and that deploy (Permissions.Set with the declared list) dropped it. The same applied to any hand-written permissions: block before its first deploy. Like other undeployed edits, such a block now follows the remote ACL on sync. Replacements and removals there count toward overwritten_local_edits in telemetry, since the block has no deployed baseline.

Tests

Added a step to config-remote-sync/unmanaged_permissions: after the imported permissions are saved and before they are deployed, a user is granted access to the pipeline remotely; sync writes it into the config and the deploy keeps it. The step fails on main. Added a unit test for the undeployed comparison (level change, removal, addition, current user's entry skipped) and extended the resource selection test. Added a sync before the deploy to config-remote-sync/cluster_policy_permissions: it reports no changes, so the creator's CAN_USE is not imported into the saved block. Ran the bundle/configsync unit tests and the config-remote-sync acceptance tests locally; golangci-lint clean.

Status Check Revision Comments
✅ Local review 19299a5 (latest) Codex and auto review, no P0/P1; open P2s: the telemetry count noted above, an optional changelog fragment, and three on the undeployed read (recreated parents, a read with no declared principals, a unit test for the current user's defaults)
✅ Design review 19299a5 (latest) RIGHT-SIZED: reuses the existing sync-side ACL read, deploy unchanged
➖ CI code review (Isaac) 19299a5 (latest) n/a on CLI PRs
✅ Code owners review b9f03a2 approved by Sankalp-Mittal
⚠️ E2E validation 19299a5 (latest) not run on a workspace; acceptance tests run locally
✅ CI / merge conflicts 19299a5 (latest) all checks pass, no conflicts

✅ success · ⏳ pending · ⚠️ warning · ❌ failed · ➖ n/a

This pull request and its description were written by Isaac.

ilyakuz-db and others added 2 commits November 17, 2026 10:34
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@ilyakuz-db
ilyakuz-db requested review from a team as code owners November 17, 2026 10:40
@ilyakuz-db ilyakuz-db added DABs DABs related issues software-factory Deprecated. Use factory-generated instead. ai-auto Deprecated: use factory-generated instead. labels Nov 17, 2026
@ilyakuz-db ilyakuz-db self-assigned this Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 19299a515 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=2 unreleased=0 coverage=2 failing=0
test branch main (054a0bf) latest (v1.27.0)
TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS= ✅ ✅ ✅
TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS=true ✅ ✅ ✅
TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS= ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=true ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=: branch ✅ | main (054a0bf) ❌ | latest (v1.27.0) ❌

main (054a0bf):

acceptance_test.go:1400: Diff:
        --- bundle/config-remote-sync/unmanaged_permissions/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncunmanaged_permissionsDMS=3514667922/002/output.txt
        @@ -73,26 +73,13 @@
         === Grant access remotely before the saved permissions are deployed
         
         === Detect and save changes
        -Detected changes in 1 resource(s):
        +No changes detected.
         
        -Resource: resources.pipelines.my_pipeline
        -  permissions[user_name='late@example.com']: add
         
        -
        -
         === Configuration changes
         
         >>> diff.py databricks.yml.backup databricks.yml
        ---- databricks.yml.backup
        -+++ databricks.yml
        -@@ -30,4 +30,6 @@
        -         - group_name: users
        -           level: CAN_VIEW
        -+        - level: CAN_RUN
        -+          user_name: late@example.com
         
        - targets:
        -
         === Deploy keeps the remote owner and grant
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
         Created jobs.my_job.permissions
        @@ -101,7 +82,7 @@
         Resources: 2 created, 0 changed, 0 deleted, 2 unchanged
         
         >>> [CLI] permissions get pipelines [MY_PIPELINE_ID]
        -[{"user_name":"owner@example.com","group_name":null,"level":"IS_OWNER"},{"user_name":null,"group_name":"users","level":"CAN_VIEW"},{"user_name":"late@example.com","group_name":null,"level":"CAN_RUN"},{"user_name":"[USERNAME]","group_name":null,"level":"CAN_MANAGE"}]
        +[{"user_name":"owner@example.com","group_name":null,"level":"IS_OWNER"},{"user_name":null,"group_name":"users","level":"CAN_VIEW"},{"user_name":"[USERNAME]","group_name":null,"level":"CAN_MANAGE"}]
         
         === Change a level, add and remove principals remotely

latest (v1.27.0): same output as main (054a0bf)

TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=true: branch ✅ | main (054a0bf) ❌ | latest (v1.27.0) ❌

main (054a0bf):

acceptance_test.go:1400: Diff:
        --- bundle/config-remote-sync/unmanaged_permissions/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncunmanaged_permissionsDMS=t2053223435/002/output.txt
        @@ -73,26 +73,13 @@
         === Grant access remotely before the saved permissions are deployed
         
         === Detect and save changes
        -Detected changes in 1 resource(s):
        +No changes detected.
         
        -Resource: resources.pipelines.my_pipeline
        -  permissions[user_name='late@example.com']: add
         
        -
        -
         === Configuration changes
         
         >>> diff.py databricks.yml.backup databricks.yml
        ---- databricks.yml.backup
        -+++ databricks.yml
        -@@ -30,4 +30,6 @@
        -         - group_name: users
        -           level: CAN_VIEW
        -+        - level: CAN_RUN
        -+          user_name: late@example.com
         
        - targets:
        -
         === Deploy keeps the remote owner and grant
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
         Created jobs.my_job.permissions
        @@ -101,7 +82,7 @@
         Resources: 2 created, 0 changed, 0 deleted, 2 unchanged
         
         >>> [CLI] permissions get pipelines [MY_PIPELINE_ID]
        -[{"user_name":"owner@example.com","group_name":null,"level":"IS_OWNER"},{"user_name":null,"group_name":"users","level":"CAN_VIEW"},{"user_name":"late@example.com","group_name":null,"level":"CAN_RUN"},{"user_name":"[USERNAME]","group_name":null,"level":"CAN_MANAGE"}]
        +[{"user_name":"owner@example.com","group_name":null,"level":"IS_OWNER"},{"user_name":null,"group_name":"users","level":"CAN_VIEW"},{"user_name":"[USERNAME]","group_name":null,"level":"CAN_MANAGE"}]
         
         === Change a level, add and remove principals remotely
         
        
FAIL

latest (v1.27.0): same output as main (054a0bf)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 19299a5

Run: 44732248591

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 275 20 3:57
💚​ aws-windows-latest-4core-16gb 1 274 19 4:13
💚​ azure linux-2core-8gb 1 274 20 3:29
💚​ azure-windows-latest-4core-16gb 1 273 19 4:01
💚​ gcp linux-2core-8gb 1 275 20 3:39
💚​ gcp-windows-latest-4core-16gb 1 274 19 4:17
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:16 gcp-windows-latest-4core-16gb TestAccept
4:11 aws-windows-latest-4core-16gb TestAccept
3:59 azure-windows-latest-4core-16gb TestAccept

@@ -30,4 +30,6 @@
- group_name: users
level: CAN_VIEW
+ - level: CAN_RUN

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct from a bundle perspective that they are merged?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI] Yes, I think merging is right here. A resource's permissions block is the whole ACL the bundle manages: deploy sends it with Permissions.Set (a PUT; DoCreate goes through DoUpdate) plus the deploying user's entry, so any grant missing from the block is removed. Without the late@example.com entry, the "Deploy keeps the remote owner and grant" step would drop the grant made in the UI, which is what sync is there to prevent.

It is also the same merge sync already does for a deployed block: per principal, keyed by user_name/group_name/service_principal_name, so a remote grant becomes an add, a changed level a replace and a principal removed remotely a remove (the job step further down in this test). The deploying user's own entry is skipped.

The trade-off: with nothing deployed yet there is no state to compare against, so sync cannot tell a hand-written entry from a remote change, and a hand-written block that was never deployed is reverted to the remote ACL. That is the same remote-wins sync applies to other local edits that are not deployed yet (the plan marks them Update and sync writes the remote value; that is what LocalEdit counts). If you would rather keep hand-written entries, the undeployed path can be limited to adds, so remote grants are picked up and nothing declared is replaced or removed; that is a small change.

🤖 Factory-generated comment. I still review the code and double-check every claim myself.

@Sankalp-Mittal Sankalp-Mittal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.

No blocking or major issues found.

// by readRemotePermissions.
func jobOwners(ctx context.Context, b *bundle.Bundle, plan *deployplan.Plan, unmanaged map[string][]dresources.StatePermission) map[string]string {
owners := make(map[string]string)
for resourceKey, entry := range plan.Plan {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] readRemotePermissions now also reads ACLs whose .permissions entry is Create, but jobACL still re-reads them here, so such jobs get a second Permissions.Get. Could jobACL use the passed map (and rename unmanaged) instead?

About line 124, which this PR does not change.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik added factory-generated and removed software-factory Deprecated. Use factory-generated instead. ai-auto Deprecated: use factory-generated instead. labels Nov 20, 2026
@ilyakuz-db
ilyakuz-db added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit d1d4c9c Nov 20, 2026
27 checks passed
@ilyakuz-db
ilyakuz-db deleted the config-remote-sync-undeployed-permissions branch November 20, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues factory-generated

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants