Skip to content

[DECO-28500] Fix config-remote-sync writing run_as: {} on a principal kind switch - #7493

Merged
petrapostelnicu-db merged 3 commits into
mainfrom
deco-28500-run-as-kind-switch
Nov 15, 2026
Merged

petrapostelnicu-db merged 3 commits into
mainfrom
deco-28500-run-as-kind-switch

Conversation

@petrapostelnicu-db

Copy link
Copy Markdown
Contributor

Changes

  • Let run_as keys that the plan skipped as backend defaults through to config-remote-sync (bundle/configsync/runas.go).

Why

  • The Jobs UI YAML sync runs config-remote-sync after a run-as change.
  • Switching a job's run_as kind (e.g. user_name → service_principal_name) removed the old key but dropped the new one: the plan marks a run_as key the config lacks as a backend default. The YAML ended up with run_as: {}, and the next sync and plan reported no changes.
  • Deploy is unaffected: it always sends the whole configured run_as.

Tests

  • New acceptance tests:
    • config-remote-sync/job_run_as: user → user, user → creator, user → service principal.
    • config-remote-sync/job_run_as_sources: bundle-level run_as, unchanged inherited job, ${var} reference, target override, group.
  • Unit tests for keepSkippedRunAsChange.
  • Local only: the test principals don't exist in a real workspace.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 4e889ab79 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0
test branch main (46546d1) latest (v1.26.0)
TestAccept/bundle/config-remote-sync/job_run_as/DMS= ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/job_run_as/DMS=true ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/job_run_as_sources/DMS= ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/job_run_as_sources/DMS=true ✅ ❌ ❌
TestAccept/bundle/config-remote-sync/job_run_as/DMS=: branch ✅ | main (46546d1) ❌ | latest (v1.26.0) ❌

main (46546d1):

acceptance_test.go:1374: Diff:
        --- bundle/config-remote-sync/job_run_as/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncjob_run_asDMS=1186658630/002/output.txt
        @@ -18,7 +18,6 @@
           run_as.user_name: replace
         
         Resource: resources.jobs.user_to_sp
        -  run_as.service_principal_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.user_to_user
        @@ -43,11 +42,12 @@
         +        user_name: [USERNAME]
                tasks:
                  - task_key: main
        -@@ -25,5 +25,5 @@
        +@@ -24,6 +24,5 @@
        +     user_to_sp:
                name: user-to-sp-[UNIQUE_NAME]
        -       run_as:
        +-      run_as:
         -        user_name: first-user@example.com
        -+        service_principal_name: [UUID]
        ++      run_as: {}
                tasks:
                  - task_key: main

latest (v1.26.0): same output as main (46546d1)

TestAccept/bundle/config-remote-sync/job_run_as/DMS=true: branch ✅ | main (46546d1) ❌ | latest (v1.26.0) ❌

main (46546d1):

acceptance_test.go:1374: Diff:
        --- bundle/config-remote-sync/job_run_as/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncjob_run_asDMS=true3388783330/002/output.txt
        @@ -18,7 +18,6 @@
           run_as.user_name: replace
         
         Resource: resources.jobs.user_to_sp
        -  run_as.service_principal_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.user_to_user
        @@ -43,11 +42,12 @@
         +        user_name: [USERNAME]
                tasks:
                  - task_key: main
        -@@ -25,5 +25,5 @@
        +@@ -24,6 +24,5 @@
        +     user_to_sp:
                name: user-to-sp-[UNIQUE_NAME]
        -       run_as:
        +-      run_as:
         -        user_name: first-user@example.com
        -+        service_principal_name: [UUID]
        ++      run_as: {}
                tasks:
                  - task_key: main

latest (v1.26.0): same output as main (46546d1)

TestAccept/bundle/config-remote-sync/job_run_as_sources/DMS=: branch ✅ | main (46546d1) ❌ | latest (v1.26.0) ❌

main (46546d1):

acceptance_test.go:1374: Diff:
        --- bundle/config-remote-sync/job_run_as_sources/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncjob_run_as_sourcesDMS=1394307067/002/output.txt
        @@ -17,14 +17,12 @@
         Detected changes in 4 resource(s):
         
         Resource: resources.jobs.inherited_to_sp
        -  run_as.service_principal_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.inherited_to_user
           run_as.user_name: add
         
         Resource: resources.jobs.target_override_to_group
        -  run_as.group_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.variable_to_user
        @@ -42,35 +34,37 @@
         +        user_name: second-user@example.com
              inherited_to_sp:
                name: inherited-to-sp-[UNIQUE_NAME]
        -@@ -33,8 +35,10 @@
        -             notebook_path: /Users/{{workspace_user_name}}/notebook
        -
        -+      run_as:
        -+        service_principal_name: [UUID]
        -     variable_to_user:
        +@@ -36,5 +38,5 @@
                name: variable-to-user-[UNIQUE_NAME]
                run_as:
         -        user_name: ${var.runner}
         +        user_name: second-user@example.com
                tasks:
                  - task_key: main
        -@@ -55,3 +59,3 @@
        +@@ -54,4 +56,3 @@
        +       jobs:
                  target_override_to_group:
        -           run_as:
        +-          run_as:
         -            user_name: target-user@example.com
        -+            group_name: data-engineers
        ++          run_as: {}
         
         === Second sync is a no-op
         
         >>> [CLI] bundle config-remote-sync
        -No changes detected.
        +Detected changes in 1 resource(s):
         
        +Resource: resources.jobs.inherited_to_sp
        +  run_as.user_name: remove
         
        +
        +
         === Plan after sync
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 5 unchanged
        +update jobs.inherited_to_sp
         
        +Plan: 0 to add, 1 to change, 0 to delete, 4 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.jobs.inherited_to_sp

latest (v1.26.0): same output as main (46546d1)

TestAccept/bundle/config-remote-sync/job_run_as_sources/DMS=true: branch ✅ | main (46546d1) ❌ | latest (v1.26.0) ❌

main (46546d1):

acceptance_test.go:1374: Diff:
        --- bundle/config-remote-sync/job_run_as_sources/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncjob_run_as_sourcesDMS=true2134952817/002/output.txt
        @@ -17,14 +17,12 @@
         Detected changes in 4 resource(s):
         
         Resource: resources.jobs.inherited_to_sp
        -  run_as.service_principal_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.inherited_to_user
           run_as.user_name: add
         
         Resource: resources.jobs.target_override_to_group
        -  run_as.group_name: add
           run_as.user_name: remove
         
         Resource: resources.jobs.variable_to_user
        @@ -42,35 +34,37 @@
         +        user_name: second-user@example.com
              inherited_to_sp:
                name: inherited-to-sp-[UNIQUE_NAME]
        -@@ -33,8 +35,10 @@
        -             notebook_path: /Users/{{workspace_user_name}}/notebook
        -
        -+      run_as:
        -+        service_principal_name: [UUID]
        -     variable_to_user:
        +@@ -36,5 +38,5 @@
                name: variable-to-user-[UNIQUE_NAME]
                run_as:
         -        user_name: ${var.runner}
         +        user_name: second-user@example.com
                tasks:
                  - task_key: main
        -@@ -55,3 +59,3 @@
        +@@ -54,4 +56,3 @@
        +       jobs:
                  target_override_to_group:
        -           run_as:
        +-          run_as:
         -            user_name: target-user@example.com
        -+            group_name: data-engineers
        ++          run_as: {}
         
         === Second sync is a no-op
         
         >>> [CLI] bundle config-remote-sync
        -No changes detected.
        +Detected changes in 1 resource(s):
         
        +Resource: resources.jobs.inherited_to_sp
        +  run_as.user_name: remove
         
        +
        +
         === Plan after sync
         
         >>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 5 unchanged
        +update jobs.inherited_to_sp
         
        +Plan: 0 to add, 1 to change, 0 to delete, 4 unchanged
        +
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
           delete resources.jobs.inherited_to_sp
        
FAIL

latest (v1.26.0): same output as main (46546d1)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 4e889ab

Run: 43424154462

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 17 9:26
💚​ aws-windows-latest-4core-16gb 1 272 15 5:52
💚​ azure linux-2core-8gb 1 269 17 8:18
💚​ azure-windows-latest-4core-16gb 1 271 15 3:46
💚​ gcp linux-2core-8gb 1 270 17 8:29
💚​ gcp-windows-latest-4core-16gb 1 272 15 5:00
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 4 slowest tests (at least 2 minutes):
duration env testname
4:05 aws linux-2core-8gb TestAccept
4:02 gcp linux-2core-8gb TestAccept
3:52 azure linux-2core-8gb TestAccept
2:31 aws-windows-latest-4core-16gb TestAccept

@petrapostelnicu-db
petrapostelnicu-db force-pushed the deco-28500-run-as-kind-switch branch from 30d51c4 to c2002f7 Compare November 12, 2026 15:30
@petrapostelnicu-db
petrapostelnicu-db marked this pull request as ready for review November 12, 2026 15:31
@petrapostelnicu-db
petrapostelnicu-db requested review from a team as code owners November 12, 2026 15:31
@petrapostelnicu-db
petrapostelnicu-db force-pushed the deco-28500-run-as-kind-switch branch 2 times, most recently from 8fc936c to ecaf0b7 Compare November 13, 2026 12:09
petrapostelnicu-db and others added 3 commits November 13, 2026 13:17
The plan skips a job's run_as keys that the config lacks as backend
defaults. When the remote switches principal kind, the new key is the
edit itself, so config-remote-sync removed the old key and dropped the
new one, leaving run_as: {}. Let skipped run_as keys through.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
config-remote-sync is a hidden command used only by DABs in the workspace;
changes to it do not get release notes.

Co-authored-by: Isaac <no-reply@databricks.com>
@petrapostelnicu-db
petrapostelnicu-db force-pushed the deco-28500-run-as-kind-switch branch from 043bb5d to 4e889ab Compare November 13, 2026 12:17
@petrapostelnicu-db
petrapostelnicu-db added this pull request to the merge queue Nov 15, 2026
Merged via the queue into main with commit e3aeacc Nov 15, 2026
27 checks passed
@petrapostelnicu-db
petrapostelnicu-db deleted the deco-28500-run-as-kind-switch branch November 15, 2026 17:38
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: e3aeacc

Run: 43742275395

Env ❌​FAIL 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
🔄​ aws linux-2core-8gb 3 11 1621 1319 169:54
💚​ aws-windows-latest-4core-16gb 12 1542 1346 171:59
🔄​ azure linux-2core-8gb 10 12 1593 1327 192:52
💚​ azure-windows-latest-4core-16gb 12 1522 1354 170:51
💚​ gcp linux-2core-8gb 12 1610 1325 164:55
❌​ gcp-windows-latest-4core-16gb 37 12 1492 1352 189:37
61 interesting tests: 37 FAIL, 13 flaky, 11 RECOVERED
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🔄​ TestAccept 🔄​f 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/invariant/delete_idempotent ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
💚​ TestAccept/bundle/resources/apps/lifecycle-started 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/data_quality_monitors/lifecycle 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/data_quality_monitors/lifecycle/DMS= 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/schemas/remove_principal_manage ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/schemas/remove_principal_manage/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/volume_change_name ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/volume_change_name/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/volumes/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/volumes/DMS=true ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/job_runs/failed_run/DMS= ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/job_runs/failed_run/DMS=true ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
💚​ TestAccept/bundle/resources/postgres_synced_tables/recreate 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
❌​ TestFsCpDirToDirWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/local_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/local_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToDirFileNotOverwritten/dbfs_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/dbfs_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/local_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/uc-volumes_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/uc-volumes_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileFileNotOverwritten/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/dbfs_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/dbfs_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/local_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/local_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/uc-volumes_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/uc-volumes_to_local ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestFsCpFileToFileWithOverwriteFlag/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncEnsureRemotePathIsUsableIfRepoDoesntExist ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncEnsureRemotePathIsUsableIfRepoExists ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncEnsureRemotePathIsUsableInWorkspace ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncIncrementalFileOverwritesFolder ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncIncrementalSyncFileToPythonNotebook ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncIncrementalSyncPythonNotebookDelete ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncIncrementalSyncPythonNotebookToFile ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestSyncNestedSpacePlusAndHashAreEscapedSync ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
Top 50 slowest tests (at least 2 minutes):
duration env testname
12:56 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
12:40 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:59 azure linux-2core-8gb TestAccept/bundle/resources/materialization_sets/batch/DMS=true
10:26 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:29 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:23 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:19 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:13 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:00 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:35 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:34 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
7:59 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:54 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:53 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:40 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
7:37 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:32 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:08 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
7:07 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:06 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:02 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
6:57 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:57 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:55 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
6:50 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
6:48 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:44 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
6:43 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:38 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:35 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:32 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:31 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:28 gcp linux-2core-8gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
6:25 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
6:15 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:14 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:10 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS=
6:09 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:06 aws-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
5:56 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
5:51 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:50 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:43 aws-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=true
5:41 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:36 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true
5:36 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
5:30 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
5:23 aws-windows-latest-4core-16gb TestAccept/bundle/deploy/spark-jar-task/DMS=
5:09 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
5:09 gcp linux-2core-8gb TestAccept/bundle/deploy/spark-jar-task/DMS=

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 16, 2026
…ion (databricks#7645)

## Changes
- `config-remote-sync` now applies the direct engine's lifecycle
metadata to every change, whatever action the plan gave it
(`bundle/configsync/lifecycle.go`). A remote value is never written back
when it falls under `ignore_remote_changes` (output-only, input-only,
backend-managed), or when it's a backend default the config doesn't set.
- When the plan reports a whole object (one side is nil), those fields
are left out of it, and the change is skipped if nothing else differs.
Inside a remote-only object, only backend defaults pinned to allowed
values are dropped. A field like `node_type_id` in a task added in the
UI may be the user's pick, so it's kept.
- Job `run_as` keys are still synced, so a principal kind switch keeps
working (databricks#7493).
- Moved the backend-default matcher from `bundle/direct` to
`dresources.ResourceLifecycleConfig.IsBackendDefault` so the plan and
config sync share it. The plan's behavior doesn't change.

## Why
Config sync only dropped changes the plan marked as skipped.
Metadata-covered fields still reached the YAML whenever the plan
reported an update, or reported the object that contains them:
- A genie space's `etag` is `output_only`, but `OverrideChangeDesc`
promotes etag drift to an update for deploy, so the etag ended up in the
YAML.
- Input-only fields are never returned by GET, so sync removed them when
the config had an undeployed edit (a job's
`trigger.table_update.condition`), or when the whole object was missing
remotely (a serving endpoint's `telemetry_config.table_names`).
- Output-only and backend-managed fields inside an object added in the
UI were written as-is (for example `aws_attributes` under a for-each
task's cluster).
- Backend defaults were added back: a job's `format` once the config
dropped it, or `spark.sql.ansi.enabled` inside a pipeline
`configuration` map added remotely.

## Tests
- New unit test in `diff_test.go` covering each case above, plus the
`run_as` switch and a backend default with a non-default value. It fails
without the fix.
- `go test ./bundle/...`, the `config-remote-sync` acceptance tests and
golangci-lint pass locally.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…set (databricks#7494)

## Changes
- `config-remote-sync` now writes a remote job `run_as` into a config
that doesn't set one, unless it names the job's owner (`skipJobRunAs` in
`bundle/configsync/runas.go`). It replaces the blanket
`resources.jobs.*.run_as` skip.
- The owner is the principal a job without `run_as` runs as, per the
Jobs API (`run_as_user_name`): the `IS_OWNER` entry of the job's ACL, or
`creator_user_name` when job access control is off. The ACL comes from:
  - the plan, for declared permissions;
- the unmanaged-permissions read from databricks#7536, which now returns the raw
ACL (the current user's `IS_OWNER`/`CAN_MANAGE` is still dropped before
permissions are written);
  - a direct read, for declared permissions that were never deployed.
- The whole-object `run_as` change is still skipped when:
  - the config sets `run_as` (the plan reports the edit key by key);
  - the remote has no `run_as`;
  - the owner is unknown because the ACL can't be read.
- `applyLifecycle` no longer drops a job's whole `run_as` as a backend
default; `skipJobRunAs` decides it.

## Why
- A run-as change in the Jobs UI never reached a YAML without `run_as`,
which is most bundles.
- The Jobs UI always shows a principal (the owner when `run_as` is
unset) and only saves a different one, so the owner check doesn't drop
UI edits.
- Jobs deployed with terraform carry `run_as` set to the owner
([provider
code](https://github.com/databricks/terraform-provider-databricks/blob/76f95e4/jobs/resource_job.go#L844-L857)),
so they are skipped.
- A `run_as` removed from the config and deployed is written back:
`jobs/reset` without `run_as` keeps the previous value, so the job still
runs as that principal.
- This replaces the `--sync-run-as-ids` flag from an earlier revision,
so the caller needs no new flag.

## Tests
- `config-remote-sync/job_run_as`:
  - remote `run_as` on a job without one → synced;
- remote `run_as` naming the owner → not synced, both with the creator
as owner and after an ownership transfer;
  - `run_as` removed from config and deployed → written back;
- `run_as` added locally, not deployed, then changed remotely → one
key-level change;
  - the kind-switch cases from databricks#7493 are unchanged;
  - a second sync is a no-op.
- `config-remote-sync/terraform_state`: a terraform-pinned `run_as`
naming the owner isn't reported.
- Unit tests for `skipJobRunAs` and `jobOwners`.

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants